LLM RAG Engineer

Job Summary
Do you want a chance to become part of the AI revolution with a market-leading and highly visionary software? We are seeking candidates who aspire to becoming an LLM RAG Expert to lead and optimize the integration of Retrieval-Augmented Generation techniques into our AI-powered solutions. The ideal candidate can come from any background but will be able to demonstrate a genuine interest and passion for LLM technology as well as machine learning, natural language processing.

You will collaborate closely with cross-functional teams to create state-of-the-art AI systems that efficiently retrieve and generate accurate, contextually relevant information.
Key Responsibilities
• Design and Development: Architect, implement, and optimize RAG workflows by integrating LLMs (e.g., OpenAI GPT, Llama, etc.) with retrieval mechanisms (e.g., vector search, Elasticsearch, FAISS, Weaviate).
• Data Pipeline Management: Build robust data pipelines for document ingestion, indexing, and retrieval to support scalable RAG solutions.
• Model Tuning: Fine-tune pre-trained language models to enhance performance on domain-specific tasks.
• Query Optimization: Develop and optimize retrieval strategies (e.g., dense or sparse retrieval, ranking algorithms) to maximize system accuracy and relevance.
• Tool Integration: Integrate external databases, APIs, and knowledge graphs into RAG systems to improve contextualization and response generation.
• Experimentation: Conduct experiments to evaluate the effectiveness of RAG workflows, analyze results, and iterate to achieve optimal performance.
• Collaboration: Work closely with product, engineering, and data science teams to deliver solutions aligned with business objectives.
• Documentation: Maintain clear and comprehensive documentation of models, pipelines, and workflows.
Qualifications
Required:
• Bachelor's or Master's degree in Computer Science, Data Science, Machine Learning, or a related field.
• Strong experience with LLMs and familiarity with frameworks like Hugging Face Transformers, LangChain, or similar tools.
• Proficiency in information retrieval systems and vector search technologies (e.g., FAISS, Pinecone, Elasticsearch, Milvus).
• Solid programming skills in Python and experience with machine learning libraries (e.g., TensorFlow, PyTorch).
• Experience working with large-scale datasets, indexing, and retrieval.
• Deep understanding of retrieval-augmented generation techniques and best practices.
• Strong problem-solving skills and ability to work with unstructured data.
Preferred:
• Hands-on experience in implementing RAG for specific use cases such as chatbots, search engines, or recommendation systems.
• Familiarity with cloud platforms (AWS, GCP, Azure) and MLOps tools for model deployment and monitoring.
• Knowledge of knowledge graphs and advanced information retrieval methods.
• Publication or research experience in LLMs, NLP, or IR.
Soft Skills:
• Strong communication and teamwork skills.
• Ability to work in a fast-paced, collaborative environment.
• Detail-oriented mindset and commitment to high-quality output.

About the team:

We are a collaborative team of engineers with a culture of continuous learning and development. We are well established in the market but still enjoy a high-energy startup-style environment. Software engineers work closely with QA and DevOps teams on projects. We are passionate about our work and everyone on the team is hands-on regardless of their role. We focus heavily on unit and integration testing. Our QA testing is automated, helping us maintain high quality in our software and catch defects early in the development cycle. We work in two-week sprints utilizing continuous integration and weekly deployments. We use serverless solutions on AWS when possible. We focus on high availability for our services, minimizing downtime, and service interruptions. We're constantly optimizing our performance and resource consumption.