Senior GenAI Lead

Job Overview:

We are looking for a Generative AI Lead to join our team and help us to design, build, and deploy state-of-the-art generative AI solutions. As a Generative AI Lead, you will be

• Responsible for leading and mentoring a team of engineers in the development of innovative generative AI solutions to real-world problems
• Work closely with pre-sales and business stakeholders to understand their needs and develop Solutions/PoCs that address those needs
• Leverage your knowledge in LangChain, LangSmith, prompt engineering, and cloud technologies to build state-of-the-art AI systems

Key Responsibilities:

Architectural Design:

• Design and develop scalable generative AI systems and frameworks using Python and related libraries
• Evaluate and select appropriate generative models and algorithms for various use cases
• Ensure architectural decisions align with organizational goals and technical requirements
• Design and integrate end-to-end use cases as per the requirements

Generative Model Understanding:

• Implement and optimize generative models, including but not limited to transformers, Open AI
• Develop and refine techniques for improving model performance, such as fine-tuning and transfer learning
• LangChain, LangSmith Integration: Utilize LangChain, LangSmith to build and deploy complex AI systems that integrate multiple language models
• Design workflows and pipelines using LangChain, LangSmith for effective prompt management and model orchestration

Prompt Engineering and Methods:

• Develop and implement prompt engineering strategies to enhance model outputs
• Experiment with and optimize various prompt methods to address specific application needs
• Collaborate with stakeholders to create and refine prompts for different scenarios
• Document Vectorization and Indexing for RAG:
• Design and implement document vectorization strategies to transform text documents into meaningful vector representations
• Develop and manage indexing techniques to support efficient retrieval of relevant information in a RAG setup
• Integrate vector databases and retrieval mechanisms to enhance the performance of RAG systems
• Optimize document indexing processes to ensure quick and accurate retrieval of contextually relevant information
• Prompt management and evaluations using tools like LangFuse

Technical Implementation:

• Write and maintain Python code for model training, evaluation, and deployment
• Implement best practices for code quality, version control, and testing
• Cloud and Infrastructure:
• Design and manage cloud-based AI solutions using platforms such as AWS, Azure, or Google Cloud
• Ensure the scalability and reliability of AI systems in cloud environments
• Optimize resource usage and manage costs associated with cloud infrastructure

Collaboration and Leadership:

• Lead and mentor team of data scientists and AI/ML engineers
• Collaborate with cross-functional teams to define project requirements and deliver solutions
• Communicate technical concepts and results to non-technical stakeholders

Research and Development:

• Stay updated with latest advancements in generative AI and related technologies
• Contribute to research publications and participate in relevant conferences or forums

Qualifications:

Education:

• Bachelor’s or Master’s degree in Computer Science, Data Science, Machine Learning, or related field

Experience:

• Proven experience as a GenAI Lead or in a similar role, focusing on generative AI
• Strong portfolio of projects involving LangChain, LangSmith, prompt engineering, RAG and cloud-based AI solutions
• Knowledge of AI Testing tools

Technical Skills:

• Expertise in generative AI models, including transformers, GANs, and VAEs
• Proficiency in LangChain, LangSmith for integrating and managing language models
• Exposure to major LLMs, HuggingFace, Kaggle models
• Advanced knowledge of prompt engineering techniques and prompt methods
• Strong programming skills in Python, with experience in relevant libraries (e.g., TensorFlow, PyTorch)
• Experience with cloud platforms (AWS, Azure, Google Cloud) and management of cloud-based AI infrastructures
• High level understanding of containerization and orchestration technologies (e.g., Docker, Kubernetes) is a plus

Soft Skills:

• Excellent problem-solving and analytical skills
• Strong leadership and team management capabilities
• Effective communication skills with ability to present complex concepts clearly to internal and external stakeholders
• Ability to work collaboratively in a fast-paced, dynamic ‘Can Do’ environment