- Implement and optimize AI orchestration frameworks like crewAI to manage complex, multi-agent AI workflows and improve task delegation efficiency.
- Develop and maintain a comprehensive testing pipeline using industry-standard tools for experiment tracking, model versioning, and performance visualization.
- Design and execute A/B tests for prompt engineering, comparing different versions to maximize output quality and consistency.
- Establish best practices for prompt templating, version control, and documentation to ensure scalability and maintainability of AI systems.
- Develop a robust evaluation framework for assessing AI model outputs, including automated metrics and human-in-the-loop feedback mechanisms.
- Implement prompt chaining and reasoning techniques to break down complex tasks into manageable sub-tasks for improved accuracy and explainability.
- Create and maintain a knowledge base of effective prompting techniques, including examples and case studies, to facilitate team learning and consistency.
- Develop strategies for handling edge cases and out-of-distribution inputs, including fallback mechanisms and graceful degradation of AI system performance.
- Integrate AI systems seamlessly with the existing Node (TypeScript) backend and React frontend, ensuring full-stack implementation and optimization of AI features within the application architecture.
- Familiarity with Large-language models, and creating other Generative AI techniques in a SaaS environment
- Experience with COTS Generative AI solutions
- Experience with Weights and Biases platform for measuring and tuning as it relates to Generative AI
- Ability to quickly create Proof-of-Concept code and engage with multiple stake holders to drive the PoC productization
- Programming experience with Javascript or TypeScript
- A customer-centric approach and a relentless drive for excellence, aligned with our foundational values.
- Exceptional written and verbal communication skills, capable of inspiring and engaging others
- Self-motivation, integrity, and a track record of delivering impactful results with a high degree of autonomy
- Agility and adaptability to thrive in fast-paced work environments
- A degree in computer science or a related field, with a preference for advanced qualifications
- Multiple Medical plans (one with $0 employee premium option), Dental & Vision plans
- 401k with company match and immediate vesting
- Generous paid time off package and 11 paid holidays
- And much more!