- Develop and maintain our LLM testing infrastructure
- Improve model performance
- Ensure quality across their AI agent deployments
- Use customer feedback to identify patterns and areas for improvement
- Develop test cases based on these real-world examples to improve our AI agentβs performance
- Iterate and improve on our LLM prompts to achieve our quality targets
- Gather and curate high-quality training data for model fine-tuning
- Prior experience working with large language models, with a focus on prompt engineering.
- Strong written communication skills in English, emphasizing clarity and precision.
- Strong scripting/programming skills in Python or Typescript
- Experience with testing frameworks and automation tools (preferred)
- Excellent analytical and problem-solving abilities (preferred)
- Strong documentation and communication skills (preferred)
- Understanding of ML concepts and fine-tuning processes (preferred)
TBD