🌎 Applied AI Engineer Full time

With the rise of large language models (LLMs) and the evolving landscape of AI tooling, we believe a new kind of engineering is emerging—one that blends the best of classical ML research with backend engineering. We’re looking for someone who not only has deep experience working with LLMs but also understands how to harness these tools to create new, valuable product experiences. This role sits at the intersection of traditional skill sets, requiring both the expertise of ML research and the practical know-how of backend engineering, along with proficiency in the latest AI technologies.

- You understand that AI-based applications thrive on data-driven feedback loops, which will be central to any system you develop. These loops will capture and instrument user data, synthesize core use cases, and implement/test strategies with LLMs to enhance performance.
- You will be responsible for integrating LLMs into software products at Zapier, which will include setting up the necessary infrastructure to ensure performance, scalability, and reliability.
- You will explore new and divergent LLM systems that can create measurable improvements over our current implementations.
- You will design and implement prompting strategies for LLMs to help users create new automations, making the process more intuitive and efficient.
- You will design and develop comprehensive evaluation suites to assess model performance and reliability.
- You will monitor the performance and health of AI systems, proactively detecting and addressing issues such as system failures and performance degradation.
- You will collaborate with Data and cross-functional teams to refine and deploy LLM-based features.

- You have 5+ years of experience in software engineering, with at least 3 of those years dedicated to building distributed, scalable cloud based web applications. You possess strong communication skills, problem-solving abilities, and a drive to deliver outstanding customer experiences for both external users and internal stakeholders.
- You’re familiar with underlying technologies like transformer networks, attention mechanisms, and how they contribute to models’ abilities to generate coherent responses, generate function calls, and perform other language tasks.
- You have at least 1 year of experience working with large language models (LLMs) to perform complex tasks in production environments. You likely have some experience in advanced prompt techniques for LLM grounding, such as chain-of-thought, static few-shot examples, and/or dynamic few-shot examples. You may also have experience fine-tuning LLMs to meet specific needs, focusing on training data optimization and bias mitigation.
- You have deployed evaluation frameworks for LLMs, with an understanding on performance, reliability, and bias assessment.
- You likely have experience with Retrieval-Augmented Generation (RAG) systems and understand how to optimize knowledge retrieval for improved model accuracy and speed. You likely have experience with different indexing and chunking strategies based on the system’s data and goals, as well as semantic search and vector databases, and how they differ from traditional retrieval methods and databases.
- You have experience of working through the full lifecycle of building, testing, deploying, and scaling LLM architectures.
- You can identify and document trade-offs made during the development process. You also have experience building with cloud infrastructure technologies.
- You love shipping to customers. You’ll be on a team focused on understanding customers' needs and translating those needs from specifications into functional, production-ready code. You know how to balance speed versus quality to support the features we build for our
- You embody our values. At Zapier, our values are at the heart of how we work together and how we think about our customers. In our remote setting, they help develop trust and ensure we work and collaborate to democratize automation.

TBD