Ai Evaluation Engineer

OverviewAbout FirstIgnite: FirstIgnite is the AI-powered business development platform for university technology transfer offices (TTOs).
We help research institutions turn breakthroughs into partnerships, licenses, and companies by combining deep LLM-driven workflows with the relationships that actually move deals forward.
Our product suite spans expert discovery, grants search, and AI-driven outreach — all built on a modern, agentic stack.
We ship fast, we measure everything, and we believe evaluations are the difference between AI features that demo well and AI features that work in production.ResponsibilitiesBuild evaluation infrastructure: Design and maintain eval suites using Promptfoo, LLM-as-judge methodologies, and custom harnesses for features like our expert search system, natural language grants search, and AI SDR agents.Define what good means: Partner with product and domain experts to translate fuzzy customer outcomes (for example, +does this surface the ri...