Engineering Manager
Software Engineering, Other Engineering
New York, NY, USA
We're hiring an Engineering Manager to lead our Observe/Evaluation team — the team responsible for making sure our voice agents actually work in production. Your core focus is two-fold: the observability tooling that gives us visibility into how agents behave, and the evaluation frameworks that let us systematically measure and improve agent quality over time. You'll also touch the integrations that connect our systems to the broader ecosystem, and a customer-facing copilot product that lets customers iterate on their own agents easily.
RESPONSIBILITIES:
- Lead and grow a team of 3-5 engineers — set expectations, support career development, and hold a high hiring bar
- Own execution of the observability and evaluation roadmap: tooling that surfaces how voice agents are performing and frameworks that let teams evaluate agent quality systematically
- Stay hands-on enough to jump into Python code, debug production issues, and make tactical contributions when needed
- Partner with the other product teams to translate voice-agent quality signals into tooling and metrics that are actually actionable
- Own the reliability and scalability of the team's integrations — the connections that link our systems to complex external and internal systems, and that need to scale as the business grows
- Support the copilot product's roadmap in collaboration with Product — the tool that lets customers iterate on their voice agents easily
- Remove blockers for your team — whether that means adjusting process, pairing on a hard debugging session, or resolving cross-functional friction
- Be actively involved in hiring: sourcing, interviewing, and onboarding
WHAT YOU'LL OWN:
- Observability tooling: Systems that give visibility into how voice agents are behaving in production — what's working, what's degrading, and why
- Evaluation frameworks: Tooling that lets the org systematically evaluate and improve voice agent quality over time, rather than relying on ad hoc checks
- Integrations: The complex external and internal integrations that connect our systems to the broader ecosystem, built to scale as usage grows
- Copilot product: A customer-facing product that lets customers iterate on their voice agents easily — this is a product we ship, not an internal tool
ABOUT YOU:
- 6+ years of experience in a relevant engineering role
- 1-3 years of experience managing a team
- Hands-on experience with Python and Typescript — you can still write and debug code at a senior level, even if you now do it less often
- Solid experience with AWS, including services like Lambda and SQS, and streaming systems (e.g. Kinesis)
- Comfortable managing ambiguity — this space (observability + eval for AI agents) is still being defined, and you'll help define it
- Strong written and verbal communication skills; able to work with both technical and non-technical stakeholders
- A track record of mentoring engineers and helping them grow
NICE TO HAVE:
- Experience with MCP (Model Context Protocol) or building agentic applications
- Background in observability, monitoring, or evaluation tooling specifically
- Experience working adjacent to applied AI or ML teams
- Prior startup experience at the Series B–D stage
BENEFITS/PERKS
- We care about your health!
- Medical, Dental, and Vision plans - 80% covered by the company
- Flexible PTO & 11 paid holidays/year
- We care about future you!
- 401k Plan
- Paid parental leave
- Pre-tax commuter benefits
- We care about connection!
- In-office breakfast and snacks daily
- Happy hours, team outings, & annual off-sites
- Complete laptop workstation
- & more to come!
190000 - 225000 USD a year