Applied AI & Backend Engineering
Preauth / Victoria
Production voice and text AI, evaluation and observability
Overview
At Preauth, I build production workflows for Victoria, a WhatsApp assistant with voice and text. My work spans Python and LangChain agents, structured extraction, tool calling, conversation evaluation and the operational tooling needed to investigate failures.
Reporting and evaluation
I built and deployed a scheduled AWS reporting pipeline that combines operational metrics with sampled LLM evaluations. Lambda, EventBridge, Bedrock, DynamoDB and S3 support report generation, durable cost controls and duplicate-send protection.
The evaluation workflow uses strict structured outputs, source-evidence validation, reproducible sampling and explicit abstention when customer feedback is insufficient. This makes the findings easier to inspect and act on.
Real-time voice observability
I instrumented calls with LangSmith and fixed tracing issues involving asynchronous transcript ordering, tool execution, session renewal and shutdown, with regression coverage. These contributions sit within the team's broader voice platform.
Engineering focus
- Production Python/LangChain routing, extraction, safety logic and tool calling.
- Conversation evaluation with LangSmith and Opik across thousands of conversations.
- Operational reporting with durable state, controlled retries and evidence-grounded results.
Employer code and customer conversations remain private. This case study describes my contributions without exposing internal data.