AI Receptionist Agent
Voice agent for local businesses that runs the whole phone call: transcribe, decide, take an action against a real integration, respond in voice. Provider-swappable across STT, TTS, LLM, and transport.
Small businesses can't afford a 24/7 receptionist. A tool-calling voice agent can handle inbound calls end-to-end: pick up, book the appointment, qualify the lead, log to the CRM, escalate emergencies to a human. Built to swap between local free providers in dev and production APIs live, without changing the brain.
A call lands on Twilio, Vapi, or a browser widget. Audio streams into the same voice endpoints regardless of transport. Silero VAD detects speech end, audio flows to a swappable STT provider, and the resulting text enters the brain (packages/core_agent/brain.py). The brain runs a tool-calling loop: emergency intercept first, then input guard against jailbreaks, then the LLM with tool bindings, up to MAX_TOOL_ITERATIONS = 2 (brain.py:184, iterated at :380) before falling back to plain text, then output sanitization for speech (numerics, abbreviations, phone-number spelling). TTS streams the reply back as it's synthesized. Every turn is PII-redacted and written to SQLite; every call fires a CRM sink at the end.
Demo mode: browser mic, local Whisper for STT, Piper for TTS, Ollama for the LLM, a fake calendar. Zero API cost, no phone needed. Production: Twilio for PSTN, Deepgram Nova-3 for STT, Cartesia Sonic-3 for TTS, Groq or a frontier LLM for reasoning, real Google Calendar and GoHighLevel CRM. Same tool-calling brain, same prompts, same tests. The provider factory reads env vars and hands the brain whichever adapter is configured.
When Groq rate-limits mid-call, dead air is not an option. The router (apps/api/app/providers/llm/router_llm.py) iterates LLM_ROUTER_ORDER (DEFAULT_ORDER at :57 = "groq,cerebras,mistral,gemini,nvidia") with DEFAULT_TIMEOUT_S = 8.0 per provider (:59) and DEFAULT_COOLDOWN_S = 30.0 on failure (:58). All three are env-overridable at runtime (:293-295). On a failed provider, `_cool_until[provider_name] = time.time() + cooldown_s` (:330) — the router skips a burned provider until the cool-down passes. Missing API keys skip a provider silently. Beyond the router, 15 distinct LLM provider adapters live under apps/api/app/providers/llm/ (anthropic, cerebras, cloudflare, deepseek, fireworks, gemini, groq, mistral, nvidia_nim, ollama, openai, openai_responses_ws, openrouter, sambanova, together) — the ecosystem width is deliberate so no single provider outage takes the platform down.
Three layers between the caller and the LLM. Emergency classifier (packages/core_agent/emergency_classifier.py:46) runs first as 34 regex patterns across 8 categories: cardiac (4 patterns), respiratory (7), bleeding_trauma (6), neurological (5), overdose_poisoning (3), self_harm (5), anaphylaxis (2), and self-declared emergency (2) — pattern-to-category map at :122. Self-harm hits get a specific escalation message pointing at 988 (the Suicide and Crisis Lifeline), everything else gets a canned "call nine one one" (:104-117). Input guard (input_guard.py) runs next with 26 regex patterns catching jailbreaks, fake authority claims, minor-caller signals, and prompt exfil attempts — costs zero LLM tokens. Write guard runs after the LLM before a booking gets persisted: a second LLM verifies the extracted name, phone, and date against the transcript so hallucinated details don't hit the calendar.
Four vertical tool sets ship today: clinic (check_availability, book_appointment, lookup_faq, escalate_to_human), restaurant (add check_price, send_sms, check_reservations_by_date), real estate (lookup_property, book_viewing, qualify_lead), and wholesaler outbound (log_disposition, send_sms). Each vertical's tool handler is built from a BusinessProfile so the same brain code powers all of them.
First-turn TTS took 2 to 3 seconds cold. The greeting cache warms all business greetings at startup, killing that gap. Tool calls take 800 ms to 1.2 s while the LLM decides — a filler pool of pre-synthesized clips ("one sec", "let me check") masks the pause. Cartesia's SSE streaming means the browser plays chunk N while chunk N+1 is still synthesizing. End-to-end p50 is around 1.5 to 2.5 seconds per turn.
An adversarial harness runs 34 LLM-simulated caller scenarios against the brain, judged by another LLM: booking a doctor for tomorrow, asking a restaurant a compliance-adjacent question, trying to jailbreak the system prompt. Per the README, the pytest suite passes 477 tests (897 test-defs total across 92 test files — some skipped in the current config), and the scenario harness clears 18 of 34 hard cases with 0 hard failures.
PII redactor scrubs phone numbers, SSNs, DOBs, and credit card numbers before anything is written to SQLite. Redaction has a Presidio NER mode for higher accuracy and a regex mode for speed. Every audit trail goes through it; nothing bypasses.