How it works
The design behind the agent: what it does to fail safely, how the model and the configuration stay swappable, and where each decision is written down. To try it first, see Running it locally.
Why it races the model against an 8-second deadline, and why the losing call completes into an outbox rather than being cancelled, is the problem this solves in the README.
What makes it production-shaped
Fails closed, toward a human. Low model confidence, a schema violation, an exhausted budget, a rate limit, or a provider outage all escalate. The agent never invents a price — an unanswerable question is a handoff, by design.
Untrusted input, treated as such. Contact text is fenced and labelled as data, with the fence markers stripped first so they cannot be forged. Model output is validated before any of it reaches a customer, and replies that leak the prompt are dropped. The golden set includes injection attempts.
A contact's history is theirs, not whoever names them. The shared secret proves a request came from the flow, not which contact it speaks for. Each contact has a random token kept in their own ManyChat custom field and stored here only as a hash. A request without it is answered from its own message alone, so a leaked secret does not expose anyone's conversation (spec 019).
Spend is bounded. Every turn records tokens, cache hits, and estimated cost. Daily token and dollar caps degrade to escalation rather than to an error — a bill cap that fails into a human is the correct failure mode for a business.
Channel constraints are data, not memory. Quick replies silently do nothing on WhatsApp; ManyChat accepts them and the contact never sees them. That lives in a ChannelCapabilities value with a test, not in a comment someone forgets. One deployment answers WhatsApp, Instagram and Messenger, so the value is chosen per request from the channel it names, never fixed at boot (spec 048).
No PII in logs. Redaction is configured at the logger, so a new log statement cannot opt out. LangChain's own tracing switches stop boot, because they would send prompts out of the process; a deployment that sets TRACE_TURNS=langsmith sends its own LangSmith workspace the shape of each turn and never its text (spec 052). Log lines name a conversation by its random ID, never by anything derived from the subscriber.
An eval harness. pnpm eval replays a labelled golden set through the current prompt and asserts escalation behavior, price grounding, prompt leakage, and latency — then prints every reply, because a green suite whose tone has drifted is still a failure and only a person can see that. agent eval conversations scripts the contact's side of whole conversations and runs them twice through the production turn path, so a stage the agent cannot reach in a real conversation fails there. Silence moves a test clock, so follow-ups and playing flows take moments, and a branch inventory counts only the branches a passing step reached, against the prompt as it is now. A calibrated judge grades what a conversation's assertions cannot, quoting the step its evidence comes from, and a person's review resolves what it leaves; --gate decides whether a run and its reviews are evidence a change may ship.
Model agnosticism
No module imports a provider package except src/agent/registry.ts. The active model is a string:
AGENT_MODEL=anthropic:claude-haiku-4-5 # default
AGENT_MODEL=openai:gpt-5-mini # same behavior, zero code changed
AGENT_MODEL=google:gemini-2.5-flash
AGENT_MODEL=ollama:llama3.1:8b # local, free — see running-locally.md
AGENT_MODEL=mock:demo # offline, deterministic, freeCallers depend on an AgentRunner port, not on a model library, so replacing the library is contained behind it: the turn moved from the Vercel AI SDK to LangChain.js and LangGraph.js without a caller changing (ADR-0026). The rest followed (ADR-0030, spec 053), and the service now depends on one model library: the tools are LangChain.js tools, the eval judges and the learning analyst call the same registry's chat model and validate their own JSON, voice notes are transcribed through the openai client behind the Transcriber port, and the offline mock: model and every test fake are LangChain.js chat models.
Configuration is files, not code: config/catalog.json holds every fact the agent may state, so a price change is a JSON edit and kill -HUP. Nothing in config/ is ever committed. The files are described in config/README.md.
Design decisions
Every non-obvious choice is written down in docs/adr/:
| ADR | Decision |
|---|---|
| 0001 | Race the model against a deadline instead of choosing sync or async |
| 0002 | One registry module may import providers; everything else uses a port (superseded) |
| 0003 | Zod schemas generate both the types and the OpenAPI document (superseded) |
| 0004 | A Postgres outbox, not Redis — one datastore, transactional enqueue |
| 0005 | Define the channel port, ship exactly one adapter |
| 0006 | ManyChat does not sign webhooks; the compensating controls, written down (superseded) |
| 0007 | generateObject, not a tool loop, while there are no tools (superseded) |
| 0008 | Ports are implemented by classes; functions stay for pure transformation |
| 0009 | Identifiers are at least two characters, enforced by lint |
| 0010 | A two-step tool loop whose tools stage actions; the server performs them after guardrails |
| 0011 | A judge model grades evals only in a run where it first agrees with hand-labelled calibration |
| 0012 | Each contact has a token held by ManyChat; only a request carrying it reads that contact's history |
| 0013 | History reaches back 30 days; the turn cap resets after 24 hours of silence |
| 0014 | Logs identify a contact by the conversation's random ID, not a hash of the subscriber ID |
| 0015 | The agent takes a lead to the payment link itself; every C6 limit on what it may claim stays |
| 0016 | Read tools are performed when called and the loop grows to four steps; writes are still staged |
| 0017 | Free text only in declared note fields no flow renders, length-capped and stripped of identifiers |
| 0018 | Port to Python on LangChain, LangGraph, FastAPI and Pydantic, cut over beside TypeScript (superseded) |
| 0019 | On an inbound turn a flow is sent when the model calls it, so the reply follows it; other writes stage |
| 0020 | Learning is offline, from paid outcomes; a tactic reaches the prompt only after approval and an eval |
| 0021 | A tenant depends on one published package and starts from a scaffolder; nobody forks to deploy |
| 0022 | Built-in tools stay in the agent; plugins get only what 036 allows, never what a built-in needs |
| 0023 | The turn handler only sequences; TurnScope builds what needs the admitted turn |
| 0024 | However a turn ends, one recorder records it and one delivery sends it |
| 0025 | Python is a second implementation in its own repository, maintained beside TypeScript, not a replacement |
| 0026 | The TypeScript model layer moves to LangChain.js behind the same registry, to rehearse the Python implementation |
| 0027 | The bounded tool loop is a LangGraph.js graph the service draws, never a prebuilt agent |
| 0028 | LangSmith traces the shape of a live turn, never its text; full text only for fixture evals |
| 0029 | Each run is projected by our tracer, then uploaded by the official LangSmith client behind its own hide functions |
| 0030 | The AI SDK leaves the service; where LangChain.js has no counterpart, a port of our own fills the gap |
Behavior is specified before it is implemented, in specs/ — a constitution of non-negotiables, the agent's behavior, and the channel contract. Tests cite the clause they enforce.
Layout
specs/ behavior specified before implementation
docs/adr/ why each decision was made
src/
contracts/ Zod schemas — the single source of truth
agent/ registry (the only provider import), runner, prompt, guardrails
channels/ the port, plus the ManyChat adapter and a local simulator
conversation/ store, rate limits, budget caps
media/ voice notes, images and videos: resolver and ffmpeg splitter
outbox/ deferred delivery
routes/ auth, the turn handler with the race
evals/ golden set + runner
packages/create/ `npm create manychat-ai-agent`: scaffolds a tenant project
test/ unit and integration (PGlite: real Postgres, no container)