Tenant configuration
Everything in this directory except *.example and this README is gitignored and must never be committed (Constitution C1).
cp config/prompt.md.example config/prompt.md
cp config/catalog.json.example config/catalog.json
cp config/rules.json.example config/rules.jsonpnpm bootstrap copies the examples for you (it never overwrites an existing file). Invalid config fails at startup rather than at the first customer message. A failed reload (kill -HUP <pid>) keeps the previous config, so a typo cannot take down a running bot.
prompt.md: persona
The system prompt preamble. Write it as instructions to the model: what tone to use, what it may and may not do. The rest of the system prompt — the catalog, the rules, the security fence — is assembled by the server.
The file is read as-is: there is no template substitution. The example uses as a placeholder for the person writing the prompt to replace with the actual business name.
catalog.json: facts the agent may cite
The only source of factual claims the agent may make. A price change is an edit here plus kill -HUP <pid>, with no prompt editing and no deploy.
| Field | Type | Notes |
|---|---|---|
businessName | string | Interpolated into prompt.md |
currency | string | ISO 4217, e.g. USD, ARS |
offerings[] | array (min 1) | What you sell; at least one is required |
offerings[].id | string | Unique identifier |
offerings[].name | string | Display name |
offerings[].description | string | |
offerings[].price | { amount, currency } | amount is in minor units (cents) to avoid float drift |
offerings[].durationHours | number | null | Optional |
offerings[].schedule | string | null | Optional |
offerings[].url | URL | null | Optional |
faq[] | array | Optional (defaults to []) |
faq[].question | string | |
faq[].answer | string | |
paymentOptions[] | array | Optional (defaults to []); see below |
paymentOptions[].id | string | Unique identifier |
paymentOptions[].description | string | How the option works, in your contacts' language |
paymentOptions lists the ways to pay you offer: instalments, a deposit, a private-class rate. The agent presents one when a contact says the price is too high or asks to pay in parts. A request it does not cover, such as a discount, still goes to a person (specs/023-sales-funnel.md).
rules.json: behaviour and limits
messages (required, no defaults)
Every message is required and has no default, because a missing value must fail at boot rather than silently emitting English at a contact who does not read it (Constitution C9).
| Key | When it is sent |
|---|---|
acknowledgement | The model lost the race and the reply is deferred to the outbox |
escalation | The turn hands off to a human (any reason) |
mediaFallback | The contact sent a voice note, image or video the agent cannot read (specs/020) |
closer | The model's reply repeats an earlier one word for word (specs/013) |
mediaFallback is optional: without it, an unreadable media message hands off with escalation instead of asking the contact to type.
closer is optional too: without it, a reply that repeats an earlier one is recorded as duplicate_detected and sent as it is.
Thresholds and limits
| Key | Default | Purpose |
|---|---|---|
confidenceThreshold | 0.6 | Below this, force escalation |
maxTurnsPerConversation | 25 | After this many turns the conversation escalates |
idleResetHours | 24 | Hours of silence after which the turn cap resets (018) |
historyDays | 30 | How far back the model's history reaches (018) |
escalationKeywords (default [])
Substring matches checked before the model runs — an instant, free handoff.
openingTrigger (optional)
A sentinel the channel flow sends to start a conversation, and the scripted reply it produces verbatim. The model never runs for it, so the opening is instant and free and cannot be turned into a handoff by a low confidence score. Unlike escalationKeywords, which match substrings, this matches the whole message — a contact who mentions the phrase must not be able to replay the opening. See specs/001-agent-behavior.md.
"openingTrigger": { "keywords": ["start workflow"], "message": "…" }budget
| Key | Default | Purpose |
|---|---|---|
dailyTokenCap | 1000000 | Max tokens per day |
dailyCostCapUsd | 5 | Max spend (USD) per day |
rateLimit
| Key | Default | Purpose |
|---|---|---|
turnsPerSubscriberPerHour | 60 | Per-subscriber rate limit |
foldBursts
Contacts often split one message over several bubbles. Bubbles that arrive while the agent is still answering an earlier one are always folded: only the last of them calls the model, and its reply answers them all. With "foldBursts": true, a reply the model finished while the contact kept writing is dropped as well, so a burst gets one reply instead of two. The dropped call is still paid for. Off by default. See specs/046-one-reply-to-a-burst-of-bubbles.md.
"foldBursts": trueburstWindowMs
With foldBursts a burst is answered once only while the model is still answering its first bubble. If your contacts write more slowly than the model answers, set burstWindowMs and every turn waits that long for another bubble before it calls the model: a bubble that arrives in time folds the earlier one, and the contact gets one reply once they go quiet. The wait comes out of the 8-second race, so replies that need more of it go out from the outbox after the holding line. Up to 4000; 0, the default, waits for nothing. See specs/047-a-burst-window-before-the-model.md.
"burstWindowMs": 3000tools.json: actions the agent can take (optional)
Without tools.json the agent only replies. With it, the agent can also act on the contact in ManyChat while it answers: send one of your flows, tag the contact, record a choice they made, read what is recorded on them, or write a note for your team. It can also schedule one follow-up for a contact who goes quiet (see Follow-ups). The design is in specs/012-agent-tools.md, specs/024-contact-read-and-notes.md, specs/025-in-window-nudge.md, specs/029-flows-before-the-reply.md, docs/adr/0010-bounded-tool-loop-with-staged-actions.md and docs/adr/0019-flows-are-sent-inside-the-loop.md.
pnpm bootstrap does not create this file, because tools are opt-in. To turn them on:
cp config/tools.json.example config/tools.jsonThen edit the file and restart, or send kill -HUP <pid>.
What you configure
| Tool | What it does in ManyChat | Configured by |
|---|---|---|
send_flow | Sends the contact one of your flows | flows |
add_tag | Adds one of your tags to the contact | tags |
remove_tag | Removes one of your tags from the contact | tags |
set_field | Writes one allowed value to a field | fields |
get_contact | Reads the contact's tags, fields and notes | tags, fields, readable and notes |
write_note | Writes a short free-text note to a note field | notes |
schedule_nudge is a fifth tool, configured by nudge. It makes no ManyChat request when the model calls it; see Follow-ups.
A list you leave empty or omit offers no tool for it. Each entry has:
id: the name the model uses for the entry. Lowercase letters, digits,_and-, unique within its list. It is also what the turn record and the logs show.flowNs,tagorfield: the object's name in your ManyChat account. The model never sees it, so renaming the object in ManyChat only needs an edit here.flowNsper channel: a ManyChat flow is built for one channel. A plain string is the flow forCHANNEL(WhatsApp unless you set it). A deployment that also answers Instagram or Messenger (CHANNELS) names each channel's flow in a map, as every flow and event in the example does:{ "whatsapp": "…", "instagram": "…", "messenger": "…" }. Leave out a channel you have no flow for yet. A turn is offered and sent only the flows its channel has, and never another channel's. That includes the opening, the stage flows, the payment link andevents, so a channel with no payment flow never reacheslink_sent(specs/049).values(fields only): the only values the agent may write. The model picks one of them and can never write free text.description: what the entry is and when to use it. This is the only guidance the model gets, so write it as a condition, for example "Send when the contact asks for details, a syllabus or something to read". It may be in your contacts' language.
The agent always acts on the contact whose message it is answering. No tool takes a contact as a parameter.
The sales funnel (optional)
Four keys turn on the sales funnel of specs/023-sales-funnel.md:
intent: trueon one field marks where the agent records whether the contact means to buy (specs/034-intent-before-the-sale.md). Itsvaluesmust be exactlynot_prospect,prospect, in that order, and the funnel needs it. Until the agent recordsprospect, it sends no flow, sets no other field or tag and schedules no follow-up: an existing customer, a supplier or a wrong number gets answers from the catalog and a person for the rest, never a sales pitch.prospectis final. A contact the funnel had already moved pastnewbefore you added the field counts as a prospect.funnel: trueon one field marks it as the place the agent records where the sale stands. Itsvaluesmust be exactlynew,qualifying,nurturing,offered,link_sent, in that order. The stage only moves forward, and the agent never writeslink_sentitself.role: "payment_link"on one flow marks the flow that sends your payment link. When it is performed, the server writes the funnel field tolink_sent.silent: trueon a flow says it sends the contact nothing: it only records something in ManyChat, such as a tracking event. On the payment-link flow it means your links live in the catalog alone: the agent writes theurlof the contact's offering in its reply, and the server adds it when the reply leaves it out. Every offering then needs aurl. A silent flow takes nosettleSeconds.repeatable: trueon a flow lets the agent send it more than once to the same contact. Every other flow is sent at most once per contact. The payment link is the usual case.
A flow the agent chooses is sent the moment it decides to, before it writes its reply, so the contact receives the flow, then the reply and its question (specs/029-flows-before-the-reply.md). ManyChat starts a flow when it accepts it and takes its own time to play it: a few seconds for an image, a text and a card, longer with a Smart Delay. Tell the agent how long with settleSeconds on the flow, from 0 to 30, and the reply waits that long after the flow is sent (specs/030-reply-waits-for-the-flow.md). Without it, the reply does not wait and can land in the middle of the flow. Time the flow on a phone and round up; a flow edited to play longer needs its value raised.
When the wait ends before the 8-second deadline, the reply goes back in the response as usual. When it ends later, the response says nothing, since the flow is already the contact's answer for now, and the reply is delivered through your reply flow when the flow has played.
Each content flow must be a leaf: it must not start another flow. Nothing here can see inside your flows, so this is yours to check. And turn off any drip sequence that sends the same flows on a timer, or contacts will get each piece twice.
Every other field in fields is a qualification question. The agent learns the answers before it sends the first content flow, one question per turn, so write each description as what to ask, for example "Whether the contact has studied the subject before. Ask before sending content." The example file has two.
Add paymentOptions to catalog.json if you offer instalments or a deposit; without them, "can I pay in parts?" goes to a person as price_negotiation.
Rolling it out
Most of this happens in your ManyChat account, not here:
- Measure the baseline first. Record your conversion rate over the four weeks before rollout, and the dates it covers. Without it, nothing can show the funnel changed anything.
- Create the intent and funnel fields in ManyChat with the names you put in
field, plus one field per qualification question. Create them before deploying: atools.jsonwith a funnel field and no intent field fails to load. Use a name your own flows do not already write. Create a tag for the person who confirms payments, such aspaid; the agent never sets it. - Make each content flow a leaf. Open every flow listed in
flowsand remove any step that starts another flow. - Retire the drip. Change the entry flow so it hands the contact to the agent's Dynamic Block instead of starting a sequence. An agent and a drip on the same contact send every piece twice.
- Tell whoever answers handoffs about the new
payment_reportedreason: those contacts say they have paid and are waiting for a person to check. - Add eval cases for your own flows and objections, and run
pnpm evalagainst the real model (specs/009-tenant-eval-suites.md).
Measuring it
Two rates, reported separately and never combined:
| Measure | Over contacts whose first turn fell in the week… | Where it comes from |
|---|---|---|
| Link-sent rate | …whose payment-link flow was performed | turns.actions, here |
| Paid conversion | …that a person tagged as paid after seeing payment | Your ManyChat account |
Link-sent rate is the one this service can compute. Replace enrolment_link with your payment-link flow's id:
WITH firsts AS (
SELECT c.id, date_trunc('week', min(t.created_at)) AS week
FROM conversations c JOIN turns t ON t.conversation_id = c.id
GROUP BY c.id
), linked AS (
SELECT DISTINCT t.conversation_id AS id
FROM turns t, jsonb_array_elements(t.actions) AS a
WHERE a ->> 'tool' = 'send_flow'
AND a ->> 'id' = 'enrolment_link'
AND a ->> 'status' = 'performed'
)
SELECT f.week, count(*) AS contacts, count(l.id) AS link_sent,
round(100.0 * count(l.id) / count(*), 1) AS link_sent_pct
FROM firsts f LEFT JOIN linked l USING (id)
GROUP BY f.week ORDER BY f.week;Since specs/034, two more figures sit beside it. Replace intent with your intent field's id, and enrolment_link with your payment-link flow's:
WITH firsts AS (
SELECT c.id, date_trunc('week', min(t.created_at)) AS week
FROM conversations c JOIN turns t ON t.conversation_id = c.id
GROUP BY c.id
), prospects AS (
SELECT DISTINCT t.conversation_id AS id
FROM turns t, jsonb_array_elements(t.actions) AS a
WHERE a ->> 'tool' = 'set_field' AND a ->> 'id' = 'intent'
AND a ->> 'value' = 'prospect' AND a ->> 'status' = 'performed'
), linked AS (
SELECT DISTINCT t.conversation_id AS id
FROM turns t, jsonb_array_elements(t.actions) AS a
WHERE a ->> 'tool' = 'send_flow' AND a ->> 'id' = 'enrolment_link'
AND a ->> 'status' = 'performed'
)
SELECT f.week, count(*) AS contacts, count(p.id) AS prospects,
round(100.0 * count(p.id) / count(*), 1) AS prospect_share_pct,
round(100.0 * count(l.id) FILTER (WHERE p.id IS NOT NULL)
/ nullif(count(p.id), 0), 1) AS prospect_link_sent_pct
FROM firsts f LEFT JOIN prospects p USING (id) LEFT JOIN linked l USING (id)
GROUP BY f.week ORDER BY f.week;A prospect share that falls while the number of contacts holds steady means the agent is reading leads as non-leads: read those conversations.
A rising link-sent rate with a flat conversion rate means the agent is asking too early, or too hard. Only the conversion rate tells a better agent from a pushier one, so read real conversations after rollout as well: the golden set only catches the pressure phrasings someone thought to write down.
Several offerings in one funnel (optional)
To sell more than one catalog offering through the same funnel, record the contact's offering in a field and tie each offering's content to it (specs/028-multi-course-funnels.md). Two keys turn it on:
offering: trueon one field marks where the contact's offering is kept. Itsvaluesmust be exactly theids of the offerings incatalog.json, in any order, and it may not be the funnel field.offeringon a flow names the catalog offering it belongs to. The agent can send that flow only once the contact is on that offering. A flow without anoffering(an intro, testimonials, the payment link) is available for every offering.
The agent places a contact who has no offering yet, by asking or once the fit is clear, before it sends any offering's content. It may move them to another offering until the stage is offered. From then on the offering is locked, and a contact who asks to switch goes to a person as explicit_request. A flow the contact already received stays sent when the offering changes.
Most of the setup is in your ManyChat account:
- Create the offering field with the name you put in
field. - Send it to the agent. Add
"offering": ""to the Dynamic Block's request body, using your field's name inside the braces. The agent adds the key to its own follow-up callbacks, but the Dynamic Block your entry flow calls is configured by you. A body that still sends"course"keeps working;offeringwins when both are sent (specs/042). - Set it in each advert's entry flow, so a lead who comes from one offering's advert starts on that offering. Without it the agent asks.
- Branch the payment flow on it. One payment-link flow serves every offering. It reads the offering field and sends that offering's link. Nothing here can see that branch, so test it by hand for every offering.
Reading the contact and writing notes (optional)
get_contact lets the agent read what is recorded on the contact right now, including what your own flows set. It returns ids only: the tags you list that the contact has, the value of each field you list, and each note's text. It never returns the contact's name, phone, email or anything you did not list. A field holding a value that is not one of its values reads as other.
readablelists tags and fields your flows set that the agent may see but never write, in the same shape astagsandfields. For example a tag your ad flow adds.noteslists free-text fields the agent may write, for whoever follows up. Each hasid,field,description,maxLength(at most 500) and"neverRendered": true. The last is your promise that no flow ever shows that field to a contact: nothing here can check it, so the file refuses to load without it.onEscalation: trueon a note, usually a handoff summary, means it is still written when the agent hands the contact to a person. Every other action is dropped on a handoff.
Before a note is written, links, emails, phone numbers and long numbers are replaced by [removed], whitespace is collapsed, and the text is cut to maxLength at a word. A name or any other detail in prose is not removed. A note replaces what the field held. The turn record and the logs hold the note's length, never its text.
The agent reads only on a turn that carries the contact's token, at most twice a turn, and gives up on a read after 1.5 seconds. A failed read is logged at warn as contact read failed and the turn goes on without it.
Follow-ups (optional)
A nudge section lets the agent follow up once with a contact who stops replying, while WhatsApp's 24-hour window is still open. The design is in specs/025-in-window-nudge.md. A timer does not decide this: the agent decides whether a follow-up would help, and what it says. The server only checks that a follow-up is still allowed when it is due.
{
"nudge": {
"delays": [
{ "id": "later_today", "minutes": 120 },
{ "id": "tomorrow", "minutes": 1200 },
],
"humanActiveTag": "human-handling",
},
}delays: the waits the agent may choose from, at least one.idfollows the same rules as every otherid.minutesmay be at most 1380 (23 hours). The hour of margin covers the worker's poll, a deferred delivery and the model call, so the follow-up still lands inside the 24-hour window. Any more fails the boot.humanActiveTag(optional): the name of a tag your team or your flows set in ManyChat while a person handles the contact. When it is due, a follow-up for a contact with that tag is cancelled. The model never sees the name. It may not be one of yourtags[].tag, so the agent can never add or remove the tag that silences it.
Without a nudge section the agent is offered no schedule_nudge tool, and the follow-up worker has nothing to do.
How a follow-up runs
- The agent schedules it. On an ordinary turn the model may call
schedule_nudgewith a delay id, usually after an offer, an objection or a question of its own. Like every write but a flow, it is only staged. It is discarded if the turn hands off, and it is performed after the reply has gone out. - One waits at a time. Performing it writes a row in the
nudgestable, dueminutesafter that moment. Scheduling again replaces the one that is waiting. The database allows only one pending row per conversation. - Anything that makes it wrong cancels it. See the table below. All of these checks run before the model is called, so a cancelled follow-up costs nothing.
- At due time it is a model turn. The worker checks every 15 seconds, claims due rows with
FOR UPDATE SKIP LOCKED, and runs the agent with the conversation's history. In place of a contact message it gets a note from the system:[no reply from the contact since 2026-01-15T10:00:00.000Z; decide whether to follow up]. No contact wrote that note, so it is not fenced. The contact never sees it. A follow-up turn is not offeredschedule_nudge, so it cannot schedule another one. After a contact's last message they get at most one follow-up, and then nothing until they write again. - The model may decline. It declines by escalating. Nothing is sent, not even your
escalationmessage, because the contact asked nothing. No person is notified, and any action it staged is dropped. The turn is recorded asnudge_skippedand never appears in later history. A failed model call ends the same way. - Otherwise it is delivered like a deferred reply. Nobody is waiting on a Dynamic Block, so the reply goes straight to the outbox: written to
MANYCHAT_REPLY_FIELD, rendered byMANYCHAT_REPLY_FLOW_NS, then any actions it staged. The turn is recorded asnudge_sent. It counts against the budget, the hourly rate and the turn cap like any model turn.
Why a follow-up is cancelled
cancel_reason | When |
|---|---|
contact_replied | The contact sent any message, before the follow-up was due or while its model call was running |
escalated | A turn of the conversation handed off to a person |
link_sent | The funnel field reached link_sent: the sale is closed |
window_closing | At due time, more than 1380 minutes have passed since the contact's last message |
human_active | At due time, the contact has the humanActiveTag tag |
read_failed | At due time, the contact's tags could not be read from ManyChat |
cap_reached | At due time, the daily budget, the hourly rate or the turn cap would refuse the turn |
A cap never sends your escalation message for a follow-up. The contact asked nothing, so there is nothing to hand off. A failed tag read cancels rather than sends: an unprompted message on top of a person's conversation is worse than a missed follow-up.
Seeing what happened
Each row of nudges holds the conversation id and no contact data. Its status is pending, running (claimed by the worker), sent, skipped or cancelled, and cancel_reason is set when it is cancelled. A row left running means the process stopped during the turn. That follow-up is lost, never sent twice.
-- What became of follow-ups in the last week
SELECT status, cancel_reason, count(*)
FROM nudges
WHERE created_at > now() - interval '7 days'
GROUP BY 1, 2 ORDER BY 3 DESC;
-- Reply rate: follow-ups the contact answered within 24 hours
SELECT count(*) FILTER (WHERE EXISTS (
SELECT 1 FROM turns r
WHERE r.conversation_id = n.conversation_id AND r.role = 'user'
AND r.created_at BETWEEN n.created_at AND n.created_at + interval '24 hours'
))::float / NULLIF(count(*), 0) AS reply_rate
FROM turns n
WHERE n.outcome = 'nudge_sent';The reply rate is the only evidence that follow-ups help rather than annoy. Report it next to the link-sent rate of specs/023. It does not say whether a follow-up read as helpful or as pressure. For that, read the follow-up conversations themselves.
Before turning follow-ups on
Tag every takeover. A person who takes a conversation over without setting
humanActiveTagis invisible to this check. So is a tag spelt differently fromhumanActiveTag: the match is exact, and nothing warns you when it never fires. Without ahumanActiveTagthere is no takeover check at all.The window is computed from what this service saw. A contact who wrote to you through a flow that never reached this service has a later window than the one computed here. The error is on the safe side: the follow-up is cancelled, never sent outside the window.
Reads need
MANYCHAT_API_TOKEN. Without it, every follow-up for a tenant with ahumanActiveTagis cancelled asread_failed.Test the follow-ups. An eval case with
"nudge": trueruns as a follow-up turn on itshistory, andtextis ignored. Incases.jsonlit is one line:json{ "id": "nudge-after-instalments", "history": [ { "role": "user", "text": "can i pay in instalments?" }, { "role": "agent", "text": "Yes, any course can be paid in three equal monthly instalments." } ], "nudge": true, "text": "", "expect": { "escalate": false }, "must_contain": ["instalment"], "review": "Follows up on paying in instalments, in one short message that does not press." }expect.escalate: trueasserts the model declines. Run them withpnpm evalagainst the real model.Turn off timed reminders. A ManyChat sequence that messages quiet contacts on a timer sends a second follow-up on top of this one.
Startup checks
tools.json is validated like the other config files: a malformed file fails the boot, and a malformed reload is refused and the previous config is kept. A file is also refused if it names an object this service already uses for delivery:
- a flow or event whose
flowNs, or any value of its map, isMANYCHAT_REPLY_FLOW_NS,MANYCHAT_REPLY_FLOW_NS_INSTAGRAMorMANYCHAT_REPLY_FLOW_NS_MESSENGER; - a field whose
fieldisMANYCHAT_REPLY_FIELDorMANYCHAT_TOKEN_FIELD; - a note whose
fieldis either of those, afieldsentry'sfield, or another note's.
It is also refused if more than one field is marked funnel, if a funnel field's values are not the five stages in order, or if more than one flow has role: "payment_link". With offerings, it is refused if more than one field is marked offering, if one field is marked both funnel and offering, if the offering field's values are not exactly the catalog's offering ids, if a flow names an offering the catalog does not have or has an offering with no offering field, or if the payment-link flow has an offering. A note without "neverRendered": true or with a maxLength over 500 is refused too, and so is a nudge delay over 1380 minutes, or a humanActiveTag that is empty or is one of your tags[].tag.
What happens on a turn
- The model chooses. It may call tools before writing its reply. A turn takes at most 8 actions, and identical calls count once. A call past the limit is dropped. The model gets up to three rounds of tool calls before it must write the reply, so it can read, act and read again.
- A flow is sent when the model calls it (
specs/029). The contact receives it before the reply, and the model is told whether ManyChat accepted it. A payment link takes the turn's earlier funnel writes with it, solink_sentis the last stage written. On a follow-up turn (nudge) flows are staged like everything else. - Every other call only stages the action: tags, fields, notes and follow-ups reach ManyChat after the reply.
- A flow is sent when the model calls it (
- The model writes the reply. It is told which flows went out, so it writes a reply that follows them and does not repeat their content, and what was staged, so it says what it is doing rather than claiming it is done.
- The guardrails decide. If the turn ends in a handoff for any reason, every staged action is discarded. The reasons are: the model escalated, confidence was too low, a prompt leak, an invalid reply, a failed model call, or
MODEL_ABORT_MS. A note markedonEscalationis the one exception: it is still written when the model escalated or confidence was too low. A flow already sent cannot be recalled: the contact receives it, then the handoff message. - The text goes first, then the staged actions. A reply that follows a flow waits for the flow's
settleSecondsfirst. On an inline reply, they run after the response to ManyChat has been sent. On a deferred reply, the outbox worker runs them after it has delivered the text, and if the reply is dead-lettered they are dropped. They run in the order the model staged them, one request each, with a 10-second timeout. Each action, sent or staged, gets one attempt: a failure is logged atwarnasaction failedand is not retried. - Later turns remember. History shows the model a line such as
[actions performed: send_flow foundation_brochure]under the reply that performed it, so it does not send the same flow again. Onlyperformedactions are listed. The server writes that line, and the guardrails remove it if the model copies it into a reply.
Turns where the model does not run (the scripted opening, escalation keywords, budget, rate and turn caps) offer no tools and stage nothing.
Seeing what the agent did
Every agent turn records its actions in the actions column of turns. It is null when no tool was offered and [] when tools were offered and none was chosen. Otherwise it has one entry per action, sent or staged, in the order the model made them:
[{ "tool": "send_flow", "id": "foundation_brochure", "status": "performed" }]How an action reaches each status. The rounded boxes are the statuses you will see in the column:
status | Meaning |
|---|---|
staged | Waiting to be performed, after the response or the outbox |
performed | ManyChat accepted the request |
failed | ManyChat refused it or it timed out; error says why |
discarded | The turn ended in a handoff (never a flow already sent) |
dropped_over_cap | Staged past the limit of 8 and never sent |
dead_lettered | Its deferred reply was dead-lettered, so it was never sent |
Entries hold ids and values only, never ManyChat names or contact text. A note's entry holds its length instead of its text. An entry that stays staged means the process stopped before the action ran. That action is never retried.
SELECT t.created_at, t.outcome, t.actions
FROM turns t JOIN conversations c ON c.id = t.conversation_id
WHERE c.subscriber_id = '<subscriber id>' AND t.role = 'agent'
ORDER BY t.seq;Before turning tools on in production
Expect slower replies. A tool turn takes two to four model calls. In a local run on 2026-09-28 with
openai:gpt-5-miniat low reasoning effort, four two-call tool turns took 8.4 to 15.5 seconds. All of them missed the 8-second deadline, so each contact got the holding message first and the answer through the outbox. The same model answered a question with no tools offered in 5.8 seconds.Test the descriptions. No check can tell whether a
descriptionleads the model to the right flow. Add eval cases that state the expected choice withexpect.actions, as the golden set does:jsonl{"id":"flow-brochure","text":"can you send me something to read about the foundation course?","expect":{"escalate":false,"actions":["send_flow foundation_brochure"]}} {"id":"flow-not-for-schedule","text":"when does the weekend intensive run?","expect":{"escalate":false,"actions":[]}}[]asserts that tools were offered and none was chosen. Run them withpnpm evalagainst the real model.performedis ManyChat's answer, not the contact's. It means ManyChat accepted the request. A flow that is unpublished, or points at a missing file, can still be accepted. Send each flow to yourself once from ManyChat before listing it here.Actions need
MANYCHAT_API_TOKEN. Without it every action is recorded asfailed.
prompt-overrides.json: replace a prompt section (optional)
The server appends its own conversational sections after prompt.md. When one of them says something your business handles differently, replace that section's body here rather than arguing with it from the persona. Only the replacement reaches the model; the default it displaces is gone.
{
"closing": "Ask a next-step question while the contact is deciding. After a farewell, return null."
}pnpm bootstrap does not create this file, because overrides are opt-in. config/prompt-overrides.json.example replaces two sections for the demo tenant. To start from it:
cp config/prompt-overrides.json.example config/prompt-overrides.jsonThen edit the file and restart, or send kill -HUP <pid>.
| Key | Replaces | In the prompt when |
|---|---|---|
operating | Response and escalation policy | Always |
format | Style, language and presentation | Always |
closing | When to ask a closing question or send null | Always |
actions | When to choose an available action | Any tool is offered |
intent | How to read and record purchase intent | A field is marked intent |
sales | Qualification, objections and sales progress | A field is marked funnel |
offerings | How to select and continue the contact's offer | A field is marked offering |
followUps | When and how to propose a follow-up | tools.json configures nudge |
Each value is the whole new body: plain text, with no templates, includes or variables. Headings and the order of sections are the server's. Leave a key out to keep its default; a section whose capability is not configured stays out of the prompt whatever you write. Where the persona and an override disagree, the override wins, so keep the two consistent.
Some instructions are never replaceable: security and input fencing, grounding in the catalog, the output fields and confidence threshold, and the RULES block after each tool section, which describes what a tool actually does. An override cannot make a refused action succeed.
Unknown keys, blank or non-string values and malformed JSON fail the boot, and a reload with them keeps the previous configuration. Error messages name the key, never your text. Specification: specs/044-tenant-prompt-section-overrides.md.
Learning from outcomes (optional)
With a learning block in rules.json, a weekly job compares the conversations of contacts who paid with those who did not, and an analyst model proposes selling tactics. Nothing it proposes reaches the agent until a person approves it and an eval shows no regression. How it works, how to set it up and the weekly routine are in docs/guides/learning-from-outcomes.md; the design is specs/031-learning-from-outcomes.md and docs/adr/0020-learning-is-offline-and-human-approved.md.
"learning": {
"language": "English",
"convertedTag": "paid",
"maxRunCostUsd": 5
}convertedTag: the ManyChat tag your team sets on a contact once they have paid. It is how the job knows who converted.language: what the proposals are written in: yours.maxRunCostUsd: the most one weekly analyst call may cost. Required. It is never charged to the dailybudget.
It needs a field marked funnel in tools.json, and INSIGHT_MODEL in .env names the analyst. Then, each week:
pnpm insights:review # approve, edit, reject or retire
PLAYBOOK_VERSION=<id> pnpm eval # a real model, not eval:mock
pnpm insights:activate <id> # refused if a case newly fails
pnpm insights:report # conversion rate by versionThe first activation is compared against the prompt with no playbook, so run pnpm eval once without PLAYBOOK_VERSION as well. A running server picks up an activation within a minute; a learning block added later needs a restart.
plugins.json: tools of your own (optional)
When tools.json cannot express an action, a write to your own CRM say, a plugin package adds a tool the agent stages and performs exactly as it does its own. config/plugins.json lists the packages to load:
{ "plugins": ["agent-plugin-example-crm"] }There is no example file, because every package listed must be installed or the server refuses to start. Plugins load at boot, so a change to one is a restart, not kill -HUP. How to write one: docs/guides/writing-a-plugin.md; the design: specs/036-plugins-extend-through-the-ports.md.