Letting an AI assistant run a calling business without letting it guess

Most in-app assistants answer questions about the product. The one I built for XCall, a calling platform for Bangladeshi shops, does the work. An owner types "call these 300 contacts about the Eid offer, then SMS the ones who are interested", and the assistant builds the contact sheet, the call script or AI voice agent, and the automation, places a test call to the owner, and then runs the campaign.

That is a very different risk profile. This agent can spend the shop's wallet money and ring real customers. Here is what made it safe enough to ship.

Use the product's own services, not a side door

The tempting shortcut is a separate "assistant API" with its own simplified endpoints. I went the other way: every action goes through the same services the pages call. If the assistant creates a call script, it is exactly the call script the editor would have created, with the same validation and the same permission checks.

The cost is real. The assistant inherits every quirk of those services and has to be retested when a page changes. But a parallel API drifts, and drift is how an agent ends up doing something the owner never saw coming.

One registry, filtered by role

Every capability lives in a single registry: what it does, which role it needs, and how dangerous it is. The assistant only ever sees the actions the current member is allowed to run. A call agent cannot talk their way into billing, because billing is not in their catalogue at all, and route tests assert that for every module.

The registry also keeps the prompt small. Area catalogues load on demand: a question about SMS pulls in the SMS tools and nothing else. Covering every area of the product grew the fixed part of the prompt by less than 10%.

Safety tiers instead of "are you sure?" everywhere

Asking for confirmation on every step trains people to click yes. So actions are tiered:

• Reads run freely and come back as cards with real data, not prose.
• Builds (a sheet, a script, an automation) are shown as a plan and need one approval.
• Spending (calls, SMS) needs a second approval, with the cost shown in taka before anything runs.

Every change records an Undo. The target was concrete: "300 contacts, AI calls, SMS the interested ones" finishes from one chat with exactly two approvals and no page visits.

Keep customer data out of the prompt

Pasted contact lists never enter the model's context beyond a three-row sample. The model decides what to do with "the sheet"; the services do it to the actual rows. A test asserts that no pasted number beyond the sample ever appears in prompt text. It is a privacy rule and a cost rule at once, because 300 phone numbers are a lot of tokens to pay for on every turn.

The same tools over MCP

Because every capability already sits in one registry with role checks and tiers attached, publishing it as a Model Context Protocol server was a thin layer, not a second implementation. An external AI client can operate a store through exactly the same rules as the in-app chat.

Measure it like software

Shop owners write in Bangla, English, and Banglish, often in one message. The evaluation set is about 40 real-style requests across every area, scored by an LLM judge, with a target of 90% passing at a median of four model rounds or fewer. Every turn logs prompt, cached, and completion tokens, and a scripted scenario test fails the build if a job goes over its token budget.

What I'd tell someone starting

• Let the agent act through your real services; never give it a shortcut the UI does not have.
• Put permissions and danger levels in one registry, and derive both the prompt and the MCP server from it.
• Make approvals rare and meaningful, and make every action undoable.
• Write the eval set before you tune the prompt.