Three patterns we use to keep AI agents reliable in production
Scoped tools, evaluation before launch and human approval for risky actions: the engineering habits that turn an AI demo into a dependable product.
Finixcode team1 min read
An AI demo takes a weekend. An AI agent your business can rely on takes engineering. These are the three patterns we apply on every agent we put into production.
1. Give agents small, well-defined tools
Agents fail most often when they are given broad access and vague instructions. We expose each capability as a narrow, typed tool with validated inputs and least-privilege credentials, often through a Model Context Protocol (MCP) server.
const issueRefund = tool({
name: "issue_refund",
description: "Refund an order up to the policy limit.",
input: z.object({
orderId: z.string(),
amount: z.number().max(policy.maxAutoRefund),
}),
requiresApproval: (input) => input.amount > policy.autoApproveLimit,
run: ({ orderId, amount }) => payments.refund(orderId, amount),
});The limit lives in code, not in a prompt. The model can ask; the tool decides what is allowed.
2. Evaluate before you launch, and on every change
We build a set of real, anonymised cases with known correct outcomes before we write the agent. Every prompt change, model upgrade or new tool runs against that set, and we track accuracy, cost and latency over time.
- Golden datasets drawn from your own history
- Automatic checks for format, policy and tone
- Model-graded and human-graded scores side by side
- A regression gate in CI: no deploy if quality drops
3. Put humans where the risk is
Not every action needs approval. Reading data rarely does; sending money or messages to customers usually does. We classify each tool by risk and route high-risk actions to a review queue, with the agent's reasoning and evidence attached so approval takes seconds.
Over time, as the numbers prove the agent reliable, approval thresholds can be raised safely, based on data rather than hope.
Bonus: log everything
Every run is traced: the input, the plan, each tool call, the output and who approved it. When something goes wrong, we can replay it, fix it and add it to the evaluation set so it never happens again.