How to make an AI agent actually useful

Every AI tool has a great first week. The demo lands. The team is excited. Someone shares it in the all-hands. Then it's the third Monday and traffic to the thing has dropped by 80% and nobody's saying anything.

Roughly half of B2B AI pilots are dead by the six-month mark. The LLM didn't get worse. What happened is that the agent stopped fitting the team, and nobody had a way to make it fit again. What separates the agents that survive from the ones the team quietly stops opening is not the model, or the demo polish, or the marketing budget. It's three things: context, reach, and trust.

If your agent has all three, it becomes furniture — always there, always used, gets updated when the team changes. If it's missing any one, it goes into the pilot graveyard.

1. Context that isn't prompt engineering

The most common failure mode: the agent's behavior lives in a text field labelled "System Prompt" that only your one ML person is willing to touch. Every time the team wants to change something — the tone of an email draft, an escalation rule, an exception for a specific customer segment — they have to file a ticket with the ML person. The ticket takes two weeks. By week three the team stops filing them and starts working around the agent instead.

This is not an agent problem. It's a context ownership problem.

The pattern that works is playbooks: plain-language rules per workflow, written by the people who own the workflow. A CS lead writes the refund escalation. An SDR writes the discovery-to-handoff runbook. An SRE writes the on-call triage flow. Each playbook says, in whatever level of detail feels right, this is how we handle X.

Bad:

[System prompt, 4KB, edited by the ML team]
You are a customer support assistant for Acme Corp. When
handling refunds, first check the customer's subscription
history, then their internal notes, then draft a reply
matching the tone described in the "Voice" section below...

Good:

# Refund escalation
When a customer emails about a refund:
1. Check subscription history (last 12 months)
2. Read internal notes for context
3. If refund < $200: draft a polite reply, apply credit
4. If refund > $200 or account age < 30 days: escalate to
   the on-call CS lead in #cs-escalations
5. Match tone: warm but firm. Don't apologise for policy.

Both encode roughly the same rules. The first requires an ML-savvy editor because a wrong word breaks the whole prompt. The second is a document the CS lead can update at 3pm on a Tuesday without asking anyone. Same information, entirely different governance.

The test is: who on your team can safely update the agent's behavior tomorrow, without a review from an engineer? If the answer is "nobody," you have a prompt, not a playbook.

2. Reach — real tool access, not token gestures

The second failure mode is agents that can't actually do anything. Every AI vendor page says the same thing: "connects to Slack." What that usually means is the agent can search public messages. That's a demo, not a tool.

Real reach means the agent can perform every action a human on the team can — read a thread, post a reply, react with an emoji, edit its own past post, pin a message, add a user to a channel, upload a file, kick off a workflow. The value of an agent scales roughly with the surface area of what it can actually do. Halve the action surface and you halve the utility, usually by more.

The concrete test isn't "does it integrate with X" — it's "walk me through a task that requires it to write four things to three systems in the right order, and recover if one of them fails." If the demo can't do that, the "integration" is a checkbox, not a tool.

Real reach also means write access, not just read. Read-only "agents" are chatbots pointed at your data. Useful ones categorise, tag, reassign, draft, notify, post digests, close tickets, update fields, dispatch on-call. If the vendor's page has a list of integrations and each one says read: yes, write: no — you're looking at a search engine, not an agent.

3. Trust — audit, observability, and constraints

The third and most under-appreciated ingredient. You cannot deploy something to production that you can't observe, can't bound, and can't override. Three constraints, in order of importance:

Audit chain. Every LLM turn, every tool call, every decision — timestamped, reviewable, exportable. When the 3am scheduled routine makes a bad call, you need to know within the hour what happened and why. Not "the agent decided to do X because the model thought it was best." A real chain: at 03:12:04 the agent read ticket #4823, retrieved the customer profile via the CRM tool, called the refund tool with amount=$150, got a 200 response, drafted an email, posted a summary to #cs-log. Every link auditable, every input redacted for secrets, every output preserved.

Without an audit chain, you cannot let the agent near anything you'd have to defend to a board.

Spend limits. Bound $ per workspace per month. Bound $ per user per day. Kill runaway loops before they run away. Every mature LLM deployment eventually hits a stuck-in-a-loop condition where the agent, confused about the state of the world, calls the same tool a thousand times before someone notices. If there's no ceiling, you find out via the invoice at month-end.

The right knob is per-workspace and per-tier caps, plus alert thresholds ("you're at 80% of the monthly allotment, three weeks left"). The wrong knob is nothing, or "call sales if you have questions about billing."

Human-in-the-loop for high-stakes actions. Not everything the agent proposes should be an action. Sending an external email should require a human OK, at least until the team has calibrated trust. Deleting data should be irreversible-with-consent. Refunds over a threshold should route to a lead. This is not "the agent isn't ready for autonomy" — it's "you don't want the training wheels off before you've watched it ride." Even mature agents keep a HITL rail on the high-blast-radius actions, because the cost of one mistake there is higher than the tedium of one approval click.

The failure mode all this fixes

The pilot graveyard is full of agents that had two of the three ingredients. The prepackaged vendor agent had reach and audit but no real context — the team could never make it speak their language. The internal LangChain project had context and reach but no audit — one weekend an intern deployed a change and blew the LLM budget in a day, and it became easier to shut it down than to fix. The read-only assistant had context and audit but no reach — cute in demos, ignored in the real workflow.

All three, or none.

The good news: you can build for this. The three ingredients are compatible with each other. Playbooks in plain markdown don't preclude real tool integration; real tool integration doesn't preclude audit; audit doesn't preclude spend limits. If a vendor tells you these are in tension, that's a design failure on their end, not a fundamental constraint.

The "would a new hire trust this?" test

Concrete checklist for evaluating any agent deployment — including your own. If you hired someone tomorrow and handed them the agent, could they:

1. Understand what it can and can't do? ("Here are its tools, here are its playbooks, here are the paths it will and won't take.")
2. Change what it does when they need to? ("Update the playbook, no ML review required.")
3. Audit what it did last week? ("Here's the log, filterable, exportable.")
4. Bound what it could cost this month? ("Here's the spend cap, alerts fire at 80%.")

If any answer is "no," the deployment isn't ready for real work yet. It might be ready for a controlled pilot; it isn't ready for the team to bet on.

Where the market is heading

The AI agent market is bifurcating into two lanes, and it's worth being clear which one you're in.

Agent-as-feature — AI features bolted onto SaaS you already own. Notion AI in Notion. Copilot in Office. Slack AI in Slack. Good for narrow, SaaS-adjacent tasks. The vendor knows their product; they can build a competent agent for the workflows their product owns. Don't fight this lane with a general-purpose tool.

Agent-as-platform — a place to build your own agents around your own workflows. Good for anything that crosses systems, requires team-specific context, or isn't the shape of what any single vendor sells. Prepackaged vendor personas will never fit here, because the persona is generic and the work isn't.

Most teams need both. The right question isn't which lane, it's for this workflow, which lane. If the workflow is unique to how your team operates — CS escalation rules, GTM handoffs, on-call triage, sales-follow-up nuances — no vendor persona will fit. That's where a platform your team builds into whatever agent it actually needs, with playbooks in plain language and real tools and a real audit chain, becomes the answer.

The rest is just picking the tools that let your team keep owning the thing.

— The Gauss team