What is an AI agent, really?

The word "agent" has been shredded of meaning in the last twelve months. Every SaaS vendor now has an "AI agent" — most of them are chatbots with a system prompt and a marketing budget. This makes it genuinely hard to evaluate what you're buying, or what to build.

Here's the working definition we'll use for the rest of this post: an AI agent is a system that can decide what to do next, take real actions to do it, string those actions together toward a goal, and adapt when a step doesn't go as planned.

That's four properties: decision, action, sequence, adaptation. If a system only has one or two of them, it's something else — probably a good something else, but not an agent.

The four tiers, from bot to agent

The whole spectrum, from most basic to most autonomous:

1. Chatbot — an LLM behind a chat window. You type; it types back. It has a system prompt telling it how to behave, and maybe a knowledge base to search. It doesn't do anything to the outside world. GPT with a persona, or a support chatbot on a marketing site.

2. Assistant / copilot — a chatbot with narrow, human-supervised tool access. GitHub Copilot suggesting code; Grammarly suggesting edits; Notion AI drafting inside a document. You're driving; the AI is co-piloting. You accept, reject, or edit every action.

3. AI agent — an assistant that has been handed the wheel for well-defined tasks. Given a goal, it decides the steps itself, calls tools itself, checks the results, and adjusts. You don't approve every action; you set the constraints and review the outcome. Most of what's usefully marketed as "agents" today lives here.

4. Autonomous agent — a long-running system that pursues open-ended goals with minimal human check-in. Aspirationally: "keep the pipeline healthy" or "grow our newsletter." In practice, this is mostly research and prototypes today. Anyone selling you this in production is oversimplifying.

The gap between (2) and (3) is the important one. Everything below it is a chatbot with extra steps. Everything above it starts producing real economic value — but also real risk if you don't have the right constraints in place.

What actually separates an agent from a chatbot

Four criteria, in rough order of importance:

Tool use. A real agent can call APIs, update records, send messages, run code, query databases. Not just retrieve documents to answer a question. If a system can only read the world and not change it, it's an assistant. Agents write.

Multi-turn planning. After it calls a tool, it looks at what came back and decides what to do next. It doesn't just answer a single question; it plans steps 2, 3, 4 based on the result of step 1. If the system's whole loop is user turn → LLM turn → user turn, it's a chatbot no matter how good the prompt.

State. It remembers what happened earlier in a conversation. Ideally it also carries state across conversations — the customer's history, the team's conventions, the last time it did this task. Without state, every interaction is Groundhog Day.

Adaptation. When a tool call fails or returns something unexpected, it tries something else. It doesn't hallucinate a fake success ("Successfully updated the record ✓" when the API returned 500) or give up ("I can't help with that"). Failures are where agents earn their money — a rigid chatbot with a workflow can handle the happy path fine.

Why most "AI agents" you'll see are useless

Every second AI startup you see demoed lives in a specific failure mode. Watch for these:

• Prompt-only prototypes. Impressive in a five-minute demo, brittle the moment reality gets weird. The team wrote the prompt against three example inputs; anything outside those hits garbage output. Any agent whose entire behavior lives in a single system prompt is one edge case away from embarrassment.
• No connection to real systems. The agent can talk about updating the CRM. It can't actually update the CRM. Read-only "agents" are chatbots.
• No context about your team. It speaks generic best-practices English. Your escalation rules, your naming conventions, the three edge cases your ops lead cares about — none of that is in the box. You get a customer support agent, not a customer support agent for your team.
• Invisible failure modes. You never know if it did the right thing or lied confidently. No audit trail, no way to inspect what tools it called, no way to trace a bad output back to a decision point.
• Vendor-imposed personas. Prepackaged "roles" the vendor picked. You can maybe tweak the system prompt at the margins. You can't fundamentally change how it thinks — because the vendor doesn't want you to.

If a demo can't survive the questions "what tools can it call, in writing?" and "show me the audit log for what it just did," the answer is that it doesn't do very much.

What makes an AI agent actually useful

The useful ones look pretty different. Five properties that sort the toys from the tools:

It lives where the work happens. The best AI teammates aren't in a separate app that requires switching tabs. They're in Slack, in the CRM, in the IDE, in the ticketing system — wherever the humans they're helping already work. An agent that requires you to open a browser tab to talk to it will be forgotten in a week.

It knows your team's context. Not through prompt engineering that requires an ML background — through playbooks, written in plain language by the people who own the workflow. "When a customer emails us about a refund, first check their subscription history, then the internal notes, then draft a reply matching the tone we usually take." Any team lead can write that. No agent should require a prompt engineer.

It has real tool access. Read and write to the systems the team already uses. Not a token gesture ("can search Slack channels") but genuine action ("can post to any channel, respond in threads, edit its own previous posts, add reactions to escalate"). The value of an agent is roughly proportional to the surface area of what it can actually do.

It's on a leash. Audit trails for every action. Spend limits so a runaway loop doesn't burn a month's LLM budget in an afternoon. Human-in-the-loop for high-stakes actions (sending an external email; making a database change; approving a refund over $500). Constraints are what let you deploy an agent to production instead of a demo.

It enhances humans, not replaces them. The best agents take the tedium — the reads, the triage, the digests, the routine follow-ups — and leave humans with the judgment calls, made better by everything underneath already being sorted. The framing "AI will replace this job" produces the wrong products; the framing "AI will handle the 60% of this job nobody enjoys" produces useful ones.

How to evaluate what you're buying

If you're picking an agent product, five questions cut through the demo theatre:

1. What tools can it actually call? Not "connects to Slack" — can it POST messages, read threads, add reactions, edit its own past posts? Get specifics in writing.
2. Who can teach it new tricks? If the honest answer is "our prompt engineering team, on request," walk away. You'll be dependent on the vendor for every new use case.
3. Can you audit what it did? A full, timestamped log of tool calls, LLM turns, decisions, and outputs. If the vendor can't show you one, don't put the agent anywhere important.
4. How does it fail? Silently is bad ("Successfully updated ✓" when nothing happened). Loudly is good ("Tool call returned 500, retried twice, escalated to human"). Ask for a demo of what happens when a tool call fails.
5. What happens when the LLM has an off day? Fallback models? Retry logic? Or does the whole thing break the day OpenAI throttles you? Multi-provider support is table stakes.

A practical closing note

This is a fast-moving field. The definition of "agent" will keep shifting; today's autonomous agent will be tomorrow's assistant, and something we don't have a word for yet will be the new frontier. The useful posture is to stop caring about the label and start asking what the system actually does.

Think of AI agents as coworkers, not features. Coworkers you'd hire have context, tools, judgment, and accountability. If the thing you're evaluating is missing any of those four, it's not the coworker — it's the intern you're going to have to babysit until they figure it out.

Custom AI agents built by your team — with your context, your tools, your leash, your judgment about when to defer — are the ones that actually earn their pay.

— The Gauss team