"AI agent" is the hottest label in software right now, which means it's also the most abused. Slap it on a product and the price goes up and the demos get more impressive.
So I made a checklist and went through fifty tools calling themselves agents. The results were… clarifying.
Most tools marketed as "AI agents" are actually chatbots with a fresh coat of paint. A real agent must be able to take multiple actions toward a goal without asking you at each step.
The fastest test: if it can only respond, it's a chatbot. If it can act — using tools, chaining steps, adjusting based on results — it's an agent.
For each tool, I asked five yes/no questions:
| Test | What it reveals |
|---|---|
| Can it use external tools (not just chat)? | Whether it can act at all |
| Can it chain multiple steps unprompted? | Whether it's autonomous |
| Does it adapt when a step fails? | Whether it's reactive or scripted |
| Can it run without a human in the loop? | Whether it's truly an agent |
| Does it have guardrails for actions? | Whether the vendor took agency seriously |
A real agent scores yes on at least the first four. Of my fifty, eleven did. The rest were chatbots, or scripted automations wearing the word "agent" for the valuation bump.
The fakes fell into three costumes:
The rebranded chatbot. Genuinely just a chat interface. Helpful! Not an agent. These were the majority.
The scripted automation. A fixed if-this-then-that flow with an AI-generated label. It can act, but it can't decide — change the situation and it breaks. Useful, but it's automation, not agency.
The demo special. Looks fully autonomous in the canned demo, falls apart the moment you give it a task the demo didn't rehearse. The tell: the demo always uses the same three examples.
The "agent" label commands agent prices. If you're paying for autonomy and getting a chatbot, you're overpaying for capability you don't receive — and you may be missing a genuine assistant that would've served you better and cheaper.
The reverse risk: you buy a real agent for a job that only needed a chatbot, then drown in guardrails and supervision you didn't need. Matching the tool to the job is the whole game, and the marketing actively works against you doing it.
I keep coming back to the same distinction I drew in the difference between AI assistants and AI agents — get that clear and the marketing stops fooling you.
Run your own audit in fifteen minutes:
If it passes, it's real. If it only ever hands the work back to you with nice instructions, it's a chatbot, and you should pay chatbot prices.
To be clear: chatbots and assistants are genuinely valuable. The problem isn't that they exist — it's that they're sold as something they're not. A great assistant openly sold as an assistant is a fantastic purchase. The trench coat is the issue, not the tool underneath.
Q: Are the eleven "real" ones always the better buy? No. Real agency is only worth paying for if your job needs autonomy. For many jobs, a well-built assistant is the smarter, cheaper choice. Buy the capability you'll actually use.
Q: Why do vendors do this? Because "agent" sells. It's the premium label of the moment, the way "AI-powered" was a few years ago. Follow the pricing and the incentive is obvious.
Q: What's the single fastest tell? Give it a task slightly outside its demo. Real agents flex. Costumed chatbots freeze or hand the work back to you.
Most "AI agent" tools are chatbots in a trench coat — useful, but not what the label promises or the price implies. Run the five-point test, give the tool a task it didn't rehearse, and buy the capability you'll actually use.
The label is marketing. The behavior is truth. Test the behavior.
Behind a lot of lean, profitable companies is the same small stack of AI tools. Here's what's actually running the show.
I went from 200 to 11,000 subscribers without hiring anyone. AI didn't write my newsletter — it did everything around it.
I chased big, audacious goals for years and burned out every time. Then I built my whole life around wins so small they felt like cheating.
Comments
Sign in to join the conversation
No comments yet. Be the first to share your thoughts!