AI Agents in Production: 7 Real Examples I've Shipped

The first agent I put into production did not fail on day one. It failed on day nine, at 3 a.m., when an upstream API changed a field name and the agent kept cheerfully producing confident output from nothing. Nobody was awake. That night taught me more than any demo ever did, and it shaped how I build everything now.

Below are seven agent systems I've built and run, what each one does, where it breaks, and what I'd keep or throw away. I'm deliberately not publishing success-rate percentages. I haven't got a public benchmark to stand behind, and I'd rather describe the failure shapes than hand you numbers you can't verify.

First, what I mean by "agent"

Most of what I ship in production is a workflow with one or two genuinely agentic steps, not a free-roaming agent. I use the same line Anthropic draws in its Building effective agents post: workflows are systems where LLMs and tools are orchestrated through predefined code paths, while agents are systems where the LLM dynamically directs its own process and tool usage.

Anthropic also notes that for many applications, optimizing a single LLM call with retrieval and in-context examples is enough, no agent needed. I take that seriously. Roughly half of my "agent" candidates turn out to be one well-prompted call plus a deterministic wrapper. The seven examples below are the ones where a loop earned its keep.

1. The content pipeline: research, write, optimize, publish

This is the backbone of BizFlowAI ContentStudio, an autonomous content and SEO/AEO machine that publishes across multiple sites without anyone pressing a button.

The shape is a pipeline of narrow roles, not one big agent:

topic queue -> researcher -> writer -> optimizer -> verifier -> publisher
                   |                                    |
              fact store  <---------------------------- rejects back to writer

The researcher is the only step with real autonomy: it decides which searches to run and when it has enough. Everything downstream is constrained. The writer receives a fixed fact list and is told that anything outside it must be written descriptively, never filled in from memory.

The part that matters is the verifier. It is a separate model call with a different prompt, and its only job is to reject: unsupported numbers, invented links, banned phrasing. A rejected draft goes back to the writer with the specific violation attached. The writer gets a bounded number of retries, then the item is parked for review instead of published. That stopping condition mirrors Anthropic's point that agents often include a maximum number of iterations to maintain control.

Failure shape: the model doesn't hallucinate wildly, it hallucinates plausibly. A real-sounding statistic with a real-sounding source. Catching that is a verification problem, not a writing problem, and it's why I treat the verifier as the product.

2. The self-learning SEO loop

This is the second half of ContentStudio, and the one I'm most proud of: measure, learn, target, generate, publish, repeat.

Search performance data flows in. A scoring step identifies which existing pages have impressions but weak positions, which topics are producing clicks, and which formats lose. That feeds a targeting step that decides what to write or refresh next. Then the content pipeline from example one runs.

Two design decisions made it trustworthy:

• The learning step writes rules, not weights. What it learns lands as plain-language guidance in a file I can read and edit. If the loop drifts, I can see it and fix it with a text diff.
• Judgment is grounded in real search data, not model opinion. The agent doesn't decide what's good. Impressions, clicks and average position decide.

The honest caveat: with low click volumes, many of these signals are noisy. A page earning a handful of clicks a month can't prove much. I treat early lessons as tentative and weight impressions and position over click counts until the sample is meaningful. An agent that overfits to noise will confidently steer you wrong, so the loop has to know how little it knows.

3. Support triage on AWS and Zendesk

This one wasn't a flashy LLM project, and it produced the first-ever SLA compliance for the team I built it for. It's a serverless integration: events from the ticketing system trigger Lambda functions via EventBridge, which classify, prioritize and route tickets.

The agentic part is small on purpose. Classification and urgency scoring use an LLM call; routing and SLA timers are deterministic code. Anthropic cites customer support as a natural agent use case because it needs external data and actions, and success is measurable through user-defined resolutions. I agree with that, with one amendment: the actions need hard permissions. This system could re-prioritize and route. It could not close tickets or message customers on its own.

A cheap model handles the high-volume, easy classification, and anything ambiguous escalates to a stronger one. That's the same routing pattern Anthropic describes of sending common questions to smaller models and hard ones to more capable models. It kept cost proportional to difficulty, and the whole thing scales to zero when the queue is empty.

Failure shape: the dangerous case isn't a misclassified ticket, it's a silently misclassified ticket nobody reviews. So every low-confidence decision lands in a visible queue instead of being quietly guessed.

4. Code review agent

A review agent reads a diff, pulls surrounding files, and leaves comments. It is genuinely agentic: it decides which files to open and how deep to go.

What I learned wasn't about prompts, it was about tools. Anthropic reports spending more time optimizing tools than the overall prompt while building their SWE-bench agent, and that matches my experience exactly. One of their examples stuck with me: tool errors with relative file paths disappeared once they required absolute paths, after which the model used the tool flawlessly. I hit the same class of bug. My file-reading tool accepted loose paths, the agent guessed, and reviews went sideways. Tightening the tool contract fixed more than any prompt rewrite.

My rules for this agent:

• Comments must cite a specific line range, or they're dropped.
• It reviews for defects and risk, not style. Linters own style.
• It never approves or blocks. It advises, a human decides.

Failure shape: confident nitpicking. The model wants to say something, so it invents a concern. A "no issues found" path that is easy and respectable cut this down more than anything else.

5. Document intake and extraction

An agent that takes messy inbound documents, extracts structured fields, validates them against business rules, and writes clean records. The loop here is validation: if extraction fails a check (totals don't sum, a required field is missing), the agent re-reads the source and tries again, up to a limit.

This is where grading by state beats grading by text. The output isn't judged by whether it sounds right, it's judged by whether the resulting record passes deterministic checks. That's the same idea behind tau-bench, which grades agents by the database state after a conversation rather than by the transcript. I borrow that principle for every agent I build: define success as a verifiable end state wherever possible.

Failure shape: documents that are valid but weird. The agent either forces them into the schema or, worse, normalizes away the weirdness that was the whole point. I route unrecognized formats to review instead of coercing them.

6. RAG assistant over internal knowledge

A retrieval agent on pgvector with hybrid search (vector plus keyword, merged with reciprocal rank fusion), able to run on a local LLM through Ollama when data can't leave the machine.

The agentic piece is query handling: it can reformulate a question, run several retrievals, and decide whether the evidence is sufficient before answering. The decision I'm strictest about is the refusal path. If retrieval doesn't support an answer, it says so and shows what it found. A RAG system that answers anyway is worse than none, because it looks authoritative.

Failure shape: retrieval that returns something plausible but wrong, and a model that happily builds an answer on it. Most RAG quality problems I've debugged were retrieval problems wearing a generation costume. I log retrieved chunks next to every answer so I can tell which half failed.

7. The nightly operations worker

The least glamorous agent and probably the most valuable: a scheduled worker that runs unattended, checks the health of the other systems, reconciles state, retries what failed, and writes a short report. It is part of the 4-system automation ecosystem that saved 73+ hours a month, with a 192% Year-1 ROI.

It has the narrowest tool access of all seven, no destructive actions, and an explicit escalation path. When it can't resolve something, it opens an item for a human instead of improvising. This is the lesson from my day-nine, 3 a.m. failure: an unattended agent needs to fail loudly and safely, not creatively.

What the failures have in common

Across all seven, the pattern is the same. The model is rarely the weakest link. The weak links are:

Reliability also compounds badly across repeated runs. The tau-bench paper introduced pass^k, the probability that all k repeated trials of a task succeed, as opposed to pass@k, where at least one succeeds. As an idealized example with round numbers: if an agent succeeded independently 90% of the time per run, the chance of eight clean runs in a row would be 0.9 to the eighth power, about 43%. Real runs aren't independent, so treat that as an illustration only. The point stands: a demo success rate tells you little about a system that runs every night.

What I'd do

If you're about to build your first production agent, here is the order I'd work in:

1. Start with the workflow, add agency only where the path genuinely can't be predicted. Most steps don't need a loop.
2. Write the verifier before the generator. Decide how you'll know it's wrong first.
3. Spend your effort on tools. Strict inputs, clear errors, absolute identifiers.
4. Cap every loop and define the "give up" path. Parking an item for review is a feature.
5. Grade by end state. Records, database rows, passing checks, not fluent text.
6. Log retrieval, tool calls and decisions next to outputs. You will need them at 3 a.m.
7. Give agents the least permission that works. Advise before act, act before delete.

I'd also stay skeptical of the hype cycle. Gartner has predicted that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. That forecast drew on a self-selected webinar poll, so I wouldn't treat the exact figure as gospel, but the three causes ring true to me. They're all engineering and scoping failures, and they're all avoidable.

If you're weighing an agent project and want a second opinion on scope, reliability or architecture, I'm happy to talk it through at lazar-milicevic.com/#contact. Otherwise, the rest of the blog goes deeper on the individual pieces above.