From Fannie Mae Systems to Autonomous AI Agents

A batch job that reconciles mortgage-backed securities does not care about your model choice, your framework, or how clever your prompt is. It runs at 2am, it either produces the right number by 6am, or someone in operations is on a bridge call by 7. That constraint, more than any tutorial or paper, is what shaped how I now build autonomous AI systems. When people ask why my agents tend to stay up when other pilots fall over, the honest answer is: I learned to build software in an environment where "it usually works" was a firing offense.

The reliability bar in financial systems is not optional

In large financial infrastructure, "reliability" is not a slide in a deck. It is a number your team owns, and it shows up in a monthly review with names attached to it. When I worked on systems that touched mortgage servicing and securitization data, the baseline expectations looked something like this:

• Batch jobs complete inside a fixed window, every night, or a documented incident opens automatically.
• Every record has a lineage: where it came from, which job touched it, what version of the code ran.
• Reprocessing is idempotent. You can rerun yesterday's job today and get the same answer.
• If something breaks, you can prove within an hour whether the number that went to a regulator was affected.

Contrast that with the current state of most "AI agent" pilots. The typical demo is a Jupyter notebook, a single prompt, and a screenshot. When it fails in production, nobody can answer basic questions: which model version ran, what tools were called, what the input context actually was, whether the failure is reproducible.

I now build agents with the same posture I used for batch pipelines. Every agent run gets a run ID. Every tool call is logged with inputs, outputs, latency, and cost. Every prompt template is versioned. If the agent produced a wrong answer last Tuesday, I can replay that exact run against a new model or a new prompt and see whether my fix actually fixes it. That is not exotic. It is table stakes for anything that runs unattended.

Compliance-minded design ages better than clever design

The other thing financial systems teach you is that clever, one-off solutions are a liability. A senior engineer once told me that the goal was not to write the smartest code, it was to write code that a stranger could pick up in two years, read once, and safely change. Systems that touch regulated data get audited, forked, and modified by people who were not in the original room. If your architecture depends on tribal knowledge, it will break the first time you rotate off.

I carry that into AI work in three specific ways.

Boring interfaces at the boundaries. My agents talk to the outside world through plain HTTP endpoints, SQL, and message queues. Not through framework-specific abstractions that will be deprecated in nine months. When Claude 4 replaces Claude 3.7, or when I swap in a local model for a batch step, the surrounding system does not care.

Explicit data contracts. Every tool an agent can call has a typed input and output schema, validated at the boundary. If the model hallucinates a field, the call fails loudly instead of silently writing garbage. This is the same discipline as a message schema in a financial pipeline. It sounds tedious. It saves you weeks of debugging.

Separation of the deterministic and the probabilistic. Anything that has to be right (a total, a currency conversion, a database write) runs as ordinary code. The LLM decides what to do and produces structured arguments. The code does it. When the model is wrong, the worst case is that a valid tool ran with the wrong inputs, which is auditable and reversible. Compare that to letting the model do arithmetic in a text response and hoping.

What "scales to zero" really means

Large financial systems taught me to respect fixed costs. A batch cluster that sits idle for 20 hours a day is still on the invoice. When I moved to my own projects, and later to building BizFlowAI, I inverted that constraint: nothing runs unless it is doing work.

The stack for most of what I build now looks like this:

The trade-off is real: cold starts on Lambda add a few hundred milliseconds, and Step Functions is not the tool if you need sub-second orchestration. For a nightly content pipeline, or a customer support classifier that runs on inbound webhooks, those costs are invisible. For a serverless AWS + Zendesk integration I built to close an SLA compliance gap, this shape was what made "first-ever SLA compliance" possible: the system responded to events instantly, and there was no server to forget to patch.

Where financial engineering habits pay off in AI agents

Three specific habits transfer almost one-to-one from enterprise batch systems to autonomous agents.

1. Design for reruns from day one

In a mortgage pipeline, you assume you will rerun jobs. Bad input file, upstream correction, a bug found next week, a regulator asks for a restated report. So every step is idempotent, keyed on natural business identifiers, and safe to execute twice.

Agents need the same property. A content generation pipeline that publishes to a CMS should never publish the same article twice because a Lambda timed out and retried. In BizFlowAI ContentStudio, every generation task has a stable task ID derived from the target site, the topic, and the run window. Downstream steps upsert by that ID. If a step fails and retries, the system converges to one correct outcome. If I want to regenerate an article with a new prompt, I bump a version and the old artifact is preserved for comparison.

2. Observability is not logging

Logging is print with a timestamp. Observability is being able to ask a question you did not anticipate when you wrote the code. Financial systems get this right because auditors ask weird questions and you have to answer them.

For agents, I record for every run:

• The model, temperature, and prompt template version
• The full tool call trace with arguments and results
• The retrieved context (chunk IDs, similarity scores, which retriever)
• Token counts and dollar cost per step
• Wall-clock latency per step
• The final output and any downstream side effects

This lets me answer questions like: "Why did quality drop on Tuesday?" (Answer: retrieval started returning chunks from a new source with bad OCR.) Or: "Why did costs jump 30% this week?" (Answer: a prompt change caused the model to call the search tool twice per turn.) Without this, you are guessing.

3. Change control that does not slow you down

The stereotype of enterprise change control is a ticket queue and a two-week approval. The good version, which the best financial teams actually practice, is different: fast changes, but every change is small, reversible, and observable. Ship a hundred small things a week, each behind a feature flag or a version bump, each measurable.

For agent systems this looks like: prompt changes deploy as versioned templates, and I can A/B two versions on the same live traffic. Model swaps go through a shadow-mode phase where the new model runs on real inputs but its output is only logged, not used. New tools are added with a kill switch. This is the same discipline that lets a large bank change a production pipeline without breaking Monday's close. It works just as well for a solo engineer running an autonomous system.

The self-learning content loop, as an example

To make this concrete, here is the shape of the self-learning content pipeline I built. It runs unattended, week after week, and improves itself based on real search performance.

1. Measure. A scheduled job pulls per-URL query and impression data from Search Console for every managed site. Data lands in Postgres.
2. Learn. An analysis step identifies queries where a page is ranking but underperforming, and gaps where impressions exist without matching content.
3. Target. An agent selects the next batch of topics, weighted by opportunity and by clustering with existing content.
4. Generate. A multi-step agent researches, drafts, edits, and optimizes each piece, with retrieval over the site's existing content to keep internal linking coherent.
5. Publish. Content is pushed to the CMS through a boring HTTP API with idempotent keys.
6. Wait. The loop closes when the next measurement cycle catches the new URLs.

Every one of those steps is versioned, logged, and independently rerunnable. If step 4 produces a bad article, I can requeue just that task without touching anything upstream. If I want to try a new model on step 3, I shadow it for a week before promoting. This is the same shape as a nightly batch pipeline. The only difference is that the "batch job" writes English instead of a fixed-format file.

What I'd do if I were starting an AI system today

If you are a CTO or a founder standing up your first serious AI agent in production, here is what I would push for on day one, based on both the enterprise work and the autonomous systems I run now.

• Pick your reliability target first, then design backwards. "The agent should run every night and produce X, and if it does not, an incident opens." That single sentence eliminates half the architectural debates.
• Version everything the model touches. Prompts, tool schemas, retrieval indexes, model IDs. If you cannot answer "what exactly ran?" for a given output, you cannot debug or improve.
• Keep the deterministic core deterministic. The LLM chooses. Code executes. Never let the model do arithmetic, formatting, or side effects directly if you can help it.
• Instrument cost and latency per step, not per run. The 80/20 of cost optimization lives in one or two steps. You cannot find them without per-step numbers.
• Build the smallest possible thing that actually runs unattended. A pilot that requires a human to click "go" every morning is not a pilot. It is a demo. The transition from demo to unattended is where 90% of the real engineering lives.

Closing

The unglamorous habits from large financial systems (reliability targets, idempotency, versioning, observability, change control) are exactly the habits that make autonomous AI agents work in production. There is no shortcut, and there is no framework that gives them to you for free. The good news is that once you internalize them, you can build small, focused systems that quietly run for months and produce measurable business outcomes with no one watching.

If you are working on something in this space, an agent that needs to graduate from pilot to production, an automation you want running unattended, or a RAG system that has to be trustworthy, I am happy to talk. You can reach me at lazar-milicevic.com/#contact, or read more of how I think about these systems on the blog.