Building AI Agents From Scratch in Python, No Framework

I've built agents that run unattended, and when I later moved similar workloads onto a framework, the biggest surprise was how little the framework changed about the core loop. Most of what an agent does is one while statement, a list of messages, and functions you wrote yourself. This post builds that version step by step, then covers where it stops being enough.

The loop is one rule

An agent is a loop that keeps calling the model until the model stops asking for tools. Anthropic's docs reduce the client-side version to one rule: while stop_reason is "tool_use", run the tools and continue the conversation (How tool use works).

Client tools run in your own application. Claude replies with a tool_use block, your code executes the operation, and you send back a tool_result (Tool use with Claude). That's the whole protocol. Here is the skeleton:

import anthropic

client = anthropic.Anthropic()
MODEL = "..."  # check Anthropic's models page for the current ID
MAX_TURNS = 10  # my choice, not a documented default

def run_agent(user_goal: str, tools: list, registry: dict) -> str:
    messages = [{"role": "user", "content": user_goal}]

    for turn in range(MAX_TURNS):
        resp = client.messages.create(
            model=MODEL,
            max_tokens=2048,
            tools=tools,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": resp.content})

        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text")

        results = [execute(b, registry) for b in resp.content if b.type == "tool_use"]
        messages.append({"role": "user", "content": results})

    raise RuntimeError(f"Agent did not finish in {MAX_TURNS} turns")

Two details matter more than they look. First, stop reasons: when Claude stops to call your own tools it is tool_use, not pause_turn, which belongs to the server-side tool loop (Stop reasons). Second, the turn cap. The 10 above is an arbitrary number I picked. Anthropic's agent guidance says agents should end on a stopping condition such as a maximum number of iterations. The right cap depends on your task, so measure how many turns your real workloads need and set it a bit above the p95.

Tool schemas are the real interface

The model never sees your Python function. It sees a name, a description, and a JSON Schema. OpenAI's function calling uses the same shape: name, description, and a parameters field holding the JSON Schema, plus an optional strict flag (Function calling). Anthropic's field is named input_schema instead of parameters, but the idea is identical.

I keep tools and schemas together so they can't drift apart:

def read_invoice(path: str) -> dict:
    ...

TOOLS = [{
    "name": "read_invoice",
    "description": "Read a parsed invoice from an absolute file path. "
                   "Returns vendor, line items and total.",
    "input_schema": {
        "type": "object",
        "properties": {"path": {"type": "string", "description": "Absolute path"}},
        "required": ["path"],
        "additionalProperties": False,
    },
}]
REGISTRY = {"read_invoice": read_invoice}

Note the word "absolute" in that description. Anthropic reported that while building its SWE-bench agent it spent more time optimizing the tools than the overall prompt, and one fix was to make a tool always require absolute filepaths after the model made mistakes with relative ones (Building effective AI agents). I have seen the same thing in my own systems. When an agent misbehaves, I look at tool descriptions and parameter names before I touch the system prompt. A vague tool name costs more than a vague prompt.

On strictness: OpenAI says strict: true makes arguments adhere to your schema on supported models, but only if your schema uses the supported JSON Schema subset (OpenAI help). Even then, the shape being valid is not the same as the value being sensible. A path can match "type": "string" and still not exist. Validate semantics inside the tool.

The executor: parallel calls and strict ordering

Claude can request several tools in one response, and all the tool_use blocks arrive in a single assistant message (Tool use with Claude). A loop that handles only the first call will silently drop work. That is why the loop above iterates over every tool_use block and returns all results in one user message.

The ordering rule is unforgiving. If tool results are not formatted correctly, the API returns an error about tool_use ids found without tool_result blocks immediately after, and the docs describe a 400 for this class of mistake (Handle tool calls). Every tool_use id needs exactly one matching tool_result, directly after the assistant message. Here is the executor:

def execute(block, registry) -> dict:
    fn = registry.get(block.name)
    try:
        if fn is None:
            raise KeyError(f"Unknown tool: {block.name}")
        output = fn(**block.input)
        content = json.dumps(output) if not isinstance(output, str) else output
        return {"type": "tool_result", "tool_use_id": block.id, "content": content}
    except Exception as e:
        return {
            "type": "tool_result",
            "tool_use_id": block.id,
            "content": f"{type(e).__name__}: {e}",
            "is_error": True,
        }

A caveat on the is_error flag: I would not rely on it from memory, so check the current Anthropic documentation on handling tool calls to see how error results are meant to be flagged. Even without the flag, the pattern holds: never let an exception escape the executor. Convert it into a tool result so the model can read the error and recover. An unhandled exception that kills the loop leaves you with a half-finished conversation and no way to resume it.

Memory is just the messages list (until it isn't)

In a hand-rolled agent, short-term memory is the messages list. That's it. It is easy to inspect, trivial to log, and trivial to replay. In my own setups I persist it after every turn, which gives me an audit trail and a way to resume a run that crashed.

The problems start when runs get long. Here is how I think about the options:

I start with truncation because tool output is where bloat comes from: a tool that returns a 40-page document will dominate your context. Cap tool outputs and tell the model in the tool description that results are truncated and how to ask for more. Long-term memory (facts that survive across runs) belongs in a database, not in the prompt. In my RAG work that means pgvector plus keyword search, but for a first agent a plain table of key-value notes the agent can read and write via two tools is enough.

Error handling and the failure modes I actually see

After shipping agents that run without a human watching, my short list of real failures is consistent:

1. Loop without progress. The model calls the same tool with the same arguments repeatedly. The turn cap catches it eventually, but I also hash (tool_name, arguments) and abort or inject a "you already tried this" message on the second repeat.
2. Wrong tool, plausible arguments. Fix with better descriptions and tighter schemas, not prompt pleading.
3. Transient API or network errors. Retry with backoff around the messages.create call, separately from the tool executor. These are different failure classes and mixing them makes debugging miserable.
4. Silent partial success. The agent returns text claiming it finished when a tool errored mid-run. I log every tool result and check final claims against recorded results in the tests.
5. Side effects on retry. If a tool sends an email or writes a record, a replayed run will do it twice. Make side-effecting tools idempotent with a key, or gate them behind a dry-run flag.

To verify the whole thing, I write a handful of scripted goals with known correct outcomes, run them against the real model, and assert on the recorded tool calls rather than on the final prose. Checking which tools were called, in what order, with what arguments, is far more stable than matching text.

Where the hand-rolled version stops

I reach for a framework when I need what this loop does not give me. LangGraph 1.0 advertises durable state, built-in persistence to save and resume workflows, and first-class human-in-the-loop pause and approval (LangChain and LangGraph 1.0). That is a fair summary of what I'd otherwise build by hand: checkpointing and approval gates. Persisting the message list works for linear runs, but branching graphs with approvals are where I'd rather not reinvent it.

CrewAI sits at a different layer. Its README describes Flows as the production-oriented, event-driven workflow layer, with Crews (collaborating agents) as the other building block (CrewAI on GitHub). A crew's process defaults to sequential, and the hierarchical process adds a manager agent that delegates and validates, which requires a manager_llm or manager_agent (CrewAI docs). If your problem really is several role-specialized agents handing work to each other, that structure helps. If it's one agent with five tools, it is overhead.

What I'd do

• Start with the loop above, always. Even if you plan to use a framework, writing it once means you can debug the framework when it misbehaves.
• Spend your time on tools. Names, descriptions, parameter constraints and error messages move reliability more than prompt tweaks.
• Log the full messages list per run. It is your trace, your test fixture and your resume point.
• Cap turns, cap tool output, retry the API separately from tools.
• Adopt a framework when you need persistence, branching or human approval, not because the tutorial did. I'd pick LangGraph for stateful graphs with approvals, and CrewAI only when the multi-agent role structure matches the problem.
• Test on tool-call traces, not on how nice the final answer reads.

If you're deciding between a hand-rolled agent and a framework for something with real stakes, I'm happy to talk it through. You can reach me at lazar-milicevic.com/#contact, or browse the other agent engineering posts on the blog.