Sovereign AI PoCs: What Innovate UK Means for Builders

The sovereign AI conversation used to be a policy panel topic. Now it shows up in the first client call. A head of engineering at a UK-funded startup asked me last month whether their PoC could use Claude via Bedrock in eu-west-2, or whether "sovereign" meant they had to run Llama on a box in Slough. The answer is not obvious, and getting it wrong burns weeks.

I have shipped local-LLM RAG pipelines, hybrid search in Postgres, and multi-agent systems that run unattended. The technical patterns that satisfy sovereign AI mandates are not exotic, but they force real architectural choices early. Here is how I think about it when scoping a PoC that has to survive both a technical review and a funder's compliance questionnaire.

What "sovereign AI" actually means for a PoC

Sovereign AI, in the context of Innovate UK's recent funding pushes and similar programs across the EU, is not a single specification. It is a cluster of requirements: data residency (your training data, embeddings, and inference logs stay in-country or in-region), model provenance (you can point at where the weights came from and who owns them), operational control (a foreign vendor cannot switch you off or exfiltrate your prompts), and often supply-chain auditability (the whole stack, from GPU driver to orchestrator, is inspectable).

For a PoC, this rarely means "you must self-host everything." It usually means you must be able to demonstrate that for a defined set of sensitive workloads, no data leaves an approved boundary, and that you have a credible path to remove any non-sovereign dependency without rewriting the system. That second half is where most PoCs quietly fail their compliance review.

The practical implication: you architect for portability from day one, even if your PoC runs on a hyperscaler in an approved region.

The three deployment tiers I use when scoping

Not every part of a system needs the same sovereignty guarantee. I split workloads into three tiers on the first whiteboard.

The mistake I see: teams treat the whole system as tier 1 and try to run a 70B model on a single A100 because "sovereign." Then latency is awful, cost is absurd, and the PoC dies before anyone reviews it. Or they treat it all as tier 3, ship on OpenAI direct, and get a compliance red card in month three.

Sort your data flows into these tiers on day one. It changes what you build.

The local-LLM RAG pattern that actually works

For tier 1 workloads, the pattern I keep coming back to is boring and it works: Ollama or vLLM serving an open-weight model, embeddings from a small local model (bge-m3 or nomic-embed-text), Postgres with pgvector plus full-text search for hybrid retrieval, and Reciprocal Rank Fusion to merge them. I wrote about the hybrid search side in more depth in an earlier post on pgvector + FTS with RRF.

A minimal, sovereign-safe RAG loop looks like this:

# All components run inside the sovereign boundary
from ollama import Client
import psycopg

llm = Client(host="http://llm.internal:11434")   # vLLM or Ollama
db  = psycopg.connect("postgresql://rag@db.internal/rag")

def answer(question: str) -> str:
    q_emb = llm.embeddings(model="bge-m3", prompt=question)["embedding"]

    with db.cursor() as cur:
        cur.execute("""
            WITH vec AS (
              SELECT id, 1 - (embedding <=> %s::vector) AS score
              FROM chunks ORDER BY embedding <=> %s::vector LIMIT 40
            ),
            fts AS (
              SELECT id, ts_rank_cd(tsv, plainto_tsquery('english', %s)) AS score
              FROM chunks WHERE tsv @@ plainto_tsquery('english', %s)
              ORDER BY score DESC LIMIT 40
            )
            SELECT c.text
            FROM (
              SELECT id, SUM(1.0 / (60 + rn)) AS rrf FROM (
                SELECT id, row_number() OVER (ORDER BY score DESC) AS rn FROM vec
                UNION ALL
                SELECT id, row_number() OVER (ORDER BY score DESC) AS rn FROM fts
              ) x GROUP BY id ORDER BY rrf DESC LIMIT 8
            ) top JOIN chunks c ON c.id = top.id;
        """, (q_emb, q_emb, question, question))
        context = "\n\n".join(r[0] for r in cur.fetchall())

    return llm.chat(model="llama3.1:8b-instruct", messages=[
        {"role": "system", "content": "Answer only from the provided context."},
        {"role": "user", "content": f"Context:\n{context}\n\nQ: {question}"}
    ])["message"]["content"]

A few things about this shape that matter for a sovereign PoC:

• The LLM endpoint is a variable, not a hardcoded provider. Swap http://llm.internal:11434 for a Bedrock endpoint and the rest is unchanged. That is the portability guarantee auditors want to see.
• Embeddings are generated in-boundary. A common leak: teams use OpenAI embeddings "just for the PoC." Now every document you ever indexed has been sent to a third party. You cannot un-send that.
• Postgres, not a hosted vector DB. Pinecone or a US-region managed vector store is a sovereignty problem you do not need. Postgres with pgvector handles millions of chunks with acceptable latency, and it runs anywhere.
• Retrieval logs live in the same Postgres. Every query, every retrieved chunk, every model response goes to an audit table in the same database. This is the artifact a reviewer will actually ask for.

Model selection: what actually fits on the hardware you have

The honest reality of local LLMs in 2026: an 8B or 12B instruction-tuned model is enough for most RAG workloads if your retrieval is good. Teams reach for 70B because it feels safer, then hit a wall on GPU cost.

Rough numbers from what I have run and what I recommend for PoC-scale work:

The counterintuitive move: spend your first two weeks improving retrieval quality, not model size. A well-tuned hybrid retriever plus an 8B model beats a 70B model on garbage retrieval, and it costs a tenth as much to run. For a PoC that has to demo on a laptop or a modest cloud VM, this is the difference between shipping and not shipping.

The compliance evidence pack (build this alongside the code)

Every sovereign AI PoC I have shipped has needed a documentation artifact that engineers routinely forget. Build it as you go, not the night before the review.

The pack I hand over usually contains:

1. Data flow diagram with the sovereign boundary drawn as a box. Every arrow crossing that box is annotated with what data, what encryption, what jurisdiction.
2. Model card for each model in use, including the base model, its license, the source of the weights (Hugging Face repo hash), any fine-tuning data, and where the weights are stored at runtime.
3. Dependency inventory with SBOM (generate it with syft or similar). Reviewers increasingly ask about the origin of every container image.
4. Prompt and response logging policy: what is stored, for how long, where, who can read it. If you use a tier 2 provider, quote the specific zero-retention terms and the region.
5. Exit plan: a written paragraph explaining exactly which components would need to change if the tier 2 vendor became unusable, and how long that swap would take. If the answer is "we would need to rebuild," you failed portability.

That last item is the one funders actually care about. Sovereignty is not just where the bytes live today, it is whether you are strategically captured.

Where builders trip up

A few patterns I have seen repeatedly on sovereign PoCs, in rough order of pain caused:

• Assuming "EU region" equals sovereign. A US-headquartered vendor with EU infrastructure is not automatically sovereign under most funder definitions. Read the specific criteria of the program you are applying to. Innovate UK, Horizon Europe, and national schemes all differ.
• Fine-tuning on the wrong data. If your fine-tune dataset was pushed to a US-hosted training service, the resulting weights may be considered contaminated. Fine-tune locally, or use adapter methods (LoRA) you can regenerate in-boundary.
• Ignoring inference telemetry. Some open-weight serving frameworks phone home for anonymous usage metrics. Turn it off explicitly and document that you did.
• Treating GPUs as fungible. Availability of specific SKUs in specific regions varies wildly. Confirm the exact instance type is available in your target region before you architect around it, not after.
• Building a demo that only runs on your laptop. A sovereign PoC that cannot be deployed to the target environment is not a PoC, it is a prototype. Deploy to a real sovereign-eligible environment in the first two weeks, even if it costs a bit.

What I'd do if I were starting a sovereign PoC on Monday

Concrete plan, compressed:

1. Day 1-3: Classify every data source into the three tiers above. Draw the boundary. Get sign-off from whoever will review the compliance side before you write code.
2. Day 4-7: Stand up Postgres with pgvector and full-text search in a sovereign-eligible region. Load a representative slice of the real data. Build hybrid retrieval with RRF. Measure retrieval quality on 30-50 real questions with a simple hit@k metric.
3. Week 2: Add a local LLM (start with Llama 3.1 8B or Qwen 2.5 7B via Ollama or vLLM). Wire it to retrieval. Get end-to-end answers flowing. Log everything to a Postgres audit table.
4. Week 3: Introduce tier 2 (a regional Claude or GPT endpoint) for the reasoning steps that genuinely need it, behind an interface that lets you swap back to the local model with an environment variable. Benchmark quality difference honestly.
5. Week 4: Write the evidence pack. Deploy to the target sovereign environment. Run the demo there, not on your laptop.

Four weeks gets you a defensible sovereign PoC with real evidence, real numbers, and a real portability story. Anything longer is usually a scoping problem, not an engineering one.

Closing

Sovereign AI is not a constraint that makes AI worse. It is a constraint that forces you to be honest about which parts of your system actually need the sovereignty guarantee, and to design cleaner boundaries as a result. The PoCs I have shipped under these constraints tend to be more portable, better logged, and easier to hand over than the ones built without them.

If you are scoping a sovereign AI PoC and want a second pair of eyes on the architecture, the tier split, or the evidence pack, I am at lazar-milicevic.com/#contact. More build notes and architecture posts on the blog.