Hybrid Search in Postgres: pgvector + FTS with RRF

The retrieval layer behind my content system is boring on purpose: PostgreSQL 16 in Docker, pgvector for embeddings, tsvector for lexical, all orchestrated from Python. It currently indexes 1,115 published pieces across 7 sites. That is the scale I'll be talking about, not a benchmark toy.

Here is the actual indexation snapshot from the last sync:

Everything below came out of running this stack against real content, watching it fail, and fixing it. I'll walk through why I run hybrid search at all, the SQL I actually use, the failure that made me instrument it differently, and the numbers that came out the other side.

Why hybrid at all

Pure vector search is great at paraphrase and terrible at proper nouns, SKUs, and exact-match jargon. Pure FTS is the reverse: it nails the exact string and misses anything the writer phrased differently.

Two cases from my own corpus made this obvious.

Query: "pgvector RRF" (a specific technique, specific library). Vector-only search returned three posts about "vector databases and hybrid retrieval" that never mentioned pgvector or RRF at all. Cosine distance thought they were close. A reader looking for that exact combination would bounce.

Query: "how I stop the agent from shipping garbage" (natural phrasing). FTS returned nothing usable because the actual post is titled "How I Stop an Autonomous AI Agent From Publishing Garbage". The lexical overlap with "shipping" and "garbage" is thin; plainto_tsquery didn't stem its way there. Vector search found it in first place.

Neither side is good enough alone. The whole point of hybrid is that the union of two weak signals is stronger than either one, and the fusion decides who to trust when they disagree.

The SQL, exactly as it runs

The pattern is two CTEs, one per retriever, ranked independently, then a final SELECT that fuses by rank position. k = 60 is the value from the original RRF paper (Cormack, Clarke, Buettcher 2009) and I have not found a reason to move it.

WITH vec AS (
  SELECT
    id,
    ROW_NUMBER() OVER (ORDER BY embedding <=> $1) AS rank_vec
  FROM chunks
  WHERE site_id = $3
  ORDER BY embedding <=> $1
  LIMIT 100
),
fts AS (
  SELECT
    id,
    ROW_NUMBER() OVER (
      ORDER BY ts_rank_cd(tsv, plainto_tsquery('english', $2)) DESC
    ) AS rank_fts
  FROM chunks
  WHERE site_id = $3
    AND tsv @@ plainto_tsquery('english', $2)
  ORDER BY ts_rank_cd(tsv, plainto_tsquery('english', $2)) DESC
  LIMIT 100
)
SELECT
  c.id,
  c.title,
  c.url,
  COALESCE(1.0 / (60 + v.rank_vec), 0) AS s_vec,
  COALESCE(1.0 / (60 + f.rank_fts), 0) AS s_fts,
  COALESCE(1.0 / (60 + v.rank_vec), 0)
    + COALESCE(1.0 / (60 + f.rank_fts), 0) AS rrf_score
FROM chunks c
LEFT JOIN vec v ON v.id = c.id
LEFT JOIN fts f ON f.id = c.id
WHERE v.id IS NOT NULL OR f.id IS NOT NULL
ORDER BY rrf_score DESC
LIMIT 20;

A few things worth calling out:

• $1 is the query embedding (a vector(1536) for OpenAI text-embedding-3-small, or whatever your model gives you). $2 is the raw query string. $3 scopes to a site.
• The vector CTE uses cosine distance (<=>) and an IVFFlat index with lists = 100. HNSW is fine too; I stayed with IVFFlat because rebuilds are cheaper and my corpus is small.
• The FTS CTE requires a GIN index on tsv. Without it, every query does a sequential scan and the whole thing collapses under any real load.
• Both sides are capped at 100 rows. That is the fusion window. Anything past position 100 in either list is effectively invisible, which is the correct default.
• The LEFT JOIN with COALESCE(..., 0) matters. A document that appears in only one list still gets fused, it just gets zero contribution from the other side.

Why RRF beats score normalization

I tried min-max normalization first, like everyone does. It works for two queries and fails for the third.

The problem is that cosine distance and ts_rank_cd are on incommensurable scales, and worse, the scales change per query. A rare query with one exact match returns FTS scores in the 0.3 range. A common query returns 0.01. Cosine distances cluster tighter on some embedding domains than others. Any linear combination like 0.6 * vec_norm + 0.4 * fts_norm requires you to fit weights against ground truth you probably don't have, and even then the weights drift when you swap the embedding model.

RRF sidesteps all of that. It only cares about rank position, not raw scores. A document at position 3 in both lists gets the same fused score regardless of whether the underlying similarities were 0.9 or 0.4. That property is what makes it survive a model swap and a corpus change without re-tuning.

The trade-off is that RRF is coarser: it can't distinguish "clearly the best hit" from "narrowly the best hit". For retrieval feeding an LLM re-ranker or an internal-linking system, that has been the right trade for me. If you were serving direct user results and needed to promote a runaway winner, you would want a two-stage pipeline where RRF gives you the top 20 and a cross-encoder re-ranks.

What broke, and the detector I should have written on day one

The first version of this ran for weeks looking fine. Result quality on spot checks was solid, my content clusters looked coherent, internal linking was placing links I would have written by hand.

Then I noticed the internal-linking system had, over about two weeks, quietly stopped linking to any post on one of the smaller sites. Not zero recommendations, just a slow drift toward the same 15 or so URLs.

The cause was ugly: I had shipped a change to the ingestion pipeline that updated the embeddings for new chunks but left the tsv column stale for a subset of rows because the trigger was scoped to inserts, not updates. FTS was returning zero rows for a chunk of the corpus. RRF didn't complain. It just fused vector rank + zero, and vector rank was enough to keep things looking normal on average, but the ordering was subtly worse and the diversity collapsed.

This is the same failure mode I hit in the July 2026 auth incident: 17 days of silent failure because the failure event wasn't in the sync list. If nothing is watching the specific thing that broke, you don't find out until a downstream symptom shows up.

The fix at the query layer is simple and I now consider it non-negotiable:

-- log alongside the query
SELECT
  COUNT(*) FILTER (WHERE v.id IS NOT NULL) AS vec_hits,
  COUNT(*) FILTER (WHERE f.id IS NOT NULL) AS fts_hits,
  COUNT(*) FILTER (WHERE v.id IS NOT NULL AND f.id IS NOT NULL) AS overlap
FROM ...;

Every hybrid query logs three numbers: how many hits each side contributed, and how many overlapped. Then two alarms:

1. Per-source zero-contribution rate. If either side returns zero hits for more than 5% of queries in a rolling 100-query window, page me. Ninety-five percent of queries should get contributions from both sides.
2. Overlap floor. If the average overlap between the two top-20 lists drops below 3, something is drifting. Either the corpus is degenerate for the queries being asked, or one side is broken.

I also added a nightly probe: 50 fixed queries with known-good answers run against the corpus, and if either the RRF top-1 or the top-5 recall drops relative to the previous night, I get a notification before users notice.

The point is not the specific thresholds. It is that RRF's biggest strength, its tolerance for one weak retriever, is also what lets a broken retriever go undetected. You have to instrument the input signals, not just the fused output.

The other things I got wrong

A short list of the mistakes worth naming, so you don't repeat them:

• Chunking too small. I started at 256 tokens. Embeddings on very short chunks are noisy and RRF amplifies that noise because the same document appears multiple times in the vector list under different chunk ids. Moving to 800 tokens with 150 overlap fixed it.
• Not deduplicating by document. Fusion at the chunk level meant one document could occupy three positions in the top 5. I now aggregate to document_id after fusion, keeping the max RRF score per document.
• Using the default English text search config on multilingual content. One of my sites has Serbian posts. to_tsvector('english', ...) on Serbian text is close to random. I switched to 'simple' for that site and let vector search carry the semantic load. Do not assume the default config is safe.
• Not filtering by site earlier. My first version fused across all sites then filtered. That threw away good hits because they got pushed past position 100 by unrelated-site noise. Filter in the CTE, not after.

The numbers

The retrieval layer isn't shipped as a product feature; it feeds internal linking, topic clustering, and the "what to write next" signal for the content system. So the honest way to measure it is the downstream effect over time.

Google Search Console, 28-day comparison:

The impressions lift is bigger than the clicks lift, which is what you would expect when better retrieval helps the site cover more query variants (broader topical footprint) faster than it improves per-query CTR. Both moved together, which is what I care about.

I am not going to claim RRF caused all of that. The content itself matters more. But the retrieval layer is what lets the automation identify gaps, link related pieces without duplicating, and cluster topics without me hand-curating. Every one of those depends on hybrid search returning the right neighbors.

What I'd do if I were starting today

1. Start with FTS only. It is one GIN index and one query. Prove the query patterns before you add embeddings.
2. Add pgvector when FTS visibly misses paraphrases on real queries you can name. Not before.
3. Use RRF from the first hybrid query. Do not build a weighted score, do not tune weights, do not normalize. RRF with k=60, ship it.
4. Instrument per-source hit counts on day one. Log vec_hits, fts_hits, overlap for every query. Alarm on drift.
5. Chunk at 600-800 tokens with overlap, dedupe to document after fusion. Do not fuse at the chunk level and hand raw chunks to whatever consumes the top-k.
6. Keep the corpus, embeddings, and tsv in the same Postgres. The operational simplicity of one database, one backup, one connection pool is worth more than any specialized vector store at this scale.

Hybrid search in Postgres is not exotic. It is a small amount of SQL, a couple of indexes, and a serious commitment to watching the two retrievers stay honest. The systems that hold up in production are the ones where the failure mode is loud, not the ones where the happy path is elegant.

If you are building something similar and want to compare notes, or you have a retrieval problem that is not quite fitting into either pure-vector or pure-BM25, I am reachable at lazar-milicevic.com/#contact. More field reports from the same stack are on the blog.