GraphRAG vs Vector RAG: Which Retrieval Architecture Actually Fits

The actual difference

Vector RAG retrieves text that resembles your question. GraphRAG retrieves entities connected to your question's entities. That one sentence predicts almost every behavioural difference between the two.

How vector RAG works, and where it breaks

Chunk, embed, index, and at query time take the top-k nearest neighbours (usually blended with BM25 and re-ranked). It is cheap, it is fast, and for "what does our documentation say about retries" it is the correct answer. Do not replace it out of fashion.

The four queries where it collapses

• Multi-hop. "Which of our suppliers depend on a vendor that had an outage last quarter?" The answer exists in no single chunk, so no chunk is similar to the question.
• Global / thematic. "What are the main themes of complaints this year?" Top-k over 200,000 chunks samples, it does not summarise.
• Aggregation. "How many incidents involved this component?" Counting is not a similarity operation.
• Negation and absence. "Which customers have no active contract?" Absence has no embedding.

How GraphRAG works

1. Index time: extract entities and relationships from documents into a knowledge graph, keeping the source text as provenance on every edge.
2. Community detection: cluster the graph (Leiden or similar) and pre-generate a summary per cluster, hierarchically.
3. Query time (local): map the question to seed entities, expand one to two hops, and serialise that neighbourhood — plus the source snippets — as context.
4. Query time (global): map the question over community summaries, then reduce the partial answers into one.

The expensive part is index time: every document goes through an LLM extraction pass, and community summaries are regenerated as the graph changes. That is the trade — you pay up-front so that traversal at query time is cheap and exact.

MATCH (e:Entity) WHERE e.key IN $seedKeys
CALL {
  WITH e
  MATCH p = (e)-[r*1..2]-(n:Entity)
  RETURN p, r
}
RETURN DISTINCT
  e.name AS seed,
  [rel IN r | { type: type(rel), evidence: rel.evidence, doc: rel.docId }] AS facts
LIMIT 200;

Local vs global search

Global search is the genuinely novel capability. It is also the one people benchmark enthusiastically and then quietly disable, because a single global query can cost dollars. Reserve it for analyst-facing surfaces, not a consumer chatbox.

Comparison table

The hybrid pipeline most teams need

In production the answer is rarely one or the other. The pattern that works:

1. Classify the question — lookup, multi-hop, or thematic. A small model with a closed enum does this reliably and cheaply.
2. Lookup → vector path. Hybrid search, re-rank, answer. Most traffic lands here.
3. Multi-hop → graph path. Entity-link the question, traverse, serialise the subgraph with evidence.
4. Thematic → community summaries, gated behind a budget and cached hard.
5. Merge and cite. Feed both chunk text and graph facts to the generator, and require citations for each claim.

Cost, latency and freshness

• Index-time LLM spend dominates GraphRAG. Estimate it before committing: documents × tokens × extraction passes. For a 100k-document corpus this is a real budget line.
• Incremental ingestion is not free. New documents can change community structure; plan for periodic re-clustering, not just appends.
• Cache global answers aggressively. Thematic questions repeat, and their answers change slowly.
• Set a freshness SLA per path. Vector chunks can be minutes fresh; graph facts are often hours behind. Say which, out loud, in the product.

How to evaluate the choice

Do not benchmark on generic QA sets — they are dominated by single-hop lookup, which flatters vector search. Build a 60-question set from your own logs, stratified: 20 lookup, 20 multi-hop, 20 thematic. Then measure per stratum:

• Answer correctness (human or LLM judge with a rubric).
• Citation precision — is each claim actually supported by the retrieved evidence?
• p95 latency and cost per query, per path.

The usual result: vector wins lookup on cost and ties on quality, graph wins multi-hop decisively, and thematic is graph-only. That table is your architecture, and it is far more persuasive to a reviewer than an opinion.

Decision guide

Worth reading: Microsoft Research on GraphRAG, the Neo4j GraphRAG overview, and the original "From Local to Global" paper for the map-reduce summarisation detail.