Build Your Own Knowledge Graph: A Practical End-to-End Guide

When a knowledge graph is the right answer

A knowledge graph earns its complexity when your questions are about relationships, not documents. "Which merchants were paid by accounts that shared a device with a flagged customer in the last 30 days?" is three hops of joins over sparse, heterogeneous data. A vector index cannot answer it, and a relational schema answers it slowly and with a query nobody wants to maintain.

Before you build, say the query shape out loud. If every real question is "find me the passage that talks about X", you want a vector store and you are done. If the questions involve chains, paths, shared attributes, or aggregation across connected entities, keep reading.

The five-stage pipeline

The order matters more than the tools. Teams that start at stage 4 — "let's put Neo4j in and figure out the model later" — end up with a graph that has 400 relationship types, half of them synonyms, and no query that runs in under a second.

Stage 1: design the ontology

The ontology is your schema: which node labels exist, which relationship types connect them, and which properties live where. Write it before you write code, and keep it small. A first version with 5–8 node labels and 8–12 relationship types is healthy; 40 labels means you modelled the source system instead of the domain.

Three modelling rules that save you later

• Nouns are nodes, verbs are edges. If you find yourself creating a PaymentEvent node just to hang three properties off a relationship, that is fine — reification is legitimate — but do it deliberately, not by accident.
• Direction is semantic. (:Customer)-[:OWNS]->(:Account) reads one way only. Traversal can go either direction at query time, so pick the direction that makes the sentence true and never flip it per-row.
• Every node needs a stable natural key. Not a UUID you minted during ingest — something derivable from the source, so re-running ingestion is idempotent.

Stage 2: extract entities and relations

For structured sources (databases, CSVs, APIs), extraction is a mapping exercise — write the transform and move on. The interesting case is unstructured text, where an LLM does the extraction. The single biggest quality lever is constraining the model to your ontology instead of asking for "entities and relationships".

const SYSTEM = `
Extract entities and relationships from the text.
Allowed node labels: Customer, Account, Transaction, Merchant, Device
Allowed relationships:
  (Customer)-[OWNS]->(Account)
  (Account)-[SENT]->(Transaction)
  (Transaction)-[PAID_TO]->(Merchant)
  (Customer)-[USED]->(Device)
Rules:
- Never invent a label or relationship outside the list.
- Copy entity names verbatim from the text; do not normalise.
- If a relationship is implied but not stated, omit it.
- Return JSON only.
`;

const Extraction = z.object({
  nodes: z.array(z.object({
    id: z.string(),              // natural key from the text
    label: z.enum(["Customer","Account","Transaction","Merchant","Device"]),
    properties: z.record(z.string()),
  })),
  edges: z.array(z.object({
    from: z.string(),
    to: z.string(),
    type: z.enum(["OWNS","SENT","PAID_TO","USED"]),
    evidence: z.string(),        // the sentence that supports it
  })),
});

Note the evidence field. Provenance is not optional: when someone asks "why does the graph say this customer used that device", you need the sentence and the document id. Store both as edge properties. It also gives you a cheap quality metric — edges with no evidence are extraction hallucinations.

Chunking for extraction is not chunking for retrieval

Retrieval chunks are sized for embedding quality. Extraction chunks should be sized so a relationship never straddles a boundary — bigger windows, generous overlap, and a second pass that resolves pronouns before extraction. A 2,000-token window with 200-token overlap is a reasonable default for prose.

Stage 3: resolve entities

"Acme Corp", "ACME Corporation", and "Acme" are one company. If you skip this stage your graph is not a graph — it is a pile of disconnected stars, and every multi-hop query returns nothing. Entity resolution is where most of the real engineering hides.

1. Normalise. Case-fold, strip legal suffixes, collapse whitespace, expand known abbreviations. This alone merges a surprising share of duplicates.
2. Block. Never compare all pairs. Bucket candidates by a cheap key — first three characters, phonetic code, shared domain — then compare only within buckets.
3. Score. Combine string similarity (Jaro-Winkler for names), embedding cosine for descriptions, and structural signals such as shared neighbours.
4. Decide with three bands. Auto-merge above the high threshold, auto-reject below the low one, and queue the middle for review. Do not chase a single magic cutoff.
5. Merge non-destructively. Keep a canonical node with aliases and a SAME_AS trail, so a bad merge is reversible.

Stage 4: load the graph

Loading should be idempotent and batched. In Cypher that means MERGE on the natural key, SET for properties, and UNWIND over a batch parameter rather than one statement per row.

// one-time: make MERGE fast and safe
CREATE CONSTRAINT customer_key IF NOT EXISTS
FOR (c:Customer) REQUIRE c.key IS UNIQUE;

// per batch (parameter $rows is an array of 1,000 records)
UNWIND $rows AS row
MERGE (c:Customer {key: row.customerKey})
  ON CREATE SET c.createdAt = datetime()
  SET c.name = row.name, c.updatedAt = datetime()
MERGE (a:Account {key: row.accountKey})
MERGE (c)-[r:OWNS]->(a)
  SET r.source = row.docId, r.evidence = row.evidence;

• Create constraints before the first load; MERGE without a unique index is a full label scan.
• Batch 1,000–10,000 rows per transaction. Bigger blows heap, smaller wastes round trips.
• Keep a raw staging copy of every extraction. Reloading from staging is how you recover from an ontology change without re-running the LLM.

Stage 5: query it

Now the payoff. The three-hop question from the introduction is one readable statement:

MATCH (flagged:Customer {status: 'FLAGGED'})-[:USED]->(d:Device)
MATCH (d)<-[:USED]-(other:Customer)-[:OWNS]->(:Account)-[:SENT]->(t:Transaction)
MATCH (t)-[:PAID_TO]->(m:Merchant)
WHERE t.createdAt > datetime() - duration('P30D')
  AND other <> flagged
RETURN m.name AS merchant, count(t) AS payments, sum(t.amount) AS total
ORDER BY total DESC LIMIT 20;

Write the five questions your product actually needs before you finish stage 1, then check each one against the ontology. If a question needs a hop you have not modelled, you found a schema gap for free.

Wiring it into an LLM

Two integration styles, and they compose:

For text-to-Cypher, run generated queries as a read-only user with a query timeout and a row cap. Treat generated Cypher exactly like generated SQL: never with write privileges, never without a limit.

Mistakes that kill graph projects

• Skipping the ontology. The graph accumulates near-duplicate relationship types and nobody can write a reliable query.
• Skipping entity resolution. Multi-hop queries silently return empty results, and the team concludes graphs do not work.
• Modelling everything. A graph of your whole warehouse is a slow copy of your warehouse. Model the connected slice.
• No freshness story. A graph that lags reality by a week is a liability in fraud or ops. Decide on batch vs streaming ingest on day one.
• No provenance. Without evidence and source ids you cannot audit, debug, or delete on request.

Weekend build checklist

1. Pick a domain with ~50 documents you understand well.
2. Write the ontology in one markdown page; list five target questions.
3. Run constrained LLM extraction with evidence capture; store raw JSON.
4. Normalise + block + score for entity resolution; eyeball 50 merges.
5. Create constraints, then batch MERGE into Neo4j.
6. Answer your five questions in Cypher; fix schema gaps.
7. Add a text-to-Cypher endpoint with a read-only user and a row cap.

Useful references while you build: the Cypher manual, the Neo4j LLM Graph Builder for a working reference pipeline, and Microsoft Research's GraphRAG write-up for how extraction quality shows up downstream.