GraphRAG

Intent

Represent entities and their relationships as a graph and walk those relationships to answer questions that depend on connections, not on a single passage. It is for domains where the answer is spread across several linked documents.

Problem

Bruna is a reliability engineer at a company with 300 microservices. When an alert fires on the payments service, the right question is rarely “what does the payments documentation say?”. It is “which services does payments call, which of those changed today, and who owns each one?”.

The company’s RAG retrieves passages similar to the question. It finds the payments page, but it does not follow the dependencies. The fact that the anti-fraud service (called by payments) was deployed at 2 p.m. lives in another document, one that looks nothing like the question.

That kind of answer is not in any passage. It is in the link between passages.

The same gap shows up in contracts and electronic invoices (NF-e in Brazil): “is this clause still valid?” depends on who repealed what, not on a paragraph similar to the question. Passage-based RAG does not walk.

The question this pattern answers: how do you answer questions whose answer depends on relationships between several things, spread across several sources?

Solution

Build a graph with the domain’s entities (services, people, contracts, clauses, laws) and the relationships between them (calls, owns, cites, repeals, depends on). Relationships come from structured sources whenever possible (service catalog, database) and, when needed, are extracted from text by a model.

At question time:

1. Identify the entities mentioned.
2. Walk the graph from them, with a query (neighbors up to N hops, filtered by relationship type).
3. Combine the facts along the path with the text passages linked to each entity.
4. The model answers from that subgraph, citing the relationships.

Microsoft Research published GraphRAG (Edge et al., 2024) as an approach for questions that need a view of a whole corpus: a graph is extracted from the documents and dense clusters get summaries. The production lesson, even outside that paper, is the same: extract relationships from a trustworthy source first; model extraction comes in only where the source does not exist, with review.

Structure

flowchart LR
  Q(["1. Question"]):::io --> E(["2. Identify entities"]):::llm
  E --> T["3. Walk the graph"]:::tool
  G[("Graph: entities and relationships")]:::store --> T
  T --> C["4. Subgraph and linked passages"]:::tool
  D[("Documents")]:::store --> C
  C --> M(["5. Answer citing the path"]):::llm

1. The question arrives.
2. A small model, or a name dictionary, identifies the entities mentioned.
3. A graph query brings back linked entities, within the relevant radius and relationship types.
4. The subgraph’s facts join the document passages linked to each entity.
5. The model answers showing the path (“payments calls anti-fraud, which was deployed at 2 p.m.”).

Worked example

Question: “Which services called by payments were deployed in the last 6 hours?”.

The name resolver maps “payments” to the payments node. The query walks at most 2 hops on CALLS, then crosses recent DEPLOYED and OWNS:

payments -CALLS-> antifraud   owner: risk-team     deploy: r-8812 at 14:05
payments -CALLS-> ledger      owner: finance-team  deploy: (none in 6h)
payments -CALLS-> notify      owner: msg-team      deploy: r-8801 at 09:12

The context the model receives is not “the 5 passages most similar to the question”. It is this list of facts, plus a paragraph from the runbook of each service that changed. The answer cites the path. If antifraud is not in the graph, the system says the entity did not resolve, instead of inventing an owner.

Compare with passage RAG on the same question:

Pseudocode

schema Entities { names: list[string], intent: "dependencies" | "ownership" | "impact" | "other" }

function answer(question):
    ents = llm(ENTITY_PROMPT, question, model="small") -> Entities            # 2
    nodes = graph.resolve(ents.names)                                         # name to node
    sub = graph.query("""
        MATCH (s:Service)-[:CALLS*1..2]->(d:Service)
        WHERE s.id IN $ids
        OPTIONAL MATCH (d)<-[:OWNS]-(t:Team)
        OPTIONAL MATCH (d)-[:DEPLOYED]->(r:Release) WHERE r.at > now() - duration('PT6H')
        RETURN s, d, t, r""", ids=nodes)                                      # 3
    passages = docs.for_entities(sub.nodes, k_per_entity=2)                   # 4
    trace.span("graph_rag", nodes=len(sub.nodes), edges=len(sub.edges))
    return llm(ANSWER_PROMPT, question, render(sub), passages)               # 5

Real-world scenarios

• Legal assistant. Laws, articles, precedents, and contracts form a graph of citations and repeals. “Is this clause still valid?” depends on following who repealed what.
• On-call incident agent. The graph of dependencies between services, owners, and recent deploys narrows down an alert’s likely causes.
• Invoice and contract extraction. Tax ids, contracts, price-adjustment clauses, and invoices tied to the same supplier form a small, stable graph, useful when the question crosses documents.

Applicability

• Questions involve relationships (“who depends on”, “what changed between”, “which are affected by”).
• The domain’s entities and relationships are fairly stable and identifiable.
• Structured sources of the relationships already exist (catalog, CMDB, database).
• Passage-based RAG fails on multi-hop questions.

When not to use

• Questions are about one document or one passage. RAG solves it for less.
• There is no reliable source of relationships, and extracting them with a model would add too much noise. A wrong graph is worse than none.
• The domain changes too fast to keep the graph current.
• An agent with query tools can solve it by searching in steps. See Agentic RAG.

A graph extracted from text only, with no review, often looks smart in the demo and lies in production. If the only source is a contract PDF, extract a few relationships (parties, term, index) with a schema and a human in the loop, as in the invoice scenario.

How to implement

1. List the questions the current RAG cannot answer and the relationship types they require.
2. Define the graph schema: entity types, relationship types, attributes.
3. Populate it from structured sources first; use model extraction only where there is no source, with review.
4. Link each entity to the documents it appears in.
5. Write traversal queries for the most common question types, with a depth limit.
6. Build the context with the subgraph as readable text plus the linked passages.
7. Evaluate with multi-hop questions, comparing against plain RAG.
8. Keep the graph current with the same events that change the sources (deploys, registrations).

Start with a graph that fits in memory, fed by the catalog you already have. A graph database comes in when queries get richer than a 2-hop neighborhood, not on day one.

Common mistakes

• Unbounded traversal. Two hops on CALLS in a dense mesh explode the subgraph. LIMIT and a max depth are part of the pattern, not an optimization.
• A name that does not resolve. “payments”, “Payments”, “svc-pay”. Without aliases and a resolver, the subgraph comes back empty and the model improvises.
• A stale graph. The 2 p.m. deploy landed in git and not in the graph. The explainable answer stays wrong until the lag ends. Treat lag as an incident.
• Sending the subgraph as unreadable JSON. The model errs less when each edge becomes a sentence: payments CALLS antifraud.

Variations

• Curated knowledge graph. Entities and relationships from official systems. More reliable.
• Model-extracted graph. The model reads documents and extracts triples (entity, relationship, entity). Quick to build, needs cleaning.
• Community summaries. Dense clusters of the graph get summaries, used for broad questions (“what are the main themes in these contracts?”). That is the central piece of Microsoft’s GraphRAG (2024).
• Graph as an agent tool. The agent decides when to query the graph, with parameterized queries.

Cost, latency, and failure modes

How to evaluate

• Multi-hop questions, with the right answer and the expected path.
• Comparison with passage-based RAG on the same questions.
• Graph quality: samples of relationships checked against their source.

{ "question": "Which services called by payments were deployed in the last 6 hours?", "expected_path": ["payments -CALLS-> antifraud", "antifraud -DEPLOYED-> r-8812"], "expected_answer_contains": ["antifraud"] }

Also evaluate the negative case: a service that payments does not call must not appear on the path. A false-positive edge is worse than a “I don’t know”.

In production

• A span with resolved entities, subgraph size, and the query used.
• An alert for frequent entities that fail to resolve.
• Monitoring of the lag between sources and the graph.

If nodes on the span goes above a few dozen, the query is too wide for the question type. Tune depth and relationship type, not the answer prompt.

Security

The graph concentrates an org map: who calls whom, who owns what, what changed. Permission belongs in the traversal. An intern who asks about payments must not receive the payroll subgraph. Filtering nodes after the model answers is too late.

Relationships extracted from an outside document (a supplier PDF, an open wiki) are persisted untrusted content. A clause that “establishes that the supplier owns the payments service” must not become an OWNS edge without review. See Untrusted Instructions.

Pros and cons

Code

Both tabs resolve the question’s entities, walk the relationships, and build the context for the answer.

import anthropic
import networkx as nx

client = anthropic.Anthropic()


def build_graph(catalog: list[dict], deploys: list[dict]) -> nx.DiGraph:
    g = nx.DiGraph()
    for svc in catalog:  # structured source: the service catalog
        g.add_node(svc["name"], kind="service", owner=svc["owner"])
        for dep in svc["calls"]:
            g.add_edge(svc["name"], dep, rel="CALLS")
    for d in deploys:  # recent events
        g.nodes[d["service"]].setdefault("deploys", []).append(d)
    return g


def neighborhood(g: nx.DiGraph, start: str, depth: int = 2) -> list[str]:  # 3
    lines = []
    for node, dist in nx.single_source_shortest_path_length(g, start, cutoff=depth).items():
        data = g.nodes[node]
        recent = [d["at"] for d in data.get("deploys", [])]
        lines.append(f"{node} (owner: {data.get('owner')}, distance {dist}, recent deploys: {recent})")
    return lines


def answer(question: str, g: nx.DiGraph) -> str:
    names = [n for n in g.nodes if n in question.lower()]  # 2. simple resolution by name
    context = "\n".join(line for n in names for line in neighborhood(g, n))  # 4
    reply = client.messages.create(  # 5
        model="claude-sonnet-5-5", max_tokens=2000,
        system="Answer using only the listed relationships. Show the dependency path that supports the answer.",
        messages=[{"role": "user", "content": f"<graph>\n{context}\n</graph>\n\n{question}"}],
    )
    return reply.content[-1].text

import Anthropic from "@anthropic-ai/sdk";
import neo4j from "neo4j-driver";

const client = new Anthropic();
const driver = neo4j.driver(process.env.NEO4J_URL!, neo4j.auth.basic(process.env.NEO4J_USER!, process.env.NEO4J_PASSWORD!));

export async function answer(question: string, serviceIds: string[]) {
  const session = driver.session({ defaultAccessMode: neo4j.session.READ });
  try {
    const res = await session.run( // 3. traversal limited to 2 hops
      `MATCH (s:Service)-[:CALLS*1..2]->(d:Service) WHERE s.id IN $ids
       OPTIONAL MATCH (t:Team)-[:OWNS]->(d)
       OPTIONAL MATCH (d)-[:DEPLOYED]->(r:Release) WHERE r.at > datetime() - duration('PT6H')
       RETURN s.id AS from, d.id AS to, t.name AS owner, collect(r.version) AS releases LIMIT 50`,
      { ids: serviceIds },
    );
    const facts = res.records
      .map((r) => `${r.get("from")} -> ${r.get("to")} (owner: ${r.get("owner")}, recent releases: ${r.get("releases")})`)
      .join("\n"); // 4
    const reply = await client.messages.create({ // 5
      model: "claude-sonnet-5-5",
      max_tokens: 2000,
      system: "Answer using only the listed relationships. Show the dependency path that supports the answer.",
      messages: [{ role: "user", content: `<graph>\n${facts}\n</graph>\n\n${question}` }],
    });
    return reply.content.find((b) => b.type === "text")?.text ?? "";
  } finally {
    await session.close();
  }
}

Check your understanding