paulserban.eu

Writing Edition

Paul Serban

Writing, snippets & book notes

Vol. 2026

August 17, 2026

GraphRAG Explained: What It Is, How It Works, and When to Use It Over Traditional RAG

From knowledge graphs to community summaries - a practical guide to retrieval-augmented systems that reason over relationships, plus five hands-on projects to build real expertise.

Introduction

Retrieval-Augmented Generation (RAG) solved a real problem: large language models are fluent but forgetful, and grounding their answers in external documents reduces hallucination and keeps responses current. For the last two years, "RAG" has effectively meant one pattern - chunk your documents, embed the chunks, store them in a vector database, and retrieve the top-k nearest neighbors for a query. This works remarkably well for lookup-style questions where the answer lives in a single passage. It works far less well when the answer depends on connecting facts scattered across many documents, or when the question itself is broad and exploratory rather than pointed.

GraphRAG is the industry's answer to that gap. Instead of treating a knowledge base as an unordered pile of text chunks, it builds an explicit knowledge graph - entities, relationships, and higher-level groupings of related entities - and uses that structure to guide retrieval and reasoning. The term was popularized by Microsoft Research's 2024 paper and open-source project of the same name, but the underlying idea predates it: combine structured knowledge representation with the generative power of LLMs so that retrieval is relationship-aware, not just similarity-aware. This article walks through why traditional RAG struggles with certain classes of questions, how GraphRAG addresses them technically, what trade-offs you take on in exchange, and five concrete projects you can build to internalize the concepts, from a weekend prototype to a production-grade system.

Context: Why Traditional RAG Falls Short

Vector-based RAG is fundamentally a nearest-neighbor search problem. You embed a query, you embed your corpus in advance, and you retrieve chunks whose embeddings are closest in vector space. This is excellent when the question maps cleanly onto a localized piece of text - "What is the maximum file size for uploads in our API?" will almost certainly be answered by a single paragraph in a single document, and semantic similarity will find it. The failure mode appears with multi-hop or aggregative questions: "How did our churn rate trend differ between enterprise and SMB customers after the Q3 pricing change, and which features were most associated with retention in each segment?" No single chunk contains that answer. It is synthesized from dozens of scattered facts - pricing change dates, churn figures by segment, feature usage logs, support tickets - and a vector search will retrieve chunks that are topically similar to the question's wording without necessarily retrieving the ones that, combined, answer it.

A second failure mode is what the GraphRAG research team called "query-focused summarization" over an entire corpus. Questions like "What are the main themes in these 10,000 support tickets?" or "Summarize the key risks mentioned across this quarter's incident reports" require a holistic view of the dataset, not a handful of nearest-neighbor chunks. Standard RAG retrieves a fixed, small number of passages - typically 3 to 20 - which is structurally incapable of representing the full breadth of a large corpus. You can increase k, but you quickly run into context window limits and the "needle in a haystack" degradation that occurs when models are given large amounts of loosely related text.

A third, subtler issue is that vector similarity has no concept of relationship direction or type. If chunk A mentions "Project Atlas was delayed because of a dependency on Project Helios," a vector search for "why was Project Atlas delayed" might retrieve that chunk, but it has no structured way to then traverse to documents about Project Helios to understand the root cause further upstream. Traditional RAG treats every chunk as an island; it has no native graph-traversal capability, because it never built a graph in the first place.

What Is GraphRAG?

GraphRAG is an architectural pattern that augments or replaces vector-only retrieval with a knowledge graph extracted from your source documents. The pipeline uses an LLM to read through source text and extract entities (people, organizations, products, concepts) and the relationships between them (works-for, depends-on, caused-by, competes-with), producing a structured graph rather than a flat index of embeddings. Once that graph exists, it can be queried in ways vector search cannot: traversing multiple hops between entities, aggregating information about everything connected to a given node, or summarizing entire clusters of related entities at once.

The defining technical contribution of Microsoft's GraphRAG implementation is the addition of a community detection step on top of the extracted graph. Using graph clustering algorithms - the Leiden algorithm is the common choice - the system groups densely connected entities into "communities," then has an LLM generate a natural-language summary of each community. This produces a hierarchy: individual entities and relationships at the base, community summaries in the middle, and higher-level summaries of communities-of-communities near the top. That hierarchy is what makes corpus-wide, query-focused summarization tractable - instead of trying to cram the whole corpus into a context window, you retrieve pre-computed summaries at the right level of granularity for the question being asked.

How GraphRAG Works: The Technical Pipeline

The GraphRAG pipeline can be broken into four stages, each of which is a meaningful engineering decision point in its own right. The first stage is indexing, where source documents are chunked (much like traditional RAG) and passed to an LLM with a prompt instructing it to extract entities and relationships in a structured format, typically JSON or a delimited tuple format. This is not a trivial prompt-engineering exercise - extraction quality directly determines graph quality, and poorly tuned extraction prompts produce noisy graphs with duplicate entities under slightly different names ("Microsoft" versus "MSFT" versus "Microsoft Corporation") or missed relationships.

The second stage is graph construction and entity resolution. Extracted entities and relationships from every chunk are merged into a single graph, which requires deduplicating entities that refer to the same real-world thing - a problem known as entity resolution or record linkage, which predates GraphRAG by decades in the database literature. Simple approaches use string similarity and LLM-based disambiguation; more rigorous approaches use embedding similarity on entity descriptions combined with graph structure (two entities connected to many of the same neighbors are more likely to be duplicates).

The third stage is community detection and summarization. The Leiden algorithm (an improvement on the earlier Louvain method, published by Traag, Waltman, and van Eck in 2019) partitions the graph into communities that are more densely connected internally than externally. For each community, an LLM generates a summary describing the entities, relationships, and themes within it. This is done at multiple levels of the hierarchy - Microsoft's implementation typically produces several levels, from fine-grained local communities up to broad thematic clusters - so that queries can be answered at whatever level of abstraction they require.

The fourth stage is query time, where GraphRAG offers two distinct modes. "Global search" answers broad, corpus-wide questions by retrieving and synthesizing relevant community summaries - useful for "what are the main themes" style questions. "Local search" answers specific, entity-centric questions by starting from relevant entities, traversing their immediate graph neighborhood, and pulling in connected relationships and source text - closer to how traditional RAG behaves, but relationship-aware. Choosing which mode to invoke (or routing dynamically based on query classification) is itself a system design decision.

# Simplified illustration of the GraphRAG indexing stage
# (conceptually aligned with microsoft/graphrag's pipeline, not copied code)

from dataclasses import dataclass
from typing import List
import json

@dataclass
class Entity:
    name: str
    type: str
    description: str

@dataclass
class Relationship:
    source: str
    target: str
    description: str
    weight: float

EXTRACTION_PROMPT = """
You are extracting a knowledge graph from the text below.
Identify entities (people, organizations, products, concepts) and the
relationships between them. Return strict JSON with this shape:
{{
  "entities": [{{"name": str, "type": str, "description": str}}],
  "relationships": [{{"source": str, "target": str, "description": str, "weight": float}}]
}}

TEXT:
{chunk}
"""

def extract_graph_from_chunk(llm_client, chunk: str) -> dict:
    response = llm_client.complete(
        prompt=EXTRACTION_PROMPT.format(chunk=chunk),
        max_tokens=1500,
        temperature=0.0,  # deterministic extraction reduces downstream noise
    )
    try:
        return json.loads(response)
    except json.JSONDecodeError:
        # Production systems need a repair/retry loop here, not a silent failure
        return {"entities": [], "relationships": []}

def merge_entity_mentions(all_extractions: List[dict]) -> dict:
    """Naive entity resolution by normalized name; real systems add
    embedding similarity and LLM-based disambiguation on top of this."""
    merged = {}
    for extraction in all_extractions:
        for e in extraction["entities"]:
            key = e["name"].strip().lower()
            if key in merged:
                merged[key]["description"] += f" | {e['description']}"
            else:
                merged[key] = e
    return merged

Implementation: A Practical Example

To make the pipeline concrete, consider a worked example: building GraphRAG over a company's internal incident postmortems. A traditional RAG system over the same corpus would let you ask "what caused the March outage" and retrieve the specific postmortem - fine for pointed lookups. But leadership's actual question is usually broader: "What are our recurring root causes across the last year of incidents, and which teams are most affected?" That is exactly the query-focused summarization problem GraphRAG targets.

After running the indexing pipeline, the graph would contain entities like services, teams, root-cause categories (e.g., "configuration drift," "third-party dependency failure"), and incidents, connected by relationships such as "caused-by," "owned-by," and "mitigated-by." Community detection would likely surface clusters such as "database-related incidents owned by the platform team" or "third-party API failures affecting the payments team," each with an LLM-generated summary. A global search query can now retrieve those community summaries directly rather than trying to read fifty postmortems end to end.

# Simplified local search at query time
def local_search(query: str, graph, vector_index, llm_client, hops: int = 2) -> str:
    seed_entities = vector_index.find_similar_entities(query, top_k=5)

    context_entities = set(seed_entities)
    frontier = set(seed_entities)
    for _ in range(hops):
        next_frontier = set()
        for entity in frontier:
            next_frontier.update(graph.neighbors(entity))
        context_entities.update(next_frontier)
        frontier = next_frontier

    relationships = graph.get_relationships_among(context_entities)
    source_text = graph.get_source_chunks(context_entities, max_chunks=15)

    context = build_context_block(context_entities, relationships, source_text)
    return llm_client.complete(
        prompt=f"Answer using only this context:\n{context}\n\nQuestion: {query}"
    )

This local search function illustrates the key structural difference from vector RAG: retrieval expands outward from seed entities along graph edges, rather than purely by embedding distance. The hops parameter is a real tuning knob - too few hops and you miss relevant upstream causes; too many and you pull in noise and blow the context budget. Production implementations typically weight traversal by relationship strength and prune low-relevance branches rather than doing a naive breadth-first expansion.

Advanced Concepts

Once the basic pipeline is working, several advanced techniques separate a toy implementation from a production-grade one. The first is the choice of community detection algorithm and resolution parameter. The Leiden algorithm, used in Microsoft's reference implementation, takes a resolution parameter that controls community granularity - too coarse and communities become meaninglessly broad ("everything about the company"); too fine and you lose the summarization benefit. Running Leiden at multiple resolutions to produce a multi-level hierarchy, as the original GraphRAG paper does, is generally better than picking a single cut.

The second advanced concept is hybrid retrieval: combining graph traversal with vector search rather than choosing one exclusively. DRIFT search, introduced in later Microsoft Research work, blends global community-level context with local entity-level detail and iterative follow-up queries, aiming to capture the strengths of both global and local search modes within a single query. In practice, many production systems route queries to vector search, graph traversal, or both based on a lightweight classifier that distinguishes lookup-style questions from exploratory ones.

A third advanced concept is incremental graph updates. The naive GraphRAG pipeline assumes a batch indexing job over a static corpus, but real knowledge bases change continuously. Re-running full extraction and community detection on every document change is prohibitively expensive at scale. Production systems need incremental update strategies - appending new entities and relationships, re-running community detection only on affected subgraphs, and invalidating stale community summaries - which is an active area of engineering work rather than a solved problem with a standard recipe.

A fourth is graph storage and query infrastructure. Early GraphRAG implementations often used in-memory graph structures or Parquet files for prototyping, but production deployments typically move to a dedicated graph database - Neo4j is the most common choice, with its Cypher query language and native support for the kind of multi-hop traversal GraphRAG requires - paired with a vector index (often the same database, since Neo4j and others now support vector search natively) for the hybrid retrieval pattern described above.

Trade-offs and Pitfalls

GraphRAG's benefits come at a real and often underestimated cost, and the first cost is indexing price and latency. Extracting entities and relationships from every chunk requires an LLM call per chunk (sometimes multiple, for multi-pass extraction and gleaning), and community summarization requires further LLM calls per community per hierarchy level. For a corpus of any meaningful size, this indexing cost can run into hundreds or thousands of LLM calls before a single user query is ever answered - a sharp contrast to vector RAG, where indexing is a single embedding-model pass per chunk. Teams evaluating GraphRAG should budget for this explicitly rather than discovering it mid-project.

The second pitfall is extraction quality sensitivity. The entire system rests on the LLM correctly identifying entities and relationships from source text, and errors compound: a missed relationship means a traversal path that should exist doesn't, a hallucinated relationship introduces false connections, and inconsistent entity naming fragments what should be a single node into several. Unlike vector RAG, where a mediocre embedding still places semantically related text reasonably close together, a mediocre knowledge graph can actively mislead downstream reasoning by presenting a confidently wrong structure.

The third pitfall is reaching for GraphRAG when it isn't the right tool. If your actual query patterns are lookup-style - "what does section 4.2 of the contract say" - you are paying the full cost of graph construction and community summarization for a problem vector RAG already solves cheaply and well. GraphRAG earns its cost on multi-hop, relationship-heavy, or corpus-wide summarization questions; teams that adopt it reflexively, because it is the newer and more sophisticated-sounding pattern, often end up with a more expensive system that performs no better - and sometimes worse, due to extraction noise - than the vector baseline they replaced.

Best Practices

Start by profiling your actual query distribution before choosing an architecture. Pull a representative sample of real or anticipated user questions and categorize them: how many are single-fact lookups versus multi-hop reasoning versus corpus-wide summarization? If the overwhelming majority are lookups, a well-tuned vector RAG system with good chunking and reranking will likely outperform GraphRAG on cost and latency while matching it on quality. GraphRAG is justified when a meaningful share of real queries require connecting facts across documents or summarizing themes across the whole corpus - and it's worth validating this with evaluation sets rather than assuming it from the nature of the domain.

When you do build GraphRAG, invest disproportionately in the extraction and entity resolution stages, since they are the foundation everything else rests on. Use low-temperature, deterministic extraction prompts, validate output against a strict schema with retry logic for malformed JSON, and build an entity resolution step that goes beyond exact string matching - embedding-based similarity on entity descriptions catches most near-duplicates that naive normalization misses. It is also worth keeping both a vector index and the graph available at query time, even if your primary retrieval path is graph-based, since a hybrid approach that falls back to vector search for pure lookup queries avoids forcing every question through expensive multi-hop traversal.

Five Practice Projects, From Basic to Advanced

The fastest way to internalize GraphRAG is to build it, starting small and adding complexity deliberately rather than attempting the full Microsoft reference architecture on day one. Each project below builds on skills from the previous one, and together they cover extraction, graph construction, community detection, hybrid retrieval, and production concerns like incremental updates.

The projects are scoped so that a working engineer can complete the first two in a weekend each, with the later ones representing one to three weeks of focused work depending on prior graph database experience.

  1. Entity-relationship extractor over a small document set (Basic). Take 10-20 short documents (Wikipedia articles on related topics work well) and write a pipeline that uses an LLM to extract entities and relationships into a structured JSON format, then loads them into a simple in-memory graph using a library like networkx in Python. Visualize the resulting graph. The goal here is purely to feel the extraction step's sensitivity to prompt design - try the same documents with a loose extraction prompt and a strict, schema-constrained one, and compare the resulting graph quality.

  2. Local search over a real knowledge graph (Basic-Intermediate). Extend project one by adding a vector index over entity descriptions (using something like FAISS or a lightweight vector store) and implement a local search function similar to the one shown earlier in this article: find seed entities by similarity, traverse N hops, and feed the resulting subgraph to an LLM to answer a question. Test it against questions that require connecting two or three documents, and compare answer quality against a plain vector-RAG baseline on the same questions.

  3. Community detection and global search (Intermediate). Scale up to a larger corpus (100+ documents - a good source is a few months of a public Slack export, GitHub issue history, or a set of blog posts on a shared topic) and implement the full Microsoft GraphRAG-style pipeline: extraction, entity resolution, community detection using the Leiden algorithm (the python-igraph or leidenalg packages implement this directly), LLM-generated community summaries, and a global search function that answers corpus-wide questions using those summaries. This is the point at which you'll directly feel the indexing cost trade-off discussed earlier.

  4. Hybrid retrieval with query routing (Advanced). Build a system that classifies incoming queries (using a lightweight classifier or even a prompted LLM call) as lookup, multi-hop, or summarization style, and routes each to the appropriate retrieval strategy - pure vector search, local graph search, or global community search - before passing results to a generation step. Evaluate on a mixed query set and measure whether routing actually improves answer quality and latency versus always using one strategy.

  5. Production GraphRAG service with incremental updates (Advanced). Migrate the graph from an in-memory structure to Neo4j (or another graph database with native vector index support), and build an incremental ingestion pipeline that can add new documents without rebuilding the entire graph and community structure from scratch - this means appending new entities and relationships, identifying which existing communities are affected, and re-running summarization only on those. Wrap the whole thing in a query API with observability (log retrieval paths, token costs per query, and latency breakdowns by pipeline stage) so you can reason about cost and quality in production the way you would any other service.

Key Takeaways

Analogies and Mental Models

A useful mental model is the difference between a library's card catalog and a research librarian who has actually read every book and can explain how they relate. Vector RAG is the card catalog: fast, cheap, and excellent at telling you which book contains a specific fact, provided you can describe that fact well enough to match a card. GraphRAG is closer to the librarian: slower and more expensive to produce, because someone had to actually read and cross-reference everything, but capable of answering "which of these authors influenced each other" or "summarize the major schools of thought across this collection" in a way no card catalog ever could.

A second useful analogy is the difference between a search engine and an encyclopedia's "see also" and category structure. A search engine (vector RAG) returns the pages most similar to your query. An encyclopedia's category hierarchy (GraphRAG's community structure) lets you ask "what are the major sub-topics under machine learning" and get a curated, synthesized answer, because someone already did the work of organizing the underlying material into a hierarchy rather than leaving it as a flat, unordered set of articles. Neither structure is strictly better - a search engine is faster to build and often faster to query for pointed questions, while the encyclopedia's structure pays off precisely when the question is about the shape of the whole collection rather than a single fact within it.

The 80/20 Insight

If you only take three things from GraphRAG, take these. First, the single highest-leverage decision is correctly diagnosing whether your query patterns actually need graph structure - most teams that adopt GraphRAG without this diagnosis end up paying its costs without reaping proportional benefit, because the majority of real-world queries against most corpora are still lookup-style. Second, extraction quality dominates everything downstream; a team that spends 80% of its engineering effort tuning entity extraction and resolution, and only 20% on fancier retrieval logic, will end up with a better system than the reverse allocation, because no amount of clever traversal logic can recover information that was never correctly extracted into the graph.

Third, you do not need the full Microsoft reference architecture to get most of the value. A large fraction of GraphRAG's practical benefit - multi-hop reasoning and relationship-aware retrieval - is achievable with just entities, relationships, and local search (project two above), without implementing the full community detection and multi-level summarization hierarchy. Reserve the full pipeline, including Leiden-based community detection and global search, for the specific case where corpus-wide, query-focused summarization is a real and recurring requirement, since that is the piece of the architecture that carries the most additional complexity and cost relative to the value it adds for any given use case.

Conclusion

GraphRAG is not a replacement for vector-based RAG; it's a complementary pattern suited to a specific class of problems that vector similarity search handles poorly - multi-hop reasoning across scattered facts and query-focused summarization over large corpora. Understanding when you actually face those problems, rather than adopting GraphRAG because it is architecturally interesting, is the most important decision in this entire space, and it is one best made with real query data rather than intuition about your domain.

The five projects outlined above are designed to build that judgment empirically. By the time you've implemented local search, community detection, hybrid routing, and an incrementally updatable production graph, you will have a concrete, firsthand sense of exactly where GraphRAG earns its cost and where it doesn't - which is a far more durable and transferable skill than knowing how to wire up the Microsoft reference implementation. As with most architectural patterns in this field, the value is less in the specific tool and more in the sharpened intuition for when a problem's structure demands it.

References