You built a RAG pipeline. You embedded your documents, set up a vector database, and wired everything to an LLM. The answers are decent, but something keeps going wrong. When a user asks a question that requires connecting multiple facts across different documents, the model misses it. It finds the right chunks individually but fails to connect the dots.

This is not a model problem. It is a retrieval problem. Classical vector search finds similar text. It does not understand relationships between entities, like how a company connects to its executives, or how one regulation affects another. The knowledge is in your documents, but the links between pieces of knowledge are invisible to the retriever.

That is exactly the gap that GraphRAG and Entity-Relation Fusion are designed to close. Instead of storing isolated text chunks, they build a structured map of entities and the relationships between them, giving your LLM a connected picture rather than a pile of fragments.

---

Classical Vector Search: Mechanics and Limitations

Classical vector search, often called dense retrieval, converts text into numerical vectors (embeddings) and stores them in a vector database. At query time, the query is also converted to a vector, and the database returns the chunks whose vectors are closest to the query vector.

This works well for surface-level similarity. If your question is about "climate change impacts on agriculture," it will reliably surface paragraphs that discuss that topic.

Here is a basic setup using OpenAI embeddings and FAISS:

import faiss
import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(text: str) -> list[float]:
    response = client.embeddings.create(
        input=text,
        model="text-embedding-3-small"
    )
    return response.data[0].embedding

# Index your chunks
chunks = ["Apple was founded by Steve Jobs.", "Tim Cook became CEO in 2011.", ...]
vectors = np.array([embed(c) for c in chunks]).astype("float32")

index = faiss.IndexFlatL2(len(vectors[0]))
index.add(vectors)

# Query
query_vec = np.array([embed("Who runs Apple?")]).astype("float32")
distances, indices = index.search(query_vec, k=3)
results = [chunks[i] for i in indices[0]]

The limitation here is clear. The question "Who runs Apple?" might return the chunk about Tim Cook AND the chunk about Steve Jobs with similar scores. There is no way for the retriever to know that Steve Jobs is historical context and Tim Cook is the current answer, because it does not model the relationship "succeeded" between them.

---

What Is GraphRAG and Entity-Relation Fusion?

GraphRAG replaces the flat list of text chunks with a knowledge graph. During indexing, an LLM or NLP pipeline extracts entities (people, organizations, concepts, events) and the relations between them (founded, acquired, works for, contradicts, etc.) from your documents.

At query time, instead of finding similar vectors, the system traverses this graph to find connected entities and retrieves their surrounding context.

Entity-Relation Fusion is the step that combines the graph traversal results with standard vector retrieval. You get both structural understanding and semantic similarity.

A simplified entity extraction step looks like this:

import anthropic
import json

client = anthropic.Anthropic()

def extract_entities_and_relations(text: str) -> dict:
    response = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": f"""Extract entities and relations from this text.
Return JSON with this structure:
{{
  "entities": [{{ "id": "e1", "name": "...", "type": "..." }}],
  "relations": [{{ "source": "e1", "target": "e2", "relation": "..." }}]
}}

Text: {text}"""
            }
        ]
    )
    return json.loads(response.content[0].text)

# Example
text = "Sam Altman leads OpenAI, which was co-founded by Elon Musk in 2015."
graph_data = extract_entities_and_relations(text)
# Result:
# {
#   "entities": [
#     {"id": "e1", "name": "Sam Altman", "type": "person"},
#     {"id": "e2", "name": "OpenAI", "type": "organization"},
#     {"id": "e3", "name": "Elon Musk", "type": "person"}
#   ],
#   "relations": [
#     {"source": "e1", "target": "e2", "relation": "leads"},
#     {"source": "e3", "target": "e2", "relation": "co-founded"}
#   ]
# }

These entities and relations are stored in a graph database like Neo4j or a lightweight in-memory graph like NetworkX.

---

GraphRAG vs. Vector Search: Feature Comparison

Feature	Classical Vector Search	GraphRAG
Retrieval method	Semantic similarity (cosine/L2 distance)	Graph traversal + semantic search
Handles multi-hop reasoning	Poor	Strong
Understands entity relationships	No	Yes
Setup complexity	Low	High
Indexing cost	Low	High (LLM extraction needed)
Query latency	Fast	Slower
Best for	Single-topic Q&A, document search	Complex reasoning, knowledge-heavy domains
Scales easily	Yes	Requires graph database management
Hallucination risk	Moderate	Lower (grounded in structured facts)

The right choice is almost always about your use case, not about which technology sounds more advanced.

---

Architectural Layout of a GraphRAG Pipeline

A full GraphRAG system has two distinct phases: indexing and querying.

INDEXING PHASE
Raw Documents
    --> Chunker (split into passages)
        --> Entity Extractor (LLM or NLP)
            --> Relation Extractor (LLM or NLP)
                --> Graph Store (Neo4j, NetworkX, etc.)
                --> Vector Store (for chunk embeddings)

QUERYING PHASE
User Query
    --> Named Entity Recognition (identify query entities)
        --> Graph Traversal (find related nodes and edges)
            --> Vector Search (find semantically similar chunks)
                --> Entity-Relation Fusion (merge both results)
                    --> LLM Generation (final answer)

A minimal project layout for this:

graphrag_project/
  indexing/
    chunker.py         # splits documents into passages
    extractor.py       # LLM-based entity and relation extraction
    graph_builder.py   # loads entities into graph store
    embedder.py        # embeds chunks into vector store
  querying/
    graph_retriever.py  # traverses graph for related entities
    vector_retriever.py # finds similar chunks
    fusion.py           # merges graph + vector results
    generator.py        # calls LLM with fused context
  config.yaml
  main.py

---

Implementing Entity-Relation Fusion in Python

The fusion step is what makes GraphRAG powerful. Here is a simple implementation that merges graph traversal results with vector search results before sending context to the LLM:

import networkx as nx
from typing import list

# Assume G is a NetworkX graph built during indexing
G = nx.DiGraph()

def graph_retrieval(query_entities: list[str], hops: int = 2) -> list[str]:
    """Traverse the graph up to N hops from each query entity."""
    context_nodes = set()
    for entity in query_entities:
        if entity in G:
            # Get all nodes within N hops
            neighbors = nx.single_source_shortest_path_length(G, entity, cutoff=hops)
            context_nodes.update(neighbors.keys())
    
    # Return the stored text for each found node
    return [G.nodes[n].get("text", "") for n in context_nodes if "text" in G.nodes[n]]

def fused_retrieval(query: str, query_entities: list[str], vector_results: list[str]) -> str:
    """Combine graph and vector results into a single context block."""
    graph_results = graph_retrieval(query_entities)
    
    # Deduplicate and combine
    all_context = list(set(vector_results + graph_results))
    return "\n\n---\n\n".join(all_context)

# Usage
query = "Who co-founded OpenAI and what role do they have now?"
vector_chunks = ["Sam Altman became CEO of OpenAI in 2019...", ...]
query_entities = ["OpenAI", "Sam Altman", "Elon Musk"]  # extracted from query

fused_context = fused_retrieval(query, query_entities, vector_chunks)

This fused context gives the LLM both the semantically similar passages AND the relationship-aware graph context, resulting in more complete and accurate answers.

---

How to Choose Between GraphRAG and Vector Search

Use classical vector search when:

• Questions are about a single topic or document section
• Your knowledge base is small to medium (under 100k chunks)
• You need fast setup and low infrastructure cost
• Users ask direct, factual questions

Use GraphRAG when:

• Questions require connecting facts across multiple documents
• Your domain has dense entity relationships (legal, medical, financial, research)
• You need to reduce hallucinations on fact-heavy queries
• Users ask "how," "why," or "what is the relationship between" questions

The hybrid approach (Entity-Relation Fusion) is usually the right long-term answer for production systems. Start with vector search, then layer in graph retrieval for the query patterns where vector search consistently fails.

---

Selecting and Querying Graph Databases (Neo4j & Cypher)

For small projects, NetworkX (in-memory Python graph) works fine. For production, use a dedicated graph database:

# docker-compose.yml for Neo4j
services:
  neo4j:
    image: neo4j:5
    ports:
      - "7474:7474"   # browser UI
      - "7687:7687"   # bolt protocol
    environment:
      NEO4J_AUTH: neo4j/your_password
    volumes:
      - neo4j_data:/data

Querying Neo4j with Cypher to find multi-hop relationships:

-- Find all people connected to OpenAI within 2 hops
MATCH (org:Organization {name: "OpenAI"})<-[r*1..2]-(person:Person)
RETURN person.name, [rel in r | type(rel)] AS relationship_chain

---

Q&A

1. Do I need GraphRAG if I already have a good vector search setup?

Not necessarily. If your users' questions are mostly single-topic and your retrieval quality is already high, classical vector search is enough. Add GraphRAG when you see a clear pattern of multi-hop reasoning failures.

2. How expensive is GraphRAG to build and maintain?

The indexing phase is the costly part. Extracting entities and relations requires LLM calls per document chunk, which adds both time and API cost. Ongoing maintenance also requires keeping the graph in sync with new documents.

3. What graph database should I start with?

For prototyping, use NetworkX (Python, in-memory). For production with large graphs, Neo4j is the most mature option. For cloud-managed options, look at Amazon Neptune or Neo4j Aura.

4. Can I use GraphRAG with any LLM?

Yes. The LLM is only used during indexing (for entity extraction) and generation (for answering). The graph traversal and fusion logic are independent of the model you choose.

5. What is the difference between a knowledge graph and a vector index?

A vector index stores text as numbers and finds similar text by distance. A knowledge graph stores named entities and the labeled relationships between them. One finds "what looks similar," the other finds "what is connected and how."

6. How do I extract entities without spending too much on LLM calls?

Use a smaller, faster model for extraction (like GPT-4o-mini or Claude Haiku). Alternatively, use traditional NLP tools like spaCy for entity recognition and only use an LLM for complex relation extraction.

7. Will GraphRAG always give better answers than vector search?

No. For simple factual lookups, classical vector search is often faster and equally accurate. GraphRAG's advantage shows specifically on questions that require reasoning across connected facts.

8. What is "community summarization" in GraphRAG?

Microsoft's GraphRAG implementation clusters related entities into "communities" and generates a summary for each cluster. This lets the system answer broad, high-level questions by summarizing entire topic clusters rather than individual chunks.

9. How do I keep the graph updated when new documents arrive?

Design your indexing pipeline to be incremental. Process only new documents, extract new entities, and merge them into the existing graph. Check for duplicate entities by matching on canonical names before inserting.

10. Is GraphRAG suitable for real-time applications?

The indexing phase is offline and not real-time. The query phase can be fast enough for real-time use if your graph is well-indexed. Graph traversal adds some latency compared to pure vector search, so benchmark against your latency requirements before committing.

My SaaS

Acluebox

Build modular and reusable system prompts with my SaaS,

Acluebox

. Also, free prompt template generators there.

References

• Edge, D. et al. From Local to Global: A Graph RAG Approach to Query-Focused Summarization - https://arxiv.org/abs/2404.16130

• Neo4j Graph Database Documentation - https://neo4j.com/docs/

• FAISS: A Library for Efficient Similarity Search - https://faiss.ai/