PathRAG Service

Overview

PathRAG is a graph-based retrieval-augmented generation technique that explores a knowledge graph via multi-hop beam search, guided by semantic scoring and structural priors. Instead of retrieving isolated document fragments (like classical RAG), PathRAG retrieves connected paths through the graph, capturing relational context between entities.

Service files: - Interface: src/interfaces/pathrag_interface.py - Implementation: src/services/pathrag_service.py

Entry point:

from src.interfaces.pathrag_interface import PathRagInterface
from src.services.pathrag_service import get_pathrag_service

pathrag_service: PathRagInterface = get_pathrag_service()
results = pathrag_service.pathrag("In which village was Bob when the milk was spilled?")

Conceptual Model

Graph Data

The knowledge graph consists of nodes (entities such as Events, Objects, Places, Persons) connected by typed edges (relationships). For example:

(kitchen_incident) -[:participant]->  (Bob)
(kitchen_incident) -[:participant]->  (Jane)
(kitchen_incident) -[:participant]->  (Martha)
(kitchen_incident) -[:takes_place]->  (kitchen)
(kitchen_incident) -[:involved]->     (cup)          # cup of milk
(bobs_house)       -[:contains]->     (kitchen)
(bobs_house)       -[:located_in]->   (Whiskerville)

What PathRAG Retrieves

Given a query, PathRAG returns paths — ordered sequences of nodes and edges — that are most relevant to answering the query.

Example queries and their target paths:

Query Path retrieved
"What did Bob spill off a table?" kitchen_incident -[:involved]-> cup
"In which room did Bob deal with the milk?" cup <-[:involved]- kitchen_incident -[:takes_place]-> kitchen
"In which village was Bob when the milk was spilled?" cup <-[:involved]- kitchen_incident -[:takes_place]-> kitchen <-[:contains]- bobs_house -[:located_in]-> Whiskerville

Algorithm: Beam-Style Multi-Hop Exploration

High-Level Steps

  1. Intent detection — classify the query to determine expected answer type (e.g., place/village, object, person).
  2. Seed selection — entity-link the query to the top-N most relevant graph nodes.
  3. Beam initialisation — create one single-node path per seed node, scored immediately.
  4. Multi-hop expansion — iteratively extend paths one hop at a time up to max_depth.
  5. Final selection — collect all explored paths, deduplicate, and return the top-N by score.

Pseudocode

function query_to_top_paths(prompt, topN, seedN=30, max_depth=4, min_depth=1,
                            beam_width=80, expand_per_path=40,
                            local_keep_m=3, global_keep_threshold=-inf,
                            no_revisit=true) -> list[Path]:

  intent        = infer_intent(prompt)
  expected_types = expected_answer_types(intent)

  # 1) Seed selection
  seed_nodes = get_topN_seed_nodes(prompt, seedN, expected_types)

  # 2) Initialise beam (one 0-hop path per seed)
  beam = []
  all_candidates = {}  # canonical_path -> best_score

  for s in seed_nodes:
    p0    = Path(nodes=[s], edges=[])
    score = score_path(p0, prompt, intent, expected_types)
    beam.append((p0, score))
    all_candidates[canonical(p0)] = max(all_candidates.get(canonical(p0), -inf), score)

  beam = dedup_and_keep_top(beam, beam_width)

  # 3) Multi-hop expansion
  for depth in 1..max_depth:
    next_pool = []

    for (path, score) in beam:

      # 3a) Expand one hop (filtered by whitelist, caps, no-revisit)
      ext_paths = extend_path_by_one_hop(path, prompt, intent,
                    limit=expand_per_path,
                    relation_whitelist=relations_allowed(intent, depth),
                    cap_per_relation=cap_per_relation(intent),
                    no_revisit=no_revisit)

      # 3b) Score each extension
      scored_ext = []
      for ep in ext_paths:
        s = score_path(ep, prompt, intent, expected_types)
        if depth >= min_depth and s < global_keep_threshold:
          continue
        scored_ext.append((ep, s))
        all_candidates[canonical(ep)] = max(all_candidates.get(canonical(ep), -inf), s)

      # 3c) Local keep: guarantee each beam element contributes top-m continuations
      scored_ext = sort_desc(scored_ext)
      keep = scored_ext[0 : local_keep_m]
      if depth < min_depth:
        keep = scored_ext[0 : 2 * local_keep_m]
      next_pool.extend(keep)

    # 3d) Global keep: dedup + trim to beam_width
    next_pool = dedup_and_keep_top(next_pool, beam_width)
    if not next_pool:
      break
    beam = next_pool

  # 4) Return top-N paths from all explored candidates
  return top_n(sort_desc(all_candidates.items()), topN)

Scoring Function

Current Implementation (Option D — raw reranker)

Path scores are driven entirely by the cross-encoder reranker score, with no per-depth normalisation and no depth penalty:

score(path) = WEIGHT_RERANK * s_rerank + WEIGHT_LENGTH_PENALTY * s_length_penalty
# Currently: WEIGHT_RERANK=1.0, WEIGHT_LENGTH_PENALTY=0.0
Term Description
s_rerank Cross-encoder reranker score: reranker(query, serialize(path)) — the dominant signal
s_length_penalty Set to 0.0; within a depth batch every path has the same hop count so this term is a constant offset with no ranking effect. Retained as a fallback tie-breaker only.

Terms deferred to Step 6: s_type, s_coverage, s_relation_prior are all weighted at 0 until intent detection is implemented.

Scoring Mode: Option D vs. A3 — Empirical Comparison

Two scoring modes were evaluated against the 10-query test suite (see Evaluation Results below):

Mode NORMALIZE_WITHIN_DEPTH WEIGHT_DEPTH_PENALTY Hit@1
Option D (current) False 0.0 10/10
A3 True 0.2 4/10

Why A3 fails on multi-hop queries:
A3 min-max normalises every depth batch to [0,1] and then subtracts depth × 0.2. This enforces a ceiling: depth-1 paths max at 0.80, depth-2 at 0.60, etc., regardless of reranker confidence. When the correct answer requires 3+ hops (e.g., spilled_milk_incident → kitchen → whyskerville), the 5-hop path containing whyskerville necessarily scores ≤0.20 and is buried by irrelevant 0- and 1-hop paths that happen to be the highest-normalised entry at their depth. Tests 3, 4, 5, 7, 9, and 10 (all multi-hop) fail under A3 for exactly this reason.

Why Option D works:
The cross-encoder already encodes both semantic match and path length in its raw score. A 4-hop path containing the exact answer phrase scores ~0.93; a 1-hop path mentioning unrelated context scores ~0.24. There is no need for an artificial length bias on top of the reranker's own calibrated confidence.

Path Expansion

function extend_path_by_one_hop(path, prompt, intent, limit,
                                relation_whitelist, cap_per_relation,
                                no_revisit) -> list[Path]:

  frontier = path.last_node()
  edges    = graph.out_edges(frontier) ∪ graph.in_edges(frontier)

  # Filter by intent-based relation whitelist
  edges = [e for e in edges if e.rel in relation_whitelist]

  # Cap per relation type to prevent hub blow-up
  edges = cap_by_relation(edges, cap_per_relation)

  # No-revisit: prevent cycles
  if no_revisit:
    edges = [e for e in edges if e.other_node not in path.nodes]

  # Cheap heuristic pre-sort before reranking
  edges = heuristic_sort(edges, prompt, intent)[0 : limit]

  return [path + e for e in edges]

Key Design Decisions

Decision Rationale
No marginal-gain pruning Cross-encoder scores are not reliably comparable across different path lengths; marginal gain would cut off valid long paths
Local top-m per parent Guarantees each beam element contributes continuations, preventing "vital info at hop 2" from being lost
Global beam dedup after local keep Keeps exploration tractable while avoiding redundant paths
Multi-term scoring Pure semantic similarity is insufficient; type-match and coverage guide the search toward structurally relevant paths
Cycle/hub control No-revisit + relation caps prevent combinatorial explosion on high-degree nodes

Tuning Parameters

Current Values

Parameter Current Value Notes
DEFAULT_TOP_N 5 Paths returned to the caller
DEFAULT_SEED_N 30 Seed nodes fed into the beam
DEFAULT_MAX_DEPTH 6 Raised from 4 — needed for 4–5-hop answers
DEFAULT_BEAM_WIDTH 30 Beam size across depths
DEFAULT_EXPAND_PER_PATH 20 Max neighbours explored per frontier node
DEFAULT_LOCAL_KEEP_M 2 Best continuations guaranteed per beam element
WEIGHT_RERANK 1.0 Full weight on cross-encoder
WEIGHT_DEPTH_PENALTY 0.0 No per-hop discount (Option D)
WEIGHT_LENGTH_PENALTY 0.0 Retired — no ranking effect within a depth
NORMALIZE_WITHIN_DEPTH False Raw reranker scores used directly (Option D)
MIN_SCORE 0.10 Paths below this score are discarded

Recommended / Tuning Guidance

Parameter Recommended Range Notes
DEFAULT_SEED_N 20–50 Depends on entity-linking quality
DEFAULT_MAX_DEPTH 4–6 6 needed when chains span 4+ hops
DEFAULT_BEAM_WIDTH 20–100 Scale with graph branching factor
DEFAULT_LOCAL_KEEP_M 2–5 Higher for noisy/dense graphs
DEFAULT_EXPAND_PER_PATH 15–50 Limits per-parent fan-out before scoring
WEIGHT_DEPTH_PENALTY 0.0–0.1 Non-zero only if multi-hop paths saturate rankings and crowd out 0-hop direct answers

Integration with Thalabus

PathRAG is intended to complement the existing fragment-based RAG (RagService) by operating over the Neo4j knowledge graph rather than vector-indexed document fragments.

Implementation Path

Step 1 — Data model & graph introspection ✅ COMPLETE

Goal: Map the existing graph structures onto PathRAG concepts. Much of this is already in place — no raw schema audit is needed.

What already exists: - Documentation/Documents/PATHRAG_SERVICE_DATA.md — complete catalog of the test entities, relationships, community, and ontology currently loaded in Neo4j (the Adventures of Bob the Cat dataset). Use this as the reference for all development and validation work; do not use EA_LABELS.md (production data). - OntologyInterface.get_ontology_entities(ontology_id) — returns all EntityOntology definitions (label, entity_description, consolidated attributes) for a given ontology. This is the authoritative list of entity types available for traversal. - OntologyInterface.get_ontology_relationships(ontology_id) — returns all RelationshipOntology definitions (label, entity_description, entity_source_labels, entity_target_labels). This is the authoritative list of relationship types for the whitelist. - Community nodes group Entity instances via (Community)-[:AGGREGATES]->(Entity). A Community is also a Dataset; its permanent id is the only community identifier and it has no separate name property. Entities carry name_consolidated, description_consolidated, and other *_consolidated attributes as their canonical property values. The Neo4j label (minus reserved labels such as Entity, Ontology, EntityOntology) is the entity type. - A vector index ontology_entity on Entity.embedding already exists and is maintained by OntologyInterface.create_semantic_search_index() / update_semantic_search_all().

What to do in this step: - Read PATHRAG_SERVICE_DATA.md to understand which entity types (animal, person, place, event) and relationship types (involve, takes_place, is_friend_of, owns, is_child_of) are present in the test dataset. - Define a canonical entity serialization format by delegating to OntologyInterface.get_entities_as_string(community_id=None, entity_ids=[id], include_relationships=False). This returns all consolidated attributes stored on the entity node (name, description, and any domain-specific properties such as fur colour, role, location type, etc.) — not just the three fields held in the in-memory entity dict. Results are cached in a class-level _entity_text_cache dict keyed by entity ID to avoid repeated DB round-trips when the same entity appears across many paths in a beam iteration. - Define a canonical path serialization format that expands every node via _serialize_entity() so the cross-encoder reranker receives the full consolidated attribute text for each entity, separated by directed edge arrows. Example output: Entity 'animal' {id:'bob'}: attributes={'name'=`Bob`, 'description'=`A grey and white tabby cat with distinctive silver fur`, 'fur_color'=`silver grey`} --[involve]--> Entity 'event' {id:'kitchen_incident'}: attributes={'name'=`kitchen_incident`, 'description'=`Bob knocks over a glass of milk on the kitchen counter`}

Rationale: Bare names like "kitchen_incident" or "Bob" give the cross-encoder virtually no signal for a query such as "What colour is Bob's fur?". Full consolidated attributes — including 'fur_color'=\silver grey`— provide exactly the vocabulary the reranker needs to score relevance correctly. - Identify which relationship types should be excluded from traversal (e.g., internal metadata edges not returned byget_ontology_relationships()). Useget_ontology_relationships()as the initial whitelist. For the test dataset this gives:. Add a hard-coded exclusion list for any structural/meta edges that appear in Neo4j but not in the ontology (e.g.CONTAINS,BASED_ON`).

Deliverable: _serialize_entity(entity) (delegates to get_entities_as_string(), cached) and _serialize_path(path) (composes full entity texts with directed edge arrows) helper methods in pathrag_service.py. A default relationship whitelist derived from get_ontology_relationships().

Implementation notes: - _entity_dict_to_nl(entity) emits "{name} is a {label}." as an intro sentence followed by "{name}'s {attr} is {value}." per non-system attribute. str(value).rstrip('.') prevents double dots. - _serialize_path(path, entity_cache) returns a single-line string "{nl_summary} ||| {entity_texts}" using ||| as separator (not \n) so the full text stays on one log line without being truncated by log filters. - NL summary sentences use active/passive voice: "{A} involves {B}", "{A} takes place {B}", "{A} is located in {B}", "{A} is friend of {B}", "{A} owns {B}" — plus (attr_key is attr_value, ...) appended in parentheses when rel_attributes are present on the edge. - Relationship attributes (rel_attributes) are extracted from the Cypher query via properties(r_out) / properties(r_in), filtered to keys ending in CONSOLIDATION_SUFFIX with reserved keys stripped, then attached to each edge dict and rendered in the path text. - _ENTITY_SYSTEM_KEYS = frozenset({"id", "label", "name", "outgoing_relationships", "incoming_relationships", "seed_score", "_relationships_loaded"}) — these keys are skipped during attribute serialization.

Step 2 — Seed selection ✅ COMPLETE

Goal: Given a free-text query, find the top-N graph nodes most likely to contain the answer or to be starting points for traversal.

What already exists: - OntologyInterface.semantic_search_entity(prompt, community_ids, limit) performs exactly this: it embeds the query using EmbeddingsManager, calls the ontology_entity vector index, and returns a dict[entity_id -> entity_dict] with name, description, score, label, all attributes (consolidated), and pre-fetched outgoing_relationships / incoming_relationships.

What to do in this step: - Implement _get_seed_nodes(query, seed_n, community_ids) as a thin wrapper around get_ontology_service().semantic_search_entity(query, community_ids, seed_n). - Convert the returned dict into an ordered list sorted by descending score. - Return a list of entity dicts in the PathRAG-internal format: { id, label, name, description, score, attributes, outgoing_relationships, incoming_relationships }. - Test via CLI: run pathrag_service.py as __main__ with a sample query and verify that seeds are plausible entities.

Deliverable: _get_seed_nodes() private method implemented and tested — largely a wrapper, minimal new code.

Implementation notes: - Thin wrapper around get_ontology_service().semantic_search_entity(query, community_ids=[community_id], limit=seed_n). Returns list sorted by descending seed_score. - Depth-0 paths (one node, no edges) are created directly from seeds and scored via reranker before the expansion loop starts, so the 0-hop bob entity (containing "Bob's fur is orange and fluffy") can rank #1 for direct attribute queries.

Step 3 — Single-hop expansion (Cypher traversal) ✅ COMPLETE

Goal: Given a frontier node, retrieve all directly connected nodes and edges up to the configured caps.

What already exists: - OntologyInterface.get_entities_by_ids(entity_ids, include_relationships=True) already fetches both outgoing_relationships and incoming_relationships for a list of entity IDs in a single Cypher round-trip. Each relationship entry contains rel_type, target_id / source_id, target_name / source_name, and target_labels / source_labels. - The seed nodes returned by semantic_search_entity() (Step 2) already carry pre-fetched relationships, so the first expansion hop can reuse that data without an extra query.

What to do in this step: - Implement _extend_one_hop(path, relation_whitelist, cap_per_relation, no_revisit) by: 1. Taking the frontier node ID from path. 2. If the frontier node already has relationships attached (from Step 2 or a previous call), reuse them. Otherwise call get_ontology_service().get_entities_by_ids([frontier_id], include_relationships=True). 3. Merge outgoing_relationships and incoming_relationships into a flat list of candidate edges. 4. Filter by relation_whitelist (relationship type in whitelist). 5. Apply cap_per_relation (keep at most N edges per relationship type). 6. Apply no_revisit in Python (discard edges leading to nodes already in the path). - Return the list of new candidate (edge, neighbor_entity) pairs. - Test in isolation with a known entity ID and verify correct edge enumeration.

Deliverable: _extend_one_hop() private method implemented and tested — builds on the existing get_entities_by_ids() API.

Implementation notes: - get_entities_by_ids() Cypher was extended to collect properties(r_out) and properties(r_in) as the 5th element of each relationship array, enabling rel_attributes to flow through the pipeline. - _norm_rel_from_ids(rel) normalises the raw relationship dict and attaches "rel_attributes": {...} extracted by stripping CONSOLIDATION_SUFFIX and filtering RESERVED_ONTOLOGY_ATTRIBUTES. - Each edge dict in a path carries {"rel_type": str, "direction": "out"|"in", "rel_attributes": dict}. - Direction-agnostic deduplication: _canon_key(path) resolves the actual graph from_id:rel_type:to_id for each edge (swapping source/target based on direction), sorts all edge keys, and joins them. This collapses reverse traversals of the same physical path (e.g., A→B→C and C←B←A) to the same canonical key, preventing duplicate paths in the results.

Step 4 — Path scoring ✅ COMPLETE

Goal: Score each candidate path so the beam can be pruned toward relevant results.

Deliverable: _score_path() implemented; reranker integration verified.

Implementation notes: - Batch scoring via _score_paths_batch(paths, query, entity_cache, normalize=False) calls rerank_strings once per depth batch, then optionally min-max normalises the scores. - normalize=cls.NORMALIZE_WITHIN_DEPTH is passed at each depth — currently False (Option D). See Scoring Function for the comparative analysis. - flat_scores = [s - depth * cls.WEIGHT_DEPTH_PENALTY for s in raw_scores] — currently a no-op since both constants are 0. The line is retained so A3 mode can be re-enabled by changing the two class-level constants without modifying algorithm code.

Step 5 — Beam search loop ✅ COMPLETE

Goal: Wire together seeds, expansion, scoring, and beam pruning into the full multi-hop loop.

Deliverable: pathrag() fully functional (single-query, no intent detection yet).

Implementation notes: - pathrag(query, community_id, ontology_id, top_n, max_depth, seed_n) is the public method; all parameters have defaults. - Seeds are scored as depth-0 paths before the expansion loop; their raw reranker scores are stored in all_candidates. - At each depth, _extend_one_hop() produces candidate extensions, which are batch-scored, then local top-m per beam element is kept before a global beam dedup prunes to beam_width. - _canon_key() is used both to deduplicate within a depth and to store best scores in the all_candidates dict across all depths. - DEFAULT_MAX_DEPTH raised to 6 after discovering that the kitchen → whyskerville chain requires depth 4 from certain seeds.

Evaluation Results

Test Suite — Adventures of Bob the Cat dataset

Config: community_id=DefaultCommunity_FictionalStories, ontology_id=FictionalStories, max_depth=6, top_n=5

# Query Option D #1 score Answer at #1? A3 #1 score Answer at #1?
1 What color is Bob the Cat's fur? 0.9114 ✅ fur text in entity cache 0.8863 ✅ 0-hop bob entity
2 Who is Jane? 0.9405 ✅ 0-hop jane entity 0.9693 ✅ 0-hop jane entity
3 Who owns Bob the Cat? 0.9485 ✅ Jane owns Bob in NL 0.8000 ❌ garden→garden irrelevant
4 Who are Bob's friends? 0.8433 ✅ Joy via event co-occurrence 0.8000 ❌ welcome_home→house irrelevant
5 Where did the milk spilling incident take place? 0.9795 ✅ spilled_milk_incident→kitchen 0.8000 ❌ bonding_moment→house wrong
6 Who was involved when Bob knocked the milk over? 0.9761 ✅ milk_mishap all actors present 0.9495 ✅ 0-hop milk_mishap entity
7 Where was Bob when the kitchen was cleaned up? 0.8357 ✅ clean_up→kitchen + bob→attic 0.8000 ❌ milk_mishap→bob (no attic)
8 Where did Bob first meet Joy the squirrel? 0.9915 ✅ unexpected_encounter + garden 0.9781 ✅ 0-hop unexpected_encounter
9 In which house did the milk spilling incident happen? 0.8618 ✅ 3-hop via Martha → house 0.8000 ❌ kitchen only (no house)
10 In which village was Bob when the milk was spilled? 0.9390 ✅ kitchen→whyskerville; #3 drops to 0.24 0.8000 ❌ village only at #2 (0.60)
Hit@1 10 / 10 4 / 10

Test 10 score cliff (Option D): Ranks 1–2 score ≥0.87 (contain whyskerville); rank 3 drops to 0.24. This separation gives an LLM consumer a clear quality signal about which paths to trust.

Step 6 — Intent detection & advanced scoring terms

Goal: Improve retrieval quality by steering the search based on the expected answer type.

What already exists: - OntologyInterface.get_ontology_relationships(ontology_id) returns all relationship types with their entity_description, entity_source_labels, and entity_target_labels. This provides the full semantic vocabulary needed to build intent-based relation whitelists without hard-coding. - OntologyInterface.get_ontology_entities(ontology_id) returns entity type definitions with descriptions, which can be used to build the expected-answer-type mapping. - OntologyInterface.get_ontology_for_query(query) can be used to auto-detect the relevant ontology for a query, avoiding the need to specify it manually.

What to do in this step: - Implement _infer_intent(query) — a lightweight classifier (rule-based or small LLM call) that maps a query to an intent category. Use the entity and relationship definitions returned by get_ontology_entities() / get_ontology_relationships() as the vocabulary for classification. - Build relation whitelists per intent by grouping relationship types from get_ontology_relationships() according to their entity_source_labels and entity_target_labels. Cache the result at service startup to avoid repeated graph queries. - Map intents to expected answer entity types using the labels from get_ontology_entities(). - Add s_type and s_relation_prior scoring terms with calibrated weights. - Add s_coverage scoring using lightweight entity extraction from the query (e.g., matching query tokens against name_consolidated fields of seed nodes).

Deliverable: Intent-aware beam search; measurable improvement on multi-hop queries. Relation whitelists driven by the live ontology rather than hard-coded strings.

Step 7 — Integration into Thalabus agents & CLI

Goal: Make PathRAG available to the rest of the system.

Deliverable: PathRAG callable from CLI, API, and at least one agent.

Step 8 — Evaluation & tuning

Goal: Establish a baseline and tune parameters systematically.

Deliverable: Evaluation report; final parameter configuration committed to app_config.py.