RAG and Reranking Service Architecture

Last Updated: July 30, 2026 Status: ✅ Fragment-Level Implementation Complete | ✅ Single Fast-Path Output (No LLM Reranking in AgentRAG) | ✅ Traditional Embeddings Path Removed | ✅ Goal-Level Dataset Fill-Up

Recent Updates (April 2026)

Table of Contents

  1. Overview
  2. Fragment-Level Architecture
  3. Boolean Query Language
  4. Reranking Service
  5. Semantic Search Implementation
  6. Data Flow
  7. Integration Points

Overview

The RAG (Retrieval-Augmented Generation) service orchestrates semantic search, text search, and reranking operations while maintaining fragment-level granularity throughout the pipeline. Fragments are only aggregated to documents at the final output stage.

Key Architecture Principle

"We must stay at FRAGMENT level for rerankings. We never go to document level neither in RAG nor in RERANK."

Core Components

Fragment-Level Architecture

Fragment Structure

All fragments maintain a consistent structure throughout the pipeline:

{
    'fragmentid': str,      # Unique fragment identifier (format: "docid-XXXX-cNN", "docid-XXXX-pNN", or "docid-XXXX-sNN")
    'documentid': str,      # Parent document ID (format: "docid-XXXX")
    'text': str,           # Fragment text content
    'score': float,        # Relevance score (0.0-1.0)
    'source': str          # Source type: 'chapters', 'paragraphs', 'summaries', 'keywords', 'category'
}

Fragment ID Format: - Chapters: docid-3894-c1, docid-3894-c2, etc. - Paragraphs: docid-3894-p1, docid-3894-p2, etc. - Summaries: docid-3894-s1, docid-3894-s2, etc.

The reranking service extracts document IDs from fragment IDs by removing the suffix after the last hyphen.

Search Methods Return Fragments

_semantic_search()

_text_search()

Fragment Combination Operators

All combination operations preserve fragment-level structure:

_combine_fragments_and()

def _combine_fragments_and(
    left_fragments: List[Dict[str, Any]], 
    right_fragments: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:

Logic: Returns fragments from documents present in both branches

Scoring Strategy: - Groups fragments by document ID - If both branches are semantic (not pure filters): - Normalizes scores relative to each branch's max - Combines using geometric mean to favor documents scoring well on both criteria - Result: combined_score = sqrt((left_norm * right_norm)) * sqrt(left_max * right_max) - If one branch is a pure filter (all scores = 1.0), preserves semantic scores - If both branches are filters, takes max score

Example:

"kitchen floor" AND "cleanup"
→ Returns fragments only from documents matching both conditions

_combine_fragments_or()

def _combine_fragments_or(
    left_fragments: List[Dict[str, Any]], 
    right_fragments: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:

Logic: Returns union of all fragments

Scoring Strategy: - De-duplicates fragments by fragment ID - Keeps the maximum score for each fragment - Preserves all unique fragments from both branches

Example:

"cat adventure" OR "squirrel friend"
→ Returns fragments from documents matching either condition

_combine_fragments_not()

def _combine_fragments_not(
    all_fragments: List[Dict[str, Any]], 
    exclude_fragments: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:

Logic: Returns fragments excluding those from documents in exclude list

Scoring Strategy: - Extracts document IDs from exclude list - Filters out all fragments from those documents - Preserves scores from remaining fragments

Example:

NOT COUNTRY_OF_ORIGIN=SY
→ Returns fragments from documents excluding those with COUNTRY_OF_ORIGIN=SY

Query AST Evaluation

The _evaluate_thalabus_ast() method traverses the query AST working exclusively with fragments:

Method Signature:

def _evaluate_thalabus_ast(
    node: Dict[str, Any], 
    ontology_id: Optional[str], 
    subset_of_datasetids: Optional[List[str]], 
    all_fragments: Optional[List[Dict[str, Any]]], 
    automatically_rerank: bool
) -> Optional[List[Dict[str, Any]]]

Returns: Optional[List[Dict[str, Any]]] (fragments or None for TRUE)

Node Type Handling:

Node Type Operation Returns
AND _combine_fragments_and(left, right) Fragment list from documents in both branches
OR _combine_fragments_or(left, right) Fragment list from documents in either branch
NOT _combine_fragments_not(all, operand) Fragment list excluding operand documents
CATEGORY ontology_service.categories_query() + convert to fragments Placeholder fragments with score=1.0
SEMANTIC Short query: _semantic_search() + _text_search() combined; Long query: _semantic_search() only Fragments from vector/text indexes
TEXT _text_search() Fragments from fulltext index
TRUE No-op None

Branch-Level Reranking

The _rerank_branch_results() method reranks fragments at branch level before combining:

Method Signature:

def _rerank_branch_results(
    query: str, 
    fragments: List[Dict[str, Any]], 
    subset_of_datasetids: Optional[List[str]]
) -> List[Dict[str, Any]]:

Process: 1. Accept fragment list from a query branch 2. Prepare for reranking service: {id: fragmentid, score, text} 3. Call reranking_service.rerank(query, fragments) 4. Update scores in original fragments from reranked results 5. Return reranked fragment list

Fragment Level: ✅ YES - Input and output are fragments

Integration Point: Called during AST traversal after each branch evaluation:

if automatically_rerank and left_fragments is not None and len(left_fragments) > 0:
    left_query = self._serialize_ast_to_query(node['left'])
    left_fragments = self._rerank_branch_results(left_query, left_fragments, subset_of_datasetids)

Final Aggregation Point

thalabus_query() aggregates fragments to documents only when return_documents=True (the default). When called with return_documents=False, fragments are returned as-is directly from the AST evaluator, skipping the aggregation step and the Neo4j document-detail fetch.

AgentRAG always calls thalabus_query(return_documents=False) so that it receives raw fragments with per-fragment scores. The fragment list (e.g. 160 chapters from a semantic search) is passed directly as rag_fragments — no secondary Neo4j chapter re-fetch is needed.

Dataset creation always happens before the optional aggregation (i.e., the dataset is created from raw fragments regardless of return_documents).

Aggregation Logic (return_documents=True only, not used by AgentRAG):

if fragments:
    # Group fragments by document
    fragments_by_doc = {}
    for frag in fragments:
        doc_id = frag.get('documentid')
        if doc_id:
            if doc_id not in fragments_by_doc:
                fragments_by_doc[doc_id] = []
            fragments_by_doc[doc_id].append(frag)

    # Calculate max score per document
    doc_scores = {}
    for doc_id, frags in fragments_by_doc.items():
        doc_scores[doc_id] = max(f.get('score', 0.0) for f in frags)

    # Fetch document details and return document-level results
    # Only at this point do we work with documents

AgentRAG Output — Single Fast-Path

AgentRAG has one retrieval path: the Thalabus query path via _rag_using_service(). After retrieval fragments go through:

  1. Score filter — Thalabus query fragments carry document-level normalized scores (typically 0.01–0.5); filtered by MIN_RERANKING_SCORE_DOC
  2. Per-document fragment cap — keeps only the top MAX_FRAGMENTS_PER_DOC fragments per document (by score), preventing a single large document from crowding out others in the global top-N
  3. Global sort + top-N truncation — sorts by score descending, returns at most MAX_RAG_OUTPUT_FRAG fragments

LLM-based reranking, disambiguation, and traditional embeddings-based retrieval have been removed from AgentRAG entirely. Reranking is handled downstream by AgentReranker as a separate plan step.

_rag_using_service (Thalabus path) logs two result types based on fragment source:

Branch Condition Content
Category-only All fragments have source='category' and score=1.0 Document-level stubs (text=None, fragmentid=documentid)
Semantic/text Fragments have source='chapters' (or 'paragraphs', etc.) and varied scores Actual chapter text with per-fragment relevance scores

In both cases the dataset returned by thalabus_query is preserved and forwarded in the agent response.

Boolean Query Language

The Thalabus query language supports boolean operators for composing complex queries.

Query Types

Semantic Search (Double Quotes)

"search phrase"

Text Search (Single Quotes)

Case-Insensitive:

'search text'

Case-Sensitive:

@'search text'

Category Queries

CATEGORY_NAME=VALUE
CATEGORY_NAME=VALUE1,VALUE2,VALUE3

Boolean Operators

AND
"condition1" AND "condition2"
"asylum seeker" AND COUNTRY_OF_ORIGIN=IQ

Behavior: - Requires all sub-conditions to be satisfied - Only documents in all branches are included - Scores are combined using geometric mean for semantic branches - Filters (score=1.0) are combined with semantic branches by preserving semantic scores

Example Results:

"asylum seeker" AND COUNTRY_OF_ORIGIN=IQ
→ Documents semantically similar to "asylum seeker" 
  AND have category COUNTRY_OF_ORIGIN=IQ
OR
"condition1" OR "condition2"
COUNTRY_OF_ORIGIN=IQ OR COUNTRY_OF_ORIGIN=SY

Behavior: - Any sub-condition being satisfied includes a document - Union of documents from all branches - Keeps maximum score when same fragment appears in multiple branches

Example Results:

"Bob the Cat" OR "adventure"
→ Documents matching either condition, scored by max relevance
NOT
NOT "condition"
NOT CATEGORY=VALUE
"asylum seeker" AND NOT COUNTRY_OF_ORIGIN=SY

Behavior: - Excludes documents matching the condition - Determines all documents for ontology/dataset - If subset_of_datasetids is provided, considers only documents in those parent datasets (OR logic) - If ontology_id is provided, considers only documents in that ontology - If both are provided, applies both filters (documents must be in ontology AND in at least one parent dataset) - Removes documents from the exclude condition - Preserves scores of remaining documents

Example Results:

"asylum seeker" AND NOT COUNTRY_OF_ORIGIN=SY
→ Documents similar to "asylum seeker" 
  EXCLUDING those with COUNTRY_OF_ORIGIN=SY
Parentheses
("condition1" OR "condition2") AND "condition3"
(CATEGORY1=VAL1 AND CATEGORY2=VAL2) OR CATEGORY3=VAL3

Behavior: - Group operations to control evaluation order - Support nested complex expressions - Standard operator precedence: NOT > AND > OR (without parentheses)

Reranking Service

Important: Retrieval and Reranking Scores Are Different Scales

The score field is reused at different stages, but the values do not have the same meaning or calibration:

Stage Producer Meaning Typical observed range
Retrieval RagService.thalabus_query() / vector search Embedding similarity, optionally combined across semantic branches Up to about 0.78 in the docid-43613 case
Reranking External /score API called by RerankingService Raw pairwise relevance score for (reranking query, fragment text) About 0.01 to 0.06 in the docid-43613 dataset
Dataset AgentReranker The reranker score persisted as AGGREGATES.weight. The hidden property (if set) controls dataset-specific visibility. Same value as the reranker score

For complete details about dataset relationships, including the hidden visibility flag, see Documentation/Documents/DATASETS.md.

The external reranker score is not a calibrated replacement for the embedding score. RerankingService.rerank() sends each fragment independently to the configured /score endpoint and stores the response as both score and ranking. The incoming retrieval score is retained only as original_score in the transient result and is not used to rescale the returned value.

Consequently, a document can be correctly ranked first while still receiving a small absolute score. This is exactly what happened for docid-43613: the retrieval dataset contained fragments around 0.78, while luca-s-2#dataset_1_reranker contained the same document's reranker weights around 0.06, 0.05, 0.03, and lower. The rank ordering is meaningful within that reranker invocation; comparing 0.06 directly with the retrieval cutoff or with 0.78 is not meaningful without calibration.

End-to-End Reranking Flow

For a semantic goal, the relevant path is:

  1. AgentRAG receives raw fragments from thalabus_query(return_documents=False).
  2. AgentRAG applies the document-level retrieval cutoff MIN_RERANKING_SCORE_DOC (0.05 by default), then caps and truncates the fragment list to MAX_RAG_OUTPUT_FRAG (1000 by default).
  3. AgentReranker gathers the dataset fragments and processes them in rounds of MAX_RERANKING_INPUT_FRAG (5000 by default). This is a per-round transport bound, not a relevance limit; inputs larger than 5000 are no longer silently discarded.
  4. For each round, AgentReranker loads fragment text and extracts the double-quoted semantic clauses from the Thalabus query. Deterministic category predicates are omitted because they have already been evaluated by retrieval. The semantic clauses are joined with AND and sent to RerankingService.rerank().
  5. RerankingService calls the external /score API in parallel. The query is the semantic query selected by AgentReranker, rather than the full compound query containing category predicates.
  6. Results within each round are sorted by the external score and filtered by MIN_RERANKING_SCORE_FRAG (0.20 by default). If nothing passes, the service keeps a top-fragment fallback for diversity; this fallback does not make every below-threshold result survive.
  7. Round results are deduplicated and globally sorted before AgentReranker persists the resulting score as the dataset relationship weight. Later consumers apply the same 0.20 minimum when building RAG messages, with a limited fill-up path for some low-score datasets.

Input and Output Limits

The limits have different meanings:

The external API is called once per fragment, so multiple rounds increase latency and request volume approximately linearly. They improve recall only for fragments that reach AgentReranker; they cannot recover fragments already removed by AgentRAG's 1000-fragment output cap.

This explains the apparent contradiction in the histogram: retrieval says that the document is a strong candidate, while reranking says that its best fragment has only a modest absolute pairwise score. The reranker is changing both the ordering and the score representation, not merely refining the original embedding score.

Why Scores Can Be Low

The reranker receives one fragment at a time, not the complete document. A fragment may discuss the conversion and interview inconsistency in German while the reranking prompt is English. Even with a multilingual model, the pairwise score depends on the model's calibration, the fragment boundary, the language, and the exact prompt. It is therefore possible for the relevant document to rank first but remain below 0.20.

The semantic-clause reduction addresses the query-shape problem, but it does not translate the query or fragment text. AgentReranker still uses the single- language rerank() path. Multilingual reranking remains a separate improvement and should be evaluated after this change so that its effect is measurable.

Possible Per-Clause Reranking

Reranking each semantic clause independently and combining the results can help when clauses describe different evidence types, such as religious conversion and interview contradictions. However, it changes the meaning of the query: an AND query requires joint evidence, while taking a maximum across separate clause scores rewards a fragment matching only one clause. A safer design is to rerank each clause independently for diagnostics or candidate expansion, then combine scores with an explicit AND rule, such as a normalized geometric mean or a minimum/weighted minimum, rather than an unrestricted maximum.

The recommended sequence is therefore: first measure the reduced combined query; then compare multilingual combined-query reranking; only afterward evaluate per-clause reranking with a combination rule and labelled relevance examples.

The current code also truncates each fragment sent to the external API at 24,000 characters. A relevant passage near the end of a long fragment is not visible to the reranker.

Operational Interpretation

The current 0.20 threshold is therefore a policy choice for the external reranker distribution, not a universal relevance boundary. For multilingual legal data, it should be validated against a labelled sample or replaced with a relative rule such as top-k, percentile, score-gap, or per-query normalization.

Architecture Overview

The reranking system enforces service separation:

Parallel Chapter Processing

The reranking service processes chapters in parallel:

MAX_PARALLELISM = 10  # Configurable max workers
max_workers = min(MAX_PARALLELISM, len(chapters))
with ThreadPoolExecutor(max_workers=max_workers) as executor:
    futures = [
        executor.submit(self._call_reranker_api, query, chapter_text, endpoint, model)
        for chapter_text in chapters
    ]

Benefits: - Faster reranking for multi-chapter documents - Efficient scaling for large documents - Thread-safe with proper error handling

Max-Pool Aggregation

Each document's final ranking score is the maximum of all its chapter ranking scores:

for doc_id in doc_ids:
    max_ranking = max([scores[chapter_id] for chapter_id in chapters_for_doc[doc_id]])

Rationale: A document is relevant if any chapter is highly relevant. This prevents long documents from being unfairly penalized by less relevant chapters.

Reranking Integration with RAG

Branch-Level Reranking

Called during AST evaluation before combining branches: - Reranks each query branch independently - Maintains fragment-level structure - Updates fragment scores in place

Fragment Format for Reranking

fragments_for_reranking = [
    {
        'id': fragment_id,      # fragmentid
        'score': float,         # RAG score
        'text': str            # Fragment text
    },
    ...
]

Reranking Service Interface

def rerank(query: str, fragments: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    """
    Rerank fragments by relevance to query.

    Args:
        query: The query text
        fragments: List of {id, score, text} dicts

    Returns:
        Same list with updated scores based on reranking
    """

Semantic Search Implementation

Semantic Search Process

  1. Get Embedding
  2. Use EmbeddingsManager to get embedding for search term

  3. Handles embedding cache and fallback

  4. Query Multiple Fragment Sources

  5. Chapters: Vector similarity via 'typical_rag' index
  6. Paragraphs: Vector similarity via paragraph index
  7. Summaries: Cosine similarity (manual calculation)

  8. Keywords: Fulltext or vector search

  9. Apply Score Factors

  10. Each source has configurable score factor

  11. Factors configured as constants in RagService

  12. Filter by Minimum Score

  13. MIN_SCORE = 0.10 (minimum similarity threshold)

  14. Removes low-relevance results

  15. Return Fragments

  16. All fragments maintain structure
  17. No document-level aggregation
  18. Ready for boolean operations or reranking

Score Factors (Configurable)

SCORE_FACTOR_PARAGRAPHS = 1.0
SCORE_FACTOR_QUESTIONS = 1.0
SCORE_FACTOR_SUMMARIES = 1.0

SCORE_CAP_CHAPTERS = 1.0
SCORE_CAP_PARAGRAPHS = 1.0
SCORE_CAP_QUESTIONS = 1.0
SCORE_CAP_SUMMARIES = 1.0

MIN_SCORE = 0.10

Fulltext Search Process

  1. Normalize Search Term
  2. Handle apostrophes: "Jane's" → "Jane s"
  3. Remove quotes and special punctuation

  4. Collapse multiple spaces

  5. Query Fulltext Indexes

  6. Chapters fulltext index (TEXT_INDEX_NAME)

  7. Keyword fulltext index

  8. Return Fragments

  9. Fragments from matching chapters/keywords
  10. Scores from fulltext match relevance
  11. Case-sensitive/insensitive configurable

Data Flow

Complete Pipeline Example

User Query: "kitchen floor" AND "cleanup"
    │
    └─→ thalabus_query(return_documents=False, create_dataset=True)
            │
            └─→ tokenize() → Parse into AST
                    │
                    └─→ AST with AND node:
                        left: SEMANTIC "kitchen floor"
                        right: SEMANTIC "cleanup"
                    │
                    └─→ _evaluate_thalabus_ast(AND node)
                            │
                            ├─ Evaluate LEFT: SEMANTIC "kitchen floor"
                            │   └─ _semantic_search("kitchen floor")
                            │       ├─ Get embedding
                            │       ├─ Query chapters vector index → [frag1, frag2, ...]
                            │       ├─ Query paragraphs vector index → [frag3, frag4, ...]
                            │       └─ Return combined fragments [FRAGMENT LEVEL]
                            │
                            ├─ Rerank LEFT branch (if enabled in RagService)
                            │   └─ _rerank_branch_results("kitchen floor", left_fragments)
                            │
                            ├─ Evaluate RIGHT: SEMANTIC "cleanup"
                            │   └─ _semantic_search("cleanup") → fragments [FRAGMENT LEVEL]
                            │
                            ├─ Rerank RIGHT branch (if enabled in RagService)
                            │
                            └─ Combine AND: _combine_fragments_and(left, right)
                                ├─ Group by documentid, find common documents
                                ├─ Combine scores (geometric mean)
                                └─ Return combined fragments [FRAGMENT LEVEL]
            │
            ├─→ Create dataset from raw fragments
            │
            └─→ return (fragments, dataset_ids)   ← no aggregation, return_documents=False
    │
    └─→ AgentRAG fast-path
            ├─ Score filter (MIN_RERANKING_SCORE_DOC)
            ├─ Per-document fragment cap (MAX_FRAGMENTS_PER_DOC)
            ├─ Global sort + top-N truncation (MAX_RAG_OUTPUT_FRAG)
            └─ return rag_fragments  [FRAGMENT LEVEL]

Note: Document-level aggregation (return_documents=True) is available in thalabus_query for non-AgentRAG callers but is not used in the main plan execution flow.

Integration Points

With OntologyService

With RerankingService

With DocumentsService

With EmbeddingsManager

Configuration

RAG Settings

TOP_N_RAG = 200
TOP_N_CHAPTERS = TOP_N_RAG
TOP_N_PARAGRAPHS = TOP_N_RAG * 3
TOP_N_QUESTIONS = TOP_N_RAG
TOP_N_SUMMARIES = TOP_N_RAG

MIN_SCORE = 0.10
TEXT_INDEX_NAME = "chapters_fulltext"
SHORT_LIST_MAX = 10

Reranking Configuration

Fragment-Level Constants:

MIN_RERANKING_SCORE_FRAG = 0.02  # Minimum weight for a fragment to pass the goal-level filter
MAX_RERANKING_OUTPUT_FRAG = 100  # Maximum fragments to return from reranking
MIN_RERANKING_OUTPUT_FRAG = 50   # Minimum fragments for the RerankingService diversity tier
                                 # (Ensures enough fragments to cover MIN_RERANKING_OUTPUT_DOC after aggregation)

Document-Level Constants:

MIN_RERANKING_SCORE_DOC = 0.05   # Minimum score for document-level operations
MAX_RERANKING_OUTPUT_DOC = 10    # Maximum documents in final aggregated results;
                                 # also used as the fill-up target in Goal._collect_fragments_from_goal_datasets
MIN_RERANKING_OUTPUT_DOC = 5     # Minimum distinct documents to ensure diversity (safety net)

Three-Tier Fragment Selection Strategy:

The reranking service uses a three-tier approach to guarantee document diversity:

Tier 1: Score-Based Selection - Keep all fragments scoring ≥ MIN_RERANKING_SCORE_FRAG (0.1) - Ideal case: high-quality fragments pass naturally

Tier 2: Fragment Diversity (when Tier 1 insufficient) - If no fragments meet score threshold: - Select top MIN_RERANKING_OUTPUT_FRAG (50) fragments - Extract distinct document IDs from fragment IDs - If < MIN_RERANKING_OUTPUT_DOC (5) distinct documents: - Add lower-scoring fragments to cover missing documents - Ensures 50 fragments from at least 5 distinct documents - Apply limit of MAX_RERANKING_OUTPUT_FRAG (100 fragments)

Tier 3: Document Aggregation Safety Net - Final aggregation enforces MIN_RERANKING_OUTPUT_DOC (5 documents minimum) - Handles cases where fragment-to-document ratio is unfavorable

How It Works (Detailed Flow):

Reranking Input: 219 fragments (candidate results)
    ↓
Tier 1 - Score Filter:
  MIN_RERANKING_SCORE_FRAG = 0.1
  Result: 0 fragments ≥ 0.1 (all below threshold)
    ↓
Tier 2 - Fragment Diversity:
  Select top 50 fragments: [fragment1(0.0237), ..., fragment50(0.0012)]
  Extract document IDs: 8 distinct documents
  Check: 8 docs ≥ MIN_RERANKING_OUTPUT_DOC(5)? YES ✓
  Expand to cover diversity: Add fragments until 5 docs minimum
  Result: 8 fragments covering 5 distinct documents
    ↓
Tier 3 - Document Aggregation:
  Group fragments by document ID
  Calculate max score per document
  Enforce MIN_RERANKING_OUTPUT_DOC: Keep at least 5 documents
  Result: Final 5 documents returned

Historical Example:

Query: German text "Der Antragsteller hat einen Artikel für eine Zeitschrift verfasst"
Documents: 22 matching category filters
Reranking results:
  - 5 fragments from docid-4975 (German text, high scores: 0.14-0.32)
  - 10 fragments from docid-3894 (French text, low scores: 0.01-0.03)
  - 7 fragments from docid-3782 (German text, medium scores: 0.08-0.12)

Without MIN_RERANKING_OUTPUT_FRAG (old behavior):
  Tier 1: Only 5 fragments from docid-4975 ≥ 0.1
  → Result: Only docid-4975 returned (WRONG - language bias)

With MIN_RERANKING_OUTPUT_FRAG=50 (new behavior):
  Tier 1: 0 fragments ≥ 0.1 (threshold not met)
  Tier 2: Select top 50 fragments → covers docid-4975, docid-3894, docid-3782
  Tier 3: Aggregate → ensures at least 5 documents
  → Result: All 3 documents returned + MIN_OUTPUT_DOC safety net
  → Ensures cross-language results despite score differences

Rationale: - Prevents language bias (e.g., German query excluding French documents) - Ensures diversity in results for exploratory queries - Balances precision (MIN_RERANKING_SCORE) with recall (MIN_RERANKING_OUTPUT)

Goal-Level Dataset Consumption

After AgentReranker runs, it stores its output as a reranked dataset (e.g. {session}#rerank_N) linked to the goal via goal_datasets. When AgentAssistant (or another downstream agent) calls build_messages(include_rag=True), Goal._collect_fragments_from_goal_datasets reads those datasets and builds the RAG context.

Fill-Up Mechanism

In production the weight distribution across fragments in a reranked dataset can be very flat — all weights may fall below MIN_RERANKING_SCORE_FRAG (0.02). Without a fallback this would leave AgentAssistant with no RAG context.

Algorithm (per dataset):

  1. Normal pass — iterate fragments; those with weight >= minimum_rank are added to fragments_by_id.
  2. Track below-threshold candidates — fragments that fail the threshold are collected in filtered_low_score_frags as (weight, id, dict) tuples.
  3. Fill-up check — after processing the dataset, if len(fragments_by_id) < MAX_RERANKING_OUTPUT_DOC and there are candidates and the char budget is not exhausted:
  4. Sort candidates by weight descending.
  5. Add candidates to fragments_by_id until MAX_RERANKING_OUTPUT_DOC is reached or the char budget is hit.
  6. Emit log(INFO, ...) once per dataset when fill-up is triggered.

Fill-up target: MAX_RERANKING_OUTPUT_DOC (10) — matches the maximum number of documents the reranker is expected to return, keeping context concise.

Fill-up is skipped when: - The char budget was already exhausted during the normal pass (truncated=True). - There are no below-threshold candidates. - len(fragments_by_id) already meets or exceeds MAX_RERANKING_OUTPUT_DOC.

RAG Message Budget

The RAG message budget (MAX_RAG_MESSAGE_LENGTH_DEFAULT) and the loaded-documents budget (MAX_LOADED_DOCUMENT_LENGTH_DEFAULT) are independent. _collect_fragments_from_goal_rag_fragments and _collect_fragments_from_goal_datasets both use the full MAX_RAG_MESSAGE_LENGTH_DEFAULT as their ceiling. The reduction for loaded documents is applied only in _truncate_rag_message and _build_category_only_rag_message_lines, where the final assembled message is checked.

Testing Recommendations

Unit Tests

Integration Tests

End-to-End Tests

See src/services/rag_service.py #region unit tests for comprehensive test suite covering: - Semantic search queries - Text search queries - Boolean operations (AND/OR/NOT) - Category queries - Mixed search types - Reranking integration - Bob the Cat fictional document tests

References