Assistant Long-Generation Architecture

Problem Statement

The AgentAssistant (and all agents) uses a medium-sized LLM (e.g., Qwen 3.5 35B A3B MoE). These models tend to exhibit two forms of entropy-driven degradation when producing very long outputs (≥ 4 000 tokens):

  1. Syntax decadence — complex structured output (HTML, JSON, XML) starts breaking down first.
  2. Semantic drift — reasoning quality degrades as accumulated entropy grows.
  3. Terminal chaos — in extreme cases, long sequences of unintelligible tokens are emitted.

Adjusting LLM sampling parameters (temperature, top-p, penalties) cannot eliminate this degradation because it is intrinsic to the current-generation transformer architecture. A different strategy is therefore required.

Solution: Skeleton + Placeholder + Fill Pattern

Instead of asking the LLM to produce the entire long answer in one shot, we split the work into three stages:

Stage 1 — Skeleton generation (AgentAssistant)

The AgentAssistant detects that the answer will be long (e.g., because the user asks for a detailed report, a multi-section analysis, etc.) and, instead of writing the full answer, it:

  1. Thinks about the structure of the answer (using the model's reasoning budget).
  2. Outputs a skeleton — a short, well-structured outline of the answer with placeholders where content sections will be inserted.

Placeholder format:

{{NNN|instructions|optional_info-sources|optional_dependencies}}
Field Required Description
NNN ✅ Sequential placeholder number, starting at 1
instructions ✅ Instructions for the filling agent: what to write and in which format
optional_info-sources ❌ Comma-separated list of source hints (doc://…, kg://…, https://…)
optional_dependencies ❌ Comma-separated list of prior placeholder numbers that must be completed first

Current implementation: only NNN and instructions are used. info-sources and dependencies are reserved for a later version.

The skeleton is stored as the AgentAssistant goal's internal_thought (silent — not shown to the user). The agent then injects sibling goals and returns restart_current_goals_level = True.

Stage 2 — Parallel placeholder filling (AgentAssistantFiller)

One AgentAssistantFiller goal is created per placeholder. Each filler:

Parallelization: Filler goals that have no declared dependency on each other are executed in parallel by the thalaplan engine. Filler N has goal_dependent_upon = [-N] (depends only on the AgentAssistant skeleton step).

Stage 3 — Consolidation and assembly (AgentAssistantRewriter)

A single AgentAssistantRewriter goal runs after ALL fillers have completed. It:

  1. Reads the skeleton from the AgentAssistant sibling's goal_internal_thought.
  2. Reads each filler's content from the corresponding AgentAssistantFiller sibling's goal_internal_thought.
  3. Substitutes each {{NNN|…}} placeholder with the corresponding filled content.
  4. (Future) Detects and resolves inconsistencies introduced by independent fillers running in parallel (different terminology, numbering schemes, etc.) — currently a dummy pass that only does substitution.
  5. Returns the assembled final answer as goal_summary (user-visible).

Goal Injection Sequence

When AgentAssistant detects placeholders in the LLM output:

parent (ONLY_SUBGOALS, e.g. CompositeAssistant)
  ├── ... (prior goals: Strategist, Planner, etc.)
  ├── AgentAssistant           [status: Completed; summary=None, internal_thought=skeleton]
  ├── AgentAssistantFiller#1   [dep: -1]       → fills {{1|...}}
  ├── AgentAssistantFiller#2   [dep: -2]       → fills {{2|...}}
  │   (parallel with Filler#1 — no cross-dep)
  ├── ...
  └── AgentAssistantRewriter   [dep: -N, ..., -1]  → assembles final answer

goal_dependent_upon offsets are relative to each goal's own index in the parent's subgoal list.

Module Summary

Class File Role
AgentAssistant agents/AgentAssistant.py Extended: detects placeholders, injects filler/rewriter goals
AgentAssistantFiller agents/AgentAssistantFiller.py Fills one placeholder section; output is silent (internal_thought)
AgentAssistantRewriter agents/AgentAssistantRewriter.py Assembles skeleton + filled sections; dummy consolidation pass

C# Display Handling (Session.Partial.PlanHandling.cs)

The C# rendering layer in InjectPlanGoalsDepthFirstAsync handles the new agents as follows:

The guard !subgoal.TargetAgent.Name.Equals("AgentAssistantFiller") is added to the SkillMessage rendering block to make this explicit, even though the null summary would already prevent rendering.

Future Work