LLM Parameters — Handling, Defaults, and Provider Behaviour

This document covers every numeric model parameter (temperature, top_p, top_k, frequency_penalty, presence_penalty, repetition_penalty, no_repeat_ngram_size) end-to-end: from the Blazor UI through the C# data model, JSON serialization, Python normalization, and each provider's API call.

1. Parameter Catalogue

Parameter	C# field	JSON key	Type (C#)	Default (C#)	Python default constant	Provider support
Temperature	`Temperature`	`temperature`	`float` (non-nullable)	`0.20f`	`DEFAULT_TEMPERATURE = 0.15`	All
TOP-P	`TopP`	`top_p`	`float` (non-nullable)	`0.0`	`DEFAULT_TOP_P = 0.9`	base, http, endpoint
Frequency Penalty	`FrequencyPenalty`	`frequency_penalty`	`float` (non-nullable)	`0.0`	`DEFAULT_FREQUENCY_PENALTY = 0.3`	base, http, mistral
TOP-K	`TopK`	`top_k`	`int?` (nullable)	`null`	`DEFAULT_TOP_K = 50`	http, mistral
Presence Penalty	`PresencePenalty`	`presence_penalty`	`float?` (nullable)	`null`	`DEFAULT_PRESENCE_PENALTY = 0.0`	http, mistral
Repetition Penalty	`—`	`repetitionpenalty`	not in UI	—	`DEFAULT_REPETITION_PENALTY = 1.2`	(commented out)
No-Repeat N-Gram Size	`—`	`norepeatngramsize`	not in UI	—	`DEFAULT_NO_REPEAT_NGRAM_SIZE = 3`	(commented out)

"base" = _prepare_base_request_params() in interface.py, used by groq and endpoint providers.

2. Semantic Convention (Disabled / Default / Custom)

The following 3-value convention applies to all penalty/sampling parameters (except temperature):

Stored value	Meaning	Python normalization result
`null` / absent	Not configured — parameter not sent to API	`None`
Negative (e.g. `-1`)	Explicitly disabled — parameter not sent to API	`None`
`0`	Use system default constant	`DEFAULT_*` constant value
Positive	Use this exact value	Cast to `float` or `int`

Rule: A parameter is added to an API request only when its normalized value is not None.

3. GUI Layer — `SettingsLlm1.razor`

File: ThalabusWeb/Pages/SettingsPages/SettingsLlm1.razor

Field	UI label
Temperature	`Temperature (0.01 - 1.0 - ...)`
TOP-P	`TOP-P (-1: disabled, 0: default value)`
Frequency Penalty	`Frequency Penalty (-1: disabled, 0: default value)`
TOP-K	`TOP-K (-1: disabled, 0: default value)`
Presence Penalty	`Presence Penalty (-1: disabled, 0: default value)`

Temperature does not support the disabled/default semantics (there is no normalization for it on the Python side). The range guidance (0.01 - 1.0 - ...) is intentional.

4. C# Data Model — `LlmSettings.cs`

File: ThalabusInterfaces/Data/Llm/LlmSettings.cs

public float Temperature { get; set; } = 0.20f;    // non-nullable

[JsonProperty("top_p")]
public float TopP { get; set; } = -1.0f;            // non-nullable, default -1 → Python treats as disabled (None)

[JsonProperty("frequency_penalty")]
public float FrequencyPenalty { get; set; } = -1.0f; // non-nullable, default -1 → Python treats as disabled (None)

[JsonProperty("top_k", NullValueHandling = NullValueHandling.Ignore)]
public int? TopK { get; set; }                       // nullable, omitted from JSON when null

[JsonProperty("presence_penalty", NullValueHandling = NullValueHandling.Ignore)]
public float? PresencePenalty { get; set; }          // nullable, omitted from JSON when null

Consistency note: FrequencyPenalty and TopP are non-nullable (float) but default to -1.0f, which Python normalizes to None (disabled, parameter not sent to API). This correctly represents "not configured = do not send". Setting to 0 means "use system default". TopK and PresencePenalty are nullable (?) so they are omitted entirely when not set, which Python also treats as None.

5. Python Normalization — `interface.py`

File: PythonServer/src/core/llm/interface.py, method _get_model_parameters()

The normalization helpers:

def _normalize_float(value, default):
    if value is None:   return None          # absent → None
    v = float(value)
    if v < 0:           return None          # negative (e.g. -1) → disabled = None
    if v == 0:          return default       # 0 → use default constant
    return v                                 # positive → keep

def _normalize_int(value, default):
    if value is None:   return None
    v = int(value)
    if v < 0:           return None
    if v == 0:          return default
    return v

Applied to all parameters:

top_p               = _normalize_float(top_p,               DEFAULT_TOP_P)                # 0.9
top_k               = _normalize_int(top_k,                 DEFAULT_TOP_K)                # 50
frequency_penalty   = _normalize_float(frequency_penalty,   DEFAULT_FREQUENCY_PENALTY)    # 0.3
presence_penalty    = _normalize_float(presence_penalty,    DEFAULT_PRESENCE_PENALTY)     # 0.0
repetition_penalty  = _normalize_float(repetition_penalty,  DEFAULT_REPETITION_PENALTY)   # 1.2
no_repeat_ngram_size= _normalize_int(no_repeat_ngram_size,  DEFAULT_NO_REPEAT_NGRAM_SIZE) # 3

_get_model_parameters() always returns a dictionary with all keys present, but values can be None (= disabled). Providers must check is not None before adding a parameter to the API request.

Important: Temperature is NOT normalized here. It is passed through directly and falls back to DEFAULT_TEMPERATURE = 0.15 where None.

6. Default Constants — `config.py`

File: PythonServer/src/core/llm/config.py

DEFAULT_TOP_P               = 0.9
DEFAULT_TOP_K               = 50
DEFAULT_FREQUENCY_PENALTY   = 0.3
DEFAULT_PRESENCE_PENALTY    = 0.0
DEFAULT_REPETITION_PENALTY  = 1.2
DEFAULT_NO_REPEAT_NGRAM_SIZE = 3
DEFAULT_TEMPERATURE         = 0.15
DEFAULT_TEMPERATURE_COLD    = 0.1

7. Provider Implementations

7.1 Base implementation (`interface.py` — `_prepare_base_request_params`)

Used by: LlmEndpoint (endpoint), LlmGroq (groq) via the default _client_chat_completions.

# Parameters sent
params["model"]                    # always
params["messages"]                 # always
params["temperature"]              # always
params["max_completion_tokens"]    # if not None
params["top_p"]                    # if not None  ✅
params["frequency_penalty"]        # if not None  ✅

⚠️ Known gap: top_k is extracted from _get_model_parameters() but is commented out and not added to the base params dict (# top_k = model_parameters.get("top_k")). presence_penalty is not referenced at all in _prepare_base_request_params — not even as a commented-out line. Providers that rely on _prepare_base_request_params (groq, endpoint) will never send presence_penalty or top_k, regardless of configuration.

7.2 HTTP provider (`provider_http.py` — `_prepare_request_data`)

Used by: LlmHttpRequest (direct) and LlmQwen (inherits).

Direct HTTP requests are bounded by LLM_REQUEST_TIMEOUT_SECONDS (300 seconds by default). AgentPlanner and AgentToolPreparer first check their finetune model through /models, bounded by LLM_FINETUNE_PING_TIMEOUT_SECONDS (3 seconds). If the model is unavailable, they skip the finetune call and continue with the standard model path.

# Parameters sent (all via is-not-None guards)
data["model"], data["messages"], data["temperature"]  # always
data["max_tokens"], data["max_completion_tokens"]     # always (even if None — design choice)
data["top_k"]            # if model_parameters["top_k"] is not None       ✅
data["frequency_penalty"] # if model_parameters["frequency_penalty"] is not None ✅
data["presence_penalty"]  # if model_parameters["presence_penalty"] is not None  ✅

All three penalty/sampling parameters use correct is not None guards. ✅

7.3 Mistral provider (`provider_mistral.py` — `_client_chat_completions`)

# Parameters sent (all via is-not-None guards)
params["model"], params["messages"], params["temperature"]  # always
params["max_tokens"]         # if not None
params["top_k"]              # if model_parameters["top_k"] is not None      ✅
params["frequency_penalty"]  # if model_parameters["frequency_penalty"] is not None ✅
params["presence_penalty"]   # if model_parameters["presence_penalty"] is not None  ✅

All three use correct is not None guards. ✅

Mistral-specific override (_get_model_parameters): When __PREVENT_MISTRAL_REPEAT_TOKENS = True (default), the provider sets higher anti-repetition defaults:

MISTRAL_DEFAULT_FREQUENCY_PENALTY   = 0.9    # instead of DEFAULT_FREQUENCY_PENALTY=0.3
MISTRAL_DEFAULT_NO_REPEAT_NGRAM_SIZE = 3
MISTRAL_DEFAULT_REPETITION_PENALTY  = 1.2

These Mistral defaults are applied only if the parameter is None, 0, or still at the system-level DEFAULT_* value. The == 0 condition in this guard is technically dead code (after base normalization, 0 has already been mapped to DEFAULT_FREQUENCY_PENALTY), so the check == DEFAULT_FREQUENCY_PENALTY is what actually triggers the override.

7.4 OpenAI provider (`provider_openai.py`)

Uses _prepare_request_params() (local, not the base method):

params = {
    "model": model,
    "messages": messages,
    "max_completion_tokens": max_completion_tokens,
    "temperature": temperature
}
# No sampling/penalty parameters added

⚠️ By design: OpenAI's API does support top_p, frequency_penalty, and presence_penalty, but the current OpenAI provider does not forward them. This appears to be an intentional simplification for the managed OpenAI endpoint.

7.5 Groq provider (`provider_groq.py`)

Inherits from LlmInterface and calls super().chat_completions() → base _client_chat_completions() → _prepare_base_request_params().

Sends: top_p, frequency_penalty. Does not send presence_penalty or top_k (base method limitation, see 7.1).

7.6 Endpoint provider (`provider_endpoint.py`)

Fully delegates to LlmInterface._client_chat_completions() → _prepare_base_request_params().

Sends: top_p, frequency_penalty. Does not send presence_penalty or top_k (base method limitation, see 7.1).

7.7 Qwen provider (`provider_qwen.py`)

Inherits from LlmHttpRequest (provider_http.py). Calls super()._prepare_request_data() then applies Qwen-specific overrides (thinking budget, max_tokens cap at 32768).

Inherits the correct is not None guards from HTTP provider. Sends top_k, frequency_penalty, presence_penalty if configured. ✅

8. Parameter Flow Diagram

User input in UI (Blazor)
    │  value -1 / 0 / positive / null
    ▼
C# LlmSettings (JSON serialization)
    │  float: 0 if not set  /  float?: omitted if null
    ▼
Python llmconfig dict (from JSON)
    │  key present with value 0, None, or numeric
    ▼
_get_model_parameters()  [interface.py]
    │  negative → None
    │  0        → DEFAULT_* constant
    │  positive → keep value
    │  None     → None
    ▼
model_parameters dict (all keys, values may be None)
    │
    ├─► HTTP / Qwen provider:    if value is not None: add to data{}   ✅
    ├─► Mistral provider:        if value is not None: add to params{} ✅
    ├─► Base / Groq / Endpoint:  top_p + freq_penalty only             ⚠️
    └─► OpenAI provider:         no sampling/penalty params            ⚠️ (by design)
    │
    ▼
API request to LLM endpoint

9. Issues and Known Gaps

#	Severity	Component	Description
1	Medium	`_prepare_base_request_params`	`presence_penalty` and `top_k` are never sent by base path (Groq, Endpoint providers)
2	Low	`provider_openai.py`	No sampling/penalty params forwarded; by design but undocumented

10. Adding a New Sampling Parameter — Checklist

Add [JsonProperty("key")] public float? NewParam { get; set; } to LlmSettings.cs (use nullable ? type)
Add InputNumber with label New Param (-1: disabled, 0: default value) to SettingsLlm1.razor
Add DEFAULT_NEW_PARAM = <value> constant to config.py
Add new_param = self.llmconfig.get("new_param", None) + _normalize_float/int(new_param, DEFAULT_NEW_PARAM) in _get_model_parameters() in interface.py
Add "new_param": new_param to the returned model_parameters dict
In each provider's request-building method, add: if model_parameters["new_param"] is not None: params["new_param"] = model_parameters["new_param"]
Update this document

11. File Reference

File	Role
`ThalabusWeb/Pages/SettingsPages/SettingsLlm1.razor`	Blazor UI — parameter input fields
`ThalabusInterfaces/Data/Llm/LlmSettings.cs`	C# model — JSON serialization
`PythonServer/src/core/llm/config.py`	Python default constants
`PythonServer/src/core/llm/interface.py`	Normalization (`_get_model_parameters`), base request builder (`_prepare_base_request_params`)
`PythonServer/src/core/llm/providers/provider_http.py`	HTTP provider — reference implementation for correct `is not None` pattern
`PythonServer/src/core/llm/providers/provider_mistral.py`	Mistral provider — overrides defaults, correct `is not None` pattern
`PythonServer/src/core/llm/providers/provider_openai.py`	OpenAI provider — no penalty params forwarded
`PythonServer/src/core/llm/providers/provider_groq.py`	Groq — inherits base, missing `presence_penalty` / `top_k`
`PythonServer/src/core/llm/providers/provider_endpoint.py`	Endpoint — inherits base, missing `presence_penalty` / `top_k`
`PythonServer/src/core/llm/providers/provider_qwen.py`	Qwen — inherits HTTP provider, full parameter support

LLM Parameters — Handling, Defaults, and Provider Behaviour

1. Parameter Catalogue

2. Semantic Convention (Disabled / Default / Custom)

3. GUI Layer — SettingsLlm1.razor

4. C# Data Model — LlmSettings.cs

5. Python Normalization — interface.py

6. Default Constants — config.py

7. Provider Implementations

7.1 Base implementation (interface.py — _prepare_base_request_params)

7.2 HTTP provider (provider_http.py — _prepare_request_data)

7.3 Mistral provider (provider_mistral.py — _client_chat_completions)

7.4 OpenAI provider (provider_openai.py)

7.5 Groq provider (provider_groq.py)

7.6 Endpoint provider (provider_endpoint.py)

7.7 Qwen provider (provider_qwen.py)