mirror of
https://github.com/enricoros/big-AGI.git
synced 2026-05-11 06:00:15 -07:00
Enrico Ros
120 lines
5.4 KiB
Markdown
120 lines
5.4 KiB
Markdown
# LLM Parameters System
|
|
|
|
This document describes how parameters flow through Big-AGI's LLM parameters system, from definition to API invocation.
|
|
|
|
## System Overview
|
|
|
|
The LLM parameters system operates across five layers that transform parameters from global definitions to vendor-specific API calls. Each layer serves a specific purpose in the parameter resolution pipeline.
|
|
|
|
## Parameter Flow Architecture
|
|
|
|
### Layer 1: Parameter Registry
|
|
**File**: `src/common/stores/llms/llms.parameters.ts`
|
|
|
|
The `DModelParameterRegistry` defines all available parameters with their constraints and metadata. Each parameter includes type information, validation rules, and default behavior.
|
|
|
|
**Default Value System**: The registry supports multiple default mechanisms:
|
|
- `nullable` - Parameters that can be explicitly null to skip API transmission
|
|
- `initialValue` - Parameter's base default (e.g., `llmVndOaiRestoreMarkdown: true`)
|
|
|
|
### Layer 2: Model Specifications
|
|
**File**: `src/modules/llms/server/llm.server.types.ts`
|
|
|
|
Models declare which parameters they support through `parameterSpecs` arrays. Each spec can override registry defaults:
|
|
|
|
```typescript
|
|
parameterSpecs: [
|
|
{ paramId: 'llmVndAntThinkingBudget', initialValue: 1024 }, // Override default
|
|
{ paramId: 'llmVndGeminiThinkingBudget', rangeOverride: [0, 8192] }, // Custom range
|
|
]
|
|
```
|
|
|
|
**Parameter Visibility**: The `hidden` flag removes parameters from the UI while keeping them functional. Models can also mark parameters as `required`.
|
|
|
|
### Layer 3: Client Configuration
|
|
|
|
The system provides two UI configurators with different scopes:
|
|
|
|
#### Full Model Configuration Dialog
|
|
**File**: `src/modules/llms/models-modal/LLMParametersEditor.tsx`
|
|
Shows all non-hidden parameters from model's `parameterSpecs`. Used in the models modal for complete configuration.
|
|
|
|
#### ChatPanel Quick Controls
|
|
**File**: `src/apps/chat/components/layout-panel/ChatPanelModelParameters.tsx`
|
|
Shows only parameters that are:
|
|
- In model's `parameterSpecs`
|
|
- Listed in `_interestingParameters` array
|
|
- Not marked as `hidden`
|
|
|
|
**Value Resolution**: Both UIs use `getAllModelParameterValues()` to merge:
|
|
1. **Fallback values** - Implicit parameters get their `runtimeFallback` values
|
|
2. **Initial values** - Model's `initialParameters` (populated during model creation)
|
|
3. **User values** - User's `userParameters` (highest priority)
|
|
|
|
### Layer 4: AIX Translation
|
|
**File**: `src/modules/aix/client/aix.client.ts`
|
|
|
|
The AIX client transforms DLLM parameters to wire protocol format. This layer handles parameter precedence rules and name transformations.
|
|
|
|
**Client Options**: The system supports parameter overrides through `llmOptionsOverride` and complete replacement via `llmUserParametersReplacement`.
|
|
|
|
### Layer 5: Vendor Adaptation
|
|
**Files**: `src/modules/aix/server/dispatch/chatGenerate/adapters/*.ts`
|
|
|
|
Server-side adapters translate AIX parameters to vendor APIs. Each vendor may interpret parameters differently:
|
|
|
|
- **OpenAI**: `vndEffort` → `reasoning_effort`
|
|
- **Perplexity**: Reuses OpenAI parameter format
|
|
- **OpenAI Responses API**: Maps to structured reasoning config with additional logic
|
|
|
|
## Parameter Initialization Process
|
|
|
|
When a model is loaded:
|
|
|
|
1. **Model Creation**: `modelDescriptionToDLLM()` creates the DLLM with empty `initialParameters`
|
|
2. **Initial Value Application**: `applyModelParameterInitialValues()` populates initial values from:
|
|
- Model spec `initialValue` (highest priority)
|
|
- Registry `initialValue` (fallback)
|
|
3. **Runtime Resolution**: `getAllModelParameterValues()` creates final parameter set:
|
|
- Required fallbacks (for missing required parameters)
|
|
- Initial parameters (model defaults)
|
|
- User parameters (user overrides)
|
|
|
|
## Special Parameter Behaviors
|
|
|
|
**Hidden Parameters**: Parameters like `llmRef` are marked `hidden: true` in the registry and never appear in the UI, but remain functional for system use.
|
|
|
|
**Nullable Parameters**: Parameters with `nullable` configuration can be explicitly set to `null` to prevent transmission to the API, distinct from being undefined.
|
|
|
|
**Range Overrides**: Models can override parameter ranges (e.g., different Gemini models support different thinking budget ranges).
|
|
|
|
**Parameter Interactions**: The UI implements business logic like disabling web search when reasoning effort is 'minimal'.
|
|
|
|
## Type Safety Mechanisms
|
|
|
|
The system maintains type safety through:
|
|
- `DModelParameterId` union from registry keys
|
|
- `DModelParameterValue<T>` conditional types for values
|
|
- `DModelParameterSpecAny` interfaces for specifications
|
|
- Runtime validation via Zod schemas at API boundaries
|
|
|
|
## Model Variant Pattern
|
|
|
|
Some vendors use model variants to enable features, for instance:
|
|
- **Anthropic**: Creates separate `idVariant: 'thinking'` entries forcing value of hidden parameters
|
|
- **Google/OpenAI**: Parameters directly on base models
|
|
|
|
## Migration and Compatibility
|
|
|
|
The architecture supports parameter evolution:
|
|
- **Precedence Rules**: Newer parameters take priority during AIX translation
|
|
- **Graceful Degradation**: Unknown parameters log warnings but don't break functionality
|
|
|
|
## Key Implementation Files
|
|
|
|
- **Registry**: `src/common/stores/llms/llms.parameters.ts`
|
|
- **Specifications**: `src/modules/llms/server/llm.server.types.ts`
|
|
- **UI Controls**: `src/modules/llms/models-modal/LLMParametersEditor.tsx`
|
|
- **AIX Translation**: `src/modules/aix/client/aix.client.ts`
|
|
- **Wire Types**: `src/modules/aix/server/api/aix.wiretypes.ts`
|
|
- **Vendor Adapters**: `src/modules/aix/server/dispatch/chatGenerate/adapters/*.ts` |