Compare commits

...

526 Commits

Author SHA1 Message Date
claude[bot] b8ae16eff1 AIX: add user-configurable rate limiting (RPM/TPM) at _LL choke point
Add per-model rate limiting to prevent hitting provider rate limits,
especially during Beam scatter with large contexts and many rays.

- New AixRateGate module with sliding-window token-aware queuing
- Rate gate integrated at _LL level (single choke point, before retry loop)
- Per-model params (llmRateLimitRPM/TPM) with per-service fallback
- DModelsService gains rateLimitRPM/TPM fields for service-level defaults
- Gate key scoped by serviceId:modelId for proper isolation
- Queued requests respect AbortSignal; flushGate() for bulk cancellation
- UI controls in Extra complexity mode (model options dialog)
- No limits by default; users opt-in per model as needed

Closes #979

Co-authored-by: Enrico Ros <enricoros@users.noreply.github.com>
2026-02-20 23:07:20 +00:00
Enrico Ros 88a796fd87 Tools: sweep: sync openai 2026-02-19 19:00:36 -08:00
Enrico Ros e403467d6d LLMs: Gemini 3.1 Pro. Fixes #987 2026-02-19 19:00:06 -08:00
Enrico Ros 1914a2a8a3 Tools: sweep: add sweeps for oai-thinking-depentent-temp 2026-02-18 17:19:37 -08:00
Enrico Ros 683892afef Tools: sweep: disable the no-temperature fix, as by default we don't set it, and it prevents our sweep with it 2026-02-18 17:19:37 -08:00
Enrico Ros 470f8aab70 LLMs: Together updates 2026-02-18 17:19:36 -08:00
Enrico Ros 7a561d6b42 LLMs: OpenPipe updates 2026-02-18 17:19:36 -08:00
Enrico Ros affff0df4a LLMs: Groq updates 2026-02-18 17:19:36 -08:00
Enrico Ros f5a81bdc94 LLMs: Gemini small updates 2026-02-18 17:19:36 -08:00
Enrico Ros 818ed53b53 LLMs: Sweep Alignment 2026-02-18 17:19:36 -08:00
Enrico Ros 12c875f4e3 AIX: OpenAI responses: fix for the older Deep Research models 2026-02-18 17:19:33 -08:00
Enrico Ros 6ff715c0f0 AIX: aixChatGenerateContent_DMessage_FromConversation: classify an errored outcome when the message is interrupted 2026-02-18 17:19:31 -08:00
Enrico Ros c4a89822d8 LLMs: typo 2026-02-18 15:51:18 -08:00
Enrico Ros a8a917f786 Roll AIX 2026-02-18 15:35:44 -08:00
Enrico Ros 3aa9a71a4b LLM Effort: split definition for UI namings with unified backend. #940 2026-02-18 14:55:00 -08:00
Enrico Ros 3758612ed6 LLMs: improve (Registry's) initialValue 2026-02-17 23:49:30 -08:00
Enrico Ros b71a4265f8 LLMs: dissolve requiredFallback 2026-02-17 23:07:55 -08:00
Enrico Ros 870cdb67cf Tools: sweep: update script and results 2026-02-17 22:21:03 -08:00
Enrico Ros 902c9dc3f4 AIX/LLMs: support search disablement client/server correctly 2026-02-17 22:20:59 -08:00
Enrico Ros 0d1db0a360 AIX: OpenAI Responses: remove forcing of no temperature, LLM_IF_HOTFIX_NoTemperature works well 2026-02-17 22:20:44 -08:00
Enrico Ros ddd784f041 LLM Effort: client-side domain check 2026-02-17 20:09:40 -08:00
Enrico Ros 830d45c06d LLM Effort: server-side dev check 2026-02-17 20:09:40 -08:00
Enrico Ros 6e27a31013 LLM Effort: Unified definition. #944, #940 2026-02-17 20:09:40 -08:00
Enrico Ros ed87595e17 LLMs: Anthropic: bit 2026-02-17 19:17:51 -08:00
Enrico Ros da01b59ae3 AIX: Anthropic: Effort is GA - no header needed 2026-02-17 19:17:51 -08:00
Enrico Ros 79046b808b AIX: Gemini: do not use alpha any longer 2026-02-17 19:17:51 -08:00
Enrico Ros 5a71153390 Custom Names: reset with warning. #970 2026-02-17 13:50:17 -08:00
Enrico Ros 94056cdf4b AutoBlocks: #983 option which does not improve things 2026-02-17 13:23:55 -08:00
Enrico Ros 41cb35c6b9 Custom Names: lingering. #970 2026-02-17 12:42:45 -08:00
Enrico Ros e133fc81f6 Custom Names: preserve. #970 2026-02-17 12:16:26 -08:00
Enrico Ros 418c2e496c LLMs: Anthropic: dMessageUtils 2026-02-17 12:01:46 -08:00
Enrico Ros 3690202b38 LLMs: Anthropic: Sonnet 4.6 2026-02-17 11:51:46 -08:00
Enrico Ros f069c2e5ab Fix: safe iteration over navItems.links in mobile nav
Fixes #984
2026-02-17 11:06:44 -08:00
Enrico Ros a1390b152f CC: .gitignore 2026-02-13 18:55:05 -08:00
Enrico Ros 4e8c7d46f6 Cleanup: remove ems 2026-02-13 18:44:35 -08:00
Enrico Ros 02944d2015 AIX: Add no-op method for setting provider infra label 2026-02-13 18:41:49 -08:00
Enrico Ros 58726f0425 AIX: OpenRouter: provider infra label 2026-02-13 17:30:26 -08:00
Enrico Ros 85f796fb1d AIX: ContentReassembler: note 2026-02-13 16:46:47 -08:00
Enrico Ros 311a9c2bf2 Roll AIX 2026-02-13 15:58:27 -08:00
Enrico Ros 6768917d44 Bits 2026-02-13 15:56:52 -08:00
Enrico Ros 7beb412738 AIX: Report broken messages. #980 2026-02-13 15:56:50 -08:00
Enrico Ros cf724625cc AIX: CSF: emulate tRPC's client-side abort as a response to the abortSignal being fired. #980
This is because the exception gets actually trapped locally in the deeper layers
due to client-side processing, which then created a particle for the abort,
which then is never used because the outer will discard it without notice
2026-02-13 15:56:49 -08:00
Enrico Ros f60b2410dd AIX: do not fake logical ends. #980 2026-02-13 15:56:07 -08:00
Enrico Ros bbdc16b06a LLMs: Together.AI: fix wire parser 2026-02-13 12:11:38 -08:00
Enrico Ros 0fa2d06725 AIX: logging: bits 2026-02-13 12:08:04 -08:00
Enrico Ros 36cdc4b55f AIX: Parser: capitalized STOP reason 2026-02-13 12:04:25 -08:00
Enrico Ros c2b4a50bfa AIX: Retriers: consolidated denylist 2026-02-13 12:02:05 -08:00
Enrico Ros 73f88d4715 AIX: OpenRouter: don't log on empty reasoning 2026-02-13 12:01:52 -08:00
Enrico Ros af919be2ac AIX: store end reason - for further debug. #980 2026-02-12 16:31:41 -08:00
Enrico Ros facffbc6c8 AIX: require clean connection ends. #980 2026-02-12 16:31:41 -08:00
Enrico Ros dd5b7cb8c2 AIX: dispatch: increase debugging vendor-initiated disconnect. #980 2026-02-12 14:12:25 -08:00
Enrico Ros 3dc61109d7 AIX: Server: debug recovered packets 2026-02-12 01:34:10 -08:00
Enrico Ros 9ef84260b0 Z.ai: no bits 2026-02-11 22:09:54 -08:00
Enrico Ros cf2df7d7f9 Z.ai: dMessageUtils 2026-02-11 22:09:27 -08:00
Enrico Ros 16a883526b Z.ai: readme 2026-02-11 17:44:33 -08:00
Enrico Ros 7b66b1a2eb Z.ai: readme 2026-02-11 17:44:22 -08:00
Enrico Ros a4adce5c79 Z.ai: AIX: fix reasoning effort 2026-02-11 17:43:23 -08:00
Enrico Ros 9e4174df53 Z.ai: AIX: fix dispatch 2026-02-11 17:36:04 -08:00
Enrico Ros b5975713a3 Z.ai: OCR does not support WebP 2026-02-11 17:26:23 -08:00
Enrico Ros 0cd04266b7 Z.ai: improve model spec 2026-02-11 17:26:23 -08:00
Enrico Ros 5cbd162454 Z.ai: Reasoning settings support 2026-02-11 17:26:23 -08:00
Enrico Ros bea1600358 AIX: OpenAI ChatCompletions: empty reasoning_content yields to non-empty content 2026-02-11 17:26:22 -08:00
Enrico Ros 6a2e201cf5 Z.ai: discovered + curated models support 2026-02-11 17:26:22 -08:00
Enrico Ros 960551933e Z.ai LLM vendor support
Note: we don't include server-side config anymore starting from this. To stress test the config system.
2026-02-11 17:26:22 -08:00
Enrico Ros 8b38b6416d Z.ai: icon & sprite 2026-02-11 17:26:22 -08:00
Enrico Ros fac4c39f48 Fix copying of message Sources. Fixes #977. Fixes #978. 2026-02-11 13:02:32 -08:00
Enrico Ros 4c930efbf0 Fix GC on Beams with reference collectors. 2026-02-11 12:59:36 -08:00
Enrico Ros 5a2a47cb87 AIX: Anthropic: Fast mode - unsupported message 2026-02-10 13:31:28 -08:00
Enrico Ros 4912a03250 LLMs: Anthropic: Fast mode research preview 2026-02-10 13:22:47 -08:00
Enrico Ros 3b13580613 LLMs: parameter-value-based enum price multipliers 2026-02-10 13:04:05 -08:00
Enrico Ros 95905113ac LLMs: cached isLLMChatFree_cached 2026-02-10 12:17:21 -08:00
Enrico Ros c6b34bb252 LLMs: Parameters: type guard enums 2026-02-10 11:53:31 -08:00
Enrico Ros e5387c2323 AIX: Moonshot: remove empty messages 2026-02-10 11:07:09 -08:00
Enrico Ros d3b4447669 CLAUDE.md: update 2026-02-10 01:51:56 -08:00
Enrico Ros d5c5eac9ec CC: allow git mv 2026-02-10 01:51:56 -08:00
Enrico Ros 49b61495d0 LLMs: Vendor Settings: unbreak hide advanced despite initially in CSF. Fixes #969 2026-02-09 23:30:41 -08:00
Enrico Ros e8298e9d30 workflows: CC: enable auth 2026-02-09 13:41:23 -08:00
Enrico Ros b29681e1f7 workflows: CC: cleanups 2026-02-09 13:30:35 -08:00
Enrico Ros 1e0b9a2f0c workflows: CC: do not trigger triage on assignment 2026-02-09 13:08:45 -08:00
Enrico Ros 442b8e95b1 workflows: CC: lock in the dm 2026-02-09 12:53:10 -08:00
Enrico Ros 27090d9e28 -Spaces 2026-02-09 05:41:55 -08:00
Enrico Ros c37b4fa076 Chat: option to discard all reasoning traces 2026-02-09 04:51:42 -08:00
Enrico Ros 83161bbe98 AIX: Anthropic: Parser: hotfix for 4.6 to elide the double-newline at the beginning when present 2026-02-09 04:50:14 -08:00
Enrico Ros 4b166120e6 AIX: Anthropic: Dispatch: hotFix for 4.6 interleaved reasoning blocks back-to-back 2026-02-09 04:23:28 -08:00
Enrico Ros 04494ac752 AIX: Anthropic: Dispatch: hotFix for empty text blocks produced by 4.6 - incoming from the Anthropic API 2026-02-09 04:23:27 -08:00
Enrico Ros 979809ddb1 AIX: Anthropic: Parser: rename hotFix 2026-02-09 04:23:26 -08:00
Enrico Ros 5d797c3339 AIX: Anthropic: warn if blocks come out of order, now that Anthropic has fixed it 2026-02-09 04:22:35 -08:00
Enrico Ros 2ff74f6b80 Wire: separate debug wire request and response 2026-02-09 04:22:33 -08:00
Enrico Ros 06b1195f9a workflows: CC: triade with workarounds: restore some 2026-02-09 01:48:51 -08:00
Enrico Ros c337b70a42 LLMs: Anthropic: copy 2026-02-09 01:40:11 -08:00
Enrico Ros 5047354892 CC: /code:review-inflight bits 2026-02-09 01:40:11 -08:00
Enrico Ros ce4e405fc6 workflows: CC: r/o triage 2026-02-09 01:40:11 -08:00
Enrico Ros 30c8d66cd1 workflows: CC: update model 2026-02-09 01:38:37 -08:00
Enrico Ros fb5c8aad29 workflows: CC: update dm 2026-02-09 01:38:19 -08:00
Enrico Ros 08d221d00f Attachments: Text: warn if empty 2026-02-08 17:31:40 -08:00
Enrico Ros af918178f6 Attachments: Markdown table conversion issue fallback 2026-02-08 17:31:40 -08:00
Enrico Ros ed19896e3c LLMs: llms.parameters: remove 'as const' 2026-02-08 17:31:39 -08:00
Enrico Ros 47ad135e4b CC: slashcommands: update-models catch-all 2026-02-08 17:27:08 -08:00
Enrico Ros 0eff7825c8 CC: slashcommands: xAI Reponses API sync 2026-02-08 17:27:08 -08:00
Enrico Ros 5c8baee390 CC: /code:review-inflight 2026-02-07 13:46:01 -08:00
Enrico Ros 3f71facb49 CLAUDE.md: update 2026-02-07 13:46:01 -08:00
Enrico Ros eba42cc8f2 CLAUDE.md: dev env 2026-02-07 13:46:01 -08:00
Enrico Ros 53092cee51 CC: allow tsc, eslint 2026-02-07 13:46:01 -08:00
Enrico Ros 4bf621f128 LLMs: OpenAI GPT-5.3-Codex speculative support 2026-02-07 13:42:12 -08:00
Enrico Ros 33505dbb8e LLMs: Anthropic/OpenRouter: align behavior, align UI #962 2026-02-06 22:40:55 -08:00
Enrico Ros c81e1f144f AIX: OpenRouter: protocol bits 2026-02-06 20:56:39 -08:00
Enrico Ros ee788b967b Roll AIX 2026-02-06 20:11:10 -08:00
Enrico Ros 38ac8733f6 AIX: OpenRouter: comment on debug: too risky 2026-02-06 20:10:48 -08:00
Enrico Ros 737a20ee06 AIX: OpenRouter: enable the stricter 'require_parametrs' mode. #948 2026-02-06 20:05:05 -08:00
Enrico Ros 19f48b8001 AIX: OpenRouter: wires for OR debug parameters 2026-02-06 19:51:50 -08:00
Enrico Ros 3471d6b4f5 Roll AIX 2026-02-06 19:30:49 -08:00
Enrico Ros 2dc7ba72b3 AIX/LLMs: bits 2026-02-06 19:30:18 -08:00
Enrico Ros e12279dab0 AIX: Anthropic: show the US inference setting when on 2026-02-06 19:24:04 -08:00
Enrico Ros 2e0c79cb64 LLMs: OpenRouter: also inherit the initial temperature from upstreams 2026-02-06 19:19:33 -08:00
Enrico Ros aa697edb8c AIX: Anthropic: minor API changes 2026-02-06 19:18:54 -08:00
Enrico Ros c72e3c58dd AIX: Anthropic: allow US servers 2026-02-06 19:17:01 -08:00
Enrico Ros 1de30c8bd5 AIX: Anthropic: accomodate some API changes 2026-02-06 18:52:58 -08:00
Enrico Ros 3a8eea6fb7 Roll AIX 2026-02-06 18:37:05 -08:00
Enrico Ros b7fd0bdba7 LLMs: OpenRouter: auto-inherit configurable parameters from Anthropic, Gemini and OpenAI.
Fixes #948: OpenAI-through-OR verbosity is sync'd with OpenAI models.

Fixes #893: Gemini-through-OR parameters are synchronized with Gemini models

Fixes #940: OpenAI-through-OR reasoning effort is synced with OpenAI models and much improved. We will have to still fix #944 for OpenAI levels to be fully sync'd with upstream (in progress)
2026-02-06 18:27:38 -08:00
Enrico Ros 58457cac50 LLMs: OR/Anthropic: support effort and adaptive.
Fixes #962
2026-02-06 18:27:38 -08:00
Enrico Ros 0fbacee7dc LLMs: Anthropic: editable Max effort. #962 2026-02-06 18:27:38 -08:00
Enrico Ros a498f28d14 LLMs: Anthropic: support for max effort. #962 2026-02-06 18:26:07 -08:00
Enrico Ros 5b9c6a2d0e LLMs: Anthropic: support adaptive thinking correctly. #962 2026-02-06 18:26:07 -08:00
Enrico Ros 4c7f50ab98 LLMs: Anthropic: inline thinking budget 2026-02-06 18:26:07 -08:00
Enrico Ros ef03d33bbf LLMs: Anthropic: GA skills 2026-02-06 18:26:07 -08:00
Enrico Ros 22c9fc56c0 LLMs: Opus 4.6: naming 2026-02-06 18:26:07 -08:00
Enrico Ros c952fd734f LLMs: Opus 4.6: remove forcing 2026-02-06 18:26:07 -08:00
Enrico Ros 310e99af23 LLMs: Opus 4.6: sort order, unhide 4.5 2026-02-06 18:26:07 -08:00
Enrico Ros e78446904a Docker: remove broken command directive. Fixes #964 2026-02-06 18:25:24 -08:00
Enrico Ros 760e9d8279 CC: Anthropic: update sources of info 2026-02-06 18:25:24 -08:00
Enrico Ros 61a60c5b9f Markdown: bundle in main chunk instead of lazy-loading 2026-02-06 12:41:41 -08:00
Enrico Ros 3054e1b88d Node 24: add .nvmrc, drop 26 from engines 2026-02-06 12:41:41 -08:00
Enrico Ros 6f4fabf147 Claude Opus 4.6 baseline support 2026-02-05 12:02:21 -08:00
Enrico Ros b0c791a055 Sweep: bits 2026-02-05 03:35:40 -08:00
Enrico Ros 748991249a LLMs: OpenAI: Update tooling availabiltiy across models 2026-02-05 02:36:28 -08:00
Enrico Ros 1aea7122cc Sweep: improve detection of connection issues 2026-02-05 02:35:47 -08:00
Enrico Ros 9a83b428f1 AppBreadcrumbs: auto-ellipsize 2026-02-05 01:21:46 -08:00
Enrico Ros 2cd38bc02b Sweep: update baseline with improved OpenAI chatCompletion values. remove verbosity when the only value is medium (aka, no parameter) 2026-02-05 00:44:48 -08:00
Enrico Ros e586142190 AIX: OpenAI-compatible: ChatCompletions: support verbosity for all (not just openrouter) 2026-02-05 00:07:36 -08:00
Enrico Ros a10d0dcf5d LLMs: auto-inject image output 2026-02-05 00:07:36 -08:00
Enrico Ros 6fdff488a9 Sweep: neutered values 2026-02-05 00:07:36 -08:00
Enrico Ros 8af0d78127 Sweep: adapt to the interfaces like aix.client.ts 2026-02-04 23:07:21 -08:00
Enrico Ros 177686a7fc Sweep: add option to merge models instead of wiping the file 2026-02-04 23:01:40 -08:00
Enrico Ros 09b6e47036 Sweep: fix Responses interface application 2026-02-04 21:14:27 -08:00
Enrico Ros 704187ba3e Models Modal: change visibility 2026-02-04 20:49:39 -08:00
Enrico Ros 4ea8a06503 LLMs: auto-inject web search 2026-02-04 20:49:39 -08:00
Enrico Ros 80fcc7d3e3 Security: client-dominated credential isolation for OpenAI access 2026-02-04 20:09:16 -08:00
Enrico Ros a04c62da6f LLMs: OpenAI: fix verbosity (automated). Fixes #947 2026-02-04 19:57:50 -08:00
Enrico Ros fcb518a050 Security: prevent key exfil 2026-02-04 19:43:09 -08:00
Enrico Ros a222626933 CC: sweep: small note 2026-02-04 19:31:41 -08:00
Enrico Ros a3ceade738 Security: anti-dns-spoofing anthropic 2026-02-04 19:26:57 -08:00
Enrico Ros 51d58223b4 Sweep: more succinct output 2026-02-04 19:12:50 -08:00
Enrico Ros d37a603db2 LLMs: OpenAI: Auto 0-day Responses suport. Fixes e458bca1a. #937 2026-02-04 19:04:13 -08:00
Enrico Ros ea984f3ddf Security: anti-dns-spoofing matching 2026-02-04 18:49:31 -08:00
Enrico Ros a9d3e3dead CC: llms: verify-parameters 2026-02-04 18:49:31 -08:00
Enrico Ros 5499e57205 Tools: sweep: json: fold some sweeps into a 'tools' array 2026-02-04 17:45:50 -08:00
Enrico Ros 6f8ee0247f Tools: sweep: baselines 2026-02-04 17:33:23 -08:00
Enrico Ros 05ee5cc3d1 Tools: sweep: merge id-based parameters 2026-02-04 17:12:36 -08:00
Enrico Ros cb6b569330 Tools: sweep: remove unnecessary configs 2026-02-04 17:05:30 -08:00
Enrico Ros 53073ff109 Tools: sweep: remove opanti summary 2026-02-04 17:05:16 -08:00
Enrico Ros 26d362d7a6 Tools: sweep: partition per-dialect 2026-02-04 16:40:35 -08:00
Enrico Ros 91d99e1a63 Tools: sweep: improvements for Gemini and Anthropic, and to save/load of results 2026-02-04 16:17:19 -08:00
Enrico Ros a20917c971 Tools: sweep: incremental output save 2026-02-04 15:23:00 -08:00
Enrico Ros af9bf9e5b3 Tools: sweep: parallel support 2026-02-04 15:13:39 -08:00
Enrico Ros 46b473b8a0 Tools: sweep: Gemini sweeps. #953 2026-02-04 15:03:31 -08:00
Enrico Ros e2b4028223 Tools: sweep: only select from the predefined sweeps inside the config file, #944, #947, #953 2026-02-04 14:52:09 -08:00
Enrico Ros bac2a31782 Tools: sweep: add opeanai image generation and search tool presence, #944, #947, #953 2026-02-04 14:51:57 -08:00
Enrico Ros 3d20e6bf91 Tools: llm parameter sweep. #944, #947, #953 2026-02-04 14:12:44 -08:00
Enrico Ros 9337216092 tRPC fetchers: console logging on connect/response/parsing can be disabled via env 2026-02-04 14:12:44 -08:00
Enrico Ros cd35d0ca55 Add TSX as a dev dependency 2026-02-04 10:54:44 -08:00
Enrico Ros 6d591b98b8 Roll packages (deep) 2026-02-04 10:53:53 -08:00
Enrico Ros 486381ab9d Sprites: run the gen node native, as module 2026-02-04 10:34:14 -08:00
Enrico Ros c619b4debb ListItemGroupCollapser: sm everywhere 2026-02-04 01:35:55 -08:00
Enrico Ros 383a3085ec Chat Dropdown: adapt Optima Dropdown. #955 2026-02-04 01:03:18 -08:00
Enrico Ros 5a3bb3d817 Chat Dropdown: adapt llmSelect. #955 2026-02-04 01:03:02 -08:00
Enrico Ros d1ba758887 Chat Dropdown: reuse toggleable set and Collapser. #955 2026-02-04 00:55:39 -08:00
Enrico Ros 6fef149997 Sprites: port models-modal 2026-02-03 23:38:50 -08:00
Enrico Ros aad3b16ff2 Sprites: port useLLMSelect, Beam 2026-02-03 23:38:50 -08:00
Enrico Ros 819ba14523 Sprites: Generate and wire 2026-02-03 23:38:50 -08:00
Enrico Ros d3c25ca16a Sprites: update generator with class 2026-02-03 23:38:27 -08:00
Enrico Ros 99a65f72ac Sprites: generator update 2026-02-03 22:35:55 -08:00
Enrico Ros be9080d392 Sprites: generator 2026-02-03 22:35:55 -08:00
Enrico Ros f32d991413 Chat Dropdown: reusable parts. #955 2026-02-03 22:34:12 -08:00
Enrico Ros 94b68ebefa CloseablePopup: memo. #955 2026-02-03 22:33:35 -08:00
Enrico Ros 0450eaaceb CC: rel:release-open 2026-02-03 09:20:11 -08:00
Enrico Ros 408c5ce088 Readme: update counter 2026-02-02 17:13:13 -08:00
Enrico Ros d936629ead 2.0.3: update readme 2026-02-02 15:48:02 -08:00
Enrico Ros 9bd1a66208 2.0.3: update news 2026-02-02 15:43:30 -08:00
Enrico Ros 1a0c029ee8 2.0.3: update package 2026-02-02 15:26:53 -08:00
Enrico Ros e7be228703 Roll AIX 2026-02-02 15:16:57 -08:00
Enrico Ros 0ab4dc972f Dockerfile: suppress CopyIgnoredFile warning for whitelist-style .dockerignore 2026-02-02 15:16:15 -08:00
Enrico Ros 5f1ca8954f Force touch to doubleClick support (disabled) 2026-02-02 15:09:11 -08:00
Enrico Ros 3ec1b033ce BlockEdit_TextFragment: support 'xs' editing. #961 2026-02-02 14:47:56 -08:00
Enrico Ros 0caf27af9b LLMs: skip prod warning for connection errors, they're still shown in the router warnings in dev 2026-02-02 12:55:38 -08:00
Enrico Ros bd67e14fa4 Debug: Wire: off 2026-02-02 12:37:13 -08:00
Enrico Ros 494c3b542c AIX/LLMs: LMStudio: generic conversion 2026-02-02 12:26:36 -08:00
Enrico Ros 8e0884eb64 AIX/LLMs: LMStudio: convert WebP -> JPG instead 2026-02-02 12:21:54 -08:00
Enrico Ros 73c4dc4ac8 AIX/LLMs: support for WebP -> PNG conversion at the hotfix (pre-CGR) stage 2026-02-02 12:18:31 -08:00
Enrico Ros d77274058d LLMs: LMStudio: use native API for detailed model information 2026-02-02 12:07:45 -08:00
Enrico Ros 0c8460419b AIX: ImageContentPart: allow not detail 2026-02-02 11:34:10 -08:00
Enrico Ros eabb589390 AIX: relax error parsing to .error: { message: '..' } 2026-02-02 11:29:52 -08:00
Enrico Ros 62f860ae93 Debug: Wire: clip curl to 4096 inner 2026-02-02 11:26:57 -08:00
Enrico Ros 605aae873c Roll packages with net removal 2026-01-31 16:40:23 -08:00
Enrico Ros 62e9ee5b05 Roll react hook form which shall be even lighter 2026-01-31 16:32:02 -08:00
Enrico Ros d686f5d143 Roll verified changes 2026-01-31 16:26:44 -08:00
Enrico Ros 3922f232ae Roll some types and prettier 2026-01-31 16:22:31 -08:00
Enrico Ros 6735b438d3 Roll Next to fix CVE 2026-01-31 16:15:57 -08:00
Enrico Ros fb1e30ab32 Roll PostHog-node 2026-01-31 16:11:32 -08:00
Enrico Ros 0ec06edb57 Roll PostHog-js 2026-01-31 16:10:21 -08:00
Enrico Ros 2a52673c56 Merge pull request #959 from enricoros/dependabot/github_actions/docker/login-action-3.7.0
chore(deps): bump docker/login-action from 3.6.0 to 3.7.0
2026-01-31 16:09:49 -08:00
Enrico Ros cc20d00d8a Drive picker: improve token handling with expiration 2026-01-31 15:52:24 -08:00
Enrico Ros 3d9201f7dc Drive picker: add a button to close and reset 2026-01-31 15:12:26 -08:00
dependabot[bot] 176732a6c0 chore(deps): bump docker/login-action from 3.6.0 to 3.7.0
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/5e57cd118135c172c3672efd75eb46360885c0ef...c94ce9fb468520275223c153574b00df6fe4bcc9)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-31 11:32:23 +00:00
Enrico Ros 39815b3af3 CustomMarkdownRenderer: reduce log 2026-01-31 03:13:46 -08:00
Enrico Ros bcce517089 Drive: token provider interface 2026-01-31 03:01:36 -08:00
Enrico Ros a4b50d0d97 OpenAI: Raise instancelimit to 10 2026-01-31 03:01:36 -08:00
Enrico Ros 2a124e7588 Drive: text bits 2026-01-30 20:04:24 -08:00
Enrico Ros a85556ab5b Attach content (docs, images, pdf, etc.) from Google Drive. Fixes #943 2026-01-30 19:18:58 -08:00
Enrico Ros cef93d6084 Attachments: cloud types 2026-01-30 17:00:57 -08:00
Enrico Ros 207e257778 AIX: Demuxers: add recovery of incomplete buffers and server-side logging 2026-01-30 14:23:29 -08:00
Enrico Ros 12203daa22 LLMs: New Heuristic model enumeration for the OpenAI protocol 2026-01-30 12:59:14 -08:00
Enrico Ros 27f8e9248d ModelsList: hidden LLMs get faded chips 2026-01-30 12:42:37 -08:00
Enrico Ros 51384dc984 AIX: remove unused OAI Realtime IF 2026-01-30 12:36:44 -08:00
Enrico Ros bc76cbb5ad Server-side fetchers: opt-in optional headers map 2026-01-30 11:55:18 -08:00
Enrico Ros 5a1ca83f6d Server-side fetchers: debug print headers 2026-01-30 11:50:27 -08:00
Enrico Ros c9f585f808 Server/Client Wire improvement, including listModels 2026-01-30 11:39:35 -08:00
Enrico Ros 9f559e1dbf LLMs: Groq: re-add SDAIA 2026-01-30 11:10:52 -08:00
Enrico Ros e458bca1a7 Revert "LLMs: OpenAI Responses API by default for new models. Fixes #937"
This caused all unknown models to default to the responses API.
We need heuristics for determining OpenAI vs OpenAI-compatible.

This reverts commit a6862d8c58.
2026-01-30 03:09:16 -08:00
Enrico Ros 43d2226019 AIX: Fix LiteLLM breakage 2026-01-30 02:01:21 -08:00
Enrico Ros 122bc34701 AIX: don't retry insufficient balance errors 2026-01-29 23:46:19 -08:00
Enrico Ros e01358e268 LRS: add back mmmu 2026-01-29 23:00:59 -08:00
Enrico Ros 847c84c3e6 LLMs: remove cbaMmmu 2026-01-29 22:39:47 -08:00
Enrico Ros b11cac4328 LLMs: sync CB 2026-01-29 22:31:09 -08:00
Enrico Ros f617b06109 LLMs: copy over initialtemperature if/when available on a per-model basis for 'fromManualMapping' 2026-01-29 22:31:08 -08:00
Enrico Ros 345ccf3369 DLLM/ModelDescription: remove trainigDataCutoff: not used anymore in this day and age 2026-01-29 21:43:03 -08:00
Enrico Ros d111b8af62 LLMs: Update XAI verified 2026-01-29 21:23:43 -08:00
Enrico Ros 8f964c5c49 LLMs: Update OpenPipe 2026-01-29 21:22:34 -08:00
Enrico Ros b6f3f4538f LLMs: Update OpenAI Instant models 2026-01-29 21:22:02 -08:00
Enrico Ros f6dd30d5d8 LLMs: Update Mistral 2026-01-29 21:21:12 -08:00
Enrico Ros af8b79f849 LLMs: Update Groq 2026-01-29 21:20:24 -08:00
Enrico Ros 0cfccc423b LLMs: Update Gemini (deprecations) 2026-01-29 21:17:40 -08:00
Enrico Ros f9a5d582d4 LLMs: Update Ollama 2026-01-29 21:14:25 -08:00
Enrico Ros 684e00d594 Model Services: bits2 2026-01-29 21:08:20 -08:00
Enrico Ros 3cd2df0b50 Model Services: Duplicates removal 2026-01-29 21:01:44 -08:00
Enrico Ros 02197f4ee6 OpenAI: fix sequence number validation - broken by LiteLLM. Fixes #957 2026-01-29 13:49:46 -08:00
Enrico Ros f9049a3fea ModelsWizard: fix build 2026-01-29 13:36:05 -08:00
Enrico Ros 462bddc271 Model Service: Service menu 2026-01-29 13:16:25 -08:00
Enrico Ros f79000cf39 Model Service: 3 dots button 2026-01-29 13:16:25 -08:00
Enrico Ros 1d95273f4d Models Refetch: update icon 2026-01-29 13:16:25 -08:00
Enrico Ros 6c4579f434 LLM Duplication copy 2026-01-29 13:16:25 -08:00
Enrico Ros 4ef56ade21 LLMs: OpenAI: default 5.2 to medium, since we have the no-thinking model too 2026-01-29 13:05:16 -08:00
Enrico Ros 7c1369d6e9 LLM Store: allow for removal/reset service-wide 2026-01-29 13:05:16 -08:00
Enrico Ros 533d54b106 LLM Options: reflect status elsewhere 2026-01-29 12:36:33 -08:00
Enrico Ros cce0ca6560 LLM Options: Clone advanced. #941 2026-01-29 12:36:01 -08:00
Enrico Ros e87ce2593c LLM Options: bits 2026-01-29 12:32:32 -08:00
Enrico Ros 431dc8b667 LLM Temperature: allow for Vendor-set no temperature (future) 2026-01-29 12:32:32 -08:00
Enrico Ros 5caf614bf7 LLM Clones: correctly group services when updating, inserting custom models 2026-01-29 12:32:32 -08:00
Enrico Ros ecf9703570 LLM Clones: User can create clones. #941 2026-01-29 12:32:32 -08:00
Enrico Ros e7641393a0 LLMParametersEditor: fix controlled component 2026-01-29 12:32:31 -08:00
Enrico Ros 2201f6ff5a ModelsList: memo feature chips 2026-01-29 12:31:59 -08:00
Enrico Ros 557e1ce293 Clipboard Content Transform: fix copying of code 2026-01-29 11:10:56 -08:00
Enrico Ros cbe9a6b9a5 LLMOptions: style a bit 2026-01-29 00:36:16 -08:00
Enrico Ros 9bbcb038d4 LLMOptions: launch purpose (not on) 2026-01-29 00:28:28 -08:00
Enrico Ros 3602204420 LLMOptions: add commands 2026-01-28 23:52:10 -08:00
Enrico Ros 6f485e5589 GoodModal: shrink on xs 2026-01-28 23:49:29 -08:00
Enrico Ros 2f46a3dfaf LLM Options: Details/Override 2026-01-28 17:45:39 -08:00
Enrico Ros 267845bba3 LLM Options: LLM actions 2026-01-28 17:21:52 -08:00
Enrico Ros 6f33a8eebf LLMs: improve variants handling across the board. #941 2026-01-28 17:03:59 -08:00
Enrico Ros b0d2b09a2e AIX: Fix show injector 2026-01-28 17:03:58 -08:00
Enrico Ros c699b6b16b OpenAI: add a no-thinking variant for GPT-5.2, which allows to change temperature. #941 2026-01-28 17:03:58 -08:00
Enrico Ros 1789bac28d Roll AIX 2026-01-28 03:21:30 -08:00
Enrico Ros 60c05f615f AIX Debugger: unfiler 2026-01-28 03:02:12 -08:00
Enrico Ros bd84523671 LLMs: Gemini: removed models 2026-01-28 02:48:58 -08:00
Enrico Ros eb21b9c770 Speex: open config if closed 2026-01-28 02:42:24 -08:00
Enrico Ros ff3ac11afb LLMParametersEditor: survive undefined temperature 2026-01-28 02:23:39 -08:00
Enrico Ros 1ef8c3d02b LLMs: Gemini: temperature defaults to 1.0 and can be changed 2026-01-28 02:01:55 -08:00
Enrico Ros 2ebaf6279b AIX: Injector: works well 2026-01-28 01:42:50 -08:00
Enrico Ros a5ee40e184 Speex: fix build 2026-01-28 01:15:56 -08:00
Enrico Ros b17a97eac7 AIX: request body injection. #953 2026-01-28 01:09:32 -08:00
Enrico Ros 63908bfaf6 Speex: update README 2026-01-27 23:53:27 -08:00
Enrico Ros 3f9a419a19 Speex: +Inworld Config 2026-01-27 23:50:33 -08:00
Enrico Ros bae691e33e Speex: +Inworld 2026-01-27 23:50:33 -08:00
Enrico Ros 91539346ee Speex: Fix #624 2026-01-27 22:08:39 -08:00
Enrico Ros 4842ca81b3 Speex: Cancellable preview 2026-01-27 22:08:38 -08:00
Enrico Ros 9c77a1a4ab Speex: Chunk test: remove button 2026-01-27 22:08:36 -08:00
Enrico Ros 4af284be42 Speex: prevent voice/engine mismatch 2026-01-27 22:08:15 -08:00
Enrico Ros 6aec68bb3c Speex: fix unlimited chunking 2026-01-27 22:08:15 -08:00
Enrico Ros d4e2b0834f Speex: allow inner calls, for bytes access 2026-01-27 20:23:10 -08:00
Enrico Ros 24c2702f96 Speex: ghost key to 'tts' for remembering the pass 2026-01-27 20:18:05 -08:00
Enrico Ros 4691fc9bad Speex: pre-wrap errors 2026-01-27 20:17:58 -08:00
Enrico Ros 8c6c60b6f1 Speex: fix webspeech voice selection 2026-01-27 20:17:02 -08:00
Enrico Ros bc482407fe Speex: overhaul for chunking and future synchroniciy / controllability 2026-01-27 20:11:49 -08:00
Enrico Ros ff05593db8 AudioAutoPlayer: handler for any streaming or full-file play, with stop and await ending 2026-01-27 20:11:49 -08:00
Enrico Ros 3d304d9374 AudioLivePlayer: extra safety 2026-01-27 20:11:49 -08:00
Enrico Ros 1734f0c2f1 AudioLivePlayer: extra safety, if stop was called already, waitForPlaybackEnd would return immediately 2026-01-27 19:38:48 -08:00
Enrico Ros 1b25e5df85 AudioLivePlayer: anti-leak the objectUrl and resolve wait on stop 2026-01-27 19:38:48 -08:00
Enrico Ros ea8eb32b0b AudioLivePlayer: await until done 2026-01-27 19:38:48 -08:00
Enrico Ros 614a1f95de AudioPlayer: improve straight play function 2026-01-27 19:38:48 -08:00
Enrico Ros d36bc28914 blobUtils: combine arrayBuffers 2026-01-27 19:38:48 -08:00
Enrico Ros deec48d7c1 CC: enable gh issues list 2026-01-27 19:38:48 -08:00
Enrico Ros b318ec8d39 Merge pull request #951 from jayrepo/patch-3
Add middleware.ts to docker image
2026-01-26 23:11:08 -08:00
Jay Chen b4b0e2befc Update .dockerignore 2026-01-27 14:22:22 +08:00
Enrico Ros 51d3fe13da Roll AIX 2026-01-26 19:51:41 -08:00
Enrico Ros 58220216d3 LLMs/AIX: support for Kimi Thinking On/Off 2026-01-26 19:43:06 -08:00
Enrico Ros cac75cca42 LLMs: Detail Kimi-K2.5 2026-01-26 19:24:09 -08:00
Enrico Ros 47f247907f LLMs: Add Kimi-K2.5 2026-01-26 18:57:50 -08:00
Enrico Ros 81e04b7322 ChatDrawer: buckets: fewer splits, more stability 2026-01-25 19:39:05 -08:00
Enrico Ros 56a964b700 AIX debugger: highlight non-conversation frames 2026-01-25 19:37:44 -08:00
Enrico Ros 458341d79f AIX debugger: don't auto-advance frame for support operations 2026-01-25 19:37:43 -08:00
Enrico Ros d1d212b075 Copy: intercept to others too 2026-01-24 20:24:52 -08:00
Enrico Ros 59c9996489 LLM Params update: TS improvements 2026-01-24 19:33:25 -08:00
Enrico Ros bf8221a2f1 LLM Params update: MDS with int (not float) 2026-01-24 19:33:25 -08:00
Enrico Ros 787a11a040 LLM Params system: Improve types definition 2026-01-24 19:33:25 -08:00
Enrico Ros 05d114be2f Copy: redo visual copy and copy interception (Ctrl+c, etc) for Plain text and HTML 2026-01-24 19:33:25 -08:00
Enrico Ros 3c04a7dbac Copy: also disable copy of collapsed Expanders 2026-01-24 19:33:25 -08:00
Enrico Ros 1673e1148d Copy: annotate what to Not copy 2026-01-24 19:33:25 -08:00
Enrico Ros de416b035d Copy: remove overlay copy button on the message 2026-01-24 19:33:25 -08:00
Enrico Ros 08aaf2989d Beam: always show model selector for Custom fusion 2026-01-24 15:49:54 -08:00
Enrico Ros a50964060c Stop ResizeObserver issue notifications 2026-01-24 15:41:41 -08:00
Enrico Ros 54b6108719 Beam: hide Merge Model selector for Custom fusion 2026-01-24 15:32:58 -08:00
Enrico Ros 585e5c254a Roll AIX 2026-01-23 11:42:32 -08:00
Enrico Ros 477808c9bb AIX: OAI Responses: allow 'failed' on web_search_call 2026-01-23 11:42:22 -08:00
Enrico Ros 6c58a2b688 AIX: OAI Responses: allow 'failed' on web_search_call 2026-01-23 11:42:10 -08:00
Enrico Ros c9854bf30f LLMs: OpenAI: partial Reasoning Effort updates. #944 2026-01-23 11:28:03 -08:00
Enrico Ros cfed4bbd41 LLMs: OpenAI: remove restore markdown on GPT-5 models 2026-01-23 11:20:53 -08:00
Enrico Ros 2dd6485b0e LLMParametersEditor: align to XAI X search default off 2026-01-23 11:19:32 -08:00
Enrico Ros bf1dd5b860 LLMs: Toggle code execution in Params Editor only (not quick) 2026-01-23 10:52:32 -08:00
Enrico Ros 765c373f7d LLMs: OAI: allow code execution 2026-01-23 10:52:32 -08:00
Enrico Ros 32d752e82b LLMs: OAI Responses: (unused) parameter for OpenAI code execution 2026-01-23 10:35:58 -08:00
Enrico Ros 4623e438fa AIX: OAI Responses: code interpreter Hosted tool def 2026-01-23 10:35:08 -08:00
Enrico Ros 8a44ff396f AIX: XAI: relax annotation title presence 2026-01-23 10:16:59 -08:00
Enrico Ros 086d7ecae4 Speex: TTS character limit settings. Fixes #942 2026-01-23 10:05:35 -08:00
Enrico Ros d6adebb711 Attachment buttons: full name in tooltip. Fixes #946 2026-01-23 09:55:29 -08:00
Enrico Ros 8325fe7b3c Roll AIX 2026-01-23 09:29:41 -08:00
Enrico Ros 7cf83f878b AIX: XAI: Response API Request + wiretypes. Fixes #938 2026-01-23 04:29:13 -08:00
Enrico Ros 597ba26424 AIX: Code Executor wires 2026-01-23 04:29:06 -08:00
Enrico Ros 7bccea47f5 AIX: OpenAI Responses: parse Code Execution and Custom Tools 2026-01-23 04:29:06 -08:00
Enrico Ros 5770116779 DMessage: Code Executor 2026-01-23 04:29:06 -08:00
Enrico Ros 0679144f69 LLMs/AIX: XAI new parametrization 2026-01-23 04:29:05 -08:00
Enrico Ros c9fd288b52 AIX: OpenAI chatCompletions: remove obsolete X search params 2026-01-23 04:16:52 -08:00
Enrico Ros 9ae449fcfd LLMs: type check server params against the client params def 2026-01-23 00:46:07 -08:00
Enrico Ros 249f67f796 AIX: improve dispatch messaging validation for 4 protocols 2026-01-23 00:46:07 -08:00
Enrico Ros e91c0bb554 AIX: stripUndefined 2026-01-23 00:28:05 -08:00
Enrico Ros 5e306d9598 AIX: XAI: models update 2026-01-23 00:25:33 -08:00
Enrico Ros 42ebc81cbb AIX: XAI: models update 2026-01-22 23:56:11 -08:00
Enrico Ros f624c37db5 AIX: XAI: models update 2026-01-22 17:51:03 -08:00
Enrico Ros 22b6f42936 AIX: OAI: Responses wiretypes changes 2026-01-22 17:25:54 -08:00
Enrico Ros 760c66cac8 Attachments: reposition menu to see the tokens bar 2026-01-22 17:18:08 -08:00
Enrico Ros 1d91e9da03 Attachments: client-side Markdown conversion and Text/HTML cleanup & Markdown conversion 2026-01-22 17:17:57 -08:00
Enrico Ros 7eac409ec6 AIX: XAI: model removal 2026-01-22 13:54:36 -08:00
Enrico Ros 128558420c AIX: XAI: model validation 2026-01-22 13:54:17 -08:00
Enrico Ros ca3e664690 AIX: spill part type cleanup 2026-01-22 13:12:29 -08:00
Enrico Ros 7eb37462d7 LLMs: update ollama models 2026-01-22 03:06:17 -08:00
Enrico Ros 31e02c2d39 CC: slashcommands: update llms:ollama 2026-01-22 03:02:55 -08:00
Enrico Ros 003a68b9b8 CustomMarkdownRenderer.tsx: allow for <br/> tags inside Table Cells. Fixes #939 2026-01-22 02:14:42 -08:00
Enrico Ros f418708389 AIX: Inspector: improve render 2026-01-22 01:55:14 -08:00
Enrico Ros d23a564035 AIX: Inspector: reassembler 'transport' data and ui 2026-01-22 01:45:20 -08:00
Enrico Ros 7fe586244c AIX: Inspector: debugger object client sync 2026-01-22 01:38:33 -08:00
Enrico Ros f1a597cdc6 AIX: move Inspector frames selector 2026-01-22 01:36:54 -08:00
Enrico Ros 9b68c8f58c CC: slashcommands: update llms:ollama 2026-01-22 01:06:11 -08:00
Enrico Ros be5b57ea71 LLMs: shared model definition validators 2026-01-22 00:54:00 -08:00
Enrico Ros 425c82f26d CC: slashcommands: improve parsing of ollama models 2026-01-22 00:08:07 -08:00
Enrico Ros 942421c1fb LLMs: together: fix key validation 2026-01-21 23:52:15 -08:00
Enrico Ros b1184f6928 AIX: CSF: also support client-side exceptions 2026-01-21 23:39:10 -08:00
Enrico Ros ffeb6d1b98 LLMs: models bits 2026-01-21 23:34:31 -08:00
Enrico Ros b2718b56b7 CC: shashcommands: improve kimi 2026-01-21 23:16:57 -08:00
Enrico Ros 455f834957 CC: allow llms:update 2026-01-21 23:05:56 -08:00
Enrico Ros 8a14c80ff8 CC: slashcommands: llms:groq update 2026-01-21 23:04:09 -08:00
Enrico Ros e268e733c7 LLMs: Groq: overlap check 2026-01-21 23:03:29 -08:00
Enrico Ros 8933a8dfb3 LLMs: Gemini: deprecations 2026-01-21 22:53:15 -08:00
Enrico Ros 9796cc525c LLMs: xAI verified 2026-01-21 22:37:11 -08:00
Enrico Ros cdbf9a9190 Speex: CSF support - auto-detected from linked services - client-unbundled-dynamic 2026-01-21 18:38:43 -08:00
Enrico Ros c26792292d Speex: extract rpc common core 2026-01-21 18:14:03 -08:00
Enrico Ros 4698e0ee03 LLMs: OpenAI: remove /v1/ from the user input, to immedialy give feedback on the correct way of doing it 2026-01-21 16:58:57 -08:00
Enrico Ros 68afcb2f4b AIX: OpenAI: disable reasoning summaries when disabling Streaming as well. Fixes #932 2026-01-21 16:52:36 -08:00
Enrico Ros e8f61e46e3 AIX: Fetchers: don't retry on 'request too large' 2026-01-21 16:22:50 -08:00
Enrico Ros 317bb2b7c8 useLLMSelect: preserve scrolling on cat toggle 2026-01-21 16:00:43 -08:00
Enrico Ros d1b3c6b468 ContentFragments: improve zero state add text look. #934 2026-01-21 16:00:42 -08:00
Enrico Ros b35eccc984 AIX: increase resilience to new values across 3 parsers. Fixes #918
Note: relaxed throws on Anthropic and Gemini (all throws), while on OAI-Responses we are raising to throws in dev.
2026-01-21 15:08:17 -08:00
Enrico Ros a780c92047 AIX: speculative support for keepalives on chatCompletions API. Fixes #918 2026-01-21 14:36:40 -08:00
Enrico Ros 5fc65698ba Collapsible Model Groups, #936 2026-01-21 14:10:30 -08:00
Enrico Ros c923b5ec4c Restyle llm select model groups #936 2026-01-21 13:36:43 -08:00
Enrico Ros 609b2b9a7b Group models correctly by service in Beam drop down list. Fixes #936 2026-01-21 13:36:31 -08:00
Enrico Ros a257278004 dMessageUtils: OSB 2026-01-21 13:15:26 -08:00
Enrico Ros 273daed634 Chat Messages: label heuristic to ignore the vendor-N- 2026-01-21 13:13:49 -08:00
Enrico Ros a6862d8c58 LLMs: OpenAI Responses API by default for new models. Fixes #937 2026-01-21 13:07:58 -08:00
Enrico Ros 323e5b4ea7 LLMs: OpenAI OSB speculative support 2026-01-21 13:06:47 -08:00
Enrico Ros 89217a5308 CC: allow gh issue view 2026-01-21 13:03:05 -08:00
Enrico Ros a45e995d2f AIX: OpenRouter: improve reasoning through OpenAI-completions across models. Fixes #893 2026-01-20 11:32:11 -08:00
Enrico Ros 8700b4c8ca Roll AIX 2026-01-20 02:07:55 -08:00
Enrico Ros 1f7f5fb488 Data (personas): more concise default 2026-01-20 01:43:45 -08:00
Enrico Ros afde8ee864 LLMs: OpenRouter: 'verbosity' support for OpenAI gpt-5 models, and Anthropic Claude Opus 4.5 (remapped to reasoning effort by OpenRouter). Fixes #927 2026-01-20 01:42:54 -08:00
Enrico Ros 3884c26b15 LLMs: OpenAI: Reviewed 5.1 Instant / 5.2 Params support. Closes #930 2026-01-20 01:21:45 -08:00
Enrico Ros 24dce7eae9 LLMs: further improve the LLMParametersEditor's sync with the dialog. #926 2026-01-20 01:09:04 -08:00
Enrico Ros 1db4e9b771 LLMs: Anthropic: Fix Effort in the LLM Options Dialog. Fixes #926 2026-01-20 00:31:39 -08:00
Enrico Ros b2ed7eae00 CC: CMDs: llms:align-params-uis 2026-01-20 00:31:39 -08:00
Enrico Ros 3169fd67e8 LLMs: OpenRouter: fix Antrhopic thinking models. Fixes #925 2026-01-20 00:15:00 -08:00
Enrico Ros 773ceb1396 Tools: data/llm: remove obsolete registry 2026-01-19 23:40:56 -08:00
Enrico Ros 8c62ee1720 LLMs/AIX: fully remove moderation
Doesn't seem to be used anymore by anyone, nor was active in the code
2026-01-19 23:32:15 -08:00
Enrico Ros 5fa1f52922 LLMs: openai service setup: bits 2026-01-19 23:09:20 -08:00
Enrico Ros d2180c010c LLMs: Helicone: start unformize 2026-01-19 22:42:16 -08:00
Enrico Ros b73df7b2ce LLMs: OpenAI: Autocomplete + suggest hosts for Chutes, Fireworks, Novita. #921 2026-01-19 22:42:16 -08:00
Enrico Ros 971f737846 LLMs: support Novita.ai models with capability auto-detection. #921 2026-01-19 22:39:32 -08:00
Enrico Ros a393353907 LLMs: Azure: rename custom field 2026-01-19 22:39:32 -08:00
Enrico Ros 751f609554 LLMs: OpenAI: unify paths 2026-01-19 22:39:31 -08:00
Enrico Ros e8cd5c6552 LLMs: Ant: unify paths 2026-01-19 22:39:31 -08:00
Enrico Ros 86e387b270 LLMs: allow OpenAI/Azure OpenAI services renaming. Fixes #922 2026-01-19 15:54:13 -08:00
Enrico Ros 32f15aa621 FormTextField: allow for end decorator 2026-01-19 15:54:13 -08:00
Enrico Ros bfc889a9e5 LLMs: fix reset of non-declared params at models update. Fixes #924 2026-01-19 14:53:46 -08:00
Enrico Ros bd907625a8 UpDate 2026-01-15 17:48:00 -08:00
Enrico Ros 60004926d7 Recommend DC (CSF) for Local services 2026-01-15 16:28:00 -08:00
Enrico Ros ac751dfd1a Roll AIX 2026-01-14 16:42:06 -08:00
Enrico Ros 6828eee17f LLMs: Perplexity: sync 2026-01-14 16:40:01 -08:00
Enrico Ros 19c97f397b LLMs: OpenAI: sync a GPT Audio model 2026-01-14 16:38:32 -08:00
Enrico Ros 0167a8bdd8 LLMs: Mistral: update 2026-01-14 16:36:31 -08:00
Enrico Ros 93e5044603 LLMs: Groq: strings 2026-01-14 16:36:19 -08:00
Enrico Ros 024d930677 LLMs: Gemini: small update 2026-01-14 16:35:22 -08:00
Enrico Ros 98873446a8 LLMs: Ollama: update tags 2026-01-14 16:35:15 -08:00
Enrico Ros 5318b7a406 OCR: cache tesseract's import 2026-01-14 16:30:57 -08:00
Enrico Ros 4a6c3cbcd2 Roll AIX 2026-01-14 15:41:58 -08:00
Enrico Ros ac0a39c202 LLMs: OpenAI: GPT-5.2 Codex support 2026-01-14 15:40:40 -08:00
Enrico Ros 88d39345a5 Attachments: PDF: add a PDF->OCR (via interim images) and an 'Auto' (default)
The Auto mode tries plain Text, then Text to Images to OCR, then falls back to pure images.
2026-01-14 15:09:40 -08:00
Enrico Ros 7aa9cb07b2 OCR: extract util functions 2026-01-14 13:18:44 -08:00
Enrico Ros ef30c8d28d Tesseract: roll package 2026-01-14 13:17:11 -08:00
Enrico Ros 2727f690b4 Attachments: PDFs: use Density to attach images 2026-01-14 10:59:43 -08:00
Enrico Ros 5945c24301 Speex: RPC: fallback to full buffer play if streaming is unavailable (Firefox) 2026-01-14 10:32:41 -08:00
Enrico Ros 7b6aff1f95 AudioLivePlayer: Firefix doesn't support MPEG streaming 2026-01-14 10:32:39 -08:00
Enrico Ros cb0fe3aadd AIX: OpenAI Responses: support for keepalive packets 2026-01-13 19:39:47 -08:00
Enrico Ros 4f9d69f9c2 AIX: Anthropic: Fix for out-of-order block start. Fixes #917 2026-01-12 16:40:16 -08:00
Enrico Ros c18aeabe06 CC: changelog cmd update 2026-01-12 04:18:02 -08:00
Enrico Ros 550742323a CC: changelog cmd 2026-01-12 04:02:17 -08:00
Enrico Ros c71f789a08 Draw PromptComposer: fix for CJK 2026-01-12 03:44:00 -08:00
Enrico Ros a9b4b195bf Extend #916 to InlineTextArea and editing of a Text Fragment 2026-01-12 03:37:02 -08:00
Enrico Ros 52e8177f42 Simplify #916 2026-01-12 03:36:33 -08:00
Enrico Ros b0743efc48 Merge branch 'fork/tantanorange/feat/bug-Input-Method-Editor' 2026-01-12 03:30:25 -08:00
Enrico Ros 6dfd652dac LLMParametersEditor: shrink label 2026-01-12 03:23:42 -08:00
Enrico Ros 3f93cb2e6d Phone: work over an empty conversation id 2026-01-12 03:11:49 -08:00
Enrico Ros 8f7b9b7f19 Optima Page Heading: support disabled 2026-01-12 02:10:38 -08:00
Enrico Ros abff89ab6b CC: ignore temp files 2026-01-12 02:10:20 -08:00
tantanorange d4f03f743a bug(issue-784): fixed 'Enter' trigger unexpected Chat under Iput Method Editor. 2026-01-11 18:32:55 -08:00
Enrico Ros c3714f6651 Camera Capture: darker 2026-01-10 15:20:30 -08:00
Enrico Ros 9b4d0ddf2f Merge pull request #913 from enricoros/dependabot/github_actions/actions/download-artifact-7.0.0
chore(deps): bump actions/download-artifact from 4.3.0 to 7.0.0
2026-01-10 14:46:32 -08:00
Enrico Ros 2c9ac2f549 Merge pull request #914 from enricoros/dependabot/github_actions/actions/upload-artifact-6.0.0
chore(deps): bump actions/upload-artifact from 4.6.2 to 6.0.0
2026-01-10 14:46:20 -08:00
dependabot[bot] c1292de2a0 chore(deps): bump actions/upload-artifact from 4.6.2 to 6.0.0
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.2 to 6.0.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/ea165f8d65b6e75b540449e92b4886f43607fa02...b7c566a772e6b6bfb58ed0dc250532a479d7789f)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-10 11:54:05 +00:00
dependabot[bot] 21d5e4cd29 chore(deps): bump actions/download-artifact from 4.3.0 to 7.0.0
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.3.0 to 7.0.0.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/d3f86a106a0bac45b974a628896c90dbdf5c8093...37930b1c2abaa49bbe596cd826c3c89aef350131)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-10 11:54:01 +00:00
Enrico Ros a9495a3e15 Roll eslint 2026-01-09 19:47:24 -08:00
Enrico Ros bff5b3d765 Roll posthog 2026-01-09 19:46:49 -08:00
Enrico Ros a4ff37eecc Roll safe packages 2026-01-09 19:40:14 -08:00
Enrico Ros 460209f486 GH Actions: fix manifest format 2026-01-09 17:36:28 -08:00
Enrico Ros 96c68c86a4 GH Actions: fix tag case 2026-01-09 17:16:22 -08:00
Enrico Ros 8b152fdff8 GH Actions: improve parallelism of the docker image build 2026-01-09 17:10:53 -08:00
Enrico Ros 25c9a52873 Dependabot: basic configuration 2026-01-09 16:40:28 -08:00
Enrico Ros 44302d903c CC Actions: checkout v6 2026-01-09 16:24:31 -08:00
Enrico Ros c7b8668609 GH Actions: docker-image: pin versions 2026-01-09 14:36:28 -08:00
Enrico Ros 7d60df6266 Docker: save 0.5GB 2026-01-09 14:27:04 -08:00
Enrico Ros b7f898a5e5 Docker: move to Node 24 and cleanup. #907 2026-01-09 13:55:02 -08:00
Enrico Ros 04c4dbe4b8 Docker: remove compose version 2026-01-09 13:53:31 -08:00
Enrico Ros 8d04c494df Docker: negate .dockerignore 2026-01-09 13:53:31 -08:00
Enrico Ros a6aadf76f3 Revert "Fix Node 25 build..." - breaks other Node versions.
This reverts commit b70d57d878.
2026-01-09 13:53:30 -08:00
Enrico Ros a685ef97bf AIX: chatGenerate executor: object-string-ellipsize when replaying input (received from the llm) 2026-01-09 13:36:06 -08:00
Enrico Ros d46c29689f AIX: OpenRouter: support image generation through OpenAI's OR-extended API, including supporting advanced Gemini params. Fixes #906 2026-01-09 13:36:06 -08:00
Enrico Ros 65ce07395b Fix drag-to-update on mobile 2026-01-09 13:36:06 -08:00
Enrico Ros cc1542fe95 Wire: improve debug print with object-ellipsize-strings 2026-01-09 13:36:06 -08:00
Enrico Ros b70d57d878 Fix Node 25 build, by using --no-webstorage with node - otherwise Zustand's persist middleware will break the build 2026-01-09 10:45:50 -08:00
Enrico Ros 5aa857362b Merge pull request #912 from enricoros/claude/issue-909-20260109-1034
feat(deepseek): add API Host field to DeepSeek settings
2026-01-09 02:42:29 -08:00
Enrico Ros c92fc34051 Merge pull request #911 from enricoros/claude/issue-902-20251229-1701
docs: remove Midori AI Subsystem section from installation guide
2026-01-09 02:39:46 -08:00
claude[bot] b01e66f12a feat(deepseek): add API Host field to DeepSeek settings
Add the ability for users to configure a custom API host for DeepSeek,
allowing them to use alternative endpoints like https://api.deepseek.com/beta.

Changes:
- Add `deepseekHost` to DDeepseekServiceSettings interface
- Wire deepseekHost to oaiHost in transport layer
- Add API Host form field visible in advanced settings

Closes #909

Co-authored-by: Enrico Ros <enricoros@users.noreply.github.com>
2026-01-09 10:38:11 +00:00
Enrico Ros a88d20784a Roll AIX 2026-01-09 02:27:54 -08:00
Enrico Ros 63486ed6cf LLMs/AIX: support 4 levels for Gemini 3 Flash. Fixes #905 2026-01-09 02:27:53 -08:00
Enrico Ros 3ceec773f2 LLMs: DeepSeek: remove 3.2 Speciale 2026-01-09 02:08:54 -08:00
Enrico Ros 817fa56ec4 LLMs: Anthropic: remove Claude 3 Opus 2026-01-09 02:05:33 -08:00
Enrico Ros 088fb21a90 LLMs: Gemini: update cache pricing 2026-01-09 01:55:53 -08:00
Enrico Ros 79c755a469 Wire: ellipsize AixDemuxers.DemuxedEvents 2026-01-09 01:31:39 -08:00
Enrico Ros a091d3f011 OpenAI: support for gpt-image-1.5 2026-01-09 01:22:26 -08:00
Enrico Ros c7c01a5d7c AIX: Gemini: sync API: FC with multimodal responses, validated FC-Config, retrieval config, multiple voices, seed, responseId 2026-01-09 01:03:29 -08:00
Enrico Ros cdc0f48973 AIX: Gemini: support MISSING_THOUGHT_SIGNATURE 2026-01-09 00:46:10 -08:00
Enrico Ros e884f6b962 LLMs: Gemini: mark Deep Research Pro Preview as hidden for now - Interactions API coming 2026-01-08 20:46:15 -08:00
Enrico Ros 485a9bea71 LLMs: Gemini: remove removed models 2026-01-08 20:44:57 -08:00
Enrico Ros f3c3b667ca LLMs: OpenAI: remove chatgpt-image from llms 2026-01-08 17:51:58 -08:00
Enrico Ros 3b0c4f31b6 LLMs: Gemini: add 3-flash-preview 2026-01-08 17:47:06 -08:00
Enrico Ros 5e54600766 Deps: set peers 2026-01-08 17:43:41 -08:00
claude[bot] c3e54f69b7 docs: remove Midori AI Subsystem section from installation guide
The Midori AI Subsystem is being sunset as announced in issue #902.
This removes the deployment section from the installation documentation.

Closes #902

Co-authored-by: Enrico Ros <enricoros@users.noreply.github.com>
2025-12-29 17:02:49 +00:00
Enrico Ros c4022d1c9b Roll small 2025-12-20 21:53:18 +01:00
Enrico Ros 6e13a78a24 Roll next 2025-12-20 21:40:15 +01:00
Enrico Ros c7cacd9727 LLMs: Gemini: remove 'medium' thinking level support - too early. Fixes #900 2025-12-20 21:08:41 +01:00
Enrico Ros a77110f704 LLMs: remove parameters set to a former enum value if then removed 2025-12-20 21:07:38 +01:00
Enrico Ros 83a6069de5 CC: update workflows 2025-12-14 03:45:26 +01:00
Enrico Ros e9a1890e54 Beam: rays/fusions can edit, delete 2025-12-11 23:50:06 +01:00
Enrico Ros bf928aa06e AIX: 'searching' item state with 5.2 Pro 2025-12-11 20:38:54 +01:00
Enrico Ros b2dc50590c LLMs: OpenAI GPT 5.2 Pro: enable web search 2025-12-11 20:33:53 +01:00
Enrico Ros 229e53ac32 Roll AIX 2025-12-11 20:20:26 +01:00
Enrico Ros 51e8a47615 OpenAI: Support X-Hight reasoning modes for 5.2-class models 2025-12-11 20:19:57 +01:00
Enrico Ros e80b58a412 UI: support 52 / 52Pro X-High/None reasoning efforts 2025-12-11 20:19:37 +01:00
Enrico Ros 48ced8b079 LLMs: support 52 / 52Pro X-High/None reasoning efforts 2025-12-11 20:19:30 +01:00
Enrico Ros c07e2aea1e AIX: support 52 / 52Pro X-High/None reasoning efforts 2025-12-11 20:19:19 +01:00
Enrico Ros f3194aa30e LLMs: Add GPT 5.2 2025-12-11 19:30:23 +01:00
Enrico Ros cb3e4cd951 LLMs: Mistral: update models 2025-12-09 13:33:01 -08:00
Enrico Ros f5d8d029ea CC: update workflows 2025-12-08 16:54:27 -08:00
Enrico Ros 7c946c4126 CC: add cherry-pick 2025-12-07 12:12:30 -08:00
Enrico Ros ded4ea0d69 Personas: disable YouTube transcript (unsupported API) 2025-12-07 12:10:27 -08:00
Enrico Ros c180c549fe BYOM: improve message 2025-12-07 11:53:49 -08:00
Enrico Ros 1f30f1168f Friction: Model Wizard: also warn if some keys are not saved 2025-12-07 11:52:51 -08:00
Enrico Ros 9446f15922 Friction: better remember Wizard model data 2025-12-07 11:51:35 -08:00
Enrico Ros e13b2c9cd9 Tutorial: 'BYOM' message 2025-12-07 11:49:18 -08:00
Enrico Ros e9e14e0292 LLMs: OpenRouter: add to the wizard 2025-12-07 11:46:04 -08:00
Enrico Ros added19656 Roll posthogs 2025-12-05 19:37:01 -08:00
Enrico Ros 4fa3c4d479 Remove old material (wrong) path 2025-12-05 19:32:34 -08:00
Enrico Ros 690738de9a Fix CVE-2025-55182 2025-12-05 18:41:13 -08:00
Enrico Ros cb31d27e68 Copy: strip background/colors on copy (keep font size and structure) 2025-12-05 11:17:17 -08:00
Enrico Ros e6658df123 Attachment: show dl issues on console 2025-12-05 11:15:15 -08:00
Enrico Ros 0b7154a14c LLMs: OpenAI: remove obsoleted models 2025-12-04 15:52:42 -08:00
Enrico Ros 02c1838de5 LLMs: OpenAI: add gpt-5.1-codex-max 2025-12-04 15:52:18 -08:00
Enrico Ros fc455fceb8 LLMs: Mistral: rmeove obsoleted models 2025-12-02 07:56:29 -08:00
Enrico Ros 8d40cdd234 LLMs: Mistral: sort 2025-12-02 07:53:15 -08:00
Enrico Ros 40145c669a LLMs: Mistral: add Mistral-Large and Ministral 2025-12-02 07:53:09 -08:00
Enrico Ros 34d2fc233f LLMs: Mistral: adapt wires 2025-12-02 07:52:55 -08:00
Enrico Ros 670ec0381a Speex: collapse configure - recollapse with shift 2025-12-01 09:11:48 -08:00
Enrico Ros 2128f255fe Speex: collapse configure when not used 2025-12-01 09:09:50 -08:00
Enrico Ros b717bd9a9a Settings: max-height to not jump around too much 2025-12-01 08:58:20 -08:00
Enrico Ros 8aab9311f5 Roll AIX 2025-12-01 08:33:32 -08:00
Enrico Ros ff3e16ea67 DeepSeek: still images are not supported 2025-12-01 08:30:16 -08:00
Enrico Ros 1de039c315 AIX: OpenAI ChatCompletion: remove multipart hotfix (not needed anymore) 2025-12-01 08:27:55 -08:00
Enrico Ros d05e1786d7 Model namings: speciale 2025-12-01 08:16:44 -08:00
Enrico Ros e34b5a7372 AIX: support Deepseek Speciale 2025-12-01 08:16:29 -08:00
Enrico Ros a1b3d1b508 DeepSeek: 3.2 and Speciale 2025-12-01 08:16:18 -08:00
Enrico Ros 1ebccdf420 Speex: Readmes 2025-12-01 02:59:56 -08:00
301 changed files with 18089 additions and 5107 deletions
+1
View File
@@ -0,0 +1 @@
commands/code/apply-issue-main.md
+1 -1
View File
@@ -46,4 +46,4 @@ Focus on discrepancies and gaps:
Report differences in wire types, adapter logic, parser handling, or dialect-specific quirks.
Prioritize new capabilities that improve user experience (reasoning visibility, better tool use, etc.).
When making changes, add comments with date: `// [OpenRouter, 2025-MM-DD]: explanation`
When making changes, add comments with date: `// [OpenRouter, 2026-MM-DD]: explanation`
+56
View File
@@ -0,0 +1,56 @@
---
description: Sync xAI Responses API implementation with latest upstream documentation
argument-hint: specific feature to check
---
Review the xAI Responses API implementation:
- xAI wire types: `src/modules/aix/server/dispatch/wiretypes/xai.wiretypes.ts` (xAI-specific request schema, tools)
- Request adapter: `src/modules/aix/server/dispatch/chatGenerate/adapters/xai.responsesCreate.ts` (AIX → xAI Responses API)
- Response parser: `src/modules/aix/server/dispatch/chatGenerate/parsers/openai.responses.parser.ts` (shared with OpenAI Responses)
- Dispatch routing: `src/modules/aix/server/dispatch/chatGenerate/chatGenerate.dispatch.ts` (dialect='xai' routing)
- OpenAI shared types: `src/modules/aix/server/dispatch/wiretypes/openai.wiretypes.ts` (InputItem/OutputItem schemas reused by xAI)
IMPORTANT context:
- We use ONLY the xAI Responses API (`POST /v1/responses`). We do NOT use the Chat Completions API (`/v1/chat/completions`) for xAI anymore.
- xAI's Responses API is similar to OpenAI's but has key differences - the skill should find what changed since our last sync.
- Response streaming/parsing reuses the OpenAI Responses parser since the format is compatible.
- We do NOT implement: Files API, Collections Search, Remote MCP tools, Voice Agent API, Image/Video generation, Batch API, or Deferred Completions.
Then take a look at the newest API information available. Try these sources, and be creative if some are blocked:
**Primary Sources (guide pages work well with WebFetch despite being JS-rendered):**
- Responses API Guide: https://docs.x.ai/docs/guides/chat
- Stateful Responses: https://docs.x.ai/docs/guides/responses-api
- Tools Overview: https://docs.x.ai/docs/guides/tools/overview
- Search Tools (web_search, x_search): https://docs.x.ai/docs/guides/tools/search-tools
- Code Execution Tool: https://docs.x.ai/docs/guides/tools/code-execution-tool
- Function Calling: https://docs.x.ai/docs/guides/function-calling
- Streaming: https://docs.x.ai/docs/guides/streaming-response
- Reasoning: https://docs.x.ai/docs/guides/reasoning
- Structured Outputs: https://docs.x.ai/docs/guides/structured-outputs
- Models & Pricing: https://docs.x.ai/developers/models
- Release Notes: https://docs.x.ai/developers/release-notes
- API Reference: https://docs.x.ai/developers/api-reference#create-new-response
**Alternative Sources if primary blocked:**
- xAI Python SDK: https://github.com/xai-org/xai-sdk-python
- Web Search for "xai grok api changelog 2026" or "xai responses api new features"
**If all blocked:** Explain what you attempted and ask user to provide documentation manually.
$ARGUMENTS
Check carefully for discrepancies between our implementation and the current API docs:
1. **Request fields**: Compare `XAIWire_API_Responses.Request_schema` against current docs - any new, changed, or deprecated parameters?
2. **Tool definitions**: Compare `XAIWire_Responses_Tools` - any new parameters on web_search/x_search/code_interpreter? Any new hosted tool types?
3. **Input/Output item types**: Any xAI-specific output items not handled by the shared OpenAI parser (e.g., x_search_call, web_search_call, code_interpreter_call)?
4. **Streaming events**: Any xAI-specific SSE event types beyond what the OpenAI Responses parser handles?
5. **Response shape**: Usage reporting differences, new fields in the response object?
6. **Adapter logic**: Message role mapping, content type handling, system message approach - still correct?
7. **Include options**: Any new values for the `include` array?
8. **Reasoning config**: Which models support it and with what values?
Prioritize breaking changes and new capabilities that would improve the user experience.
When making changes, add comments with date: `// [xAI, 2026-MM-DD]: explanation`
**Self-update this skill**: After completing the sync, if your research reveals that assumptions in THIS skill file (`.claude/commands/aix/sync-xai-api.md`) are wrong or outdated - e.g., new APIs we now implement, new tool types added, URLs moved, file paths changed - update this skill file to stay accurate for next time.
+34
View File
@@ -0,0 +1,34 @@
---
description: Review in-flight changes for coherence, completeness, and quality
---
Review the current in-flight changes in the big-agi-private repository (dev branch, continuously rebased ~1800 commits on top of main).
**Step 1: Scope and read**
`git diff --stat` + `git status` for breadth. Then full `git diff` (if empty: `git diff --cached`, then `git diff HEAD~1`).
For every file in the diff, read surrounding context in the actual source file - the diff alone hides bugs in adjacent untouched code.
**Step 2: Reverse-engineer the intent**
From the diff, determine the **what**, **how**, and **why**. Present this concisely so the author can confirm or correct,
but don't stop here, continue to the full review in the same response.
**Step 3: Validate**
Run `tsc --noEmit --pretty` and `npm run lint` (in parallel). Report any errors with the review.
If the diff removes/renames identifiers, grep the codebase for stale references to the OLD names. This catches broken guards, stale imports, and incomplete migrations.
**Step 4: Deep review**
Evaluate every file in the diff.
Leave no rocks unturned - correctness, coherence, completeness, excess, generalization, maintenance burden,
codebase consistency, etc.
**Step 5: Prioritized next steps**
Think about what happens when the next developer touches this code.
Rank findings by severity (bug > correctness > cleanup > cosmetic). Be specific about what to change and where.
Remember: design values for this codebase: orthogonal features, features that generalize well, modularized and reusable code,
type-discriminated data, optimized code, zero maintenance burden. Minimize future pain, etc.
+63
View File
@@ -0,0 +1,63 @@
---
description: Sync LLM parameter options between full model dialog and chat side panel
---
Audit and sync LLM parameter configurations between the two UI editors. Goal: identical `value` fields in option arrays + equivalent onChange logic. Labels/descriptions can differ for UI space.
**Files to Compare:**
1. **Full Model Dialog**: `src/modules/llms/models-modal/LLMParametersEditor.tsx` (main branch)
2. **Chat Side Panel**: `src/apps/chat/components/layout-panel/ChatPanelModelParameters.tsx` (main derived branches only)
**Reference Documentation:**
- Parameter system: `kb/systems/LLM-parameters-system.md`
- Parameter registry: `src/common/stores/llms/llms.parameters.ts`
**Task: Perform a comprehensive audit**
1. **Read both files** and extract all option arrays (e.g., `_reasoningEffortOptions`, `_antEffortOptions`, `_geminiThinkingLevelOptions`, etc.)
2. **Check for missing parameters:**
- Parameters handled in `LLMParametersEditor.tsx` but NOT in `ChatPanelModelParameters.tsx`
- Parameters in `ChatPanelModelParameters.tsx`'s `_interestingParameters` array but missing UI controls
- Note: The side panel intentionally shows only "interesting" parameters - focus on those listed in `_interestingParameters`
3. **Check for value mismatches** between corresponding option arrays:
- Different number of options (e.g., 3 vs 4 options)
- Same label but different `value` (this causes the bug in issue #926)
- Different labels for the same `value`
- Missing `_UNSPECIFIED`/Default option in one but not the other
4. **Check onChange handler consistency:**
- Both should remove parameter on `_UNSPECIFIED` selection
- Both should set explicit values the same way
- Watch for conditions like `value === 'high'` that may differ
**Output Format:**
```
## Parameter Sync Audit Report
### Missing Parameters
- [ ] `llmVndXyz` - In full dialog, missing from side panel
### Value Mismatches
- [ ] `_xyzOptions`:
- Full dialog: [values...]
- Side panel: [values...]
- Issue: [description]
### Handler Inconsistencies
- [ ] `llmVndXyz` onChange differs: [explanation]
### Recommended Fixes
1. [Specific fix with code snippet if needed]
```
**Fix Direction:** Full dialog is source of truth. Update side panel to match its values when mismatched.
**Notes:**
- Side panel uses shorter descriptions (space-constrained) - that's fine
- Variable names may differ (e.g., `_anthropicEffortOptions` vs `_antEffortOptions`) - that's fine, but same is better
- `value` fields must be identical sets
- `_UNSPECIFIED` must mean the same thing in both
- onChange: remove on `_UNSPECIFIED`, set explicit value otherwise
@@ -4,17 +4,46 @@ description: Update Anthropic model definitions with latest pricing and capabili
Update `src/modules/llms/server/anthropic/anthropic.models.ts` with latest model definitions.
Reference `src/modules/llms/server/llm.server.types.ts` and `src/modules/llms/server/models.mappings.ts` for context only. Focus on the model file, do not descend into other code.
Reference files (for context only, do not modify):
- `src/modules/llms/server/llm.server.types.ts`
- `src/modules/llms/server/models.mappings.ts`
- `src/common/stores/llms/llms.parameters.ts`
**Primary Sources:**
- Models: https://docs.claude.com/en/docs/about-claude/models/overview
- Pricing: https://claude.com/pricing#api
- Deprecations: https://docs.claude.com/en/docs/about-claude/model-deprecations
**Workflow: Start with recent changes, then verify the full model list.**
**Fallbacks if blocked:** Check Anthropic TypeScript SDK at https://github.com/anthropics/anthropic-sdk-typescript, search "anthropic models latest pricing", "anthropic latest models", or search GitHub for latest model prices and context windows
**Primary Sources (append `.md` to any path for clean markdown):**
1. Recent changes: https://platform.claude.com/docs/en/release-notes/overview.md
2. Models & IDs: https://platform.claude.com/docs/en/about-claude/models/overview.md
3. Pricing (base, cache, batch, long context): https://platform.claude.com/docs/en/about-claude/pricing.md
4. Deprecations & retirement dates: https://platform.claude.com/docs/en/about-claude/model-deprecations.md
**Discovering feature docs:** The release notes and models overview markdown
contain inline links to feature-specific pages (thinking modes, effort,
context windows, what's-new pages, etc.). When a new capability is
referenced, follow those links — append `.md` to get markdown. Examples of
pages you might discover this way:
- `about-claude/models/whats-new-claude-*` — per-generation changes
- `build-with-claude/extended-thinking` — thinking budget configuration
- `build-with-claude/effort` — effort parameter levels
- `build-with-claude/adaptive-thinking` — adaptive thinking mode
**Fallback web pages** (crawl if `.md` paths break or structure changes):
- https://platform.claude.com/docs/en/about-claude/models/overview
- https://platform.claude.com/docs/en/about-claude/pricing
- https://platform.claude.com/docs/en/release-notes/overview
- https://claude.com/pricing
**Fallbacks if blocked:** Check the Anthropic TypeScript SDK at
https://github.com/anthropics/anthropic-sdk-typescript, or web-search
for "anthropic models latest pricing" / "anthropic latest models".
**Important:**
- Review the full model list for additions, removals, and price changes
- For new models: check which `parameterSpecs` are needed (thinking mode,
effort levels, 1M context, skills, web tools) by reading the linked
feature docs and comparing with existing model entries
- When thinking/effort semantics change between generations
(e.g. adaptive vs manual thinking), document in comments
- Minimize whitespace/comment changes, focus on content
- Preserve comments to make diffs easy to review
- Flag broken links or unexpected content
@@ -0,0 +1,91 @@
---
description: Update/validate dynamic vendor model parsers (OpenRouter, TogetherAI, Alibaba, Azure, Novita, ChutesAI, FireworksAI, TLUS, LM Studio, LocalAI, FastAPI)
---
Validate that the dynamic (API-fetched) vendor model parsers are up to date and not silently broken.
These vendors do NOT have hardcoded model lists - they fetch models from APIs at runtime. But their parsers, filters, heuristic detection, and capability mapping can break if upstream APIs change. This skill covers all dynamic vendors NOT covered by the other `llms:update-models-{vendor}` skills.
## Vendors to Validate
### High Risk
**OpenRouter** - `src/modules/llms/server/openai/models/openrouter.models.ts`
- Most complex parser. Vendor-specific parameter inheritance (Anthropic thinking variants, Gemini thinking/image, OpenAI reasoning effort, xAI/DeepSeek reasoning).
- Hardcoded family ordering list (lines ~24-37) - check if new leading vendors are missing.
- Hardcoded old/deprecated model hiding list (lines ~39-49) - check if stale.
- Cache pricing detection (Anthropic-style vs OpenAI-style) - verify format still valid.
- Variant injection for Anthropic thinking/non-thinking - verify still correct.
- Reference: https://openrouter.ai/docs/models
### Medium Risk
**Novita** - `src/modules/llms/server/openai/models/novita.models.ts`
- Features array mapping (`function-calling`, `reasoning`, `structured-outputs`) and input modalities parsing.
- Pricing unit conversion (hundredths of cent per million → dollars per 1K).
- Hostname heuristic: `novita.ai`.
**ChutesAI** - `src/modules/llms/server/openai/models/chutesai.models.ts`
- Custom `max_model_len` field for context window.
- Assumes all models support Vision + Functions (aggressive).
- Hostname heuristic: `.chutes.ai`.
**FireworksAI** - `src/modules/llms/server/openai/models/fireworksai.models.ts`
- Relies on provider capability flags: `supports_chat`, `supports_image_input`, `supports_tools`.
- Hostname heuristic: `fireworks.ai/`.
**TogetherAI** - `src/modules/llms/server/openai/models/together.models.ts`
- Type allow-list (`type: 'chat'`), vision detection by string match.
- Custom wire schema with pricing conversion.
**TLUS** - `src/modules/llms/server/openai/models/tlusapi.models.ts`
- Detected by response structure (`total_models`, `free_models`, `pro_models` fields).
- Capability enum mapping (`text`, `vision`, `audio`, `tool-calling`, `reasoning`, `websearch`).
- Tier-based pricing (`free` vs paid).
**Alibaba** - `src/modules/llms/server/openai/models/alibaba.models.ts`
- Model list was cleared (dynamic-only). Exclusion patterns for non-chat models.
- Assumes 128K context and Vision+Functions for all models (overly permissive).
- Check if hardcoded data should be restored now that naming has stabilized.
### Low Risk (local/generic - validate only if issues reported)
**Azure** - `src/modules/llms/server/openai/models/azure.models.ts`
- Custom deployments API, not `/v1/models`. User-specific. Deployment name fallback logic.
**LM Studio** - `src/modules/llms/server/openai/models/lmstudio.models.ts`
- Local service, native API (`/api/v1/models`). GGUF metadata parsing, capability flags.
**LocalAI** - `src/modules/llms/server/openai/models/localai.models.ts`
- Local service. String-based hide list, vision/reasoning detection by name pattern.
**FastAPI** - `src/modules/llms/server/openai/models/fastapi.models.ts`
- Generic passthrough. Detected by `owned_by === 'fastchat'`. Minimal parsing.
## Validation Checklist
For each vendor (prioritize High > Medium > Low):
1. **Read the parser file** and check for:
- Deny/allow lists that may be stale (new model families missing)
- Capability assumptions that may be wrong (e.g. "all models support vision")
- Field names that may have changed upstream
- Pricing conversion math that may use wrong units
2. **Check upstream docs** (where available) for:
- API response schema changes
- New model types or capability fields
- Deprecated fields
3. **Cross-reference with OpenRouter** (aggregator):
- OpenRouter surfaces models from many of these vendors
- If OpenRouter shows capabilities that a vendor's parser misses, the parser is stale
4. **Fix issues found** - update parsers, filters, deny lists as needed.
5. Run `tsc --noEmit` after changes.
**Important:**
- Do NOT convert dynamic vendors to hardcoded lists - the dynamic approach is intentional
- Focus on parser correctness, not model coverage
- Flag any vendor whose API response format seems to have changed substantially
+3 -3
View File
@@ -6,11 +6,11 @@ Update `src/modules/llms/server/openai/models/groq.models.ts` with latest model
Reference `src/modules/llms/server/llm.server.types.ts` and `src/modules/llms/server/models.mappings.ts` for context only. Focus on the model file, do not descend into other code.
**Primary Sources:**
- Models: https://console.groq.com/docs/models
**Primary Source:**
- Fetch https://console.groq.com/docs/models.md directly (markdown format, no search needed)
- Pricing: https://groq.com/pricing/
**Fallbacks if blocked:** Search "groq models latest pricing", "groq latest models", "groq api models", or search GitHub for latest model prices and context windows
**Do NOT use web search.** The `.md` endpoint provides structured markdown content directly.
**Important:**
- Review the full model list for additions, removals, and price changes
+2 -2
View File
@@ -6,11 +6,11 @@ Update `src/modules/llms/server/openai/models/moonshot.models.ts` with latest mo
Reference `src/modules/llms/server/llm.server.types.ts` and `src/modules/llms/server/models.mappings.ts` for context only. Focus on the model file, do not descend into other code.
**Primary Sources:**
**Primary Sources (fetch directly, no search needed):**
- Pricing: https://platform.moonshot.ai/docs/pricing/chat
- API Reference: https://platform.moonshot.ai/docs/api/chat
**Fallbacks if blocked:** Search "moonshot kimi models latest pricing", "kimi k2 models", "moonshot api models", or search GitHub for latest model prices and context windows
**Do NOT use web search.** Fetch the URLs directly, or ask the user to provide data, if unaccessible.
**Important:**
- Review the full model list for additions, removals, and price changes
@@ -8,8 +8,8 @@ Reference `src/modules/llms/server/llm.server.types.ts` and `src/modules/llms/se
**Automated Workflow:**
```bash
# 1. Fetch the HTML
curl -s "https://ollama.com/library?sort=featured" -o /tmp/ollama-featured.html
# 1. Fetch the HTML (sorted by newest for stable ordering)
curl -s "https://ollama.com/library?sort=newest" -o /tmp/ollama-newest.html
# 2. Parse it with the script
node .claude/scripts/parse-ollama-models.js > /tmp/ollama-parsed.txt 2>&1
@@ -22,15 +22,18 @@ The parser outputs: `modelName|pulls|capabilities|sizes`
- Example: `deepseek-r1|66200000|tools,thinking|1.5b,7b,8b,14b,32b,70b,671b`
**Primary Sources:**
- Model Library: https://ollama.com/library?sort=featured
- Model Library: https://ollama.com/library?sort=newest
- Parser script: `.claude/scripts/parse-ollama-models.js`
**Fallbacks if blocked:** Check https://github.com/ollama/ollama, search "ollama featured models", "ollama latest models", or search GitHub for latest model info
**Important:**
- Skip models below 50,000 pulls (parser does this automatically)
- Skip embedding models (parser does not do this automatically)
- Sort them in the EXACT same order as the source (featured models)
- Parser filtering rules:
- Top 30 newest models are always included (regardless of pull count)
- After top 30, only models with 50K+ pulls are included
- Models with 'cloud' capability are automatically excluded
- Models with 'embedding' capability are automatically excluded
- Sort them in the EXACT same order as the source (newest first, for stable ordering)
- Extract tags: 'tools' → hasTools, 'vision' → hasVision, 'embedding' → isEmbeddings (note the 's'), 'thinking' → tags only
- Extract 'b' tags (1.5b, 7b, 32b) to tags field
- Set today's date (YYYYMMDD format) for newly added models only
@@ -0,0 +1,57 @@
---
description: Verify model parameterSpecs match API-validated sweep data
argument-hint: openai | anthropic | gemini | xai (or empty for all)
---
# Verify LLM Parameters
Compare model `parameterSpecs` in definition files against API-validated sweep data.
If `$ARGUMENTS` provided, verify only that dialect, which includes reading the pair of sweep results and model defintions. Otherwise verify all four, and read the pairs in sequence.
## Files
**Sweep results** (source of truth for select parameters):
- `tools/develop/llm-parameter-sweep/llm-{dialect}-parameters-sweep.json`
By the time you see these files, the repo owner has already updated them via `tools/develop/llm-parameter-sweep/sweep.sh` (very long running, 15 min per vendor).
**Model definitions (source of truth for model defintions for the user and application, including constants, interfaces, supported parameters and sometimes allowed parameter values)**:
- OpenAI: `src/modules/llms/server/openai/models/openai.models.ts`
- Anthropic: `src/modules/llms/server/anthropic/anthropic.models.ts`
- Gemini: `src/modules/llms/server/gemini/gemini.models.ts`
- xAI: `src/modules/llms/server/openai/models/xai.models.ts`
## Task
The sweep data is the source of truth for allowed model parameter values or value ranges.
For each model in the sweep, verify the model definition exposes exactly those capabilities - no more, no less. This includes:
- The parameter is present in parameterSpecs
- The paramId variant covers exactly the values from the sweep, if applicable
- etc.
Report models where the definition doesn't match the sweep.
## Parameter Mapping
Example parameter mapping. Note that new parameters may have been added to both the definition, and the sweep.
The objective of the sweep is to hint at model definition values, but the model definitions are what matters for Big-AGI,
and need to be carefully updated, otherwise thousands of clients may break.
| Dialect | Sweep Key | Model paramId |
|-----------|--------------------------|------------------------------|
| OpenAI | `oai-reasoning-effort` | `llmVndOaiEffort` |
| OpenAI | `oai-verbosity` | `llmVndOaiVerbosity` |
| OpenAI | `oai-image-generation` | `llmVndOaiImageGeneration` |
| OpenAI | `oai-web-search` | `llmVndOaiWebSearchContext` |
| Anthropic | `ant-effort` | `llmVndAntEffort` |
| Anthropic | `ant-thinking-budget` | `llmVndAntThinkingBudget` |
| Gemini | `gemini-thinking-level` | `llmVndGemEffort` |
| Gemini | `gemini-thinking-budget` | `llmVndGeminiThinkingBudget` |
| xAI | `xai-web-search` | `llmVndXaiWebSearch` |
## Output
Report first for every model the expected values from the sweep, then the actual values from the definition, then the mismatches.
Finally make one table for each dialect listing all models with mismatches and the specific issues.
+56
View File
@@ -0,0 +1,56 @@
---
description: Generate changelog bullets for big-agi.com/changes
argument-hint: date like "2026-01-10" or empty for auto-detect
---
Generate changelog bullets for a single entry in https://big-agi.com/changes
**Step 1: Find the starting date**
IMPORTANT: This repo rebases frequently, so commits are INTERLEAVED throughout history.
New commits can appear at line 10, 500, or 1800. Use AUTHOR DATE (`%ad`) to filter - it's preserved during rebases.
If `$ARGUMENTS` provided, use it as the cutoff date.
If NO argument:
1. Fetch https://big-agi.com/changes to get the most recent changelog date
2. Use that date as the cutoff
**Step 2: Get commits by author date**
Filter commits by author date to catch ALL new commits regardless of position in history:
```bash
# For commits after Jan 10, 2026 (adjust date pattern as needed)
git log --oneline --no-merges --format="%h %ad %s" --date=short | grep "2026-01-1[1-9]\|2026-01-2\|2026-02"
# Verify interleaving by checking line numbers
git log --oneline --no-merges --format="%h %ad %s" --date=short | grep -n "2026-01-1[1-9]"
```
The line numbers prove commits are scattered (e.g., lines 14, 638, 1156, 1803 = interleaved).
**Step 3: Write bullets**
Real examples from big-agi.com/changes:
- "Gemini 3 Flash support with 4-level thinking: high, medium, low, minimal"
- "Cloud Sync launched! - long awaited and top requested"
- "Deepseek V3.2 Speciale comes with almost Gemini 3 Pro performance but 20 times cheaper"
- "Anthropic Opus 4.5 with controls for effort (speed tradeoff), thinking budget, search"
- "Login with email, via magic link"
- "Mobile UX fixes for popups drag/interaction"
**Rules:**
1. **Order by importance** - most significant changes first, minor fixes last
2. **Feature-first, no verb prefixes** - "Gemini 3 support" not "Add Gemini 3 support"
3. **Model names lead** when it's about LLMs
4. **Specific details** - "4-level thinking: high, medium, low, minimal" not "multiple thinking levels"
5. **One-liners** - short, no fluff
6. **Consolidate commits** - 10 persona editor commits = 1 bullet
7. **No corporate speak** - no "enhanced", "streamlined", "robust", "leverage"
**Skip:** WIP, internal refactors, KB docs, automation, review cleanups, trivial fixes, deps bumps, CI changes.
**Output:** Just bullets, ready to paste. 2-5 bullets but adapt depending on scope, especially
in relation to the usual https://big-agi.com/changes entries.
+113
View File
@@ -0,0 +1,113 @@
---
description: Execute the Big-AGI release process
argument-hint: version like "2.0.4" or empty to auto-increment patch
---
Execute the release process for Big-AGI. Go step-by-step, waiting for user approval between major steps.
## Step 1: Determine Version
If `$ARGUMENTS` provided, use it. Otherwise, read `package.json` and increment patch version.
## Step 2: Update Files
1. **package.json** - Update `version` field
2. **src/common/app.release.ts** - Increment `Monotonics.NewsVersion` (e.g., 203 → 204)
3. **src/apps/news/news.data.tsx** - Add new entry at top of `NewsItems` array
For the news entry, ask user for release name and key highlights.
**News entry style** - Draft is a starting point, user will refine:
- Models lead when model-heavy, grouped together
- Callout features get own bullet with colon explanation
- UX items grouped, minimal bold
- Fixes last, brief
- Release name stays subtle - don't oversell the theme
Use `<B>`, `<B issue={N}>`, `<B href='url'>`. Re-read file after user edits.
4. User runs `npm i` to update lockfile
## Step 3: README
Update `README.md`:
- Line ~46: Update model examples if new flagship models
- Line ~147: Add release bullet above previous version
**Style:** `- Open X.Y.Z: **Name** feature1, feature2, feature3`
## Step 4: Git Operations
User commits changes, then:
```bash
git tag vX.Y.Z
git push opensource vX.Y.Z
```
## Step 5: GitHub Release
Create release with `gh release create`. Structure:
```
# Big-AGI X.Y.Z - Name
## What's New
### **Headline Feature**
1-2 sentences explaining the main theme. Then bullet points for specifics.
### **Also New**
- Bullet list of other features
- Keep it scannable
**Full Changelog**: https://github.com/enricoros/big-AGI/compare/vPREV...vNEW
## Get Started
Available now at [big-agi.com](https://big-agi.com), via Docker, or self-host from source.
```
## Step 6: Announcements
Draft for user to post:
**Twitter** - Thematic, not feature dumps. Talk about what it means, not what it lists:
```
Big-AGI Open X.Y.Z is out!
[Theme - e.g., "Lots of love to models: native support, latest protocols, total configuration - puts you in control."]
[One more angle, natural prose]
[Optional link]
```
**Discord** - Structured with bold headers:
```
## :partyblob: Big-AGI **Open** X.Y.Z
**Category:** Items
**Category:** Items
**More:** Count of commits/fixes
```
## Tone Guide
**Good:**
- "Lots of love to models: native support, latest protocols, total configuration"
- "UX quality of life improvements, from Google Drive to message reorder"
- "Gemini 3 Flash support with 4-level thinking: high, medium, low, minimal"
**Bad:**
- "Rolling out the red carpet for top models!" (too salesy)
- "Enhanced and streamlined the robust model experience" (corporate speak)
- "Added support for Gemini 3 Flash model with multiple thinking levels" (verb prefix, vague)
## Reference
Find previous copy at:
- **GitHub releases:** https://github.com/enricoros/big-AGI/releases
- **News entries:** `src/apps/news/news.data.tsx`
- **README:** `README.md` release notes section
- **Changelog:** https://big-agi.com/changes
Match the existing tone - professional but human, specific not generic, features not marketing.
+41 -9
View File
@@ -1,23 +1,36 @@
#!/usr/bin/env node
/**
* Parse Ollama featured models from HTML
* Parse Ollama models from HTML (sorted by newest for stable ordering)
*
* Usage:
* 1. Fetch HTML: curl -s "https://ollama.com/library?sort=featured" -o /tmp/ollama-featured.html
* 1. Fetch HTML: curl -s "https://ollama.com/library?sort=newest" -o /tmp/ollama-newest.html
* 2. Parse: node .claude/scripts/parse-ollama-models.js
*
* Outputs: pipe-delimited format: modelName|pulls|capabilities|sizes
* Example: deepseek-r1|66200000|tools,thinking|1.5b,7b,8b,14b,32b,70b,671b
*
* Filtering rules:
* - Top 30 newest models are always included (regardless of pull count)
* - After top 30, only models with 50K+ pulls are included
* - Models with 'cloud' capability are always excluded
* - Models with 'embedding' capability are always excluded
*
* Pull counts are rounded to significant figures for stable diffs:
* - >=10M: round to 100K (e.g., 109,123,456 -> 109,100,000)
* - >=1M: round to 10K (e.g., 5,432,100 -> 5,430,000)
* - <1M: round to 1K (e.g., 88,700 -> 89,000)
*/
const fs = require('fs');
const htmlPath = process.argv[2] || '/tmp/ollama-featured.html';
const htmlPath = process.argv[2] || '/tmp/ollama-newest.html';
const TOP_N_ALWAYS_INCLUDE = 30;
const MIN_PULLS_THRESHOLD = 50000;
if (!fs.existsSync(htmlPath)) {
console.error(`Error: HTML file not found at ${htmlPath}`);
console.error('Please fetch it first with:');
console.error(' curl -s "https://ollama.com/library?sort=featured" -o /tmp/ollama-featured.html');
console.error(' curl -s "https://ollama.com/library?sort=newest" -o /tmp/ollama-newest.html');
process.exit(1);
}
@@ -25,7 +38,7 @@ const html = fs.readFileSync(htmlPath, 'utf8');
// Split into model sections - each starts with <a href="/library/
const modelSections = html.split(/<a href="\/library\//);
const models = [];
const allParsedModels = [];
for (let i = 1; i < modelSections.length; i++) {
const section = modelSections[i].substring(0, 5000); // Large enough window to capture all data
@@ -65,10 +78,27 @@ for (let i = 1; i < modelSections.length; i++) {
sizes.push(sizeMatch[1].trim());
}
// Only include models with 50K+ pulls
if (pulls >= 50000) {
models.push({ name, pulls, capabilities, sizes });
// Skip models with 'cloud' or 'embedding' capability
if (capabilities.includes('cloud') || capabilities.includes('embedding')) {
continue;
}
allParsedModels.push({ name, pulls: roundPulls(pulls), capabilities, sizes });
}
// Apply filtering: top 30 always included, rest need 50K+ pulls
const models = allParsedModels.filter((model, index) => {
return index < TOP_N_ALWAYS_INCLUDE || model.pulls >= MIN_PULLS_THRESHOLD;
});
/**
* Round pulls to significant figures for stable output.
* This reduces churn from daily fluctuations while preserving magnitude.
*/
function roundPulls(pulls) {
if (pulls >= 10000000) return Math.round(pulls / 100000) * 100000; // >=10M: round to 100K
if (pulls >= 1000000) return Math.round(pulls / 10000) * 10000; // >=1M: round to 10K
return Math.round(pulls / 1000) * 1000; // <1M: round to 1K
}
// Output in pipe-delimited format (in the order they appear on the page)
@@ -78,4 +108,6 @@ models.forEach(m => {
console.log(`${m.name}|${m.pulls}|${caps}|${tags}`);
});
console.error(`\nTotal models with 50K+ pulls: ${models.length}`);
const topNCount = Math.min(TOP_N_ALWAYS_INCLUDE, allParsedModels.length);
const thresholdCount = models.length - topNCount;
console.error(`\nTotal models: ${models.length} (top ${topNCount} newest + ${thresholdCount} with ${MIN_PULLS_THRESHOLD / 1000}K+ pulls)`);
+8 -3
View File
@@ -4,12 +4,17 @@
"Bash(cat:*)",
"Bash(cp:*)",
"Bash(curl:*)",
"Bash(eslint:*)",
"Bash(find:*)",
"Bash(gh issue list:*)",
"Bash(gh issue view:*)",
"Bash(git branch:*)",
"Bash(git cherry-pick:*)",
"Bash(git describe:*)",
"Bash(git grep:*)",
"Bash(git log:*)",
"Bash(git log:*)",
"Bash(git ls-tree:*)",
"Bash(git mv:*)",
"Bash(git show:*)",
"Bash(grep:*)",
"Bash(ls:*)",
@@ -18,13 +23,13 @@
"Bash(npm install)",
"Bash(npm install:*)",
"Bash(npm run:*)",
"Bash(npx eslint:*)",
"Bash(npx tsc:*)",
"Bash(rg:*)",
"Bash(rm:*)",
"Bash(sed:*)",
"Bash(tree:*)",
"Bash(tsc:*)",
"Read(//tmp/**)",
"Skill(llms:update-models*)",
"WebFetch",
"WebFetch(domain:big-agi.com)",
"WebSearch",
+15 -40
View File
@@ -1,43 +1,18 @@
# big-AGI non-code files
/docs/
/dist/
README.md
*
# Ignore build and log files
Dockerfile
/.dockerignore
!app/
!kb/
!pages/
!public/
!src/
!tools/
# Node build artifacts
/node_modules
/.pnp
.pnp.js
!*.mjs
!middleware_BASIC_AUTH.ts
!middleware.ts
!next.config.ts
!package*.json
!tsconfig.json
# next.js
/.next/
/out/
# production
/build
# versioning
.git/
.github/
# IDEs
.idea/
# debug
npm-debug.log*
yarn-debug.log*
yarn-error.log*
.pnpm-debug.log*
# local env files
.env*.local
# vercel
.vercel
# typescript
*.tsbuildinfo
next-env.d.ts
!LICENSE
!README.md
+69
View File
@@ -0,0 +1,69 @@
version: 2
updates:
- package-ecosystem: docker
directory: /
schedule:
interval: weekly
commit-message:
prefix: "chore(deps)"
ignore:
- dependency-name: "node"
versions: [">=25", "<26"] # Node 25 breaks the build because of a dummy localStorage object
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
commit-message:
prefix: "chore(deps)"
# Disabled npm updates for now - will need precise package pinning, as some packages changed behavior upstream
# - package-ecosystem: npm
# directory: /
# schedule:
# interval: weekly
# commit-message:
# prefix: "chore(deps)"
# cooldown:
# semver-patch: 3
# semver-minor: 7
# semver-major: 14
# # Ignore packages intentionally pinned due to upstream issues
# ignore:
# # Issue #857: v11.6+ breaks streaming; tried 11.4.4/11.6/11.7, only 11.5.1 works
# - dependency-name: "@trpc/*"
# versions: [">=11.5.1", "<12"]
# # Pinned during tRPC #857 debugging - may be safe to unpin, test first
# - dependency-name: "@tanstack/react-query"
# versions: [">=5.90.10", "<6"]
# # Pinned because 5.0.8 changes signatures so return set({ .. }) != void;
# - dependency-name: "zustand"
# versions: [">=5.0.7", "<6"]
# groups:
# next:
# patterns:
# - "@next/*"
# - "eslint-config-next"
# - "next"
# react:
# patterns:
# - "react"
# - "react-dom"
# - "@types/react"
# - "@types/react-dom"
# emotion:
# patterns:
# - "@emotion/*"
# mui:
# patterns:
# - "@mui/*"
# dnd-kit:
# patterns:
# - "@dnd-kit/*"
# prisma:
# patterns:
# - "@prisma/*"
# - "prisma"
# vercel:
# patterns:
# - "@vercel/*"
+13 -11
View File
@@ -12,27 +12,30 @@ on:
jobs:
claude-dm:
# Only allow repository owner to trigger DMs with @claude (blocks other users and bots)
if: |
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) ||
github.actor == 'enricoros' &&
github.triggering_actor == 'enricoros' &&
((github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) ||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')))
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
pull-requests: write
contents: write # Required for code creation and commits
issues: write
id-token: write
pull-requests: write
actions: read # Required for Claude to read CI results on PRs
id-token: write # required to use OIDC to authenticate to Claude Code API
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-depth: 0 # 1 -> 0: full history helps with git blame, etc.
- name: Run Claude Code DM Response
id: claude
@@ -41,6 +44,7 @@ jobs:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Security: Only users with write access can trigger (DMs allow code execution)
# Note: contents:write permission enables code creation and commits
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
@@ -49,9 +53,7 @@ jobs:
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://docs.claude.com/en/docs/claude-code/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr:*)'
# disabling opus for now claude-opus-4-1-20250805
claude_args: |
--model claude-sonnet-4-5-20250929
--model claude-opus-4-6
--max-turns 100
--allowedTools "Edit,Read,Write,WebFetch,WebSearch,Bash(cat:*),Bash(cp:*),Bash(find:*),Bash(git branch:*),Bash(grep:*),Bash(ls:*),Bash(mkdir:*),Bash(npm install),Bash(npm install:*),Bash(npm run:*),Bash(gh issue:*),Bash(gh search:*),Bash(gh label:*),Bash(gh pr:*),mcp__chrome-devtools,SlashCommand"
--allowedTools "Edit,Read,Write,WebFetch,WebSearch,Bash(cat:*),Bash(cp:*),Bash(find:*),Bash(git branch:*),Bash(grep:*),Bash(ls:*),Bash(mkdir:*),Bash(npm run:*),Bash(gh issue:*),Bash(gh search:*),Bash(gh label:*),Bash(gh pr:*),SlashCommand"
+16 -10
View File
@@ -2,7 +2,7 @@ name: Claude Code Auto-Triage Issues
on:
issues:
types: [ opened, assigned ]
types: [ opened ]
jobs:
claude-issue-triage:
@@ -17,15 +17,15 @@ jobs:
permissions:
contents: read
issues: write
pull-requests: write
id-token: write
pull-requests: read # was write, but we're not altering PRs here
actions: read
id-token: write # required to use OIDC to authenticate to Claude Code API
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-depth: 0 # 1 -> 0: full history helps with git blame, etc.
- name: Analyze issue and provide help
uses: anthropics/claude-code-action@v1
@@ -35,6 +35,7 @@ jobs:
github_token: ${{ secrets.GITHUB_TOKEN }}
allowed_non_write_users: '*'
# track_progress: true # Enables tracking comments
show_full_output: ${{ github.event.repository.private }} # security: do not log verbosely in private repo
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
@@ -54,9 +55,11 @@ jobs:
**Use web search**: When potentially outside Big-AGI (e.g. user configuration), search the web for similar errors or related issues
**Provide a solution**:
- Provide multiple solutions if uncertain, and say so
- If you can fix it in code, propose the fix
- If possible also suggest fixes or workarounds for immediate relief
- Analyze the code and suggest specific fixes with code examples
- If possible also suggest fixes or workarounds for immediate relief
- Reference specific files and line numbers
- Suggest workarounds for immediate relief if applicable
- Use web search to find similar issues and solutions
- Test selectively and even npm install and run build if needed to verify the solution
2. Always add the 'claude-triage' issue label to indicate this issue was triaged by Claude
3. Comment with:
@@ -65,13 +68,16 @@ jobs:
- Next steps or clarification needed
- Link duplicates if found
Remember: design values for this codebase: orthogonal features, features that generalize well, modularized and reusable code,
type-discriminated data, optimized code, zero maintenance burden. Minimize future pain, etc.
IMPORTANT: You are in READ-ONLY triage mode. Analyze and suggest solutions in your comment, but do NOT attempt to push code changes.
If you're uncertain, say so and suggest next steps.
If you write any code make sure that it compiles and that you push it.
Be welcoming, helpful, professional, solution-focused and no-BS.
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://docs.claude.com/en/docs/claude-code/cli-reference for available options
claude_args: |
--model claude-sonnet-4-5-20250929
--model claude-opus-4-6
--max-turns 75
--allowedTools "Edit,Read,Write,WebFetch,WebSearch,Bash(cat:*),Bash(cp:*),Bash(find:*),Bash(git branch:*),Bash(grep:*),Bash(ls:*),Bash(mkdir:*),Bash(npm install),Bash(npm install:*),Bash(npm run:*),Bash(gh issue:*),Bash(gh search:*),Bash(gh label:*),Bash(gh pr:*),mcp__chrome-devtools,SlashCommand"
--allowedTools "Edit,Read,Write,WebFetch,WebSearch,Bash(cat:*),Bash(cp:*),Bash(find:*),Bash(git branch:*),Bash(grep:*),Bash(ls:*),Bash(mkdir:*),Bash(npm run:*),Bash(gh issue:*),Bash(gh search:*),Bash(gh label:*),Bash(gh pr:*),SlashCommand"
-77
View File
@@ -1,77 +0,0 @@
name: Claude Code PR Review
on:
pull_request:
types: [ opened, synchronize, ready_for_review ]
# Limit branches
branches: [ main, dev, v1 ]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
jobs:
claude-pr-review:
# Skip draft PRs
# Optional: filter authors: github.event.pull_request.user.login != 'enricoros'
if: |
github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
pull-requests: write
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run PR Review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Security: Allow any user to trigger reviews (read-only PR analysis is safe)
github_token: ${{ secrets.GITHUB_TOKEN }}
allowed_non_write_users: '*'
# track_progress: true # Enables tracking comments
# This setting allows Claude to read CI results on PRs
additional_permissions: |
actions: read
prompt: |
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}
Please review this pull request and provide feedback on:
- Potential bugs or issues
- Adherence to Big-AGI architecture and design patterns
- Code quality and best practices, including TypeScript types, error handling, and edge cases
- Performance considerations: bundle size, React patterns, streaming efficiency
- Security concerns if applicable
Use the repository's CLAUDE.md for guidance on style and conventions.
Use `gh pr comment` with your Bash tool to leave your review as a comment on the PR.
Use `gh pr review comment` for inline suggestions on specific lines.
IMPORTANT: After completing your review, always add the 'claude-review' label to the PR to indicate it was reviewed by Claude:
gh pr edit ${{ github.event.pull_request.number }} --add-label "claude-review"
Be constructive, helpful, no-BS, and specific with file:line references.
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://docs.claude.com/en/docs/claude-code/cli-reference for available options
claude_args: |
--model claude-sonnet-4-5-20250929
--max-turns 100
--allowedTools "Edit,Read,Write,WebFetch,WebSearch,Bash(cat:*),Bash(cp:*),Bash(find:*),Bash(git branch:*),Bash(grep:*),Bash(ls:*),Bash(mkdir:*),Bash(npm install),Bash(npm install:*),Bash(npm run:*),Bash(gh issue:*),Bash(gh search:*),Bash(gh label:*),Bash(gh pr:*),mcp__chrome-devtools"
+120 -37
View File
@@ -20,29 +20,122 @@ env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build-and-push-image:
runs-on: ubuntu-latest
timeout-minutes: 60 # Max 1 hour (expected: ~25min)
# Build job: runs on native runners for each platform (no QEMU emulation)
build:
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}
name: Build ${{ matrix.platform }}
timeout-minutes: 30
permissions:
contents: read
packages: write
security-events: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
echo "IMAGE_NAME_LC=${IMAGE_NAME,,}" >> $GITHUB_ENV
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
fetch-depth: 1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
labels: |
org.opencontainers.image.title=Big-AGI Open
org.opencontainers.image.description=Big-AGI Open - Multi-model AI workspace for experts who need to think broader, decide smarter, and build with confidence.
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.documentation=https://big-agi.com
- name: Build and push by digest
id: build
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LC }}
build-args: |
NEXT_PUBLIC_GA4_MEASUREMENT_ID=${{ secrets.GA4_MEASUREMENT_ID }}
NEXT_PUBLIC_BUILD_HASH=${{ github.sha }}
NEXT_PUBLIC_BUILD_REF_NAME=${{ github.ref_name }}
outputs: type=image,push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true
provenance: false
cache-from: type=gha,scope=${{ github.repository }}-${{ matrix.platform }}
cache-to: type=gha,scope=${{ github.repository }}-${{ matrix.platform }},mode=max
- name: Export digest
run: |
mkdir -p ${{ runner.temp }}/digests
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6.0.0
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1
# Merge job: combines platform-specific images into a unified multi-arch manifest
merge:
name: Merge manifests
runs-on: ubuntu-latest
timeout-minutes: 10
needs: build
permissions:
contents: read
packages: write
steps:
- name: Prepare
run: echo "IMAGE_NAME_LC=${IMAGE_NAME,,}" >> $GITHUB_ENV
- name: Download digests
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131 # v7.0.0
with:
path: ${{ runner.temp }}/digests
pattern: digests-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Log in to the Container registry
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -50,7 +143,7 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
@@ -66,28 +159,18 @@ jobs:
# Version tags (v2.0.0, 2.0.0)
type=ref,event=tag
type=semver,pattern={{version}}
labels: |
org.opencontainers.image.title=Big-AGI Open
org.opencontainers.image.description=Big-AGI Open - Multi-model AI workspace for experts who need to think broader, decide smarter, and build with confidence.
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.documentation=https://big-agi.com
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
file: Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
NEXT_PUBLIC_GA4_MEASUREMENT_ID=${{ secrets.GA4_MEASUREMENT_ID }}
NEXT_PUBLIC_BUILD_HASH=${{ github.sha }}
NEXT_PUBLIC_BUILD_REF_NAME=${{ github.ref_name }}
# Enable build cache (future)
#cache-from: type=gha
#cache-to: type=gha,mode=max
# Enable provenance and SBOM (future)
#provenance: true
#sbom: true
- name: Create manifest list and push
working-directory: ${{ runner.temp }}/digests
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
--annotation='index:org.opencontainers.image.title=Big-AGI Open' \
--annotation='index:org.opencontainers.image.description=Big-AGI Open - Multi-model AI workspace for experts who need to think broader, decide smarter, and build with confidence.' \
--annotation='index:org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}' \
--annotation='index:org.opencontainers.image.documentation=https://big-agi.com' \
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LC }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LC }}:${{ steps.meta.outputs.version }}
+3
View File
@@ -53,3 +53,6 @@ next-env.d.ts
.env*.local
/.run/dev (ENV).run.xml
/src/modules/3rdparty/aider/scratch*
# Ignore temporary CC files
/tmpclaude*
+1
View File
@@ -0,0 +1 @@
24
+27 -12
View File
@@ -5,18 +5,31 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Development Commands
```bash
# Targeted Code Quality (safe while dev server runs)
npx tsc --noEmit # Type check without building
npx eslint src/path/to/file.ts # Lint specific file
npm run lint # Lint entire project
# Validate (~5s, safe while dev server runs, do NOT use `next build` ~45s for same checks)
tsc --noEmit --pretty && npm run lint # Type check (~3.5s) + ESLint (~2s)
eslint src/path/to/file.ts # Lint specific file
# Full build (~60s+, only when suspecting runtime/bundle issues)
npm run build # next build runs compile+lint+types but stops at first type-error file; tsc shows all at once
# Database & External Services
# npm run supabase:local-update-types # Generate TypeScript types
# npm run stripe:listen # Listen for Stripe webhooks
```
## Development Environment
- Dev servers may be running on ports 3000, 3001, 3002, or 3003 (not always this app - other projects may occupy these ports). Never start or stop dev servers, let the user do it.
- For runtime debugging, use `mcp__chrome-devtools` if present to launch a controlled Chrome instance against the running dev server - useful for console errors, network inspection, and React devtree.
## Architecture Overview
Big-AGI is a Next.js 15 application with a modular architecture built for advanced AI interactions. The codebase follows a three-layer structure with distinct separation of concerns.
### Core Directory Structure
You are started from the root of the repository (i.e. where the git folder is or scripts should be run from). You won't need to issue 'cd ...' commands.
```
/app/api/ # Next.js App Router (API routes only, mostly -> /src/server/)
/pages/ # Next.js Pages Router (file-based, mostly -> /src/apps/)
@@ -32,7 +45,7 @@ Big-AGI is a Next.js 15 application with a modular architecture built for advanc
- **Frontend**: Next.js 15, React 18, Material-UI Joy, Emotion (CSS-in-JS)
- **State Management**: Zustand with localStorge/IndexedDB (single cell) persistence
- **API Layer**: tRPC with React Query for type-safe communication
- **API Layer**: tRPC with TanStack React Query for type-safe communication
- **Runtime**: Edge Runtime for AI operations, Node.js for data processing
### Apps Architecture Pattern
@@ -51,7 +64,7 @@ Modules in `/src/modules/` provide reusable business logic:
- **`aix/`** - AI communication framework for real-time streaming
- **`beam/`** - Multi-model AI reasoning system (scatter/gather pattern)
- **`blocks/`** - Content rendering (markdown, code, images, etc.)
- **`llms/`** - Language model abstraction supporting 16 vendors
- **`llms/`** - Language model abstraction supporting 19 vendors
### Key Subsystems & Their Patterns
@@ -122,6 +135,7 @@ Located in `/src/common/layout/optima/`
2. **Per-Instance Stores** (Vanilla Zustand)
- `store-beam_vanilla`: Beam scatter/gather state
- `store-perchat_vanilla`: Chat overlay state
- `store-attachment-drafts_vanilla`: Attachment drafts
- High-performance, no React integration
3. **Module Stores**
@@ -181,14 +195,14 @@ Architecture and system documentation is available in the `/kb/` knowledge base:
### Testing & Quality
- Run `npm run lint` before committing
- Type-check with `npx tsc --noEmit`
- Type-check with `tsc --noEmit`
- Test critical user flows manually
### Adding a New LLM Vendor
1. Create vendor in `/src/modules/llms/vendors/[vendor]/`
2. Implement `IModelVendor` interface
3. Register in `vendors.registry.ts`
4. Add environment variables to `env.ts` (if server-side keys needed)
4. Add environment variables to the vendor's server file and `/src/server/env.server.ts` (if server-side keys needed)
### Debugging Storage Issues
- Check IndexedDB: DevTools → Application → IndexedDB → `app-chats`
@@ -200,9 +214,9 @@ Architecture and system documentation is available in the `/kb/` knowledge base:
### AIX Streaming Pattern
```typescript
// Efficient streaming with decimation
aixChatGenerateContent_DMessage(
aixChatGenerateContent_DMessage_FromConversation(
llmId,
request,
chatHistory,
{ abortSignal, throttleParallelThreads: 1 },
async (update, isDone) => {
// Real-time UI updates
@@ -216,7 +230,7 @@ aixChatGenerateContent_DMessage(
const MODEL_VENDOR_REGISTRY: Record<ModelVendorId, IModelVendor> = {
openai: ModelVendorOpenAI,
anthropic: ModelVendorAnthropic,
// ... 14 more vendors
// ... 17 more vendors
};
```
@@ -228,7 +242,8 @@ The server uses a split architecture with two tRPC routers:
Distributed edge runtime for low-latency AI operations:
- **AIX** - AI streaming and communication
- **LLM Routers** - Direct vendor integrations (OpenAI, Anthropic, Gemini, Ollama)
- **External Services** - ElevenLabs (TTS), Google Search, YouTube transcripts
- **Speex** - Unified TTS router (ElevenLabs, Inworld, and other TTS vendors)
- **External Services** - Google Search, YouTube transcripts
Located at `/src/server/trpc/trpc.router-edge.ts`
+19 -10
View File
@@ -1,5 +1,8 @@
# syntax=docker/dockerfile:1
# check=skip=CopyIgnoredFile
# Base
FROM node:22-alpine AS base
FROM node:24-alpine AS base
ENV NEXT_TELEMETRY_DISABLED=1
# Dependencies
@@ -39,19 +42,20 @@ ENV NEXT_PUBLIC_GA4_MEASUREMENT_ID=${NEXT_PUBLIC_GA4_MEASUREMENT_ID}
ARG NEXT_PUBLIC_POSTHOG_KEY
ENV NEXT_PUBLIC_POSTHOG_KEY=${NEXT_PUBLIC_POSTHOG_KEY}
# Optional argument to configure Google Drive Picker at build time (can reuse AUTH_GOOGLE_ID value)
ARG NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID
ENV NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID=${NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID}
# Copy development deps and source
COPY --from=deps /app/node_modules ./node_modules
COPY . .
# link ssl3 for latest Alpine
RUN sh -c '[ ! -e /lib/libssl.so.3 ] && ln -s /usr/lib/libssl.so.3 /lib/libssl.so.3 || echo "Link already exists"'
# Build the application
ENV NODE_ENV=production
RUN npm run build
# Reduce installed packages to production-only
RUN npm prune --production
RUN npm prune --omit=dev
# Runner
@@ -59,18 +63,23 @@ FROM base AS runner
WORKDIR /app
# As user
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
RUN addgroup --system --gid 1001 nodejs \
&& adduser --system --uid 1001 nextjs \
&& apk add --no-cache openssl
# Copy Built app
COPY --from=builder --chown=nextjs:nodejs /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next ./.next
COPY --from=builder --chown=nextjs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nextjs:nodejs /app/src/server/prisma ./src/server/prisma
# Instead of `COPY --from=builder --chown=nextjs:nodejs /app/.next ./.next`, we only extract some parts, excluding .next/cache which is build time only:
COPY --from=builder --chown=nextjs:nodejs /app/.next/BUILD_ID ./.next/
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
COPY --from=builder --chown=nextjs:nodejs /app/.next/server ./.next/server
COPY --from=builder --chown=nextjs:nodejs /app/.next/types ./.next/types
COPY --from=builder --chown=nextjs:nodejs /app/.next/*.json ./.next/
# Minimal ENV for production
ENV NODE_ENV=production
ENV PATH=$PATH:/app/node_modules/.bin
# Run as non-root user
USER nextjs
@@ -79,4 +88,4 @@ USER nextjs
EXPOSE 3000
# Start the application
CMD ["next", "start"]
CMD ["/app/node_modules/.bin/next", "start"]
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2023-2025 Enrico Ros
Copyright (c) 2023-2026 Enrico Ros
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+10 -9
View File
@@ -10,7 +10,7 @@
[![Discord](https://img.shields.io/discord/1098796266906980422?style=for-the-badge&label=Discord&logo=discord&logoColor=white&labelColor=000000&color=purple)](https://discord.gg/MkH4qj2Jp9)
<br/>
[![GitHub Monthly Commits](https://img.shields.io/github/commit-activity/m/enricoros/big-agi?style=for-the-badge&x=3&logo=github&logoColor=white&label=commits&labelColor=000&color=green)](https://github.com/enricoros/big-agi/commits)
[![GHCR Pulls](https://img.shields.io/badge/ghcr.io-767k_dl-12b76a?style=for-the-badge&logo=Xdocker&logoColor=white&labelColor=000&color=A8E6CF)](https://github.com/enricoros/big-AGI/pkgs/container/big-agi)
[![GHCR Pulls](https://img.shields.io/badge/ghcr.io-800k_dl-12b76a?style=for-the-badge&logo=Xdocker&logoColor=white&labelColor=000&color=A8E6CF)](https://github.com/enricoros/big-AGI/pkgs/container/big-agi)
[![Contributors](https://img.shields.io/github/contributors/enricoros/big-agi?style=for-the-badge&x=2&logo=Xgithub&logoColor=white&label=cooks&labelColor=000&color=A8E6CF)](https://github.com/enricoros/big-AGI/graphs/contributors)
[![License: MIT](https://img.shields.io/badge/License-MIT-A8E6CF?style=for-the-badge&labelColor=000)](https://opensource.org/licenses/MIT)
<br/>
@@ -37,13 +37,13 @@ You need to think broader, decide faster, and build with confidence, then you ne
It comes packed with **world-class features** like Beam, and is praised for its **best-in-class AI chat UX**.
**As an independent, non-VC-funded project, Pro subscriptions at $10.99/mo fund development for everyone, including the free and open-source tiers.**
![LLM Vendors](https://img.shields.io/badge/18+_LLM_Services-500+_Models-black?style=for-the-badge&logo=anthropic&logoColor=white&labelColor=purple)&nbsp;
![LLM Vendors](https://img.shields.io/badge/19+_LLM_Services-500+_Models-black?style=for-the-badge&logo=anthropic&logoColor=white&labelColor=purple)&nbsp;
[![Feature Beam](https://img.shields.io/badge/AI--Validation-BEAM-000?style=for-the-badge&labelColor=purple)](https://big-agi.com/beam)&nbsp;
[![Feature Inspector](https://img.shields.io/badge/Expert_Mode-AI_Inspector-000?style=for-the-badge&labelColor=purple)](https://big-agi.com/inspector)
### What makes Big-AGI different:
**Intelligence**: with [Beam & Merge](https://big-agi.com/beam) for multi-model de-hallucination, native search, and bleeding-edge AI models like Opus 4.5, Nano Banana, Kimi K2 or GPT 5.1 -
**Intelligence**: with [Beam & Merge](https://big-agi.com/beam) for multi-model de-hallucination, native search, and bleeding-edge AI models like Opus 4.5, Nano Banana Pro, Kimi K2.5 or GPT 5.2 -
**Control**: with personas, data ownership, requests inspection, unlimited usage with API keys, and *no vendor lock-in* -
and **Speed**: with a local-first, over-powered, zero-latency, madly optimized web app.
@@ -144,7 +144,8 @@ NOTE: this is a powerful tool - if you need a toy UI or clone, this ain't it.
## Release Notes
👉 **[See the Live Release Notes](https://big-agi.com/changes)**
- Open 2.0.1: **Opus 4.5** full support, **Gemini 3 Pro** w/ code exec, **Nano Banana Pro**, **Grok 4.1**, **GPT-5.1**, **Kimi K2 Thinking** + 280 fixes
- Open 2.0.3: **Red Carpet** **Kimi K2.5**, **Gemini 3 Flash**, **GPT 5.2**, Google Drive, Inworld, Novita.ai, Speech/UX improvements
- Open 2.0.2: **Speex** multi-vendor speech synthesis, **Opus 4.5**, **Gemini 3 Pro**, **Nano Banana Pro**, **Grok 4.1**, **GPT-5.1**, **Kimi K2** + 280 fixes
### What's New in 2.0 · Oct 31, 2025 · Open
@@ -312,7 +313,7 @@ For full details and former releases, check out the [archived versions changelog
## 👉 Supported Models & Integrations
Delightful UX with latest models exclusive features like Beam for **multi-model AI validation**.
> ![LLM Vendors](https://img.shields.io/badge/18_LLM_Services-500+_Models-black?style=for-the-badge&logo=openai&logoColor=white&labelColor=purple)&nbsp;
> ![LLM Vendors](https://img.shields.io/badge/19_LLM_Services-500+_Models-black?style=for-the-badge&logo=openai&logoColor=white&labelColor=purple)&nbsp;
> [![Feature Beam](https://img.shields.io/badge/AI--Validation-BEAM-000?style=for-the-badge&logo=anthropic&labelColor=purple)](https://big-agi.com/beam)
| ![Advanced AI](https://img.shields.io/badge/Advanced%20AI-32383e?style=for-the-badge&logo=ai&logoColor=white) | ![500+ AI Models](https://img.shields.io/badge/500%2B%20AI%20Models-32383e?style=for-the-badge&logo=ai&logoColor=white) | ![Flow-state UX](https://img.shields.io/badge/Flow--state%20UX-32383e?style=for-the-badge&logo=flow&logoColor=white) | ![Privacy First](https://img.shields.io/badge/Privacy%20First-32383e?style=for-the-badge&logo=privacy&logoColor=white) | ![Advanced Tools](https://img.shields.io/badge/Fun%20To%20Use-f22a85?style=for-the-badge&logo=tools&logoColor=white) |
@@ -323,16 +324,16 @@ Delightful UX with latest models exclusive features like Beam for **multi-model
### AI Models & Vendors
Configure 100s of AI models from 18+ providers:
Configure 100s of AI models from 19+ providers:
| **AI models** | _supported vendors_ |
|:--------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Opensource Servers | [LocalAI](https://localai.io/) · [Ollama](https://ollama.com/) |
| Local Servers | [LM Studio](https://lmstudio.ai/) (non-open) |
| Multimodal services | [Azure](https://azure.microsoft.com/en-us/products/ai-services/openai-service) · [Anthropic](https://anthropic.com) · [Google Gemini](https://ai.google.dev/) · [OpenAI](https://platform.openai.com/docs/overview) |
| LLM services | [Alibaba](https://www.alibabacloud.com/en/product/modelstudio) · [DeepSeek](https://deepseek.com) · [Groq](https://wow.groq.com/) · [Mistral](https://mistral.ai/) · [Moonshot](https://www.moonshot.cn/) · [OpenPipe](https://openpipe.ai/) · [OpenRouter](https://openrouter.ai/) · [Perplexity](https://www.perplexity.ai/) · [Together AI](https://www.together.ai/) · [xAI](https://x.ai/) |
| LLM services | [Alibaba](https://www.alibabacloud.com/en/product/modelstudio) · [DeepSeek](https://deepseek.com) · [Groq](https://wow.groq.com/) · [Mistral](https://mistral.ai/) · [Moonshot](https://www.moonshot.cn/) · [OpenPipe](https://openpipe.ai/) · [OpenRouter](https://openrouter.ai/) · [Perplexity](https://www.perplexity.ai/) · [Together AI](https://www.together.ai/) · [xAI](https://x.ai/) · [Z.ai](https://z.ai/) |
| Image services | OpenAI · Google Gemini |
| Speech services | [ElevenLabs](https://elevenlabs.io) (Voice synthesis / cloning) |
| Speech services | [ElevenLabs](https://elevenlabs.io) · [Inworld](https://inworld.ai) · [OpenAI TTS](https://platform.openai.com/docs/guides/text-to-speech) · LocalAI · Browser (Web Speech API) |
### Additional Integrations
@@ -388,4 +389,4 @@ When you open an issue, our custom AI triage system (powered by [Claude Code](ht
MIT License · [Third-Party Notices](src/modules/3rdparty/THIRD_PARTY_NOTICES.md)
**2023-2025** · Enrico Ros × [Big-AGI](https://big-agi.com)
**2023-2026** · Enrico Ros × [Big-AGI](https://big-agi.com)
-3
View File
@@ -2,8 +2,6 @@
#
# For more examples, such running big-AGI alongside a web browsing service, see the `docs/docker` folder.
version: '3.9'
services:
big-agi:
image: ghcr.io/enricoros/big-agi:latest
@@ -11,4 +9,3 @@ services:
- "3000:3000"
env_file:
- .env
command: [ "next", "start", "-p", "3000" ]
+3 -2
View File
@@ -29,7 +29,8 @@ How to set up AI models and features in big-AGI.
[OpenPipe](https://app.openpipe.ai/settings),
[Perplexity](https://www.perplexity.ai/settings/api),
[TogetherAI](https://api.together.xyz/settings/api-keys),
[xAI](http://x.ai/api)
[xAI](http://x.ai/api),
[Z.ai](https://z.ai/)
- **[Azure OpenAI](config-azure-openai.md)** guide
- **FireworksAI** ([API keys](https://fireworks.ai/account/api-keys), via custom OpenAI endpoint: https://api.fireworks.ai/inference)
- **[OpenRouter](config-openrouter.md)** guide
@@ -43,7 +44,7 @@ How to set up AI models and features in big-AGI.
- **[Web Browsing](config-feature-browse.md)**: Enable web page download through third-party services or your own cloud
- **Web Search**: Google Search API (see '[Environment Variables](environment-variables.md)')
- **Image Generation**: GPT Image (gpt-image-1), DALL·E 3 and 2
- **Voice Synthesis**: ElevenLabs API for voice generation
- **Voice Synthesis**: ElevenLabs, Inworld, OpenAI TTS, LocalAI, or browser Web Speech API
## Deployment & Customization
+55
View File
@@ -0,0 +1,55 @@
# Google Drive Integration
Attach files from Google Drive directly in the chat composer.
## Setup
### 1. Enable APIs
In [Google Cloud Console](https://console.cloud.google.com/):
1. Go to **APIs & Services > Library**
2. Enable **Google Drive API** and **Google Picker API**
### 2. Configure OAuth
1. Go to **APIs & Services > OAuth consent screen**
2. Create consent screen (External or Internal)
3. Add scope: `https://www.googleapis.com/auth/drive.file`
4. Add test users if in testing mode
### 3. Create Credentials
1. Go to **APIs & Services > Credentials**
2. Create **OAuth client ID** (Web application)
3. Add JavaScript origins:
- `http://localhost:3000` (dev)
- `https://your-domain.com` (prod)
### 4. Set Environment Variable
```bash
NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID=your-client-id.apps.googleusercontent.com
```
## Usage
- Click **Drive** button in attachment menu
## Supported Files
| Type | Export Format |
|-----------------|---------------------|
| Regular files | Downloaded directly |
| Google Docs | Markdown (.md) |
| Google Sheets | CSV (.csv) |
| Google Slides | PDF (.pdf) |
| Google Drawings | SVG (.svg) |
## Troubleshooting
**Picker won't open**: Check `NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID` is set and APIs are enabled.
**OAuth errors**: Verify your domain is in authorized JavaScript origins. Add yourself as test user if app is in testing mode.
**Download fails**: Check file permissions and that Drive API is enabled.
@@ -19,7 +19,6 @@ services:
- .env
environment:
- PUPPETEER_WSS_ENDPOINT=ws://browserless:3000
command: [ "next", "start", "-p", "3000" ]
depends_on:
- browserless
+6 -3
View File
@@ -66,8 +66,9 @@ HTTP_BASIC_AUTH_PASSWORD=
# Frontend variables
NEXT_PUBLIC_MOTD=
NEXT_PUBLIC_GA4_MEASUREMENT_ID=
NEXT_PUBLIC_POSTHOG_KEY=
NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID=
NEXT_PUBLIC_PLANTUML_SERVER_URL=
NEXT_PUBLIC_POSTHOG_KEY=
```
## Backend Variables
@@ -132,10 +133,11 @@ Enable the app to Talk, Draw, and Google things up.
| Variable | Description |
|:---------------------------|:------------------------------------------------------------------------------------------------------------------------|
| **Text-To-Speech** | [ElevenLabs](https://elevenlabs.io/) is a high quality speech synthesis service |
| **Text-To-Speech** | ElevenLabs, Inworld, OpenAI TTS, LocalAI, and browser Web Speech API are supported |
| `ELEVENLABS_API_KEY` | ElevenLabs API Key - used for calls, etc. |
| `ELEVENLABS_API_HOST` | Custom host for ElevenLabs |
| `ELEVENLABS_VOICE_ID` | Default voice ID for ElevenLabs |
| | *Note: OpenAI TTS and LocalAI TTS reuse credentials from your configured LLM services (no separate env vars needed)* |
| **Google Custom Search** | [Google Programmable Search Engine](https://programmablesearchengine.google.com/about/) produces links to pages |
| `GOOGLE_CLOUD_API_KEY` | Google Cloud API Key, used with the '/react' command - [Link to GCP](https://console.cloud.google.com/apis/credentials) |
| `GOOGLE_CSE_ID` | Google Custom/Programmable Search Engine ID - [Link to PSE](https://programmablesearchengine.google.com/) |
@@ -154,8 +156,9 @@ The value of these variables are passed to the frontend (Web UI) - make sure the
| `NEXT_PUBLIC_DEBUG_BREAKS` | (optional, development) When set to 'true', enables automatic debugger breaks on DEV/error/critical logs in development builds |
| `NEXT_PUBLIC_MOTD` | Message of the Day - displays a dismissible banner at the top of the app (see [customizations](customizations.md) for the template variables). Example: 🔔 Welcome to our deployment! Version {{app_build_pkgver}} built on {{app_build_time}}. |
| `NEXT_PUBLIC_GA4_MEASUREMENT_ID` | (optional) The measurement ID for Google Analytics 4. (see [deploy-analytics](deploy-analytics.md)) |
| `NEXT_PUBLIC_POSTHOG_KEY` | (optional) Key for PostHog analytics. (see [deploy-analytics](deploy-analytics.md)) |
| `NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID` | (optional) Google OAuth Client ID for Drive Picker. Can reuse `AUTH_GOOGLE_ID`. See [Google Drive](config-feature-google-drive.md) |
| `NEXT_PUBLIC_PLANTUML_SERVER_URL` | The URL of the PlantUML server, used for rendering UML diagrams. Allows using custom local servers. |
| `NEXT_PUBLIC_POSTHOG_KEY` | (optional) Key for PostHog analytics. (see [deploy-analytics](deploy-analytics.md)) |
> Important: these variables must be set at build time, which is required by Next.js to pass them to the frontend.
> This is in contrast to the backend variables, which can be set when starting the local server/container.
-5
View File
@@ -136,11 +136,6 @@ Deploy big-AGI on a Kubernetes cluster for enhanced scalability and management.
For more detailed instructions on Kubernetes deployment, including updating and troubleshooting, refer to our [Kubernetes Deployment Guide](deploy-k8s.md).
### Midori AI Subsystem for Docker Deployment
Follow the instructions found on [Midori AI Subsystem Site](https://io.midori-ai.xyz/subsystem/manager/)
for your host OS. After completing the setup process, install the Big-AGI docker backend to the Midori AI Subsystem.
## Enterprise-Grade Installation
For businesses seeking a fully-managed, scalable solution, consider our managed installations.
+1
View File
@@ -37,6 +37,7 @@ Built with tRPC, it manages the lifecycle of AI-generated content from request t
| Perplexity | ✅ | ❌ (rejected) | | ✅ | Yes + 📦 | |
| TogetherAI | ✅ | ✅ | | ✅ | Yes + 📦 | |
| xAI | | | | | | |
| Z.ai | ✅ | ✅ | Img: ✅ | ✅ | Yes + 📦 | Thinking mode |
| Ollama (2) | ❌ (broken) | ? | | | | |
Notes:
+5 -16
View File
@@ -13,12 +13,9 @@ The LLM parameters system operates across five layers that transform parameters
The `DModelParameterRegistry` defines all available parameters with their constraints and metadata. Each parameter includes type information, validation rules, and default behavior.
**Example**: `llmVndOaiReasoningEffort4` defines a 4-value enum with 'medium' as the required fallback.
**Default Value System**: The registry supports multiple default mechanisms:
- `initialValue` - Parameter's base default (e.g., `llmVndOaiRestoreMarkdown: true`)
- `requiredFallback` - Fallback for required parameters (e.g., `llmTemperature: 0.5`)
- `nullable` - Parameters that can be explicitly null to skip API transmission
- `initialValue` - Parameter's base default (e.g., `llmVndOaiRestoreMarkdown: true`)
### Layer 2: Model Specifications
**File**: `src/modules/llms/server/llm.server.types.ts`
@@ -27,7 +24,6 @@ Models declare which parameters they support through `parameterSpecs` arrays. Ea
```typescript
parameterSpecs: [
{ paramId: 'llmVndOaiReasoningEffort4' },
{ paramId: 'llmVndAntThinkingBudget', initialValue: 1024 }, // Override default
{ paramId: 'llmVndGeminiThinkingBudget', rangeOverride: [0, 8192] }, // Custom range
]
@@ -51,20 +47,14 @@ Shows only parameters that are:
- Not marked as `hidden`
**Value Resolution**: Both UIs use `getAllModelParameterValues()` to merge:
1. **Fallback values** - Required parameters get their `requiredFallback` values
1. **Fallback values** - Implicit parameters get their `runtimeFallback` values
2. **Initial values** - Model's `initialParameters` (populated during model creation)
3. **User values** - User's `userParameters` (highest priority)
### Layer 4: AIX Translation
**File**: `src/modules/aix/client/aix.client.ts`
The AIX client transforms DLLM parameters to wire protocol format. This layer handles parameter precedence rules and name transformations:
```
// Parameter precedence: newer 4-value version takes priority over 3-value
...((llmVndOaiReasoningEffort4 || llmVndOaiReasoningEffort) ?
{ vndOaiReasoningEffort: llmVndOaiReasoningEffort4 || llmVndOaiReasoningEffort } : {})
```
The AIX client transforms DLLM parameters to wire protocol format. This layer handles parameter precedence rules and name transformations.
**Client Options**: The system supports parameter overrides through `llmOptionsOverride` and complete replacement via `llmUserParametersReplacement`.
@@ -73,7 +63,7 @@ The AIX client transforms DLLM parameters to wire protocol format. This layer ha
Server-side adapters translate AIX parameters to vendor APIs. Each vendor may interpret parameters differently:
- **OpenAI**: `vndOaiReasoningEffort``reasoning_effort`
- **OpenAI**: `vndEffort``reasoning_effort`
- **Perplexity**: Reuses OpenAI parameter format
- **OpenAI Responses API**: Maps to structured reasoning config with additional logic
@@ -105,7 +95,7 @@ When a model is loaded:
The system maintains type safety through:
- `DModelParameterId` union from registry keys
- `DModelParameterValue<T>` conditional types for values
- `DModelParameterSpec<T>` interfaces for specifications
- `DModelParameterSpecAny` interfaces for specifications
- Runtime validation via Zod schemas at API boundaries
## Model Variant Pattern
@@ -117,7 +107,6 @@ Some vendors use model variants to enable features, for instance:
## Migration and Compatibility
The architecture supports parameter evolution:
- **Version Coexistence**: Both `llmVndOaiReasoningEffort` and `llmVndOaiReasoningEffort4` exist simultaneously
- **Precedence Rules**: Newer parameters take priority during AIX translation
- **Graceful Degradation**: Unknown parameters log warnings but don't break functionality
+1 -1
View File
@@ -6,7 +6,7 @@ Client-Side Fetch (CSF) enables direct browser-to-API communication, bypassing t
CSF is implemented as an opt-in setting stored as `csf: boolean` in each vendor's service settings. The vendor interface exposes `csfAvailable?: (setup) => boolean` to determine if CSF can be enabled (typically checking if an API key or host is configured). The actual execution happens in `aix.client.direct-chatGenerate.ts` which dynamically imports when CSF is active, making direct fetch calls using the same wire protocols as the server.
All 16 supported vendors (OpenAI, Anthropic, Gemini, Ollama, LocalAI, Deepseek, Groq, Mistral, xAI, OpenRouter, Perplexity, Together AI, Alibaba, Moonshot, OpenPipe, LM Studio) support CSF. Cloud vendors require CORS support from the API provider (all tested vendors return `access-control-allow-origin: *`). Local vendors (Ollama, LocalAI, LM Studio) require CORS to be enabled on the local server.
All 17 supported vendors (OpenAI, Anthropic, Gemini, Ollama, LocalAI, Deepseek, Groq, Mistral, xAI, OpenRouter, Perplexity, Together AI, Alibaba, Moonshot, OpenPipe, LM Studio, Z.ai) support CSF. Cloud vendors require CORS support from the API provider (all tested vendors return `access-control-allow-origin: *`). Local vendors (Ollama, LocalAI, LM Studio) require CORS to be enabled on the local server.
## UI
+1626 -956
View File
File diff suppressed because it is too large Load Diff
+24 -21
View File
@@ -1,6 +1,6 @@
{
"name": "big-agi",
"version": "2.0.2",
"version": "2.0.3",
"private": true,
"author": "Enrico Ros <enrico.ros@gmail.com>",
"repository": "https://github.com/enricoros/big-agi",
@@ -12,6 +12,7 @@
"start": "next start",
"lint": "next lint",
"postinstall": "prisma generate --no-hints",
"gen:icon-sprites": "node tools/develop/gen-icon-sprites/generate-llm-sprites.ts",
"db:push": "prisma db push",
"db:studio": "prisma studio",
"vercel:env:pull": "npx vercel env pull .env.development.local",
@@ -29,38 +30,39 @@
"@emotion/react": "^11.14.0",
"@emotion/server": "^11.11.0",
"@emotion/styled": "^11.14.1",
"@googleworkspace/drive-picker-react": "^0.2.0",
"@mui/icons-material": "^5.18.0",
"@mui/joy": "^5.0.0-beta.52",
"@next/bundle-analyzer": "~15.1.8",
"@next/bundle-analyzer": "~15.1.12",
"@prisma/client": "~5.22.0",
"@tanstack/react-query": "5.90.10",
"@tanstack/react-virtual": "^3.13.12",
"@tanstack/react-virtual": "^3.13.18",
"@trpc/client": "11.5.1",
"@trpc/next": "11.5.1",
"@trpc/react-query": "11.5.1",
"@trpc/server": "11.5.1",
"@vercel/analytics": "^1.5.0",
"@vercel/speed-insights": "^1.2.0",
"@vercel/analytics": "^1.6.1",
"@vercel/speed-insights": "^1.3.1",
"browser-fs-access": "^0.38.0",
"cheerio": "^1.1.2",
"csv-stringify": "^6.6.0",
"dexie": "~4.0.11",
"dexie-react-hooks": "~1.1.7",
"diff": "^8.0.2",
"eventemitter3": "^5.0.1",
"diff": "^8.0.3",
"eventemitter3": "^5.0.4",
"idb-keyval": "^6.2.2",
"mammoth": "^1.11.0",
"nanoid": "^5.1.6",
"next": "~15.1.8",
"next": "~15.1.12",
"nprogress": "^0.2.0",
"pdfjs-dist": "5.4.54",
"posthog-js": "^1.298.1",
"posthog-node": "^5.14.0",
"posthog-js": "^1.341.0",
"posthog-node": "^5.24.10",
"prismjs": "^1.30.0",
"puppeteer-core": "^24.31.0",
"puppeteer-core": "^24.36.1",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-hook-form": "^7.66.1",
"react-hook-form": "^7.71.1",
"react-markdown": "^10.1.0",
"react-player": "^3.4.0",
"react-resizable-panels": "^3.0.6",
@@ -71,29 +73,30 @@
"remark-math": "^6.0.0",
"sharp": "^0.34.5",
"superjson": "^2.2.6",
"tesseract.js": "^6.0.1",
"tesseract.js": "^7.0.0",
"tiktoken": "^1.0.22",
"turndown": "^7.2.2",
"zod": "^4.1.13",
"zod": "^4.3.6",
"zustand": "5.0.7"
},
"devDependencies": {
"@posthog/nextjs-config": "^1.6.0",
"@types/node": "^24.10.1",
"@posthog/nextjs-config": "~1.6.4",
"@types/node": "^25.2.0",
"@types/nprogress": "^0.2.3",
"@types/prismjs": "^1.26.5",
"@types/react": "^19.2.7",
"@types/react": "^19.2.11",
"@types/react-csv": "^1.1.10",
"@types/react-dom": "^19.2.3",
"@types/turndown": "^5.0.6",
"cross-env": "^10.1.0",
"eslint": "^9.39.1",
"eslint-config-next": "~15.1.8",
"prettier": "^3.6.2",
"eslint": "^9.39.2",
"eslint-config-next": "~15.1.12",
"prettier": "^3.8.1",
"prisma": "~5.22.0",
"tsx": "^4.21.0",
"typescript": "^5.9.3"
},
"engines": {
"node": "^26.0.0 || ^24.0.0 || ^22.0.0 || ^20.0.0"
"node": "^24.0.0 || ^22.0.0 || ^20.0.0"
}
}
+1 -1
View File
@@ -58,7 +58,7 @@ export function CallWizard(props: { strict?: boolean, conversationId: string | n
// derived state
const outOfTheBlue = !props.conversationId;
const overriddenEmptyChat = chatEmptyOverride || !chatIsEmpty;
const overriddenEmptyChat = outOfTheBlue || chatEmptyOverride || !chatIsEmpty;
const overriddenRecognition = recognitionOverride || recognition.mayWork;
const synthesisShallWork = !!speexGlobalEngine;
const allGood = overriddenEmptyChat && overriddenRecognition && synthesisShallWork;
+2 -1
View File
@@ -24,6 +24,7 @@ import { OptimaPanelGroupedList } from '~/common/layout/optima/panel/OptimaPanel
import { OptimaPanelIn, OptimaToolbarIn } from '~/common/layout/optima/portals/OptimaPortalsIn';
import { PhVoice } from '~/common/components/icons/phosphor/PhVoice';
import { SpeechResult, useSpeechRecognition } from '~/common/components/speechrecognition/useSpeechRecognition';
import { clipboardInterceptCtrlCForCleanup } from '~/common/util/clipboardUtils';
import { conversationTitle, remapMessagesSysToUsr } from '~/common/stores/chat/chat.conversation';
import { createDMessageFromFragments, createDMessageTextContent, DMessage, messageFragmentsReduceText, messageWasInterruptedAtStart } from '~/common/stores/chat/chat.message';
import { createErrorContentFragment } from '~/common/stores/chat/chat.fragments';
@@ -360,7 +361,7 @@ export function Telephone(props: {
<ScrollToBottom stickToBottomInitial>
<Box sx={{ minHeight: '100%', p: 1, display: 'flex', flexDirection: 'column', gap: 1 }}>
<Box onCopy={clipboardInterceptCtrlCForCleanup} sx={{ minHeight: '100%', p: 1, display: 'flex', flexDirection: 'column', gap: 1 }}>
{/* Call Messages [] */}
{callMessages.map((message) =>
+2 -1
View File
@@ -13,6 +13,7 @@ import type { ConversationHandler } from '~/common/chat-overlay/ConversationHand
import type { DLLMContextTokens } from '~/common/stores/llms/llms.types';
import { DConversationId, excludeSystemMessages } from '~/common/stores/chat/chat.conversation';
import { ShortcutKey, useGlobalShortcuts } from '~/common/components/shortcuts/useGlobalShortcuts';
import { clipboardInterceptCtrlCForCleanup } from '~/common/util/clipboardUtils';
import { convertFilesToDAttachmentFragments } from '~/common/attachment-drafts/attachment.pipeline';
import { createDMessageFromFragments, createDMessageTextContent, DMessage, DMessageId, DMessageUserFlag, DMetaReferenceItem, MESSAGE_FLAG_AIX_SKIP, messageHasUserFlag } from '~/common/stores/chat/chat.message';
import { createTextContentFragment, DMessageFragment, DMessageFragmentId } from '~/common/stores/chat/chat.fragments';
@@ -323,7 +324,7 @@ export function ChatMessageList(props: {
);
return (
<List role='chat-messages-list' sx={listSx}>
<List role='chat-messages-list' sx={listSx} onCopy={clipboardInterceptCtrlCForCleanup}>
{props.isMessageSelectionMode && (
<MessagesSelectionHeader
@@ -220,7 +220,7 @@ export function CameraCaptureModal(props: {
backdropFilter: 'none', // using none because this is heavy
// backdropFilter: 'blur(4px)',
// backgroundColor: 'rgba(11 13 14 / 0.75)',
backgroundColor: 'rgba(var(--joy-palette-neutral-darkChannel) / 0.5)',
backgroundColor: 'rgba(var(--joy-palette-neutral-darkChannel) / 0.67)',
},
},
}}
+24 -5
View File
@@ -17,7 +17,8 @@ import { useChatAutoSuggestAttachmentPrompts, useChatMicTimeoutMsValue } from '.
import { useAgiAttachmentPrompts } from '~/modules/aifn/agiattachmentprompts/useAgiAttachmentPrompts';
import { useBrowseCapability } from '~/modules/browse/store-module-browsing';
import { DLLM, getLLMContextTokens, getLLMPricing, LLM_IF_OAI_Vision } from '~/common/stores/llms/llms.types';
import { DLLM, getLLMContextTokens, LLM_IF_OAI_Vision } from '~/common/stores/llms/llms.types';
import { llmChatPricing_adjusted } from '~/common/stores/llms/llms.pricing';
import { AudioGenerator } from '~/common/util/audio/AudioGenerator';
import { AudioPlayer } from '~/common/util/audio/AudioPlayer';
import { ButtonAttachFilesMemo, openFileForAttaching } from '~/common/components/ButtonAttachFiles';
@@ -34,7 +35,7 @@ import { copyToClipboard, supportsClipboardRead } from '~/common/util/clipboardU
import { createTextContentFragment, DMessageAttachmentFragment, DMessageContentFragment, duplicateDMessageFragments } from '~/common/stores/chat/chat.fragments';
import { glueForMessageTokens, marshallWrapDocFragments } from '~/common/stores/chat/chat.tokens';
import { isValidConversation, useChatStore } from '~/common/stores/chat/store-chats';
import { getModelParameterValueOrThrow } from '~/common/stores/llms/llms.parameters';
import { getModelParameterValueWithFallback } from '~/common/stores/llms/llms.parameters';
import { launchAppCall, removeQueryParam, useRouterQuery } from '~/common/app.routes';
import { lineHeightTextareaMd, themeBgAppChatComposer } from '~/common/app.theme';
import { optimaOpenPreferences } from '~/common/layout/optima/useOptima';
@@ -63,8 +64,10 @@ import { chatExecuteModeCanAttach, useChatExecuteMode } from '../../execute-mode
import { ButtonAttachCameraMemo, useCameraCaptureModalDialog } from './buttons/ButtonAttachCamera';
import { ButtonAttachClipboardMemo } from './buttons/ButtonAttachClipboard';
import { ButtonAttachGoogleDriveMemo } from './buttons/ButtonAttachGoogleDrive';
import { ButtonAttachScreenCaptureMemo } from './buttons/ButtonAttachScreenCapture';
import { ButtonAttachWebMemo } from './buttons/ButtonAttachWeb';
import { hasGoogleDriveCapability, useGoogleDrivePicker } from '~/common/attachment-drafts/useGoogleDrivePicker';
import { ButtonBeamMemo } from './buttons/ButtonBeam';
import { ButtonCallMemo } from './buttons/ButtonCall';
import { ButtonGroupDrawRepeat } from './buttons/ButtonGroupDrawRepeat';
@@ -197,7 +200,7 @@ export function Composer(props: {
const showChatAttachments = chatExecuteModeCanAttach(chatExecuteMode, props.capabilityHasT2IEdit);
const {
/* items */ attachmentDrafts,
/* append */ attachAppendClipboardItems, attachAppendDataTransfer, attachAppendEgoFragments, attachAppendFile, attachAppendUrl,
/* append */ attachAppendClipboardItems, attachAppendCloudFile, attachAppendDataTransfer, attachAppendEgoFragments, attachAppendFile, attachAppendUrl,
/* take */ attachmentsRemoveAll, attachmentsTakeAllFragments, attachmentsTakeFragmentsByType,
} = useAttachmentDrafts(conversationOverlayStore, enableLoadURLsInComposer, chatLLMSupportsImages, handleFilterAGIFile, showChatAttachments === 'only-images');
@@ -231,9 +234,9 @@ export function Composer(props: {
if (props.chatLLM && tokensComposer > 0)
tokensComposer += glueForMessageTokens(props.chatLLM);
const tokensHistory = _historyTokenCount;
const tokensResponseMax = getModelParameterValueOrThrow('llmResponseTokens', props.chatLLM?.initialParameters, props.chatLLM?.userParameters, 0) ?? 0;
const tokensResponseMax = getModelParameterValueWithFallback('llmResponseTokens', props.chatLLM?.initialParameters, props.chatLLM?.userParameters, 0) ?? 0 /* if null, assume 0*/;
const tokenLimit = getLLMContextTokens(props.chatLLM) ?? 0;
const tokenChatPricing = getLLMPricing(props.chatLLM)?.chat;
const tokenChatPricing = React.useMemo(() => llmChatPricing_adjusted(props.chatLLM), [props.chatLLM]);
// Effect: load initial text if queued up (e.g. by /link/share_targetF)
@@ -545,6 +548,9 @@ export function Composer(props: {
// Enter: primary action
if (e.key === 'Enter') {
// Skip if composing (e.g., CJK input methods) - issue #784
if (e.nativeEvent.isComposing)
return;
// Alt (Windows) or Option (Mac) + Enter: append the message instead of sending it
if (e.altKey && !e.metaKey && !e.ctrlKey) {
@@ -620,6 +626,8 @@ export function Composer(props: {
const { openWebInputDialog, webInputDialogComponent } = useWebInputModal(handleAttachWebLinks, composeText);
const { openGoogleDrivePicker, googleDrivePickerComponent } = useGoogleDrivePicker(attachAppendCloudFile, isMobile);
// Attachments Down
@@ -799,6 +807,11 @@ export function Composer(props: {
<ButtonAttachWebMemo disabled={!hasComposerBrowseCapability} onOpenWebInput={openWebInputDialog} />
</MenuItem>
{/* Responsive Google Drive button */}
{hasGoogleDriveCapability && <MenuItem>
<ButtonAttachGoogleDriveMemo onOpenGoogleDrivePicker={openGoogleDrivePicker} fullWidth />
</MenuItem>}
{/* Responsive Paste button */}
{supportsClipboardRead() && <MenuItem>
<ButtonAttachClipboardMemo onAttachClipboard={attachAppendClipboardItems} />
@@ -828,6 +841,9 @@ export function Composer(props: {
{/* Responsive Web button */}
{showChatAttachments !== 'only-images' && <ButtonAttachWebMemo color={showTint} disabled={!hasComposerBrowseCapability} onOpenWebInput={openWebInputDialog} />}
{/* Responsive Google Drive button */}
{hasGoogleDriveCapability && showChatAttachments !== 'only-images' && <ButtonAttachGoogleDriveMemo color={showTint} onOpenGoogleDrivePicker={openGoogleDrivePicker} />}
{/* Responsive Paste button */}
{supportsClipboardRead() && showChatAttachments !== 'only-images' && <ButtonAttachClipboardMemo color={showTint} onAttachClipboard={attachAppendClipboardItems} />}
@@ -1123,6 +1139,9 @@ export function Composer(props: {
{/* Camera (when open) */}
{cameraCaptureComponent}
{/* Google Drive Picker (when open) */}
{googleDrivePickerComponent}
{/* Web Input Dialog (when open) */}
{webInputDialogComponent}
@@ -0,0 +1,49 @@
import * as React from 'react';
import { Box, Button, ColorPaletteProp, IconButton, Tooltip } from '@mui/joy';
import AddToDriveRoundedIcon from '@mui/icons-material/AddToDriveRounded';
import { buttonAttachSx } from '~/common/components/ButtonAttachFiles';
import { KeyStroke } from '~/common/components/KeyStroke';
export const ButtonAttachGoogleDriveMemo = React.memo(ButtonAttachGoogleDrive);
function ButtonAttachGoogleDrive(props: {
color?: ColorPaletteProp,
isMobile?: boolean,
disabled?: boolean,
fullWidth?: boolean,
noToolTip?: boolean,
onOpenGoogleDrivePicker: () => void,
}) {
const button = props.isMobile ? (
<IconButton color={props.color} disabled={props.disabled} onClick={props.onOpenGoogleDrivePicker}>
<AddToDriveRoundedIcon />
</IconButton>
) : (
<Button
variant={props.color ? 'soft' : 'plain'}
color={props.color || 'neutral'}
disabled={props.disabled}
fullWidth={props.fullWidth}
startDecorator={<AddToDriveRoundedIcon />}
onClick={props.onOpenGoogleDrivePicker}
sx={buttonAttachSx.desktop}
>
Drive
</Button>
);
return (props.noToolTip || props.isMobile) ? button : (
<Tooltip arrow disableInteractive placement='top-start' title={
<Box sx={buttonAttachSx.tooltip}>
<b>Add from Google Drive</b><br />
Attach files from your Drive
</Box>
}>
{button}
</Tooltip>
);
}
@@ -91,8 +91,11 @@ function InputErrorIndicator() {
const converterTypeToIconMap: { [key in AttachmentDraftConverterType]: React.ComponentType<any> | null } = {
'text': TextFieldsIcon,
'text-cleaner': CodeIcon,
'text-markdown': TextFieldsIcon,
'rich-text': CodeIcon,
'rich-text-cleaner': CodeIcon,
'rich-text-markdown': TextFieldsIcon,
'rich-text-table': PivotTableChartIcon,
'image-original': ImageOutlinedIcon,
'image-resized-high': PhotoSizeSelectLargeOutlinedIcon,
@@ -100,8 +103,10 @@ const converterTypeToIconMap: { [key in AttachmentDraftConverterType]: React.Com
'image-to-default': ImageOutlinedIcon,
'image-caption': AbcIcon,
'image-ocr': AbcIcon,
'pdf-auto': PictureAsPdfIcon,
'pdf-text': PictureAsPdfIcon,
'pdf-images': PermMediaOutlinedIcon,
'pdf-images-ocr': AbcIcon,
'pdf-text-and-images': PermMediaOutlinedIcon,
'docx-to-html': DescriptionOutlinedIcon,
'url-page-text': TextFieldsIcon, // was LanguageIcon
@@ -199,13 +204,21 @@ function attachmentIcons(attachmentDraft: AttachmentDraft, noTooltips: boolean,
function attachmentLabelText(attachmentDraft: AttachmentDraft): string {
const converter = attachmentDraft.converters.find(c => c.isActive) ?? null;
if (converter && attachmentDraft.label === 'Rich Text') {
if (converter.id === 'rich-text-table')
return 'Rich Table';
if (converter.id === 'rich-text-cleaner')
if (converter && attachmentDraft.label === 'Text') {
if (converter.id === 'text-markdown')
return 'Markdown';
if (converter.id === 'text-cleaner')
return 'Clean HTML';
}
if (converter && attachmentDraft.label === 'Rich Text') {
if (converter.id === 'rich-text')
return 'Rich HTML';
if (converter.id === 'rich-text-markdown')
return 'Markdown';
if (converter.id === 'rich-text-cleaner')
return 'Clean HTML';
if (converter.id === 'rich-text-table')
return 'Rich Table';
}
return ellipsizeFront(attachmentDraft.label, 22);
}
@@ -228,9 +241,10 @@ function LLMAttachmentButton(props: {
const isUnconvertible = !draft.converters.length;
const isOutputLoading = draft.outputsConverting;
const isOutputMissing = !draft.outputFragments.length;
const isOutputWarned = !!draft.outputWarnings?.length;
const hasLiveFiles = draft.outputFragments.some(_f => _f.liveFileId);
const showWarning = isUnconvertible || (isOutputMissing || !llmSupportsAllFragments);
const showWarning = isUnconvertible || (isOutputMissing || !llmSupportsAllFragments) || isOutputWarned;
// handlers
@@ -257,6 +271,17 @@ function LLMAttachmentButton(props: {
if (isInputLoading)
return <InputLoadingPlaceholder label={draft.label} />;
// tooltip for truncated filenames (only show when menu is closed)
const displayedLabel = attachmentLabelText(draft);
const showFilenameTooltip = !props.menuShown && !isOutputLoading && displayedLabel !== draft.label;
// label element (reused with/without tooltip)
const labelElement = (
<Typography level='title-sm' sx={{ whiteSpace: 'nowrap' }}>
{isOutputLoading ? 'Converting... ' : displayedLabel}
</Typography>
);
return (
<Button
size='sm'
@@ -280,10 +305,11 @@ function LLMAttachmentButton(props: {
{/* Icons: Web Page Screenshot, Converter[s] */}
{attachmentIcons(draft, props.menuShown, props.onViewImageRefPart)}
{/* Label */}
<Typography level='title-sm' sx={{ whiteSpace: 'nowrap' }}>
{isOutputLoading ? 'Converting... ' : attachmentLabelText(draft)}
</Typography>
{/* Label (with tooltip for truncated filenames) */}
{showFilenameTooltip
? <TooltipOutlined title={<span style={{ wordBreak: 'break-all' }}>{draft.label}</span>}>{labelElement}</TooltipOutlined>
: labelElement
}
{/* Is Converting icon */}
{isOutputLoading && <CircularProgress color='success' size='sm' />}
@@ -1,16 +1,15 @@
import * as React from 'react';
import type { SxProps } from '@mui/joy/styles/types';
import { Box, Checkbox, Chip, CircularProgress, LinearProgress, ListDivider, ListItem, ListItemDecorator, MenuItem, Radio, Typography } from '@mui/joy';
import AttachmentIcon from '@mui/icons-material/Attachment';
import { Box, Button, ButtonGroup, Checkbox, Chip, CircularProgress, Divider, LinearProgress, ListDivider, ListItem, ListItemDecorator, MenuItem, Radio, Typography } from '@mui/joy';
import ClearIcon from '@mui/icons-material/Clear';
import ContentCopyIcon from '@mui/icons-material/ContentCopy';
import DeleteForeverIcon from '@mui/icons-material/DeleteForever';
import DeleteOutlineIcon from '@mui/icons-material/DeleteOutline';
import ExpandLessIcon from '@mui/icons-material/ExpandLess';
import ExpandMoreIcon from '@mui/icons-material/ExpandMore';
import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown';
import KeyboardArrowLeftIcon from '@mui/icons-material/KeyboardArrowLeft';
import KeyboardArrowRightIcon from '@mui/icons-material/KeyboardArrowRight';
import ReadMoreIcon from '@mui/icons-material/ReadMore';
import VerticalAlignBottomIcon from '@mui/icons-material/VerticalAlignBottom';
import VisibilityIcon from '@mui/icons-material/Visibility';
@@ -18,6 +17,7 @@ import { CloseablePopup } from '~/common/components/CloseablePopup';
import { DMessageAttachmentFragment, DMessageDocPart, DMessageImageRefPart, isDocPart, isImageRefPart, isZyncAssetImageReferencePartWithLegacyDBlob } from '~/common/stores/chat/chat.fragments';
import { LiveFileIcon } from '~/common/livefile/liveFile.icons';
import { copyToClipboard } from '~/common/util/clipboardUtils';
import { humanReadableBytes } from '~/common/util/textUtils';
import { themeZIndexOverMobileDrawer } from '~/common/app.theme';
import { useUIPreferencesStore } from '~/common/stores/store-ui';
@@ -32,12 +32,20 @@ const DEFAULT_DETAILS_OPEN = true;
const SHOW_INLINING_OPERATIONS = false;
const indicatorSx = {
fontSize: '1rem',
} as const;
// const indicatorSx = {
// fontSize: '1rem',
// } as const;
//
// const indicatorGapSx: SxProps = {
// paddingLeft: '1.375rem',
// };
const indicatorGapSx: SxProps = {
paddingLeft: '1.375rem',
const actionButtonsSx: SxProps = {
ml: 'auto',
minHeight: 0,
borderRadius: '1rem',
backgroundColor: 'background.surface',
'& button': { fontSize: 'xs', fontWeight: 'md', py: 0, minWidth: 0, minHeight: 0 },
};
@@ -82,9 +90,10 @@ export function LLMAttachmentMenu(props: {
const isUnconvertible = !draft.converters.length;
const isOutputMissing = !draft.outputFragments.length;
const isOutputMultiple = draft.outputFragments.length > 1;
const isOutputWarned = !!draft.outputWarnings?.length;
const hasLiveFiles = draft.outputFragments.some(_f => _f.liveFileId);
const showWarning = isUnconvertible || isOutputMissing || !llmSupportsAllFragments;
const showWarning = isUnconvertible || isOutputMissing || !llmSupportsAllFragments || isOutputWarned;
// hooks
@@ -157,6 +166,8 @@ export function LLMAttachmentMenu(props: {
minWidth={260}
noTopPadding
placement='top'
placementOffset={[0, 15]}
boxShadow='lg'
zIndex={themeZIndexOverMobileDrawer /* was not set, but the Attachment Menu can be used from the Personas Modal */}
>
@@ -187,9 +198,10 @@ export function LLMAttachmentMenu(props: {
<ListItem sx={{ fontSize: 'sm', my: 0.75 }}>
Attach {draftSource.media === 'url' ? 'web page'
: draftSource.media === 'file' ? 'file'
: draftSource.media === 'text'
? (draftSource.method === 'drop' ? 'drop' : draftSource.method === 'clipboard-read' ? 'clipboard' : draftSource.method === 'paste' ? 'paste' : '')
: ''} as:
: draftSource.media === 'cloud' ? 'cloud file'
: draftSource.media === 'text'
? (draftSource.method === 'drop' ? 'drop' : draftSource.method === 'clipboard-read' ? 'clipboard' : draftSource.method === 'paste' ? 'paste' : '')
: ''} as:
{uiComplexityMode === 'extra' && (
<Chip component='span' size='sm' color='neutral' variant='outlined' startDecorator={<ContentCopyIcon />} onClick={(event) => handleCopyLabelToClipboard(event, draft.label)} sx={{ ml: 'auto' }}>
copy name
@@ -197,6 +209,17 @@ export function LLMAttachmentMenu(props: {
)}
</ListItem>
)}
{/* Auto-heuristics message, with explanation */}
{!!draft.outputsHeuristic?.isAuto && (
<ListItem color={draft.outputsHeuristic.isAuto ? 'primary' : undefined} sx={{ fontSize: 'sm', fontWeight: 'lg', mb: 0.5 }}>
{draft.outputsHeuristic.isAuto ? 'Auto: ' : ''}
{draft.outputsHeuristic.actualConverterId === 'pdf-text' && 'Text'}
{draft.outputsHeuristic.actualConverterId === 'pdf-images-ocr' && 'OCR'}
{draft.outputsHeuristic.actualConverterId === 'pdf-images' && 'Images'}
{draft.outputsHeuristic.actualConverterId === 'pdf-text-and-images' && 'Text + Images'}
{draft.outputsHeuristic.explain && ` (${draft.outputsHeuristic.explain})`}
</ListItem>
)}
{!isUnconvertible && draft.converters.map((c, idx) =>
<MenuItem
disabled={c.disabled || isConverting}
@@ -213,7 +236,9 @@ export function LLMAttachmentMenu(props: {
</ListItemDecorator>
{c.unsupported
? <Box>Unsupported 🤔 <Typography level='body-xs'>{c.name}</Typography></Box>
: c.name}
: (/* auto-converted */ draft.outputsHeuristic?.isAuto && c.id === draft.outputsHeuristic.actualConverterId)
? <Box component='span' sx={{ fontWeight: 'lg', color: 'primary.softColor' }}>{c.name}</Box>
: c.name}
</MenuItem>,
)}
{/*{!isUnconvertible && <ListDivider sx={{ mb: 0 }} />}*/}
@@ -261,11 +286,19 @@ export function LLMAttachmentMenu(props: {
<Typography color={isInputError ? 'danger' : 'warning'} level='title-sm'>
{isInputError ? 'Loading Issue' : 'Warning'}
</Typography>
{/* Only show 1 warning, excluding lower priorities */}
{isInputError ? <div>{draft.inputError}</div>
: isUnconvertible ? <div>Attachments of type {draft.input?.mimeType} are not supported yet. You can request this on GitHub.</div>
: isOutputMissing ? <div>File not supported. Please try another format.</div>
: !llmSupportsAllFragments ? <div>May not be compatible with the current model. Please try another format.</div>
: <>Unknown warning</>}
: draft.outputWarnings?.length ? '' /* printed below */
: <>Unknown warning</>}
{/* Explicit output warnings */}
{!!draft.outputWarnings?.length && draft.outputWarnings.map((w, widx) =>
<Box key={'ow-' + widx} sx={{ fontSize: 'sm', color: 'warning.softColor', py: 1 }}> {w}</Box>)
}
</Box>
</MenuItem>
</Box>
@@ -294,24 +327,24 @@ export function LLMAttachmentMenu(props: {
Details
</Typography>
) : (
<Box sx={{ my: 0.5 }}>
<Box sx={{ my: 1 }}>
{/* <- inputs */}
{showInputs && !!draftInput && (
<Typography level='body-sm' textColor='text.primary' startDecorator={<AttachmentIcon sx={indicatorSx} />}>
{draftInput.mimeType}{typeof draftInput.dataSize === 'number' ? ` · ${draftInput.dataSize.toLocaleString()} bytes` : ''}
<Typography level='body-sm' textColor='success.softColor'>
Input: {draftInput.mimeType}{typeof draftInput.dataSize === 'number' ? ` · ${humanReadableBytes(draftInput.dataSize)}` : ''}
</Typography>
)}
{showInputs && !!draftInput?.altMimeType && (
<Typography level='body-sm' sx={indicatorGapSx}>
{draftInput.altMimeType} · {draftInput.altData?.length.toLocaleString()}
<Typography level='body-sm' textColor='success.softColor'>
Input: {draftInput.altMimeType}{!draftInput.altData?.length ? '' : ` · ${humanReadableBytes(draftInput.altData.length)}`}
</Typography>
)}
{showInputs && !!draftInput?.urlImage && (
<Typography level='body-sm' sx={indicatorGapSx}>
{draftInput.urlImage.mimeType} · {draftInput.urlImage.width} x {draftInput.urlImage.height} · {draftInput.urlImage.imgDataUrl?.length.toLocaleString()}
{' · '}
<Chip component='span' size='sm' color='primary' variant='outlined' startDecorator={<VisibilityIcon />} onClick={(event) => {
<Typography level='body-sm' textColor='success.softColor' sx={{ display: 'flex', alignItems: 'center' }}>
Input: {draftInput.urlImage.mimeType} · {draftInput.urlImage.width}x{draftInput.urlImage.height}{!draftInput.urlImage.imgDataUrl?.length ? '' : ` · ${humanReadableBytes(draftInput.urlImage.imgDataUrl.length)}`}
&nbsp;
<Chip component='span' size='sm' color='success' variant='soft' startDecorator={<VisibilityIcon />} onClick={(event) => {
if (draftInput?.urlImage?.imgDataUrl) {
// Invoke the viewer but with a virtual 'temp' part description to see this preview image
handleViewImageRefPart(event, {
@@ -325,8 +358,8 @@ export function LLMAttachmentMenu(props: {
height: draftInput.urlImage.height || undefined,
});
}
}}>
view
}} sx={{ ml: 'auto' }}>
view input
</Chip>
</Typography>
)}
@@ -335,45 +368,79 @@ export function LLMAttachmentMenu(props: {
{/* Converters: {draft.converters.map(((converter, idx) => ` ${converter.id}${converter.isActive ? '*' : ''}`)).join(', ')}*/}
{/*</Typography>*/}
{/* Downward arrow */}
<Divider color='success'>
<KeyboardArrowDownIcon color='success' />
</Divider>
{/* -> Outputs */}
<Box sx={{ mt: 1 }}>
<Box>
{isOutputMissing ? (
<Typography level='body-sm' startDecorator={<ReadMoreIcon sx={indicatorSx} />}>...</Typography>
<Typography level='body-sm' color={isConverting ? 'primary' : 'danger'}>{isConverting ? '...' : '... nothing ...'}</Typography>
) : (
draft.outputFragments.map(({ part }, index) => {
if (isDocPart(part)) {
return (
<Typography key={index} level='body-sm' sx={{ color: 'text.primary' }} startDecorator={<ReadMoreIcon sx={indicatorSx} />}>
<span>{part.data.mimeType /* part.type: big-agi type, not source mime */} · {part.data.text.length.toLocaleString()} bytes ·&nbsp;</span>
<Chip component='span' size='sm' color='primary' variant='outlined' startDecorator={<VisibilityIcon />} onClick={(event) => handleViewDocPart(event, part)}>
view
</Chip>
<Chip component='span' size='sm' color='success' variant='outlined' startDecorator={<ContentCopyIcon />} onClick={(event) => handleCopyToClipboard(event, part.data.text)}>
copy
</Chip>
<Typography key={index} component='div' level='body-sm' textColor='primary.softColor' sx={{ display: 'flex', alignItems: 'center' }}>
<span>{part.data.mimeType /* part.type: big-agi type, not source mime */} · {humanReadableBytes(part.data.text.length)} &nbsp;</span>
{/*<Chip component='span' size='sm' color='primary' variant='outlined' startDecorator={<VisibilityIcon />} onClick={(event) => handleViewDocPart(event, part)} sx={{ ml: 'auto' }}>*/}
{/* view*/}
{/*</Chip>*/}
{/*<Chip component='span' size='sm' color='primary' variant='outlined' startDecorator={<ContentCopyIcon />} onClick={(event) => handleCopyToClipboard(event, part.data.text)}>*/}
{/* copy*/}
{/*</Chip>*/}
<ButtonGroup size='sm' color='primary' variant='outlined' sx={actionButtonsSx}>
<Button startDecorator={<VisibilityIcon sx={{ fontSize: 'md' }} />} onClick={(event) => handleViewDocPart(event, part)}>
view
</Button>
<Button onClick={(event) => handleCopyToClipboard(event, part.data.text)}/* endDecorator={<ContentCopyIcon />} */>
copy
</Button>
</ButtonGroup>
</Typography>
);
} else if (isZyncAssetImageReferencePartWithLegacyDBlob(part) || isImageRefPart(part)) {
// Unified Image Reference handling (both Zync Asset References with legacy fallback and legacy image_ref)
const legacyImageRefPart = isZyncAssetImageReferencePartWithLegacyDBlob(part) ? part._legacyImageRefPart! : part;
const { dataRef, width, height } = legacyImageRefPart;
const resolution = width && height ? `${width} x ${height}` : 'no resolution';
const resolution = width && height ? `${width}x${height}` : 'no resolution';
const mime = dataRef.reftype === 'dblob' ? dataRef.mimeType : 'unknown image';
return (
<Typography key={index} level='body-sm' sx={{ color: 'text.primary' }} startDecorator={<ReadMoreIcon sx={indicatorSx} />}>
<span>{mime /*.replace('image/', 'img: ')*/} · {resolution} · {dataRef.reftype === 'dblob' ? (dataRef.bytesSize?.toLocaleString() || 'no size') : '(remote)'} ·&nbsp;</span>
<Chip component='span' size={isOutputMultiple ? 'sm' : 'md'} color='primary' variant='outlined' startDecorator={<VisibilityIcon />}
onClick={(event) => handleViewImageRefPart(event, legacyImageRefPart)}>
view
</Chip>
{isOutputMultiple && <Chip component='span' size={isOutputMultiple ? 'sm' : 'md'} color='danger' variant='outlined' startDecorator={<DeleteForeverIcon />} onClick={(event) => handleDeleteOutputFragment(event, index)}>
del
</Chip>}
<Typography key={index} component='div' level='body-sm' textColor='primary.softColor' sx={{ display: 'flex', alignItems: 'center' }}>
<span>{mime /*.replace('image/', 'img: ')*/} · {resolution} · {
dataRef.reftype !== 'dblob' ? '(remote)'
: !dataRef.bytesSize ? 'no size'
: humanReadableBytes(dataRef.bytesSize)} &nbsp;</span>
{/*<Chip component='span' size={isOutputMultiple ? 'sm' : 'md'} color='primary' variant='outlined' startDecorator={<VisibilityIcon />}*/}
{/* onClick={(event) => handleViewImageRefPart(event, legacyImageRefPart)}>*/}
{/* view*/}
{/*</Chip>*/}
{/*{isOutputMultiple && <Chip component='span' size={isOutputMultiple ? 'sm' : 'md'} color='danger' variant='outlined' startDecorator={<DeleteForeverIcon />} onClick={(event) => handleDeleteOutputFragment(event, index)}>*/}
{/* del*/}
{/*</Chip>}*/}
<ButtonGroup size='sm' color='primary' variant='outlined' sx={actionButtonsSx}>
<Button
startDecorator={<VisibilityIcon sx={{ fontSize: 'md' }} />}
onClick={(event) => handleViewImageRefPart(event, legacyImageRefPart)}
>
view
</Button>
{isOutputMultiple && (
<Button
color='warning'
endDecorator={<DeleteOutlineIcon sx={{ fontSize: 'md' }} />}
onClick={(event) => handleDeleteOutputFragment(event, index)}
// sx={{ width: 48 }}
>
del
</Button>
)}
</ButtonGroup>
</Typography>
);
} else {
return (
<Typography key={index} level='body-sm' sx={{ color: 'text.primary' }} startDecorator={<ReadMoreIcon sx={indicatorSx} />}>
<Typography key={index} level='body-sm' textColor='primary.softColor'>
{(part as DMessageAttachmentFragment['part']).pt}: (other)
</Typography>
);
@@ -381,8 +448,8 @@ export function LLMAttachmentMenu(props: {
})
)}
{!!llmTokenCountApprox && (
<Typography level='body-xs' mt={0.5} sx={indicatorGapSx}>
~{llmTokenCountApprox.toLocaleString()} tokens
<Typography level='body-xs' mt={0.5} textColor='primary.softColor'>
&nbsp; ~ {llmTokenCountApprox.toLocaleString()} tokens
</Typography>
)}
</Box>
@@ -8,7 +8,7 @@ import SettingsIcon from '@mui/icons-material/Settings';
import { findModelVendor } from '~/modules/llms/vendors/vendors.registry';
import type { DModelsServiceId } from '~/common/stores/llms/llms.service.types';
import { DLLM, DLLMId, isLLMVisible } from '~/common/stores/llms/llms.types';
import { DLLM, DLLMId, getLLMLabel, isLLMVisible } from '~/common/stores/llms/llms.types';
import { DebouncedInputMemo } from '~/common/components/DebouncedInput';
import { GoodTooltip } from '~/common/components/GoodTooltip';
import { KeyStroke } from '~/common/components/KeyStroke';
@@ -65,7 +65,7 @@ function LLMDropdown(props: {
return true;
// filter-out models that don't contain the search string
if (lcFilterString && !llm.label.toLowerCase().includes(lcFilterString))
if (lcFilterString && !getLLMLabel(llm).toLowerCase().includes(lcFilterString))
return false;
// filter-out hidden models from the dropdown
@@ -89,7 +89,7 @@ function LLMDropdown(props: {
// add the model item
llmItems[llm.id] = {
title: llm.label,
title: getLLMLabel(llm),
...(llm.userStarred ? { symbol: '⭐' } : {}),
// icon: llm.id.startsWith('some vendor') ? <VendorIcon /> : undefined,
};
@@ -5,7 +5,7 @@ import { useModuleBeamStore } from '~/modules/beam/store-module-beam';
import type { DFolder } from '~/common/stores/folders/store-chat-folders';
import { DMessage, DMessageUserFlag, MESSAGE_FLAG_STARRED, messageFragmentsReduceText, messageHasUserFlag, messageUserFlagToEmoji } from '~/common/stores/chat/chat.message';
import { conversationTitle, DConversationId } from '~/common/stores/chat/chat.conversation';
import { getLocalMidnightInUTCTimestamp, getTimeBucketEn } from '~/common/util/timeUtils';
import { createTimeBucketClassifierEn } from '~/common/util/timeUtils';
import { isAttachmentFragment, isContentOrAttachmentFragment, isDocPart, isImageRefPart, isZyncAssetImageReferencePart } from '~/common/stores/chat/chat.fragments';
import { shallowEquals } from '~/common/util/hooks/useShallowObject';
import { useChatStore } from '~/common/stores/chat/store-chats';
@@ -235,14 +235,14 @@ export function useChatDrawerRenderItems(
break;
}
const midnightTime = getLocalMidnightInUTCTimestamp();
const getTimeBucket = createTimeBucketClassifierEn();
const grouped = chatNavItems.reduce((acc, item) => {
// derive the bucket name
let bucket: string;
switch (grouping) {
case 'date':
bucket = getTimeBucketEn(item.updatedAt || midnightTime, midnightTime);
bucket = getTimeBucket(item.updatedAt || Date.now());
break;
case 'persona':
bucket = item.systemPurposeId;
@@ -44,7 +44,7 @@ import { Release } from '~/common/app.release';
import { TooltipOutlined } from '~/common/components/TooltipOutlined';
import { adjustContentScaling, themeScalingMap, themeZIndexChatBubble } from '~/common/app.theme';
import { avatarIconSx, makeMessageAvatarIcon, messageBackground, useMessageAvatarLabel } from '~/common/util/dMessageUtils';
import { copyToClipboard } from '~/common/util/clipboardUtils';
import { clipboardCopyDOMSelectionOrFallback, copyToClipboard } from '~/common/util/clipboardUtils';
import { createTextContentFragment, DMessageFragment, DMessageFragmentId, updateFragmentWithEditedText } from '~/common/stores/chat/chat.fragments';
import { useFragmentBuckets } from '~/common/stores/chat/hooks/useFragmentBuckets';
import { useUIPreferencesStore } from '~/common/stores/store-ui';
@@ -69,7 +69,7 @@ const ENABLE_BUBBLE = true;
export const BUBBLE_MIN_TEXT_LENGTH = 3;
// Enable the hover button to copy the whole message. The Copy button is also available in Blocks, or in the Avatar Menu.
const ENABLE_COPY_MESSAGE_OVERLAY: boolean = false;
// const ENABLE_COPY_MESSAGE_OVERLAY: boolean = false;
const messageBodySx: SxProps = {
@@ -314,11 +314,17 @@ export function ChatMessage(props: {
const handleCloseOpsMenu = React.useCallback(() => setOpsMenuAnchor(null), []);
const handleOpsCopy = (e: React.MouseEvent) => {
copyToClipboard(textSubject, 'Text');
const handleOpsMessageCopySrc = React.useCallback((e: React.MouseEvent) => {
e.preventDefault();
// copy full source text (ops menu) - bypasses DOM, always gets pre-collapsed content
copyToClipboard(fragmentFlattenedText, 'Message');
handleCloseOpsMenu();
closeContextMenu();
}, [fragmentFlattenedText, handleCloseOpsMenu]);
const handleBubbleCopyDOM = (e: React.MouseEvent) => {
e.preventDefault();
// copy cleaned DOM selection (bubble) - rich text for pasting into Google Docs, etc.
clipboardCopyDOMSelectionOrFallback(blocksRendererRef.current, textSubject, 'Selection');
closeBubble();
};
@@ -893,18 +899,18 @@ export function ChatMessage(props: {
{/* Overlay copy icon */}
{ENABLE_COPY_MESSAGE_OVERLAY && !fromSystem && !isEditingText && (
<Tooltip title={messagePendingIncomplete ? null : (fromAssistant ? 'Copy message' : 'Copy input')} variant='solid'>
<IconButton
variant='outlined' onClick={handleOpsCopy}
sx={{
position: 'absolute', ...(fromAssistant ? { right: { xs: 12, md: 28 } } : { left: { xs: 12, md: 28 } }), zIndex: 10,
opacity: 0, transition: 'opacity 0.16s cubic-bezier(.17,.84,.44,1)',
}}>
<ContentCopyIcon />
</IconButton>
</Tooltip>
)}
{/*{ENABLE_COPY_MESSAGE_OVERLAY && !fromSystem && !isEditingText && (*/}
{/* <Tooltip title={messagePendingIncomplete ? null : (fromAssistant ? 'Copy message' : 'Copy input')} variant='solid'>*/}
{/* <IconButton*/}
{/* variant='outlined' onClick={handleOpsMessageCopySrc}*/}
{/* sx={{*/}
{/* position: 'absolute', ...(fromAssistant ? { right: { xs: 12, md: 28 } } : { left: { xs: 12, md: 28 } }), zIndex: 10,*/}
{/* opacity: 0, transition: 'opacity 0.16s cubic-bezier(.17,.84,.44,1)',*/}
{/* }}>*/}
{/* <ContentCopyIcon />*/}
{/* </IconButton>*/}
{/* </Tooltip>*/}
{/*)}*/}
{/* Message Operations Menu (3 dots) */}
@@ -934,7 +940,7 @@ export function ChatMessage(props: {
</MenuItem>
)}
{/* Copy */}
<MenuItem onClick={handleOpsCopy} sx={{ flex: 1 }}>
<MenuItem onClick={handleOpsMessageCopySrc} sx={{ flex: 1 }}>
<ListItemDecorator><ContentCopyIcon /></ListItemDecorator>
Copy
</MenuItem>
@@ -1162,7 +1168,7 @@ export function ChatMessage(props: {
{/* Bubble Copy */}
<Tooltip disableInteractive arrow placement='top' title='Copy Selection'>
<IconButton onClick={handleOpsCopy}>
<IconButton onClick={handleBubbleCopyDOM}>
<ContentCopyIcon />
</IconButton>
</Tooltip>
@@ -1181,7 +1187,7 @@ export function ChatMessage(props: {
minWidth={220}
placement='bottom-start'
>
<MenuItem onClick={handleOpsCopy} sx={{ flex: 1, alignItems: 'center' }}>
<MenuItem onClick={(e) => { handleOpsMessageCopySrc(e); closeContextMenu(); }} sx={{ flex: 1, alignItems: 'center' }}>
<ListItemDecorator><ContentCopyIcon /></ListItemDecorator>
Copy
</MenuItem>
@@ -24,6 +24,15 @@ export const DocSelColor: ColorPaletteProp = 'primary';
const DocUnselColor: ColorPaletteProp = 'primary';
const _styles = {
label: {
whiteSpace: 'nowrap',
fontWeight: 'md',
minWidth: 48,
},
} as const;
export function buttonIconForFragment(part: DMessageAttachmentFragment['part']): React.ComponentType<any> {
const pt = part.pt;
switch (pt) {
@@ -146,10 +155,14 @@ export function DocAttachmentFragmentButton(props: {
if (!isDocPart(fragment.part))
return 'Unexpected: ' + fragment.part.pt;
const buttonText = ellipsizeMiddle(fragment.part.l1Title || fragment.title || 'Document', 28 /* totally arbitrary length */);
const Icon = isSelected ? EditRoundedIcon : buttonIconForFragment(fragment.part);
const fullTitle = fragment.part.l1Title || fragment.title || 'Document';
const buttonText = ellipsizeMiddle(fullTitle, 28 /* totally arbitrary length */);
const showFilenameTooltip = fullTitle !== buttonText;
const labelContent = <Box sx={_styles.label}>{buttonText}</Box>;
return (
<Button
size={props.contentScaling === 'md' ? 'md' : 'sm'}
@@ -171,9 +184,10 @@ export function DocAttachmentFragmentButton(props: {
</Box>
)}
<Box sx={{ display: 'flex', flexDirection: 'column', alignItems: 'flex-start', paddingX: '0.5rem' }}>
<Box sx={{ whiteSpace: 'nowrap', fontWeight: 'md', minWidth: 48 }}>
{buttonText}
</Box>
{showFilenameTooltip
? <TooltipOutlined title={<span style={{ wordBreak: 'break-all' }}>{fullTitle}</span>}>{labelContent}</TooltipOutlined>
: labelContent
}
{/*<Box sx={{ fontSize: 'xs', fontWeight: 'sm' }}>*/}
{/* {fragment.caption}*/}
{/*</Box>*/}
@@ -1,5 +1,7 @@
import * as React from 'react';
import type { SxProps } from '@mui/joy/styles/types';
import { BlocksTextarea } from '~/modules/blocks/BlocksContainers';
import type { ContentScaling } from '~/common/app.theme';
@@ -96,6 +98,8 @@ export function BlockEdit_TextFragment(props: {
const handleEditKeyDown = React.useCallback((e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === 'Enter') {
if (e.nativeEvent.isComposing)
return;
const withControl = e.ctrlKey;
if (enterIsNewline ? e.shiftKey : !e.shiftKey) {
e.preventDefault();
@@ -120,6 +124,32 @@ export function BlockEdit_TextFragment(props: {
{ key: ShortcutKey.Esc, description: 'Cancel', level: 3, action: onEscapePressed },
], [isControlled, isEdited, isFocused, onEscapePressed, onSubmit, props.enableRestart]));
// memo style
const sx = React.useMemo((): SxProps | undefined => {
// check sources of custom, and early outs
const isXS = props.contentScaling === 'xs';
const isSquareTop = !!props.squareTopBorder;
if (!isXS && !isSquareTop) return undefined;
if (isSquareTop && !isXS) return _styles.squareTop;
return {
// scaling note: in Chat, this can go xs/sm/md, while in Beam, this is xs/xs/sm
...(isXS && {
fontSize: 'xs',
lineHeight: 'md', // was 1.75 on all
// '--Textarea-paddingBlock': 'calc(0.25rem - 0.5px - var(--variant-borderWidth, 0px))', // not used, overridden in BlocksTextarea
'--Textarea-paddingInline': '6px',
'--Textarea-minHeight': '1.75rem', // was 2rem on 'sm'
'--Icon-fontSize': 'lg', // was 'xl' on 'sm'
'--Textarea-focusedThickness': '1px',
boxShadow: 'none', // too small to show this
}),
...(isSquareTop && _styles.squareTop),
};
}, [props.contentScaling, props.squareTopBorder]);
return (
<BlocksTextarea
variant={/*props.invertedColors ? 'plain' :*/ 'soft'}
@@ -140,7 +170,7 @@ export function BlockEdit_TextFragment(props: {
onKeyDown={handleEditKeyDown}
slotProps={enterIsNewline ? _textAreaSlotPropsEnter : _textAreaSlotPropsDone}
// endDecorator={props.endDecorator}
sx={!props.squareTopBorder ? undefined : _styles.squareTop}
sx={sx}
/>
);
}
@@ -96,7 +96,17 @@ export function ContentFragments(props: {
// Content Fragments Edit Zero-State: button to create a new TextContentFragment
if (isEditingText && !props.contentFragments.some(isTextContentFragment))
return !props.onFragmentAddBlank ? null : (
<Button aria-label='message body empty' variant='plain' color='neutral' onClick={props.onFragmentAddBlank} sx={{ justifyContent: 'flex-start' }}>
<Button
aria-label='message body empty'
color={fromAssistant ? 'neutral' : 'primary'}
variant='outlined'
onClick={props.onFragmentAddBlank}
sx={{
justifyContent: 'flex-start',
backgroundColor: fromAssistant ? 'neutral.softBg' : 'primary.softBg',
'&:hover': { backgroundColor: fromAssistant ? 'neutral.softHoverBg' : 'primary.softHoverBg' },
}}
>
add text ...
</Button>
);
@@ -171,6 +171,7 @@ export function BlockPartModelAnnotations(props: {
return (
<Box
data-agi-no-copy // do not copy these buttons: has its own copy functionality
sx={{ mx: 1.5 }}
>
@@ -182,7 +182,7 @@ export function BlockPartModelAux(props: {
return <Box sx={_styles.block}>
{/* Chip to expand/collapse */}
<Box sx={{ display: 'flex', flexWrap: 'wrap', gap: 1, alignItems: 'center', justifyContent: 'space-between' }}>
<Box data-agi-no-copy /* do not copy these buttons */ sx={{ display: 'flex', flexWrap: 'wrap', gap: 1, alignItems: 'center', justifyContent: 'space-between' }}>
<Chip
color={props.isLastFragment ? REASONING_COLOR : 'neutral'}
variant={expanded ? 'solid' : 'soft'}
+6 -4
View File
@@ -1,4 +1,4 @@
import { AixChatGenerateContent_DMessageGuts, aixChatGenerateContent_DMessage_FromConversation } from '~/modules/aix/client/aix.client';
import { aixChatGenerateContent_DMessage_FromConversation, AixChatGenerateContent_DMessageGuts } from '~/modules/aix/client/aix.client';
import { autoChatFollowUps } from '~/modules/aifn/auto-chat-follow-ups/autoChatFollowUps';
import { autoConversationTitle } from '~/modules/aifn/autotitle/autoTitle';
@@ -55,7 +55,7 @@ export async function runPersonaOnConversationHead(
const parallelViewCount = getUXLabsHighPerformance() ? 0 : getInstantAppChatPanesCount();
// ai follow-up operations (fire/forget)
const { autoSpeak, autoSuggestDiagrams, autoSuggestHTMLUI, autoSuggestQuestions, autoTitleChat, chatKeepLastThinkingOnly } = getChatAutoAI();
const { autoSpeak, autoSuggestDiagrams, autoSuggestHTMLUI, autoSuggestQuestions, autoTitleChat, chatThinkingPolicy } = getChatAutoAI();
// AutoSpeak
const autoSpeaker: PersonaProcessorInterface | null = autoSpeak !== 'off' ? new PersonaChatMessageSpeak(autoSpeak) : null;
@@ -129,8 +129,10 @@ export async function runPersonaOnConversationHead(
if (!hasBeenAborted && (autoSuggestDiagrams || autoSuggestHTMLUI || autoSuggestQuestions))
void autoChatFollowUps(conversationId, assistantMessageId, autoSuggestDiagrams, autoSuggestHTMLUI, autoSuggestQuestions);
if (chatKeepLastThinkingOnly)
cHandler.historyKeepLastThinkingOnly();
if (chatThinkingPolicy === 'last-only')
cHandler.historyStripThinking(1);
else if (chatThinkingPolicy === 'discard-all')
cHandler.historyStripThinking(0);
// return true if this succeeded
return messageStatus.outcome === 'success';
+9 -7
View File
@@ -8,6 +8,8 @@ import { Is } from '~/common/util/pwaUtils';
export type ChatAutoSpeakType = 'off' | 'firstLine' | 'all';
export type ChatThinkingPolicy = 'last-only' | 'all' | 'discard-all';
export type TokenCountingMethod = 'accurate' | 'approximate';
@@ -38,8 +40,8 @@ interface AppChatStore {
autoVndAntBreakpoints: boolean;
setAutoVndAntBreakpoints: (autoVndAntBreakpoints: boolean) => void;
chatKeepLastThinkingOnly: boolean,
setChatKeepLastThinkingOnly: (chatKeepLastThinkingOnly: boolean) => void;
chatThinkingPolicy: ChatThinkingPolicy,
setChatThinkingPolicy: (chatThinkingPolicy: ChatThinkingPolicy) => void;
tokenCountingMethod: TokenCountingMethod;
setTokenCountingMethod: (tokenCountingMethod: TokenCountingMethod) => void;
@@ -110,8 +112,8 @@ const useAppChatStore = create<AppChatStore>()(persist(
autoVndAntBreakpoints: true, // 2024-08-24: on as it saves user's money
setAutoVndAntBreakpoints: (autoVndAntBreakpoints: boolean) => _set({ autoVndAntBreakpoints }),
chatKeepLastThinkingOnly: true,
setChatKeepLastThinkingOnly: (chatKeepLastThinkingOnly: boolean) => _set({ chatKeepLastThinkingOnly }),
chatThinkingPolicy: 'last-only',
setChatThinkingPolicy: (chatThinkingPolicy: ChatThinkingPolicy) => _set({ chatThinkingPolicy }),
tokenCountingMethod: Is.Desktop ? 'accurate' : 'approximate',
setTokenCountingMethod: (tokenCountingMethod: TokenCountingMethod) => _set({ tokenCountingMethod }),
@@ -189,7 +191,7 @@ export const useChatAutoAI = () => useAppChatStore(useShallow(state => ({
autoSuggestQuestions: state.autoSuggestQuestions,
autoTitleChat: state.autoTitleChat,
autoVndAntBreakpoints: state.autoVndAntBreakpoints,
chatKeepLastThinkingOnly: state.chatKeepLastThinkingOnly,
chatThinkingPolicy: state.chatThinkingPolicy,
tokenCountingMethod: state.tokenCountingMethod,
setAutoSpeak: state.setAutoSpeak,
setAutoSuggestAttachmentPrompts: state.setAutoSuggestAttachmentPrompts,
@@ -198,7 +200,7 @@ export const useChatAutoAI = () => useAppChatStore(useShallow(state => ({
setAutoSuggestQuestions: state.setAutoSuggestQuestions,
setAutoTitleChat: state.setAutoTitleChat,
setAutoVndAntBreakpoints: state.setAutoVndAntBreakpoints,
setChatKeepLastThinkingOnly: state.setChatKeepLastThinkingOnly,
setChatThinkingPolicy: state.setChatThinkingPolicy,
setTokenCountingMethod: state.setTokenCountingMethod,
})));
@@ -210,7 +212,7 @@ export const getChatAutoAI = (): {
autoSuggestQuestions: boolean,
autoTitleChat: boolean,
autoVndAntBreakpoints: boolean,
chatKeepLastThinkingOnly: boolean,
chatThinkingPolicy: ChatThinkingPolicy,
} => useAppChatStore.getState();
export const useChatAutoSuggestHTMLUI = (): boolean =>
+4
View File
@@ -101,6 +101,10 @@ export function PromptComposer(props: {
if (e.key !== 'Enter')
return;
// Skip if composing (e.g., CJK input methods) - issue #784
if (e.nativeEvent.isComposing)
return;
// Shift: toggles the 'enter is newline'
if (enterIsNewline ? e.shiftKey : !e.shiftKey) {
if (userHasText)
+14
View File
@@ -71,6 +71,20 @@ export const DevNewsItem: NewsItem = {
// news and feature surfaces
export const NewsItems: NewsItem[] = [
{
versionCode: '2.0.3',
versionName: 'Red Carpet',
versionDate: new Date('2026-02-03T12:00:00Z'),
items: [
{ text: <><B>Kimi K2.5</B>, <B>Gemini 3 Flash</B>, <B>GPT Image 1.5</B>, <B>GPT 5.2 Codex</B>, <B issue={921}>Novita.ai</B> models, and xAI search and code execution</> },
{ text: <><B issue={943}>Google Drive</B>: attach docs, sheets, images with optimal LLM conversion</> },
{ text: <>Speech: new <B href='https://inworld.ai'>Inworld</B> support, cancelable, unlimited length</> },
{ text: <>Copy as-seen, reorder messages, AI Injector, PDF auto-OCR</> },
{ text: <>Models: <B issue={941}>duplication</B>, improved parameters, cleaner UI</> },
{ text: <>Fixes, security patches, CJK/IME input</> },
{ text: <>Developers: new Docker build, faster, and smaller containers, AI request injection capabilities in the inspector</>, dev: true },
],
},
{
versionCode: '2.0.2',
versionName: 'Heavy Critters',
+2 -1
View File
@@ -12,6 +12,7 @@ import type { ContentScaling } from '~/common/app.theme';
import { GoodTooltip } from '~/common/components/GoodTooltip';
import { agiUuid } from '~/common/util/idUtils';
import { copyToClipboard } from '~/common/util/clipboardUtils';
import { getLLMLabel } from '~/common/stores/llms/llms.types';
import { useFormEditTextArray } from '~/common/components/forms/useFormEditTextArray';
import { useLLMSelect, useLLMSelectLocalState } from '~/common/components/forms/useLLMSelect';
import { useToggleableBoolean } from '~/common/util/hooks/useToggleableBoolean';
@@ -255,7 +256,7 @@ export function Creator(props: { display: boolean }) {
Embodying Persona ...
</Typography>
<Typography level='title-sm' sx={{ mt: 1 }}>
Using: {personaLlm?.label}
Using: {personaLlm ? getLLMLabel(personaLlm) : 'Loading model...'}
</Typography>
</Box>
<Box>
+38
View File
@@ -14,6 +14,37 @@ import { InlineError } from '~/common/components/InlineError';
import type { SimplePersonaProvenance } from '../store-app-personas';
// configuration
const TEMP_DISABLE_YOUTUBE_TRANSCRIPT = true;
function YouTubeDisabledCard() {
return (
<Card
variant='soft'
color='primary'
invertedColors
sx={{
p: 3,
textAlign: 'center',
border: '1px solid',
borderColor: 'primary.solidBg',
}}
>
<Typography level='title-sm' sx={{ mb: 1 }}>
Temporarily Disabled
</Typography>
<Typography level='body-sm' sx={{ mb: 2 }}>
YouTube transcript extraction is currently unavailable due to API changes.
</Typography>
<Typography level='body-xs' color='neutral'>
Download transcripts manually and use the &quot;From Text&quot; option instead.
</Typography>
</Card>
);
}
function YouTubeVideoTranscriptCard(props: { transcript: YTVideoTranscript, onClose: () => void, sx?: SxProps }) {
const { transcript } = props;
return (
@@ -109,6 +140,13 @@ export function FromYouTube(props: {
setVideoID(videoId);
};
if (TEMP_DISABLE_YOUTUBE_TRANSCRIPT)
return <>
<Typography level='title-md' startDecorator={<YouTubeIcon sx={{ color: '#f00' }} />} sx={{ mb: 3 }}>
YouTube -&gt; Persona
</Typography>
<YouTubeDisabledCard />
</>;
return <>
+11 -6
View File
@@ -13,11 +13,11 @@ import { useLLMSelect } from '~/common/components/forms/useLLMSelect';
import { useLabsDevMode } from '~/common/stores/store-ux-labs';
import { useModelDomain } from '~/common/stores/llms/hooks/useModelDomain';
import type { TokenCountingMethod } from '../chat/store-app-chat';
import type { ChatThinkingPolicy, TokenCountingMethod } from '../chat/store-app-chat';
import { useChatAutoAI } from '../chat/store-app-chat';
const _keepThinkingBlocksOptions: FormSelectOption<'all' | 'last-only'>[] = [
const _keepThinkingBlocksOptions: FormSelectOption<ChatThinkingPolicy>[] = [
{
value: 'last-only',
label: 'Most Recent',
@@ -28,6 +28,11 @@ const _keepThinkingBlocksOptions: FormSelectOption<'all' | 'last-only'>[] = [
label: 'Preserve All',
description: 'Keep all traces',
},
{
value: 'discard-all',
label: 'Discard All',
description: 'May reduce quality',
},
] as const;
const _tokenCountingMethodOptions: FormSelectOption<TokenCountingMethod>[] = [
@@ -76,7 +81,7 @@ export function AppChatSettingsAI() {
autoSuggestHTMLUI, setAutoSuggestHTMLUI,
// autoSuggestQuestions, setAutoSuggestQuestions,
autoTitleChat, setAutoTitleChat,
chatKeepLastThinkingOnly, setChatKeepLastThinkingOnly,
chatThinkingPolicy, setChatThinkingPolicy,
tokenCountingMethod, setTokenCountingMethod,
} = useChatAutoAI();
@@ -155,10 +160,10 @@ export function AppChatSettingsAI() {
<FormSelectControl
title='Reasoning traces'
tooltip='Controls how AI thinking/reasoning blocks are kept in your chat history. Keeping only in the last message (default) reduces clutter.'
tooltip='Controls how AI thinking/reasoning blocks are kept in your chat history. "Most Recent" keeps only the last message traces (default). "Discard All" removes all traces after each response, which may reduce multi-turn quality with some providers.'
options={_keepThinkingBlocksOptions}
value={chatKeepLastThinkingOnly ? 'last-only' : 'all'}
onChange={(value) => setChatKeepLastThinkingOnly(value === 'last-only')}
value={chatThinkingPolicy}
onChange={setChatThinkingPolicy}
/>
<ListDivider inset='gutter'>Automatic AI Functions</ListDivider>
+2 -1
View File
@@ -130,6 +130,7 @@ const _styles = {
// modal: undefined,
modal: {
flexGrow: 1,
backgroundColor: 'background.level1',
} as const,
@@ -209,7 +210,7 @@ export function SettingsModal(props: {
<GoodModal
// title='Preferences' strongerTitle
title={
<AppBreadcrumbs size='md' rootTitle='App'>
<AppBreadcrumbs size='md' rootTitle={isMobile ? 'App' : 'Application'}>
<AppBreadcrumbs.Leaf><b>Preferences</b></AppBreadcrumbs.Leaf>
</AppBreadcrumbs>
}
+13 -2
View File
@@ -1,10 +1,11 @@
import { SpeexConfigureEngines } from '~/modules/speex/components/SpeexConfigureEngines';
import { useSpeexEngines } from '~/modules/speex/store-module-speex';
import { useSpeexEngines, useSpeexTtsCharLimit } from '~/modules/speex/store-module-speex';
import { ChatAutoSpeakType, useChatAutoAI } from '../chat/store-app-chat';
import { FormRadioOption } from '~/common/components/forms/FormRadioControl';
import { FormChipControl } from '~/common/components/forms/FormChipControl';
import { FormRadioOption } from '~/common/components/forms/FormRadioControl';
import { FormSwitchControl } from '~/common/components/forms/FormSwitchControl';
const _autoSpeakOptions: FormRadioOption<ChatAutoSpeakType>[] = [
@@ -21,6 +22,7 @@ export function VoiceOutSettings(props: { isMobile: boolean }) {
// external state
const { autoSpeak, setAutoSpeak } = useChatAutoAI();
const { ttsCharLimit, setTtsCharLimit } = useSpeexTtsCharLimit();
// external state - module
const hasEngines = useSpeexEngines().length > 0;
@@ -39,6 +41,15 @@ export function VoiceOutSettings(props: { isMobile: boolean }) {
onChange={setAutoSpeak}
/>
{/* TTS character limit toggle */}
<FormSwitchControl
title='Speak Cost Guard'
description={ttsCharLimit !== null ? 'Max ~3 min' : 'Unlimited'}
tooltip='Limits text sent to TTS providers, helping prevent unexpected costs with cloud services. By default the limit is 4096 characters (~3 minutes of speech).'
checked={ttsCharLimit !== null}
onChange={(checked) => setTtsCharLimit(checked ? 4096 : null)}
/>
{/* Engine configuration */}
<SpeexConfigureEngines isMobile={props.isMobile} />
+3 -3
View File
@@ -1,5 +1,5 @@
/**
* Copyright (c)2024-2025 Enrico Ros
* Copyright (c)2024-2026 Enrico Ros
*
* This file is include by both the frontend and backend, however depending on the time
* of the build, the values may be different.
@@ -23,8 +23,8 @@ export const Release = {
// this is here to trigger revalidation of data, e.g. models refresh
Monotonics: {
Aix: 43,
NewsVersion: 202,
Aix: 59,
NewsVersion: 203,
},
// Frontend: pretty features
@@ -0,0 +1,146 @@
/**
* Attachment Cloud Files
*
* For future refresh capability, the output fragments should preserve:
* - provider, fileId: to identify the file
* - mimeType: the original cloud MIME type
* - the converter used (stored in outputsHeuristic.actualConverterId)
*
* Google Workspace files (Docs, Sheets, Slides) are auto-exported during
* input loading to standard formats (HTML, CSV, PDF) and then processed
* by standard converters.
*/
import type { AttachmentCloudProviderId } from './attachment.types';
// Error handling
export class CloudFetchError extends Error {
constructor(public readonly code: _CloudFetchErrorCode, public readonly details?: string) {
super(`Cloud fetch error: ${code}${details ? ` - ${details}` : ''}`);
this.name = 'CloudFetchError';
}
}
type _CloudFetchErrorCode = 'AUTH_EXPIRED' | 'NOT_FOUND' | 'FORBIDDEN' | 'RATE_LIMITED' | 'NETWORK_ERROR' | 'NOT_IMPLEMENTED' | 'FETCH_FAILED';
// Utility functions
/**
* Google Workspace files can't be downloaded directly - they must be exported.
* We prioritize AI-friendly formats (text > binary).
*
* Docs: md, docx, pdf, txt, rtf, odt, epub, html.zip
* Sheets: xlsx, pdf, csv (1st sheet), tsv, ods
* Slides: pptx, pdf, txt, png/jpg/svg (1st slide)
* Drawings: png, pdf, jpg, svg
*
* Regular files: we'll return no conversion
*
* @see https://developers.google.com/workspace/drive/api/guides/ref-export-formats
*/
const _GOOGLE_WORKSPACE_EXPORT: Record<string, { mimeType: string; ext: string, converter: string }> = {
'application/vnd.google-apps.document': { mimeType: 'text/markdown', ext: '.md', converter: 'Doc -> ' },
'application/vnd.google-apps.spreadsheet': { mimeType: 'text/csv', ext: '.csv', converter: 'Sheet -> ' },
'application/vnd.google-apps.presentation': { mimeType: 'application/pdf', ext: '.pdf', converter: 'Slides -> ' },
'application/vnd.google-apps.drawing': { mimeType: 'image/svg+xml', ext: '.svg', converter: 'Drawing -> ' },
};
export function attachmentCloudGoogleWorkspaceExportMIME(cloudMimeType: string): string | undefined {
return _GOOGLE_WORKSPACE_EXPORT[cloudMimeType]?.mimeType;
}
export function attachmentCloudConverterPrefix(cloudMimeType: string): string {
return _GOOGLE_WORKSPACE_EXPORT[cloudMimeType]?.converter || 'Drive -> ';
}
// Fetcher
/**
* Fetch a file from a cloud provider.
*
* @param provider - The cloud provider ID
* @param fileId - The file ID in the provider's system
* @param accessToken - OAuth access token
* @param exportMimeType - For native formats (Docs/Sheets), the export format
* @returns The file content as a Blob
*/
export async function attachmentCloudFetchFile(
provider: AttachmentCloudProviderId,
fileId: string,
accessToken: string,
exportMimeType?: string,
): Promise<Blob> {
switch (provider) {
case 'gdrive':
return _fetchGoogleDriveFile(fileId, accessToken, exportMimeType);
case 'onedrive':
case 'dropbox':
throw new CloudFetchError('NOT_IMPLEMENTED', `${provider} support coming soon`);
default:
throw new CloudFetchError('NOT_IMPLEMENTED', `Unknown provider: ${provider}`);
}
}
/**
* Google Drive API - Fetch file content
* https://developers.google.com/drive/api/reference/rest/v3/files/get
* https://developers.google.com/drive/api/reference/rest/v3/files/export
*/
async function _fetchGoogleDriveFile(
fileId: string,
accessToken: string,
exportMimeType?: string,
): Promise<Blob> {
// for native Google Workspace files, use export endpoint
const url = exportMimeType
? `https://www.googleapis.com/drive/v3/files/${encodeURIComponent(fileId)}/export?mimeType=${encodeURIComponent(exportMimeType)}`
: `https://www.googleapis.com/drive/v3/files/${encodeURIComponent(fileId)}?alt=media`;
const response = await fetch(url, {
headers: {
'Authorization': `Bearer ${accessToken}`,
},
}).catch((error) => {
console.log('[DEV] Network error while fetching Google Drive file:', { error });
throw new CloudFetchError('NETWORK_ERROR', error?.message || String(error));
});
// NOTE: we shall consider moving this to use fetchResponseOrTRPCThrow instead of this custom small impl..
if (!response.ok) {
const errorCode = _mapHttpStatusToErrorCode(response.status);
let details = `${response.status}: ${response.statusText}`;
try {
const errorBody = await response.text();
if (errorBody) details += ` - ${errorBody.slice(0, 200)}`;
} catch { /* ignore */
}
throw new CloudFetchError(errorCode, details);
}
return response.blob();
}
function _mapHttpStatusToErrorCode(status: number): _CloudFetchErrorCode {
switch (status) {
case 401:
return 'AUTH_EXPIRED';
case 403:
return 'FORBIDDEN';
case 404:
return 'NOT_FOUND';
case 429:
return 'RATE_LIMITED';
default:
return 'FETCH_FAILED';
}
}
@@ -59,17 +59,35 @@ export async function imageDataToImageAttachmentFragmentViaDBlob(
origin: { // User originated
ot: 'user',
source: 'attachment',
media: source.media === 'file' ? source.origin : source.media === 'url' ? 'url' : 'unknown',
url: source.media === 'url' ? source.url : undefined,
fileName: source.media === 'file' ? source.refPath : undefined,
media:
source.media === 'file' ? source.origin
: source.media === 'url' ? 'url'
: source.media === 'cloud' ? source.provider
: 'unknown',
url:
source.media === 'url' ? source.url
: source.media === 'cloud' ? source.webViewLink
: undefined,
fileName:
source.media === 'file' ? source.refPath
: source.media === 'cloud' ? source.fileName
: undefined,
},
});
// use title if available, otherwise use the source refPath/refUrl/fileName
const refTextSummary = title || (
source.media === 'file' ? source.refPath
: source.media === 'url' ? source.refUrl
: source.media === 'cloud' ? source.fileName
: undefined
);
// Future-proof: create a Zync Image Asset reference attachment fragment, with the legacy image_ref part for compatibility for the time being
return createZyncAssetReferenceAttachmentFragment(
title, caption,
nanoidToUuidV4(dblobAssetId, 'convert-dblob-to-dasset'),
title || (source.media === 'file' ? source.refPath : source.media === 'url' ? source.refUrl : undefined), // use title if available, otherwise use the source refPath or refUrl
refTextSummary,
'image',
{
pt: 'image_ref' as const,
@@ -77,7 +95,7 @@ export async function imageDataToImageAttachmentFragmentViaDBlob(
...(title ? { altText: title } : {}),
...(imageWidth ? { width: imageWidth } : {}),
...(imageHeight ? { height: imageHeight } : {}),
}
},
);
} catch (error) {
console.error('imageAttachment: Error processing image:', error);
@@ -11,12 +11,14 @@ import { convert_Base64DataURL_To_Base64WithMimeType, convert_Base64WithMimeType
import { getDomainModelConfiguration } from '~/common/stores/llms/hooks/useModelDomain';
import { htmlTableToMarkdown } from '~/common/util/htmlTableToMarkdown';
import { humanReadableHyphenated } from '~/common/util/textUtils';
import { ocrImageWithProgress, ocrPdfPagesWithProgress } from '~/common/util/ocrUtils';
import { pdfToImageDataURLs, pdfToText } from '~/common/util/pdfUtils';
import { createDMessageDataInlineText, createDocAttachmentFragment, DMessageAttachmentFragment, DMessageDataInline, DMessageDocPart, DVMimeType, isContentOrAttachmentFragment, isDocPart, specialContentPartToDocAttachmentFragment } from '~/common/stores/chat/chat.fragments';
import type { AttachmentCreationOptions, AttachmentDraft, AttachmentDraftConverter, AttachmentDraftId, AttachmentDraftInput, AttachmentDraftSource, AttachmentDraftSourceOriginFile, DraftEgoFragmentsInputData, DraftWebInputData, DraftYouTubeInputData } from './attachment.types';
import type { AttachmentsDraftsStore } from './store-attachment-drafts_slice';
import { attachmentCloudConverterPrefix, attachmentCloudFetchFile, attachmentCloudGoogleWorkspaceExportMIME, CloudFetchError } from './attachment.cloud';
import { attachmentGetLiveFileId, attachmentSourceSupportsLiveFile } from './attachment.livefile';
import { guessInputContentTypeFromMime, heuristicMimeTypeFixup, mimeTypeIsDocX, mimeTypeIsPDF, mimeTypeIsPlainText, mimeTypeIsSupportedImage, reverseLookupMimeType } from './attachment.mimetypes';
import { imageDataToImageAttachmentFragmentViaDBlob } from './attachment.dblobs';
@@ -27,6 +29,11 @@ const PDF_IMAGE_QUALITY = 0.5;
const ENABLE_TEXT_AND_IMAGES = false; // [PROD] ?
const DOCPART_DEFAULT_VERSION = 1;
// PDF text extraction quality thresholds
const IMAGE_LOW_TEXT_THRESHOLD = 80; // chars per image - below this, consider the image as low-text (photo-like) rather than document-like
const PDF_LOW_TEXT_THRESHOLD = 160; // chars per page - below this, consider the PDF as scanned/image-based
const PDF_FALLBACK_MAX_IMAGES = 32; // max pages to convert to images when auto-falling back (to respect LLM limits)
// internal mimes, only used to route data within us (source -> input -> converters)
const INT_MIME_VND_AGI_EGO_FRAGMENTS = 'application/vnd.agi.ego.fragments';
@@ -63,7 +70,8 @@ export function attachmentCreate(source: AttachmentDraftSource): AttachmentDraft
export async function attachmentLoadInputAsync(source: Readonly<AttachmentDraftSource>, edit: (changes: Partial<Omit<AttachmentDraft, 'outputFragments'>>) => void) {
edit({ inputLoading: true });
switch (source.media) {
const sourceMedia = source.media;
switch (sourceMedia) {
// Download URL (page, file, ..) and attach as input
case 'url':
@@ -141,6 +149,7 @@ export async function attachmentLoadInputAsync(source: Readonly<AttachmentDraftS
} else
edit({ inputError: 'No content or file found at this link' });
} catch (error: any) {
console.log('[DEV] Issue downloading page for attachment:', { error });
edit({ inputError: `Issue downloading page: ${error?.message || (typeof error === 'string' ? error : JSON.stringify(error))}` });
}
break;
@@ -221,6 +230,34 @@ export async function attachmentLoadInputAsync(source: Readonly<AttachmentDraftS
}
break;
case 'cloud':
const cloudLabel = source.fileName || 'Cloud File';
const cloudRef = source.webViewLink || `${source.provider}:${source.fileId}`;
edit({ label: cloudLabel, ref: cloudRef });
try {
// fetch / export to the destination mime
const exportMime = attachmentCloudGoogleWorkspaceExportMIME(source.mimeType);
const cloudBlob = await attachmentCloudFetchFile(source.provider, source.fileId, source.accessToken, exportMime);
// use export mime if we exported, otherwise use source or detected mime
const resultMime = exportMime || source.mimeType /* provided outside */ || cloudBlob.type /* connection */ || 'application/octet-stream';
edit({
input: {
mimeType: resultMime,
data: cloudBlob,
dataSize: cloudBlob.size,
},
});
} catch (error: unknown) {
const errorMessage = error instanceof CloudFetchError
? `${error.code}: ${error.details || error.message}`
: `Failed to download: ${error instanceof Error ? error.message : String(error)}`;
edit({ inputError: errorMessage });
}
break;
case 'ego':
edit({
label: source.label,
@@ -231,6 +268,10 @@ export async function attachmentLoadInputAsync(source: Readonly<AttachmentDraftS
},
});
break;
default:
const _exhaustiveCheck: never = sourceMedia;
break;
}
edit({ inputLoading: false });
@@ -251,6 +292,7 @@ export function attachmentDefineConverters(source: AttachmentDraftSource, input:
const converters: AttachmentDraftConverter[] = [];
const autoAddImages = ENABLE_TEXT_AND_IMAGES && !!options?.hintAddImages;
const fromCloud = source.media === 'cloud';
switch (true) {
@@ -258,6 +300,7 @@ export function attachmentDefineConverters(source: AttachmentDraftSource, input:
case mimeTypeIsPlainText(input.mimeType):
// handle a secondary layer of HTML 'text' origins: drop, paste, and clipboard-read
const textOriginHtml = source.media === 'text' && input.altMimeType === 'text/html' && !!input.altData;
const textOriginClipboard = source.media === 'text' && ['clipboard-read', 'paste'].includes(source.method);
const isHtmlTable = !!input.altData?.startsWith('<table');
// p1: Tables
@@ -265,12 +308,21 @@ export function attachmentDefineConverters(source: AttachmentDraftSource, input:
converters.push({ id: 'rich-text-table', name: 'Markdown Table' });
// p2: Text
converters.push({ id: 'text', name: attachmentSourceSupportsLiveFile(source) ? 'Text (Live)' : 'Text' });
if (fromCloud && input.mimeType === 'text/markdown') {
converters.push({ id: 'text', name: 'Markdown' });
} else {
converters.push({ id: 'text', name: attachmentSourceSupportsLiveFile(source) ? 'Text (Live)' : 'Text' });
if (!textOriginHtml && textOriginClipboard) {
converters.push({ id: 'text-markdown', name: 'Text -> Markdown' });
converters.push({ id: 'text-cleaner', name: 'Text -> Clean HTML' });
}
}
// p3: Html
// p3: Html -> Markdown, and Html
if (textOriginHtml) {
converters.push({ id: 'rich-text-cleaner', name: 'Cleaner HTML' });
converters.push({ id: 'rich-text', name: 'HTML · Heavy' });
converters.push({ id: 'rich-text-markdown', name: 'HTML -> Markdown' });
converters.push({ id: 'rich-text-cleaner', name: 'HTML -> Clean HTML' });
}
break;
@@ -283,16 +335,18 @@ export function attachmentDefineConverters(source: AttachmentDraftSource, input:
converters.push({ id: 'image-original', name: 'Image (original quality)', disabled: !inputImageMimeSupported });
if (!inputImageMimeSupported)
converters.push({ id: 'image-to-default', name: `As Image (${PLATFORM_IMAGE_MIMETYPE})` });
converters.push({ id: 'image-caption', name: 'Caption (Text)', disabled: visionModelMissing });
converters.push({ id: 'image-caption', name: 'AI Caption (Text)', disabled: visionModelMissing });
converters.push({ id: 'unhandled', name: 'No Image' });
converters.push({ id: 'image-ocr', name: 'Add Text (OCR)', isCheckbox: true });
break;
// PDF
case mimeTypeIsPDF(input.mimeType):
converters.push({ id: 'pdf-text', name: 'PDF To Text', isActive: !autoAddImages || undefined });
converters.push({ id: 'pdf-images', name: 'PDF To Images' });
converters.push({ id: 'pdf-text-and-images', name: 'PDF Text & Images (best)', isActive: autoAddImages });
converters.push({ id: 'pdf-auto', name: 'Auto', isActive: !autoAddImages });
converters.push({ id: 'pdf-text', name: 'PDF Text' });
converters.push({ id: 'pdf-images-ocr', name: 'PDF -> OCR (for scans)' });
converters.push({ id: 'pdf-images', name: 'PDF -> Images' });
converters.push({ id: 'pdf-text-and-images', name: 'PDF -> Text + Images', isActive: autoAddImages });
break;
// DOCX
@@ -337,6 +391,12 @@ export function attachmentDefineConverters(source: AttachmentDraftSource, input:
break;
}
// cosmetic for cloud: prepend cloud label prefixes
const cloudLabelPrefix = source.media === 'cloud' ? attachmentCloudConverterPrefix(source.mimeType) : '';
if (cloudLabelPrefix)
for (const converter of converters)
converter.name = cloudLabelPrefix + converter.name;
edit({ converters });
}
@@ -380,7 +440,8 @@ function _prepareDocData(source: AttachmentDraftSource, input: Readonly<Attachme
srcFileSize: source.fileWithHandle.size || input.dataSize,
};
switch (source.origin) {
const sourceOrigin = source.origin;
switch (sourceOrigin) {
case 'camera':
fileTitle = source.refPath || _lowCollisionRefString('Camera Photo', 6);
break;
@@ -398,6 +459,10 @@ function _prepareDocData(source: AttachmentDraftSource, input: Readonly<Attachme
case 'drop':
fileTitle = source.refPath || _lowCollisionRefString('Dropped File', 6);
break;
default:
const _exhaustiveCheck: never = sourceOrigin;
fileTitle = 'File';
break;
}
return {
title: fileTitle,
@@ -415,6 +480,25 @@ function _prepareDocData(source: AttachmentDraftSource, input: Readonly<Attachme
refString: humanReadableHyphenated(textRef),
};
// Cloud files
case 'cloud':
const cloudFileName = source.fileName || 'Cloud File';
const cloudProviderLabel = source.provider === 'gdrive' ? 'Google Drive'
: source.provider === 'onedrive' ? 'OneDrive'
: source.provider === 'dropbox' ? 'Dropbox'
: 'Cloud';
const cloudRef = `${source.provider}-${source.fileName || _lowCollisionRefString('file', 6)}`;
return {
title: cloudFileName,
caption: `From ${cloudProviderLabel}`,
refString: humanReadableHyphenated(cloudRef),
// TODO: expand this to allow future redownload - or other location but for the same purpose
docMeta: {
srcFileName: source.fileName,
srcFileSize: source.fileSize || input.dataSize,
},
};
// The application attaching pieces of itself
case 'ego':
const egoKind = source.method === 'ego-fragments' ? 'Chat Message' : '';
@@ -478,6 +562,8 @@ export async function attachmentPerformConversion(
edit(attachment.id, {
outputsConverting: true,
outputsConversionProgress: null,
outputWarnings: undefined,
outputsHeuristic: undefined,
});
// apply converter to the input
@@ -490,41 +576,81 @@ export async function attachmentPerformConversion(
switch (converter.id) {
// text as-is
// text
case 'text':
case 'text-cleaner':
case 'text-markdown':
const possibleLiveFileId = await attachmentGetLiveFileId(source);
const textContent = await _inputDataToString(input.data, 'text');
const textualInlineData = createDMessageDataInlineText(textContent, input.mimeType);
let textContent = await _inputDataToString(input.data, 'text');
let textContentMime = input.mimeType || 'text/plain';
switch (converter.id) {
case 'text-cleaner':
textContent = _cleanPossibleHtmlText(textContent);
break;
case 'text-markdown':
try {
const { convertHtmlToMarkdown } = await import('./file-converters/HtmlToMarkdown');
textContent = convertHtmlToMarkdown(textContent);
textContentMime = 'text/markdown';
} catch (error) {
console.log('[DEV] Error converting Text (HTML) to Markdown:', error);
}
break;
}
const textualInlineData = createDMessageDataInlineText(textContent, textContentMime);
newFragments.push(createDocAttachmentFragment(title, caption, _guessDocVDT(input.mimeType), textualInlineData, refString, DOCPART_DEFAULT_VERSION, docMeta, possibleLiveFileId));
break;
// html as-is
// html
case 'rich-text':
case 'rich-text-cleaner':
case 'rich-text-markdown':
let richText: string;
if (input.altData)
richText = input.altData;
else if (input.mimeType === 'text/html')
richText = await _inputDataToString(input.data, 'rich-text');
else
richText = '';
let richTextMimeType = 'text/html';
// html -> cleaner/html or markdown
switch (converter.id) {
case 'rich-text-cleaner':
richText = _cleanPossibleHtmlText(richText);
richTextMimeType = 'text/html';
break;
case 'rich-text-markdown':
try {
const { convertHtmlToMarkdown } = await import('./file-converters/HtmlToMarkdown');
richText = convertHtmlToMarkdown(richText);
richTextMimeType = 'text/markdown';
} catch (error) {
console.log('[DEV] Error converting HTML to Markdown:', error);
}
break;
}
// NOTE: before we had the following: createTextAttachmentFragment(ref || '\n<!DOCTYPE html>', input.altData!), which
// was used to wrap the HTML in a code block to facilitate AutoRenderBlocks's parser. Historic note, for future debugging.
const richTextData = createDMessageDataInlineText(input.altData || '', input.altMimeType);
const richTextData = createDMessageDataInlineText(richText, richTextMimeType);
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.VndAgiCode, richTextData, refString, DOCPART_DEFAULT_VERSION, docMeta));
break;
// html cleaned
case 'rich-text-cleaner':
const cleanerHtml = (input.altData || '')
// remove class and style attributes
.replace(/<[^>]+>/g, (tag) =>
tag.replace(/ class="[^"]*"/g, '').replace(/ style="[^"]*"/g, ''),
)
// remove svg elements
.replace(/<svg[^>]*>.*?<\/svg>/g, '');
const cleanedHtmlData = createDMessageDataInlineText(cleanerHtml, 'text/html');
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.VndAgiCode, cleanedHtmlData, refString, DOCPART_DEFAULT_VERSION, docMeta));
break;
// html to markdown table
case 'rich-text-table':
let tableData: DMessageDataInline;
try {
const mdTable = htmlTableToMarkdown(input.altData!, false);
tableData = createDMessageDataInlineText(mdTable, 'text/markdown');
// fall back to source text if the table conversion produced empty/tiny content
if (mdTable.replace(/[\s|:\-]/g, '').length < 2) {
const fallbackText = await _inputDataToString(input.data, 'rich-text-table');
tableData = createDMessageDataInlineText(fallbackText || mdTable, input.mimeType);
} else {
tableData = createDMessageDataInlineText(mdTable, 'text/markdown');
}
} catch (error) {
// fallback to text/plain
const fallbackText = await _inputDataToString(input.data, 'rich-text-table');
@@ -570,23 +696,14 @@ export async function attachmentPerformConversion(
case 'image-ocr':
if (!_expectBlob(input.data, 'Image OCR converter')) break;
try {
let lastProgress = -1;
const { recognize } = await import('tesseract.js');
const result = await recognize(input.data, undefined, {
errorHandler: e => console.error(e),
logger: (message) => {
if (message.status === 'recognizing text') {
if (message.progress > lastProgress + 0.01) {
lastProgress = message.progress;
edit(attachment.id, { outputsConversionProgress: lastProgress });
}
}
},
});
const imageText = result.data.text;
// Image -> OCR -> Inline text doc
const imageText = await ocrImageWithProgress(input.data, (progress) => edit(attachment.id, { outputsConversionProgress: progress }));
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(imageText, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'image' }));
// warn if very little text was extracted (likely a photo/diagram rather than text)
if (imageText.trim().length < IMAGE_LOW_TEXT_THRESHOLD)
edit(attachment.id, { outputWarnings: ['Very little text extracted - this image may not contain readable text.'] });
} catch (error) {
console.error(error);
console.error('[Image OCR Error]', error);
}
break;
@@ -615,26 +732,111 @@ export async function attachmentPerformConversion(
} catch (error: any) {
console.log('[DEV] Failed to caption image:', error);
const errorText = `[Captioning failed: ${error?.message || String(error)}]`;
edit(attachment.id, { outputWarnings: [errorText] });
newFragments.push(createDocAttachmentFragment(title, caption + ' (Error)', DVMimeType.TextPlain, createDMessageDataInlineText(errorText, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'image-caption' }));
}
break;
// pdf to text
case 'pdf-text':
if (!_expectBlob(input.data, 'PDF text converter')) break;
// Convert Blob to ArrayBuffer for PDF.js
const pdfText = await pdfToText(await input.data.arrayBuffer(), (progress: number) => {
edit(attachment.id, { outputsConversionProgress: progress });
});
if (pdfText.trim().length < 2) {
// Warn the user if no text is extracted
// edit(attachment.id, { inputError: 'No text found in the PDF file.' });
} else
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(pdfText, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' }));
// pdf-auto: intelligent conversion with fallback chain (text → OCR → images)
case 'pdf-auto':
if (!_expectBlob(input.data, 'PDF auto converter')) break;
try {
// Phase 1: Try text extraction (0-20% progress)
const pdfArrayBuffer = await input.data.arrayBuffer();
// [pdf-text] Extract text with quality metadata
const pdfTextResult = await pdfToText(pdfArrayBuffer, (progress: number) => {
// Reserve 0-20% for text extraction attempt, 20-100% for potential image fallback
edit(attachment.id, { outputsConversionProgress: progress * 0.2 });
});
// Check text density to detect scanned/image-based PDFs
if (pdfTextResult.avgCharsPerPage >= PDF_LOW_TEXT_THRESHOLD) {
// Good text extraction - use it
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(pdfTextResult.text, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' }));
edit(attachment.id, {
outputsHeuristic: { isAuto: true, actualConverterId: 'pdf-text', explain: `${pdfTextResult.avgCharsPerPage.toFixed(0)} chars/page` },
});
} else {
// Low text density - try OCR
// console.log(`[PDF Auto] Low text density (${pdfTextResult.avgCharsPerPage.toFixed(0)} chars/page), trying OCR...`);
// [pdf-images] Phase 2: Render pages to images (20-40% progress)
const pdfArrayBufferForImages = await input.data.arrayBuffer();
const imageDataURLs = await pdfToImageDataURLs(pdfArrayBufferForImages, PLATFORM_IMAGE_MIMETYPE, PDF_IMAGE_QUALITY, PDF_IMAGE_PAGE_SCALE, (progress) => {
edit(attachment.id, { outputsConversionProgress: 0.2 + progress * 0.2 });
});
// Limit pages for OCR (performance)
const pagesToProcess = Math.min(imageDataURLs.length, PDF_FALLBACK_MAX_IMAGES);
const imagesToOcr = imageDataURLs.slice(0, pagesToProcess);
// Phase 3: Try OCR on rendered pages (40-90% progress)
try {
// [pdf-images-ocr] OCR the images
const ocrResult = await ocrPdfPagesWithProgress(imagesToOcr, (progress) => {
edit(attachment.id, { outputsConversionProgress: 0.4 + progress * 0.5 });
});
if (ocrResult.avgCharsPerPage >= PDF_LOW_TEXT_THRESHOLD) {
// OCR yielded good text - use it
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(ocrResult.text, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' }));
const truncNote = pdfTextResult.pageCount > pagesToProcess ? ` (${pagesToProcess}/${pdfTextResult.pageCount} pages)` : '';
edit(attachment.id, {
outputsHeuristic: { isAuto: true, actualConverterId: 'pdf-images-ocr', explain: /*OCR extracted */`${ocrResult.avgCharsPerPage.toFixed(0)} chars/page${truncNote}` },
});
} else {
// OCR also yielded poor results - fall back to images
// console.log(`[PDF Auto] OCR also sparse (${ocrResult.avgCharsPerPage.toFixed(0)} chars/page), falling back to images`);
for (let i = 0; i < pagesToProcess; i++) {
const pdfPageImage = imageDataURLs[i];
const pdfPageImageF = await imageDataToImageAttachmentFragmentViaDBlob(pdfPageImage.mimeType, pdfPageImage.base64Data, source, `${title} (pg. ${i + 1})`, caption, false, false);
if (pdfPageImageF)
newFragments.push(pdfPageImageF);
}
const truncNote = pdfTextResult.pageCount > pagesToProcess ? ` (${pagesToProcess}/${pdfTextResult.pageCount} pages)` : '';
edit(attachment.id, {
outputsHeuristic: { isAuto: true, actualConverterId: 'pdf-images', explain: `not a text page${truncNote}` },
});
}
} catch (ocrError) {
// OCR failed - fall back to images
console.warn('[PDF Auto] OCR failed, falling back to images:', ocrError);
for (let i = 0; i < pagesToProcess; i++) {
const pdfPageImage = imageDataURLs[i];
const pdfPageImageF = await imageDataToImageAttachmentFragmentViaDBlob(pdfPageImage.mimeType, pdfPageImage.base64Data, source, `${title} (pg. ${i + 1})`, caption, false, false);
if (pdfPageImageF)
newFragments.push(pdfPageImageF);
}
edit(attachment.id, {
outputsHeuristic: { isAuto: true, actualConverterId: 'pdf-images', explain: 'OCR failed, attached as images' },
});
}
}
} catch (error) {
console.error('Error in PDF auto conversion:', error);
}
break;
// pdf to images
// pdf-text: strict text extraction, no fallback (honors user choice)
case 'pdf-text':
if (!_expectBlob(input.data, 'PDF text converter')) break;
try {
const pdfTextResult = await pdfToText(await input.data.arrayBuffer(), progress => edit(attachment.id, { outputsConversionProgress: progress }));
// Always output text, even if sparse (user explicitly chose this)
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(pdfTextResult.text, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' }));
edit(attachment.id, {
// warn if very little text was extracted (likely a scanned PDF)
outputWarnings: pdfTextResult.avgCharsPerPage >= 20 ? undefined : ['Very little text extracted - this PDF may be scanned. Try "Auto" or "OCR (for scans)" mode.'],
outputsHeuristic: { isAuto: false, actualConverterId: 'pdf-text', explain: `${pdfTextResult.avgCharsPerPage.toFixed(0)} chars/page` },
});
} catch (error) {
console.error('Error in PDF text extraction:', error);
}
break;
// pdf-images: render all pages as images (honors user choice)
case 'pdf-images':
if (!_expectBlob(input.data, 'PDF images converter')) break;
// Convert Blob to ArrayBuffer for PDF.js
@@ -647,11 +849,39 @@ export async function attachmentPerformConversion(
if (pdfPageImageF)
newFragments.push(pdfPageImageF);
}
edit(attachment.id, {
outputsHeuristic: { isAuto: false, actualConverterId: 'pdf-images', explain: `${imageDataURLs.length} pages` },
});
} catch (error) {
console.error('Error converting PDF to images:', error);
}
break;
// pdf-images-ocr: force OCR on all pages (for scanned documents)
case 'pdf-images-ocr':
if (!_expectBlob(input.data, 'PDF OCR converter')) break;
try {
// Render pages to images (0-40% progress)
const imageDataURLs = await pdfToImageDataURLs(await input.data.arrayBuffer(), PLATFORM_IMAGE_MIMETYPE, PDF_IMAGE_QUALITY, PDF_IMAGE_PAGE_SCALE, (progress) => {
edit(attachment.id, { outputsConversionProgress: progress * 0.4 });
});
// OCR all pages (40-100% progress)
const ocrResult = await ocrPdfPagesWithProgress(imageDataURLs, (progress) => {
edit(attachment.id, { outputsConversionProgress: 0.4 + progress * 0.6 });
});
newFragments.push(createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(ocrResult.text, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' }));
edit(attachment.id, {
// warn if very little text was extracted (likely a scanned PDF)
outputWarnings: ocrResult.avgCharsPerPage >= 20 ? undefined : ['Very little text extracted via OCR - this PDF may contain mostly images/diagrams.'],
outputsHeuristic: { isAuto: false, actualConverterId: 'pdf-images-ocr', explain: `${ocrResult.avgCharsPerPage.toFixed(0)} chars/page from ${ocrResult.pageCount} pages` },
});
} catch (error) {
console.error('Error in PDF OCR:', error);
}
break;
// pdf to text and images
case 'pdf-text-and-images':
if (!_expectBlob(input.data, 'PDF text and images converter')) break;
@@ -673,18 +903,21 @@ export async function attachmentPerformConversion(
}
// duplicated from 'pdf-text'
const pdfText = await pdfToText(pdfArrayBufferForText, (progress: number) => {
const pdfTextResult = await pdfToText(pdfArrayBufferForText, (progress: number) => {
edit(attachment.id, { outputsConversionProgress: 0.5 + progress / 2 }); // Update progress (50% to 100%)
});
if (pdfText.trim().length < 2) {
// Do not warn the user, as hopefully the images are useful
} else {
const textFragment = createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(pdfText, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' });
if (pdfTextResult.text.trim().length >= 2) {
// Add text fragment if there's meaningful text
const textFragment = createDocAttachmentFragment(title, caption, DVMimeType.TextPlain, createDMessageDataInlineText(pdfTextResult.text, 'text/plain'), refString, DOCPART_DEFAULT_VERSION, { ...docMeta, srcOcrFrom: 'pdf' });
newFragments.push(textFragment);
}
// Note: if text is sparse, images are still attached (user explicitly chose text+images)
// Add the text fragment first, then the image fragments
newFragments.push(...imageFragments);
edit(attachment.id, {
outputsHeuristic: { isAuto: false, actualConverterId: 'pdf-text-and-images', explain: `${pdfTextResult.avgCharsPerPage.toFixed(0)} chars/page + ${imageFragments.length} images` },
});
} catch (error) {
console.error('Error converting PDF to text and images:', error);
}
@@ -801,14 +1034,28 @@ export async function attachmentPerformConversion(
case 'unhandled':
// force the user to explicitly select 'as text' if they want to proceed
break;
default:
const _exhaustiveCheck: never = converter.id;
console.warn('[DEV] Unhandled converter type:', _exhaustiveCheck);
break;
}
}
// warn if any doc output fragment has empty text content (something went wrong in conversion)
// TODO: future: check if the text is a conversion error... can happen with drag & drop
const emptyOutputWarnings: string[] = [];
for (const fragment of newFragments)
if (isDocPart(fragment.part) && fragment.part.data.idt === 'text' && !fragment.part.data.text.trim())
emptyOutputWarnings.push('Converted output is empty - the source content may be missing or invalid.');
// update
replaceOutputFragments(attachment.id, newFragments);
edit(attachment.id, {
outputsConverting: false,
outputsConversionProgress: null,
...(emptyOutputWarnings.length && { outputWarnings: emptyOutputWarnings }),
});
}
@@ -843,6 +1090,19 @@ async function _inputDataToString(data: AttachmentDraftInput['data'], debugLocat
return '';
}
/**
* Simple Client-side cleaning of possible HTML
*/
function _cleanPossibleHtmlText(inputStr: string): string {
return inputStr
// remove class and style attributes
.replace(/<[^>]+>/g, (tag) =>
tag.replace(/ class="[^"]*"/g, '').replace(/ style="[^"]*"/g, ''),
)
// remove svg elements
.replace(/<svg[^>]*>.*?<\/svg>/g, '');
}
/**
* Special function to convert a list of files to Attachment Fragments, without passing through the attachments system
@@ -24,6 +24,16 @@ export type AttachmentDraft = {
outputsConversionProgress: number | null;
outputFragments: DMessageAttachmentFragment[];
// Warnings for poor conversions (e.g. scanned PDF with text extraction rather than OCR)
outputWarnings?: string[];
// Tracks what method was actually used (especially for Auto mode)
outputsHeuristic?: {
isAuto: boolean;
actualConverterId: AttachmentDraftConverterType;
explain?: string; // e.g., "42 chars/page detected"
};
// metadata: {
// creationDate?: Date; // Creation date of the file
// modifiedDate?: Date; // Last modified date of the file
@@ -33,6 +43,13 @@ export type AttachmentDraft = {
export type AttachmentDraftId = string;
export type AttachmentCreationOptions = {
/** Also attach an image representation of the attachment. Requires Release.Features.ENABLE_TEXT_AND_IMAGES as well. */
hintAddImages?: boolean;
}
export type AttachmentCloudProviderId = 'gdrive' | 'onedrive' | 'dropbox';
// 0. draft source (filled at the onset)
@@ -51,6 +68,23 @@ export type AttachmentDraftSource = {
method: 'clipboard-read' | AttachmentDraftSourceOriginDTO;
textPlain?: string;
textHtml?: string;
} | {
media: 'cloud';
origin: AttachmentDraftSourceOriginCloud;
// auth for fetching
accessToken: string;
// tokenExpiresAt?: number; // optional for staleness detection, unix ts
// recipe for fetching
provider: AttachmentCloudProviderId;
fileId: string;
mimeType: string; // cloud-native MIME (e.g., 'application/vnd.google-apps.document')
// decorative
fileName: string;
fileSize?: number;
webViewLink?: string; // link to view in cloud provider's UI
} | {
// special type for attachments thar are references to self (ego, application) objects
media: 'ego';
@@ -65,10 +99,7 @@ export type AttachmentDraftSourceOriginDTO = 'drop' | 'paste';
export type AttachmentDraftSourceOriginUrl = 'input-link' | 'clipboard-read' | AttachmentDraftSourceOriginDTO;
export type AttachmentCreationOptions = {
/** Also attach an image representation of the attachment. Requires Release.Features.ENABLE_TEXT_AND_IMAGES as well. */
hintAddImages?: boolean;
}
export type AttachmentDraftSourceOriginCloud = `picker-${AttachmentCloudProviderId}`;
// 1. draft input (loaded from the source)
@@ -135,9 +166,10 @@ export type AttachmentDraftConverter = {
}
export type AttachmentDraftConverterType =
| 'text' | 'rich-text' | 'rich-text-cleaner' | 'rich-text-table'
| 'text' | 'text-cleaner' | 'text-markdown'
| 'rich-text' | 'rich-text-cleaner' | 'rich-text-markdown' | 'rich-text-table'
| 'image-original' | 'image-resized-high' | 'image-resized-low' | 'image-ocr' | 'image-caption' | 'image-to-default'
| 'pdf-text' | 'pdf-images' | 'pdf-text-and-images'
| 'pdf-auto' | 'pdf-text' | 'pdf-images' | 'pdf-images-ocr' | 'pdf-text-and-images'
| 'docx-to-html'
| 'url-page-text' | 'url-page-markdown' | 'url-page-html' | 'url-page-null' | 'url-page-image'
| 'youtube-transcript' | 'youtube-transcript-simple'
@@ -0,0 +1,109 @@
import { default as TurndownService } from 'turndown';
// Cached Turndown service instance
let _turndownService: TurndownService | null = null;
function getTurndownService(): TurndownService {
if (!_turndownService) {
_turndownService = new TurndownService({
headingStyle: 'atx',
codeBlockStyle: 'fenced',
emDelimiter: '_',
});
// Remove script and style elements
_turndownService.remove(['script', 'style', 'noscript']);
}
return _turndownService;
}
/**
* Convert HTML string to Markdown using Turndown.
* Performs basic HTML cleaning before conversion.
*/
export function convertHtmlToMarkdown(html: string): string {
// Basic client-side HTML cleaning using DOMParser
const cleanedHtml = cleanHtmlForMarkdown(html);
return getTurndownService().turndown(cleanedHtml);
}
/**
* Client-side HTML cleaning optimized for Markdown conversion.
* Uses DOMParser (browser-native) instead of Cheerio (server-only).
*/
function cleanHtmlForMarkdown(html: string): string {
try {
const parser = new DOMParser();
const doc = parser.parseFromString(html, 'text/html');
// Remove unwanted elements
const unwantedSelectors = [
'script', 'style', 'link', 'noscript', 'iframe', 'svg', 'canvas',
'nav:not(main nav)', 'aside', 'footer:not(article footer)',
'.ad', '.ads', '.advertisement', '.banner', '.popup', '.modal', '.overlay',
'.cookie-banner', '.newsletter-signup', '.social-share', '.comments',
'.sidebar', '.widget', '.carousel', '.slider',
'[aria-hidden="true"]', '[hidden]',
'[data-analytics]', '[data-tracking]', '[data-gtm]',
];
for (const selector of unwantedSelectors) {
try {
doc.querySelectorAll(selector).forEach(el => el.remove());
} catch {
// Skip invalid selectors (e.g., complex :not() selectors may fail in some browsers)
}
}
// Remove hidden elements via inline styles
doc.querySelectorAll('[style]').forEach(el => {
const style = el.getAttribute('style') || '';
if (style.includes('display: none') || style.includes('display:none') ||
style.includes('visibility: hidden') || style.includes('visibility:hidden'))
el.remove();
});
// Clean up anchor hrefs (remove tracking parameters)
doc.querySelectorAll('a[href]').forEach(el => {
const href = el.getAttribute('href');
if (!href) return;
// Remove javascript: links
if (href.toLowerCase().startsWith('javascript:')) {
el.removeAttribute('href');
return;
}
// Remove tracking parameters
if (href.includes('?')) {
try {
const url = new URL(href, 'http://placeholder');
const cleanParams = new URLSearchParams();
url.searchParams.forEach((value, key) => {
if (!key.match(/^(utm_|fbclid|gclid|msclkid)/i))
cleanParams.append(key, value);
});
const cleanHref = `${url.pathname}${cleanParams.toString() ? '?' + cleanParams.toString() : ''}${url.hash}`;
el.setAttribute('href', cleanHref);
} catch {
// Keep original href if URL parsing fails
}
}
});
// Remove comments (HTML comment nodes)
const walker = document.createTreeWalker(doc.body, NodeFilter.SHOW_COMMENT);
const comments: Comment[] = [];
while (walker.nextNode())
comments.push(walker.currentNode as Comment);
comments.forEach(comment => comment.remove());
return doc.body.innerHTML;
} catch (error) {
console.error('HTML cleaning error:', error);
return html; // Return original if cleaning fails
}
}
@@ -12,7 +12,7 @@ import type { DMessageId } from '~/common/stores/chat/chat.message';
import { getAllFilesFromDirectoryRecursively, getDataTransferFilesOrPromises } from '~/common/util/fileSystemUtils';
import { useChatAttachmentsStore } from '~/common/chat-overlay/store-perchat_vanilla';
import type { AttachmentDraftSourceOriginDTO, AttachmentDraftSourceOriginFile, AttachmentDraftSourceOriginUrl } from './attachment.types';
import type { AttachmentDraftSource, AttachmentDraftSourceOriginDTO, AttachmentDraftSourceOriginFile, AttachmentDraftSourceOriginUrl } from './attachment.types';
import type { AttachmentDraftsStoreApi } from './store-attachment-drafts_slice';
@@ -26,6 +26,9 @@ function notifyOnlyImages(item: any) {
}
export type AttachmentStoreCloudInput = Omit<Extract<AttachmentDraftSource, { media: 'cloud' }>, 'media' | 'origin'>;
/**
* @param attachmentsStoreApi A Per-Chat or standalone Attachment Drafts store.
* @param enableLoadURLsOnPaste Only used if invoking attachAppendDataTransfer or attachAppendClipboardItems.
@@ -254,8 +257,8 @@ export function useAttachmentDrafts(attachmentsStoreApi: AttachmentDraftsStoreAp
// https://github.com/enricoros/big-AGI/issues/286
const textHtml = clipboardItem.types.includes('text/html')
? await clipboardItem.getType('text/html')
.then(blob => blob?.text() ?? '')
.catch(() => '')
.then(blob => blob?.text() ?? '')
.catch(() => '')
: '';
const heuristicBypassImage = textHtml.startsWith('<table ');
@@ -289,8 +292,8 @@ export function useAttachmentDrafts(attachmentsStoreApi: AttachmentDraftsStoreAp
// get the Plain text
const textPlain = clipboardItem.types.includes('text/plain')
? await clipboardItem.getType('text/plain')
.then(blob => blob?.text() ?? '')
.catch(() => '')
.then(blob => blob?.text() ?? '')
.catch(() => '')
: '';
// attach as URL
@@ -321,6 +324,27 @@ export function useAttachmentDrafts(attachmentsStoreApi: AttachmentDraftsStoreAp
}
}, [_createAttachmentDraft, attachAppendFile, attachAppendUrl, enableLoadURLsOnPaste, filterOnlyImages, hintAddImages]);
/**
* Append a cloud file (Google Drive, OneDrive, etc.) to the attachments.
* This is the entry point for cloud file picker integrations.
*/
const attachAppendCloudFile = React.useCallback((cloudFile: AttachmentStoreCloudInput) => {
if (ATTACHMENTS_DEBUG_INTAKE)
console.log('attachAppendCloudFile', cloudFile);
// only-images: ignore cloud files as they may not be images
if (filterOnlyImages && !cloudFile.mimeType.startsWith('image/')) {
notifyOnlyImages(cloudFile);
return Promise.resolve();
}
return _createAttachmentDraft({
media: 'cloud',
origin: `picker-${cloudFile.provider}`,
...cloudFile,
}, { hintAddImages });
}, [_createAttachmentDraft, filterOnlyImages, hintAddImages]);
/**
* Append ego content to the attachments.
*/
@@ -348,6 +372,7 @@ export function useAttachmentDrafts(attachmentsStoreApi: AttachmentDraftsStoreAp
// create drafts
attachAppendClipboardItems,
attachAppendCloudFile,
attachAppendDataTransfer,
attachAppendEgoFragments,
attachAppendFile,
@@ -0,0 +1,219 @@
import * as React from 'react';
import { createPortal } from 'react-dom';
import type { PickerCanceledEvent, PickerPickedEvent } from '@googleworkspace/drive-picker-element';
import { DrivePicker, DrivePickerDocsView } from '@googleworkspace/drive-picker-react';
import { IconButton } from '@mui/joy';
import LogoutIcon from '@mui/icons-material/Logout';
import { TooltipOutlined } from '~/common/components/TooltipOutlined';
import { addSnackbar } from '~/common/components/snackbar/useSnackbarsStore';
import type { AttachmentStoreCloudInput } from './useAttachmentDrafts';
// configuration
const GOOGLE_DRIVE_CLIENT_ID = process.env.NEXT_PUBLIC_GOOGLE_DRIVE_CLIENT_ID || '';
const MAX_FILE_SIZE_MB = 10; // skip files larger than this; 0 = no limit; note: Google Workspace files report 0 bytes
const MAX_PICKER_FILES = 8; // max files per picker session; 0 = unlimited
export const hasGoogleDriveCapability = !!GOOGLE_DRIVE_CLIENT_ID;
// -- Token Definitions --
export interface ICloudProviderTokenAccessor {
get: () => CloudProviderToken | null;
set: (value: CloudProviderToken | null) => void;
}
export interface CloudProviderToken {
token: string;
expiresAt?: number; // timestamp in ms; if missing, token is treated as valid (the downstream may clear it eventually)
}
function _getUnexpiredToken(stored: CloudProviderToken | null): string | undefined {
if (!stored?.token) return undefined;
// if expiresAt is set and expired (with 60s safety margin), return undefined
if (stored.expiresAt && Date.now() > stored.expiresAt - 60 * 1000) return undefined;
return stored.token;
}
// --- In-memory token storage ---
let _inMemoryToken: CloudProviderToken | null = null;
const _inMemoryTokenStorage: ICloudProviderTokenAccessor = {
get: () => _inMemoryToken,
set: (value: CloudProviderToken | null) => _inMemoryToken = value,
};
type _OauthResponseEvent = {
detail?: {
access_token: string; // xxxx.yyyyy....
expires_in?: string | number; // 3599
// scope?: string; // 'https://www.googleapis.com/auth/drive.file'
// token_type?: string; // 'Bearer'
};
}
type _OauthErrorEvent = {
detail?: {
error?: string; // 'access_denied', 'popup_closed_by_user', ...
} | {
type?: string; // 'popup_closed'
// message?: string; // 'Popup window closed'
// stack?: string;
} | object;
}
export function useGoogleDrivePicker(
onCloudFileSelected: (cloudFile: AttachmentStoreCloudInput) => void,
isMobile: boolean,
tokenStorage: ICloudProviderTokenAccessor = _inMemoryTokenStorage,
loginHint?: string,
) {
// state
const [isPickerOpen, setIsPickerOpen] = React.useState(false);
const openGoogleDrivePicker = React.useCallback(() => setIsPickerOpen(true), []);
const handleDeauthClick = React.useCallback(() => {
setIsPickerOpen(false);
tokenStorage.set(null);
}, [tokenStorage]);
// handle oauth events, to store the token for the picker callback
const handleOAuthResponse = React.useCallback((e: _OauthResponseEvent) => {
if (!e.detail?.access_token) return;
const expiresIn = typeof e.detail.expires_in === 'number' ? e.detail.expires_in : typeof e.detail.expires_in === 'string' ? parseInt(e.detail.expires_in, 10) : undefined;
tokenStorage.set({
token: e.detail.access_token,
expiresAt: expiresIn === undefined ? undefined : Date.now() + expiresIn * 1000,
});
}, [tokenStorage]);
const handleOAuthError = React.useCallback((e: _OauthErrorEvent) => {
setIsPickerOpen(false);
// ignore if user closed the popup
if (e?.detail && 'type' in e?.detail && e.detail.type === 'popup_closed') return;
const errorMsg = e?.detail && 'error' in e?.detail && typeof e.detail.error === 'string' ? e.detail.error : undefined;
addSnackbar({ key: 'gdrive-oauth-error', message: errorMsg === 'access_denied' ? 'Drive file access was denied' : 'Google Drive authentication failed.', type: 'issue' });
}, []);
// handler picker events
const handleCanceled = React.useCallback((_e: PickerCanceledEvent) => setIsPickerOpen(false), []);
const handlePicked = React.useCallback((e: PickerPickedEvent) => {
setIsPickerOpen(false);
const docs = e.detail?.docs;
if (!docs?.length) return;
// read token, just set by handleOAuthResponse
const currentToken = _getUnexpiredToken(tokenStorage.get());
if (!currentToken)
return addSnackbar({ key: 'gdrive-no-token', message: 'Unable to access Google Drive.', type: 'issue' });
// convert picker docs to cloud file metadata for the attachment system
const maxBytes = MAX_FILE_SIZE_MB * 1024 * 1024;
const skippedFiles: string[] = [];
for (const doc of docs) {
// skip files that are too large (note: Google Workspace files report 0 bytes)
if (MAX_FILE_SIZE_MB && doc.sizeBytes && doc.sizeBytes > maxBytes) {
skippedFiles.push(doc.name);
continue;
}
onCloudFileSelected({
accessToken: currentToken,
provider: 'gdrive',
fileId: doc.id,
mimeType: doc.mimeType,
fileName: doc.name,
fileSize: doc.sizeBytes,
webViewLink: doc.url,
});
}
if (skippedFiles.length)
addSnackbar({ key: 'gdrive-size-limit', message: `Skipped ${skippedFiles.length} file(s) over ${MAX_FILE_SIZE_MB} MB: ${skippedFiles.join(', ')}`, type: 'issue' });
}, [onCloudFileSelected, tokenStorage]);
// memo components (close button and picker) | null
const googleDrivePickerComponent = React.useMemo(() => !isPickerOpen || !GOOGLE_DRIVE_CLIENT_ID ? null : <>
{/* Top-level close button - portaled to body, above the Google Drive picker */}
{createPortal(
<TooltipOutlined title='Close and Switch Google Drive Account' placement='bottom'>
<IconButton
onClick={handleDeauthClick}
sx={{
'--IconButton-size': '2.75rem',
backgroundColor: 'background.popup',
borderRadius: '50%',
boxShadow: 'lg',
position: 'fixed',
top: '1rem',
left: '50%',
transform: 'translateX(-50%)',
zIndex: 2002, // above the Drive Picker (2001+)
}}
>
<LogoutIcon />
</IconButton>
</TooltipOutlined>,
document.body,
)}
<DrivePicker
app-id={GOOGLE_DRIVE_CLIENT_ID.split('-')[0] || ''}
client-id={GOOGLE_DRIVE_CLIENT_ID}
title='Attach files from Google Drive'
multiselect={true}
hide-title-bar='true'
// nav-hidden={true /* disables the 'Google Drive' nav */}
// mine-only={true}
login-hint={loginHint}
max-items={MAX_PICKER_FILES || undefined}
oauth-token={_getUnexpiredToken(tokenStorage.get())}
onOauthResponse={handleOAuthResponse}
onOauthError={handleOAuthError}
onPicked={handlePicked}
onCanceled={handleCanceled}
>
<DrivePickerDocsView
// file-ids='id1,id2,id3'
// include-folders='default'
// mime-types=
mode={isMobile ? 'LIST' : undefined /* LIST, GRID - if set hides the switch */}
// owned-by-me='default'
// select-folder-enabled='default' // does not work, while the one in DrivePicker does
// starred=
/>
</DrivePicker>
</>, [handleCanceled, handleDeauthClick, handleOAuthError, handleOAuthResponse, handlePicked, isMobile, isPickerOpen, loginHint, tokenStorage]);
return {
openGoogleDrivePicker,
googleDrivePickerComponent,
};
}
@@ -227,8 +227,9 @@ export class ConversationHandler {
return _chatStoreActions.historyView(this.conversationId)?.find(m => m.id === messageId);
}
historyKeepLastThinkingOnly(): void {
return _chatStoreActions.historyKeepLastThinkingOnly(this.conversationId);
/** Strips thinking fragments from assistant messages, preserving `keepCount` most recent (0 = discard all, 1 = keep last only). */
historyStripThinking(keepCount: number): void {
return _chatStoreActions.historyStripThinking(this.conversationId, keepCount);
}
title(): string | undefined {
@@ -1,4 +1,6 @@
import type { DBlobAssetId } from '~/common/stores/blob/dblobs-portability';
import type { DConversationId } from '~/common/stores/chat/chat.conversation';
import { collectFragmentAssetIds, gcRegisterAssetCollector } from '~/common/stores/chat/chat.gc';
import { ConversationHandler } from './ConversationHandler';
@@ -14,6 +16,40 @@ export class ConversationsManager {
private static _instance: ConversationsManager;
private readonly handlers: Map<DConversationId, ConversationHandler> = new Map();
private constructor() {
// Register a GC collector to protect DBlob assets referenced in active Beam stores.
// Uses inversion of control to avoid circular dependency (chat/ -> chat-overlay/).
gcRegisterAssetCollector(() => this._collectBeamAssetIds());
}
/**
* Collect DBlob asset IDs from all active Beam stores (rays, fusions, follow-ups).
*/
private _collectBeamAssetIds(): DBlobAssetId[] {
const assetIds = new Set<DBlobAssetId>();
for (const handler of this.handlers.values()) {
const { rays, fusions } = handler.getBeamStore().getState();
// Scatter rays + their follow-up messages
for (const ray of rays) {
collectFragmentAssetIds(ray.message.fragments, assetIds);
// if (ray.followUpMessages)
// for (const msg of ray.followUpMessages)
// collectFragmentAssetIds(msg.fragments, assetIds);
}
// Gather fusions + their follow-up messages
for (const fusion of fusions) {
if (fusion.outputDMessage)
collectFragmentAssetIds(fusion.outputDMessage.fragments, assetIds);
// if (fusion.followUpMessages)
// for (const msg of fusion.followUpMessages)
// collectFragmentAssetIds(msg.fragments, assetIds);
}
}
return Array.from(assetIds);
}
static getHandler(conversationId: DConversationId): ConversationHandler {
const instance = ConversationsManager._instance || (ConversationsManager._instance = new ConversationsManager());
let handler = instance.handlers.get(conversationId);
+21 -5
View File
@@ -1,12 +1,27 @@
import * as React from 'react';
import { Breadcrumbs, Typography } from '@mui/joy';
import { Breadcrumbs, BreadcrumbsSlotsAndSlotProps, Typography } from '@mui/joy';
import KeyboardArrowRightIcon from '@mui/icons-material/KeyboardArrowRight';
import { Link } from '~/common/components/Link';
const _sx = { p: 0 };
const _breadcrumbSlotProps: BreadcrumbsSlotsAndSlotProps['slotProps'] = {
root: {
sx: { p: 0 },
},
// see anatomy https://mui.com/joy-ui/react-breadcrumbs/#anatomy
ol: {
// keep it all in one line
sx: { flexWrap: 'nowrap' },
},
li: {
// undo the 'flex' on li, and auto-ellipsize contents
sx: { display: 'block' },
className: 'agi-ellipsize',
},
} as const;
export function AppBreadcrumbs(props: {
size?: 'sm' | 'md' | 'lg';
@@ -23,12 +38,13 @@ export function AppBreadcrumbs(props: {
onRootClick?.();
}, [onRootClick]);
return <Breadcrumbs size={props.size || 'sm'} separator={<KeyboardArrowRightIcon />} aria-label='breadcrumbs' sx={_sx}>
{(props.children && !!rootTitle && !!onRootClick)
? <AppBreadcrumbs.Link color='neutral' href='#' onClick={handleRootClick}>{props.rootTitle}</AppBreadcrumbs.Link>
return <Breadcrumbs size={props.size || 'sm'} aria-label='breadcrumbs' separator={<KeyboardArrowRightIcon />} slotProps={_breadcrumbSlotProps}>
{/* Title */}
{(props.children && !!rootTitle && !!onRootClick) ? <AppBreadcrumbs.Link color='neutral' href='#' onClick={handleRootClick}>{props.rootTitle}</AppBreadcrumbs.Link>
: (typeof props.rootTitle === 'string') ? <Typography>{props.rootTitle}</Typography>
: props.rootTitle
}
{/* Rest */}
{props.children}
{/*{nav.pnt === 'create-new' && <Link color='neutral' href='#'>Create New</Link>}*/}
{/*{['Characters', 'Futurama', 'TV Shows', 'Home'].map((item: string) => (*/}
+3 -2
View File
@@ -93,6 +93,8 @@ export function CloseablePopup(props: {
},
}], [props.placementOffset]);
const popperMemoSx: undefined | SxProps = React.useMemo(() => !props.zIndex ? undefined : ({ zIndex: props.zIndex }), [props.zIndex]);
const styleMemoSx: SxProps = React.useMemo(() => ({
// style
@@ -120,7 +122,6 @@ export function CloseablePopup(props: {
}), [props.boxShadow, props.maxHeightGapPx, props.maxWidth, props.minWidth, props.size, props.dense, props.bigIcons, props.noBottomPadding, props.noTopPadding, props.sx]);
return (
<Popup
role={undefined}
@@ -129,7 +130,7 @@ export function CloseablePopup(props: {
placement={props.placement}
disablePortal={false}
modifiers={modifiersMemo}
sx={props.zIndex ? { zIndex: props.zIndex } : undefined}
sx={popperMemoSx}
>
<ClickAwayListener onClickAway={handleClose}>
{props.menu ? (
@@ -20,7 +20,7 @@ const BoxCollapsee = styled(Box)({
export function ExpanderControlledBox(props: { expanded: boolean, children: React.ReactNode, sx?: SxProps }) {
return (
<BoxCollapser aria-expanded={props.expanded} sx={props.sx}>
<BoxCollapser aria-expanded={props.expanded} data-agi-no-copy={!props.expanded || undefined} sx={props.sx}>
<BoxCollapsee>
{props.children}
</BoxCollapsee>
+2
View File
@@ -40,6 +40,8 @@ export function InlineTextarea(props: {
const handleEditKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
if (e.key === 'Enter') {
if (e.nativeEvent.isComposing)
return;
const shiftOrAlt = e.shiftKey || e.altKey;
if (enterIsNewline ? shiftOrAlt : !shiftOrAlt) {
e.preventDefault();
@@ -0,0 +1,51 @@
import * as React from 'react';
import type { SxProps } from '@mui/joy/styles/types';
import { ListItem, ListItemButton } from '@mui/joy';
import ExpandMoreIcon from '@mui/icons-material/ExpandMore';
const _styles = {
headerButton: {
color: 'text.tertiary',
py: 1.5,
border: 'none',
fontSize: 'sm',
fontWeight: 'md',
justifyContent: 'space-between',
'&[aria-expanded="false"]': {
fontWeight: 'lg',
fontStyle: 'italic',
color: 'text.primary',
},
},
headerDeco: {
width: '1rem',
},
headerCollapser: {
fontSize: 'md',
transition: 'transform 0.14s',
'[aria-expanded="false"] &': {
transform: 'rotate(-180deg)',
},
},
} as const satisfies Record<string, SxProps>;
export function ListItemGroupCollapser<TId extends string>(props: {
id: TId;
label: string;
isCollapsed: boolean;
onToggleCollapse: (id: TId) => void;
}) {
return (
<ListItem>
<ListItemButton color='neutral' aria-expanded={!props.isCollapsed} onClick={() => props.onToggleCollapse(props.id)} sx={_styles.headerButton}>
{/*{serviceVendor?.id && <ListItemDecorator><LLMVendorIconSprite vendorId={serviceVendor.id} /></ListItemDecorator>}*/}
<div style={_styles.headerDeco} />
<div>{props.label}</div>
<ExpandMoreIcon sx={_styles.headerCollapser} />
</ListItemButton>
</ListItem>
);
}
@@ -28,6 +28,7 @@ export function FormTextField(props: {
tooltip?: string | React.JSX.Element,
placeholder?: string, isError?: boolean, disabled?: boolean,
value: string | undefined, onChange: (text: string) => void,
endDecorator?: React.ReactNode,
inputSx?: SxProps,
}) {
const acId = 'text-' + props.autoCompleteId;
@@ -45,6 +46,7 @@ export function FormTextField(props: {
autoComplete='off'
variant='outlined' placeholder={props.placeholder} error={props.isError}
value={props.value} onChange={event => props.onChange(event.target.value)}
endDecorator={props.endDecorator}
sx={props.inputSx ?? _styles.inputDefault}
/>
</FormControl>
@@ -1,11 +1,14 @@
import * as React from 'react';
import { FormSwitchControl } from './FormSwitchControl';
import { Box, FormControl, Switch, Tooltip } from '@mui/joy';
import { FormLabelStart } from './FormLabelStart';
/**
* Reusable toggle for enabling client-side API fetch.
* Appears with animation when client key is present.
* Shows a tooltip recommendation when local host is detected but CSF is off.
*/
export function SetupFormClientSideToggle(props: {
visible: boolean;
@@ -13,8 +16,12 @@ export function SetupFormClientSideToggle(props: {
onChange: (on: boolean) => void;
helpText: string;
disabled?: boolean;
localHostDetected?: boolean; // shows a tooltip to hint at using this
}) {
// show recommendation tooltip for local hosts when CSF is off
const showLocalRecommendation = !!props.localHostDetected && !props.checked;
return (
<div
style={{
@@ -24,14 +31,29 @@ export function SetupFormClientSideToggle(props: {
}}
>
<div style={{ overflow: 'hidden' }}>
<FormSwitchControl
title='Direct Connection'
description={props.checked ? 'Connect from browser' : 'Via server (default)'}
tooltip={props.helpText}
checked={props.checked}
onChange={props.onChange}
disabled={props.disabled}
/>
<FormControl orientation='horizontal' disabled={props.disabled} sx={{ flexWrap: 'wrap', justifyContent: 'space-between', alignItems: 'center' }}>
<FormLabelStart
title='Direct Connection'
description={props.checked ? 'Connect from browser' : 'Via server (default)'}
tooltip={showLocalRecommendation ? undefined : props.helpText}
/>
<Tooltip
open={showLocalRecommendation}
disableInteractive
arrow
variant='solid'
color='success'
placement='top-end'
title='Recommended ON for local services'
>
<Switch
checked={props.checked}
onChange={event => props.onChange(event.target.checked)}
endDecorator={props.checked ? 'On' : 'Off'}
slotProps={{ endDecorator: { sx: { minWidth: 26 } } }}
/>
</Tooltip>
</FormControl>
</div>
</div>
);
@@ -1,7 +1,7 @@
import * as React from 'react';
import { Box, Button, FormLabel } from '@mui/joy';
import SyncIcon from '@mui/icons-material/Sync';
import RefreshIcon from '@mui/icons-material/Refresh';
import type { ToggleableBoolean } from '~/common/util/hooks/useToggleableBoolean';
@@ -32,7 +32,7 @@ export function SetupFormRefetchButton(props: {
color={props.error ? 'warning' : 'primary'}
disabled={props.disabled}
loading={props.loading}
endDecorator={<SyncIcon />}
endDecorator={<RefreshIcon />}
onClick={props.refetch}
sx={{ minWidth: 120, ml: 'auto' }}
>
@@ -1,6 +1,6 @@
import * as React from 'react';
import type { DLLM, DLLMId } from '~/common/stores/llms/llms.types';
import { DLLM, DLLMId, getLLMLabel } from '~/common/stores/llms/llms.types';
import { useLLMs } from '~/common/stores/llms/llms.hooks';
import type { FormRadioOption } from './FormRadioControl';
@@ -20,8 +20,8 @@ export function useFormRadioLlmType(label: string, runModelId: DLLMId | null, in
const hidden = !runLLM || !utilLLM || runLLM === utilLLM;
const options = React.useMemo((): FormRadioOption<LlmType>[] => [
{ label: runLLM?.label ?? '[missing llm]', value: 'run' },
{ label: utilLLM?.label ?? '[missing util llm]', value: 'util' },
{ label: runLLM ? getLLMLabel(runLLM) : '[missing llm]', value: 'run' },
{ label: utilLLM ? getLLMLabel(utilLLM) : '[missing util llm]', value: 'util' },
], [runLLM, utilLLM]);
const [llmType, component] = useFormRadio<LlmType>(initialModelType, options, label, hidden);
+69 -39
View File
@@ -1,22 +1,25 @@
import * as React from 'react';
import type { SxProps } from '@mui/joy/styles/types';
import { Box, Chip, ColorPaletteProp, FormControl, IconButton, ListDivider, ListItem, ListItemButton, ListItemDecorator, Option, Select, SelectSlotsAndSlotProps, SvgIconProps, VariantProp, optionClasses } from '@mui/joy';
import { Chip, ColorPaletteProp, FormControl, IconButton, ListDivider, ListItem, ListItemButton, ListItemDecorator, Option, optionClasses, Select, SelectSlotsAndSlotProps, VariantProp } from '@mui/joy';
import ArrowForwardRoundedIcon from '@mui/icons-material/ArrowForwardRounded';
import AutoModeIcon from '@mui/icons-material/AutoMode';
import BuildCircleIcon from '@mui/icons-material/BuildCircle';
import type { IModelVendor } from '~/modules/llms/vendors/IModelVendor';
import { LLMVendorIconSprite } from '~/modules/llms/components/LLMVendorIconSprite';
import { findModelVendor } from '~/modules/llms/vendors/vendors.registry';
import { llmsGetVendorIcon, LLMVendorIcon } from '~/modules/llms/components/LLMVendorIcon';
import type { DModelDomainId } from '~/common/stores/llms/model.domains.types';
import { DLLM, DLLMId, getLLMPricing, LLM_IF_OAI_Reasoning, LLM_IF_Outputs_Audio, LLM_IF_Outputs_Image, LLM_IF_Tools_WebSearch } from '~/common/stores/llms/llms.types';
import type { DModelsServiceId } from '~/common/stores/llms/llms.service.types';
import { isLLMChatFree_cached } from '~/common/stores/llms/llms.pricing';
import { DLLM, DLLMId, getLLMLabel, LLM_IF_OAI_Reasoning, LLM_IF_Outputs_Audio, LLM_IF_Outputs_Image, LLM_IF_Tools_WebSearch } from '~/common/stores/llms/llms.types';
import { ListItemGroupCollapser } from '~/common/components/ListItemGroupCollapser';
import { PhGearSixIcon } from '~/common/components/icons/phosphor/PhGearSixIcon';
import { StarIconUnstyled, StarredNoXL2 } from '~/common/components/StarIcons';
import { TooltipOutlined } from '~/common/components/TooltipOutlined';
import { getChatLLMId, llmsStoreActions } from '~/common/stores/llms/store-llms';
import { findModelsServiceOrNull, getChatLLMId, llmsStoreActions } from '~/common/stores/llms/store-llms';
import { optimaActions, optimaOpenModels } from '~/common/layout/optima/useOptima';
import { useToggleableStringSet } from '~/common/util/hooks/useToggleableStringSet';
import { useUIPreferencesStore } from '~/common/stores/store-ui';
import { useVisibleLLMs } from '~/common/stores/llms/llms.hooks';
@@ -69,13 +72,6 @@ const _styles = {
backgroundColor: 'background.surface',
zIndex: 1,
},
listVendor: {
// see OptimaBarDropdown's _styles.separator
fontSize: 'sm',
color: 'text.tertiary',
textAlign: 'center',
my: 0.75,
},
listConfSep: {
mb: 0,
},
@@ -84,7 +80,7 @@ const _styles = {
},
} as const satisfies Record<string, SxProps>;
const _slotProps: SelectSlotsAndSlotProps<false>['slotProps'] = {
const _slotProps = {
// see the OptimaBarDropdown.listbox for a well made customization (max-height, max-width, etc.)
listbox: {
sx: {
@@ -127,7 +123,7 @@ const _slotProps: SelectSlotsAndSlotProps<false>['slotProps'] = {
minWidth: '6rem',
} as const,
} as const,
} as const;
} as const satisfies SelectSlotsAndSlotProps<false>['slotProps'];
interface LLMSelectOptions {
@@ -155,13 +151,14 @@ export function useLLMSelect(
llmId: undefined | DLLMId | null, // undefined: not set at all, null: has the meaning of no-llm-wanted here
setLlmId: (llmId: DLLMId | null) => void,
options: LLMSelectOptions,
): [DLLM | null, React.JSX.Element | null, React.FunctionComponent<SvgIconProps> | undefined] {
): [DLLM | null, React.JSX.Element | null] {
// options
const { label, larger = false, disabled = false, placeholder = LLM_TEXT_PLACEHOLDER, isHorizontal = false, autoRefreshDomain, appendConfigureModels = false, showStarFilter = false } = options;
// state
const [controlledOpen, setControlledOpen] = React.useState(false);
const { set: collapsedServices, toggle: toggleServiceCollapse } = useToggleableStringSet<DModelsServiceId>();
// external state
const starredOnly = useUIPreferencesStore(state => showStarFilter && state.showModelsStarredOnly);
@@ -174,34 +171,72 @@ export function useLLMSelect(
const isReasoning = !LLM_SELECT_SHOW_REASONING_ICON ? false : llm?.interfaces?.includes(LLM_IF_OAI_Reasoning) ?? false;
// Scroll preservation: MUI's useSelect auto-scrolls to highlighted item when options change - we want to preserve scroll instead
const listboxRef = React.useRef<HTMLUListElement>(null);
const listboxSlotPropsStable = React.useMemo(() => ({
..._slotProps,
listbox: { ..._slotProps.listbox, ref: listboxRef },
}), []);
React.useLayoutEffect(() => {
// restore scroll after collapse/expand - snapshot before MUI scrolls, restore via double RAF
const el = listboxRef.current;
if (!el) return;
const scrollTop = el.scrollTop;
const raf = requestAnimationFrame(() => {
// usually works, especially on expansion
el.scrollTop = scrollTop;
return requestAnimationFrame(() => {
// fixes the collapse too
el.scrollTop = scrollTop;
});
});
return () => cancelAnimationFrame(raf);
}, [collapsedServices]);
// memo LLM Options
const optimizeToSingleVisibleId = (!controlledOpen && _filteredLLMs.length > LLM_SELECT_REDUCE_OPTIONS) ? llmId : null; // id to keep visible when optimizing
const optionsArray = React.useMemo(() => {
// check if we have multiple services (to show collapsible headers)
const hasMultipleServices = _filteredLLMs.some((llm, i, arr) => i > 0 && llm.sId !== arr[i - 1].sId);
// create the option items
let formerVendor: IModelVendor | null = null;
let prevServiceId: DModelsServiceId | null = null;
return _filteredLLMs.reduce((acc, llm, _index) => {
if (optimizeToSingleVisibleId && llm.id !== optimizeToSingleVisibleId)
return acc;
const vendor = findModelVendor(llm.vId);
const vendorChanged = vendor !== formerVendor;
if (vendorChanged)
formerVendor = vendor;
const serviceVendor = findModelVendor(llm.vId);
const isServiceCollapsed = hasMultipleServices && collapsedServices.has(llm.sId);
// add separators if the vendor changed (and more than one vendor)
const addSeparator = vendorChanged && formerVendor !== null;
if (addSeparator && !optimizeToSingleVisibleId)
acc.push(<Box key={'llm-sep-' + llm.id} sx={_styles.listVendor}>{vendor?.name}</Box>);
// add collapsible service headers when changing services
if (hasMultipleServices && llm.sId !== prevServiceId) {
if (!optimizeToSingleVisibleId) {
const serviceLabel = findModelsServiceOrNull(llm.sId)?.label || serviceVendor?.name || llm.sId;
acc.push(<ListItemGroupCollapser key={'SID-' + llm.sId} id={llm.sId} label={serviceLabel} isCollapsed={isServiceCollapsed} onToggleCollapse={toggleServiceCollapse} />);
}
prevServiceId = llm.sId;
}
// skip models if service is collapsed (but always show selected model)
if (isServiceCollapsed && llm.id !== llmId)
return acc;
let features = '';
const isNotSymlink = !llm.label.startsWith('🔗');
const seemsFree = !!getLLMPricing(llm)?.chat?._isFree;
const isNotSymlink = !llm.label.startsWith('🔗'); // getLLMLabel exception: need access to the base
const llmLabel = getLLMLabel(llm);
const seemsFree = isLLMChatFree_cached(llm);
if (isNotSymlink) {
// check features
if (seemsFree) features += 'free ';
if (llm.isUserClone)
features += ' '; // is clone
if (llm.interfaces.includes(LLM_IF_OAI_Reasoning))
features += '🧠 '; // can reason
if (llm.interfaces.includes(LLM_IF_Tools_WebSearch))
@@ -221,16 +256,16 @@ export function useLLMSelect(
value={llm.id}
// Disabled to avoid regenerating the memo too frequently
// sx={llm.id === llmId ? { fontWeight: 'md' } : undefined}
label={llm.label}
label={llmLabel}
>
{!noIcons && (
<ListItemDecorator>
{(llm.userStarred && !starredOnly) ? <StarredNoXL2 /> : vendor?.id ? <LLMVendorIcon vendorId={vendor.id} /> : null}
{(llm.userStarred && !starredOnly) ? <StarredNoXL2 /> : serviceVendor?.id ? <LLMVendorIconSprite vendorId={serviceVendor.id} /> : null}
</ListItemDecorator>
)}
{/*<Tooltip title={llm.description}>*/}
<div className='agi-ellipsize'>{llm.label}</div>
<div className='agi-ellipsize'>{llmLabel}</div>
{/* Features Chips - sync with `ModelsList.tsx` */}
{!!features && !showModelOptions && <Chip size='sm' color={seemsFree ? 'success' : undefined} variant='plain' sx={_styles.chips}>{features.trim().replace(' ', ' ')}</Chip>}
@@ -244,7 +279,7 @@ export function useLLMSelect(
// variant='outlined'
onClick={(e) => {
e.stopPropagation();
optimaActions().openModelOptions(llm.id);
optimaActions().openModelOptions(llm.id, 'parameters');
}}
sx={_styles.configButton}
>
@@ -260,7 +295,7 @@ export function useLLMSelect(
return acc;
}, [] as React.JSX.Element[]);
}, [_filteredLLMs, llmId, noIcons, optimizeToSingleVisibleId, starredOnly]);
}, [_filteredLLMs, collapsedServices, llmId, noIcons, optimizeToSingleVisibleId, starredOnly, toggleServiceCollapse]);
const onSelectChange = React.useCallback((_event: unknown, value: DLLMId | null) => {
@@ -289,7 +324,7 @@ export function useLLMSelect(
listboxOpen={controlledOpen}
onListboxOpenChange={hasNoModels ? optimaOpenModels : setControlledOpen}
placeholder={hasNoModels ? LLM_TEXT_CONFIGURE : placeholder}
slotProps={_slotProps}
slotProps={listboxSlotPropsStable}
endDecorator={autoRefreshDomain ?
<TooltipOutlined title='Auto-select the model'>
<IconButton onClick={() => llmsStoreActions().assignDomainModelId(autoRefreshDomain, null)}>
@@ -332,12 +367,7 @@ export function useLLMSelect(
</Select>
{/*</Box>*/}
</FormControl>
), [appendConfigureModels, autoRefreshDomain, controlledOpen, disabled, hasNoModels, hasStarred, isHorizontal, isReasoning, label, larger, llmId, onSelectChange, optimizeToSingleVisibleId, options.color, options.sx, options.variant, optionsArray, placeholder, showNoOptions, showStarFilter, starredOnly]);
), [appendConfigureModels, autoRefreshDomain, controlledOpen, disabled, hasNoModels, hasStarred, isHorizontal, isReasoning, label, larger, listboxSlotPropsStable, llmId, onSelectChange, optimizeToSingleVisibleId, options.color, options.sx, options.variant, optionsArray, placeholder, showNoOptions, showStarFilter, starredOnly]);
// Memo the vendor icon for the chat LLM
const chatLLMVendorIconFC = React.useMemo(() => {
return !llm?.vId ? undefined : llmsGetVendorIcon(llm.vId);
}, [llm?.vId]);
return [llm, llmSelectComponent, chatLLMVendorIconFC];
return [llm, llmSelectComponent];
}
+10
View File
@@ -0,0 +1,10 @@
import * as React from 'react';
import { SvgIcon, SvgIconProps } from '@mui/joy';
export function InworldIcon(props: SvgIconProps) {
return <SvgIcon viewBox='0 0 141 181' width='24' height='24' fill='currentColor' {...props}>
<path d='M48.2616 34.7993C47.9981 34.8585 47.9766 34.6058 48.1379 34.5144C53.3155 31.4874 60.6866 30.767 61.235 30.353C61.7297 29.6594 57.864 29.697 56.5199 29.7024C46.74 30.0411 38.2989 33.681 31.288 40.2941C20.5242 49.4664 16.9973 64.5582 18.4866 78.1285C20.2285 92.1611 27.9384 105.344 39.385 113.635C63.2405 129.673 96.9242 122.011 114.774 100.183C136.753 73.537 130.194 35.6004 107.215 18.2613C101.252 14.1214 94.7359 10.8524 87.7035 8.98143C84.4238 8.00828 79.8592 7.07277 77.7624 7.00288C74.5472 6.90073 75.6171 9.54596 75.0472 10.1965C74.4773 10.8471 68.5901 11.1105 75.2515 13.4493C79.1925 14.8311 76.7946 14.9117 73.7891 17.6322C71.1278 20.0409 73.4504 22.7399 72.531 23.6861C68.5847 27.7508 74.6279 29.5358 77.7462 31.31C92.8542 39.3747 101.177 53.8913 98.4242 70.9831C94.5585 93.5644 60.9286 103.317 46.0464 85.462C34.5999 71.924 38.928 49.3858 55.2241 42.0361C57.4822 40.9447 62.5792 39.3532 60.3909 39.536C51.9552 40.2403 45.369 44.7297 43.2399 46.3588C43.0571 46.4986 42.8152 46.2835 42.9281 46.0846C44.4711 43.3695 53.4122 37.036 71.0041 34.8101C74.1494 34.1327 62.3157 31.6541 48.2616 34.7993Z' />
<path d='M55.4704 170.577C56.4274 170.566 57.3791 170.706 58.2716 170.797C60.5673 170.867 63.148 170.604 65.6696 170.932C71.1536 171.244 76.6807 172.475 82.1808 172.609C85.9712 172.889 89.5896 172.717 93.4607 172.862C96.3801 173.238 99.848 173.007 102.466 173.4C104.101 173.588 105.601 173.507 107.241 173.749C111.579 174.367 115.902 174.459 120.295 174.4C121.967 174.453 123.682 174.486 125.349 174.378C127.128 174.276 129.069 174.351 130.499 173.212C131.349 172.711 131.306 171.695 131.483 170.996C131.704 170.615 132.166 170.41 132.37 170.023C132.596 169.357 132.741 168.663 132.752 167.98V167.937C132.8 165.615 131.688 163.33 130.838 161.174C129.424 158.05 127.763 155.13 125.967 152.206C123.962 149.372 122.956 145.506 119.547 143.974C113.402 141.281 107.085 141.565 100.283 140.743C87.482 139.608 73.8849 138.775 61.9867 138.614C50.8843 138.775 31.2708 139.501 21.1738 140.399C3.2378 141.996 11.5875 144.84 11.023 149.872C10.8187 151.727 8.79172 151.576 9.01753 153.394C9.39388 156.028 14.3349 159.324 16.8188 159.991C27.76 162.916 13.2435 163.351 14.8672 166.276C16.2166 168.711 22.2383 168.012 24.6954 168.98L55.4597 170.577H55.4704Z' />
</SvgIcon>;
}
+13
View File
@@ -0,0 +1,13 @@
import * as React from 'react';
import { SvgIcon, SvgIconProps } from '@mui/joy';
export function ZAIIcon(props: SvgIconProps) {
return (
<SvgIcon viewBox='0 0 30 30' width={24} height={24} fill='currentColor' strokeWidth={0} {...props}>
<path d='M15.47,7.1l-1.3,1.85c-0.2,0.29-0.54,0.47-0.9,0.47h-7.1V7.09C6.16,7.1,15.47,7.1,15.47,7.1z' />
<polygon points='24.3,7.1 13.14,22.91 5.7,22.91 16.86,7.1' />
<path d='M14.53,22.91l1.31-1.86c0.2-0.29,0.54-0.47,0.9-0.47h7.09v2.33H14.53z' />
</SvgIcon>
);
}
+1 -1
View File
@@ -186,7 +186,7 @@ export function GoodModal(props: {
},
display: 'flex',
flexWrap: 'wrap',
gap: 1,
// gap: { md: 1 }, // Note: let the startButton decide how to space itself
justifyContent: 'space-between',
}}>
{props.startButton}
@@ -4,6 +4,8 @@ import type { SelectSlotsAndSlotProps } from '@mui/joy/Select/SelectProps';
import { Box, ListDivider, listItemButtonClasses, ListItemDecorator, listItemDecoratorClasses, Option, optionClasses, Select, selectClasses } from '@mui/joy';
import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown';
import { ListItemGroupCollapser } from '~/common/components/ListItemGroupCollapser';
// set to true to enable the dense mode, which is default in the rest of the app
const useDenseDropdowns = false;
@@ -138,6 +140,9 @@ function OptimaBarDropdown<TValue extends string>(props: {
showSymbols?: boolean | 'compact',
showGone?: boolean,
showFaded?: boolean,
// collapsible separators: when provided, separators become clickable toggle buttons
collapsedSeparators?: ReadonlySet<string>,
onSeparatorClick?: (key: string) => void,
}, ref: React.Ref<OptimaBarControlMethods>) {
// state
@@ -197,14 +202,25 @@ function OptimaBarDropdown<TValue extends string>(props: {
const label = (props.showSymbols && _item.symbol && !(_item.title === 'Default' && _item.symbol === '🧠')) ? `${_item.symbol} ${safeTitle}` : safeTitle;
const iconOrSymbol = _item.icon || _item.symbol || '';
return _item.type === 'separator' ? (
<ListDivider key={_itemKey || `sep-${idx}`}>
{/*<Box sx={{ display: 'flex', alignItems: 'center', gap: 1, '--Icon-fontSize': 'var(--joy-fontSize-lg)' }}>*/}
{/*{_item.icon} */}
{_item.title}
{/*</Box>*/}
</ListDivider>
) : (
if (_item.type === 'separator')
return props.onSeparatorClick ? (
<ListItemGroupCollapser
key={_itemKey}
id={_itemKey}
label={safeTitle}
isCollapsed={!!props.collapsedSeparators?.has(_itemKey)}
onToggleCollapse={props.onSeparatorClick}
/>
) : (
<ListDivider key={_itemKey || `sep-${idx}`}>
{/*<Box sx={{ display: 'flex', alignItems: 'center', gap: 1, '--Icon-fontSize': 'var(--joy-fontSize-lg)' }}>*/}
{/*{_item.icon} */}
{_item.title}
{/*</Box>*/}
</ListDivider>
);
return (
<Option key={_itemKey} value={_itemKey} label={label}>
{/* Icon / Symbol */}
{(props.showSymbols === true || (props.showSymbols === 'compact' && !!iconOrSymbol)) && <ListItemDecorator>
@@ -146,13 +146,16 @@ export function MobileNavItems(props: { currentApp?: NavItemApp }) {
Models
</Button>
{/* HARDCODED: Discord */}
<BringTheLove
text={navItems.links[0].name}
icon={navItems.links[0].icon}
link={navItems.links[0].href}
sx={_styles.button}
/>
{/* External links (e.g. Discord) */}
{navItems.links.slice(0 /* take up to 1 element, which is Discord */, 1).map((link) => (
<BringTheLove
key={link.name}
text={link.name}
icon={link.icon}
link={link.href}
sx={_styles.button}
/>
))}
</Box>
</Sheet>
@@ -19,7 +19,7 @@ const _styles = {
textDecoration: 'underline',
},
},
accentedTagline: {
taglineAccented: {
textAlign: 'start',
mt: 0.75,
},
@@ -37,9 +37,10 @@ const _styles = {
export function OptimaAppPageHeading(props: {
title: React.ReactNode;
tagline?: React.ReactNode;
accentedTagline?: boolean;
taglineAccented?: boolean;
startDecorator?: React.ReactNode;
endDecorator?: React.ReactNode;
disabled?: boolean;
noDivider?: boolean;
noMarginBottom?: boolean;
onClick?: (event: React.MouseEvent) => void;
@@ -50,13 +51,13 @@ export function OptimaAppPageHeading(props: {
return (
<Box mb={props.noMarginBottom ? undefined : 2.25} sx={{ overflow: 'hidden', display: 'grid' }}>
{!!props.title && <Typography level={isMobile ? 'h3' : 'h2'} startDecorator={props.startDecorator} endDecorator={props.endDecorator} sx={_styles.title}>
{!!props.title && <Typography level={isMobile ? 'h3' : 'h2'} startDecorator={props.startDecorator} endDecorator={props.endDecorator} textColor={props.disabled ? 'neutral.plainDisabledColor' : undefined} sx={_styles.title}>
{props.onClick
? <Box component='span' sx={_styles.textClickable} onClick={props.onClick} className='agi-ellipsize'>{props.title}</Box>
: <span className='agi-ellipsize'>{props.title}</span>
}
</Typography>}
{!!props.tagline && <Typography level='body-sm' sx={props.accentedTagline ? _styles.accentedTagline : _styles.tagline}>
{!!props.tagline && <Typography level='body-sm' sx={props.taglineAccented ? _styles.taglineAccented : _styles.tagline}>
{props.tagline}
</Typography>}
{!props.noDivider && <ListDivider sx={_styles.divisor} />}
@@ -10,6 +10,8 @@ import { OPTIMA_OPEN_DEBOUNCE, OPTIMA_PEEK_HOVER_ENTER_DELAY, OPTIMA_PEEK_HOVER_
export type PreferencesTabId = 'chat' | 'voice' | 'draw' | 'tools' | undefined;
export type ModelOptionsContext = 'full' | 'parameters';
interface OptimaState {
@@ -27,6 +29,7 @@ interface OptimaState {
showKeyboardShortcuts: boolean;
showLogger: boolean;
showModelOptions: DLLMId | false;
showModelOptionsContext: ModelOptionsContext;
showModels: boolean;
showPreferences: boolean;
preferencesTab: PreferencesTabId;
@@ -51,6 +54,7 @@ const modalsClosedState = {
showKeyboardShortcuts: false,
showLogger: false,
showModelOptions: false,
showModelOptionsContext: 'full' as ModelOptionsContext,
showModels: false,
showPreferences: false,
} as const;
@@ -102,7 +106,7 @@ export interface OptimaActions {
openLogger: () => void;
closeModelOptions: () => void;
openModelOptions: (id: DLLMId) => void;
openModelOptions: (id: DLLMId, context?: ModelOptionsContext) => void;
closeModels: () => void;
openModels: () => void;
@@ -209,7 +213,7 @@ export const useLayoutOptimaStore = create<OptimaState & OptimaActions>((_set, _
openLogger: () => _set({ ...modalsClosedState, showLogger: true }),
closeModelOptions: () => _set({ showModelOptions: false }),
openModelOptions: (id: DLLMId) => _set({ showModelOptions: id }),
openModelOptions: (id: DLLMId, context?: ModelOptionsContext) => _set({ showModelOptions: id, showModelOptionsContext: context ?? 'full' }),
closeModels: () => _set({ showModels: false }),
openModels: () => _set({ showModels: true }),
+1
View File
@@ -91,6 +91,7 @@ export function useOptimaModals() {
showKeyboardShortcuts: state.showKeyboardShortcuts,
showLogger: state.showLogger,
showModelOptions: state.showModelOptions,
showModelOptionsContext: state.showModelOptionsContext,
showModels: state.showModels,
showPreferences: state.showPreferences,
preferencesTab: state.preferencesTab,
@@ -27,6 +27,9 @@ export type GlobalOverlayId = // string - disabled so we keep an orderliness
| 'livefile-overwrite'
| 'shortcuts-confirm-close'
| 'blocks-off-enhance-code'
| 'llms-remove-clones'
| 'llms-reset-parameters'
| 'llms-reset-visibility'
| 'llms-service-remove'
| 'composer-unsupported-attachments' // The LLM does not seem to support this mime type - continue anyway?
| 'composer-open-or-attach' // Open a file or attach it to the chat?
@@ -11,6 +11,10 @@ export function setupClientUncaughtErrorsLogging(): () => void {
// Handle uncaught exceptions
const handleError = (event: ErrorEvent) => {
// Ignore benign ResizeObserver errors (browser warning, not an actual error)
if (event.message?.includes('ResizeObserver loop'))
return;
logger.error('Uncaught error', {
message: event.error?.message || event.message,
stack: event.error?.stack,
+4
View File
@@ -3,6 +3,8 @@ import * as React from 'react';
import { CacheProvider, EmotionCache } from '@emotion/react';
import { CssBaseline, CssVarsProvider } from '@mui/joy';
import { VendorIconSpriteMemo } from '~/modules/llms/components/LLMVendorIconSprite';
import { createAppTheme, createEmotionCache } from '~/common/app.theme';
import { useUIComplexityIsMinimal } from '~/common/stores/store-ui';
@@ -61,6 +63,8 @@ export const ProviderTheming = (props: { emotionCache?: EmotionCache, children:
<CacheProvider value={props.emotionCache || clientSideEmotionCache}>
<CssVarsProvider defaultMode='light' theme={theme}>
<CssBaseline />
{/* Inject sprites to be referenced by SVG rendering */}
<VendorIconSpriteMemo />
{/* Disabled for now, we don't use those */}
{/*<_GlobalSVGFiltersMemo />*/}
{props.children}
@@ -54,6 +54,8 @@ const scrollableBoxSx: SxProps = {
overflowY: 'auto',
// actually make sure this scrolls & fills
height: '100%',
// prevents pull-to-refresh on mobile when scrolling up in the chat
overscrollBehaviorY: 'none',
} as const;
+7 -7
View File
@@ -208,7 +208,7 @@ export type DMessageToolInvocationPart = {
type: 'code_execution';
language: string;
code: string;
author: 'gemini_auto_inline';
author: DMessageToolCodeExecutor;
}
};
@@ -224,12 +224,12 @@ export type DMessageToolResponsePart = {
} | {
type: 'code_execution';
result: string; // The output
executor: 'gemini_auto_inline';
executor: DMessageToolCodeExecutor;
},
environment: DMessageToolEnvironment,
};
type DMessageToolEnvironment = 'upstream' | 'server' | 'client';
type DMessageToolCodeExecutor = 'gemini_auto_inline' | 'code_interpreter';
type DVoidModelAnnotationsPart = {
pt: 'annotations',
@@ -403,7 +403,7 @@ export function create_FunctionCallInvocation_ContentFragment(id: string, functi
return _createContentFragment(_create_FunctionCallInvocation_Part(id, functionName, args));
}
export function create_CodeExecutionInvocation_ContentFragment(id: string, language: string, code: string, author: 'gemini_auto_inline'): DMessageContentFragment {
export function create_CodeExecutionInvocation_ContentFragment(id: string, language: string, code: string, author: DMessageToolCodeExecutor): DMessageContentFragment {
return _createContentFragment(_create_CodeExecutionInvocation_Part(id, language, code, author));
}
@@ -411,7 +411,7 @@ export function create_FunctionCallResponse_ContentFragment(id: string, error: b
return _createContentFragment(_create_FunctionCallResponse_Part(id, error, name, result, environment));
}
export function create_CodeExecutionResponse_ContentFragment(id: string, error: boolean | string, result: string, executor: 'gemini_auto_inline', environment: DMessageToolEnvironment): DMessageContentFragment {
export function create_CodeExecutionResponse_ContentFragment(id: string, error: boolean | string, result: string, executor: DMessageToolCodeExecutor, environment: DMessageToolEnvironment): DMessageContentFragment {
return _createContentFragment(_create_CodeExecutionResponse_Part(id, error, result, executor, environment));
}
@@ -553,7 +553,7 @@ function _create_FunctionCallInvocation_Part(id: string, functionName: string, a
return { pt: 'tool_invocation', id, invocation: { type: 'function_call', name: functionName, args } };
}
function _create_CodeExecutionInvocation_Part(id: string, language: string, code: string, author: 'gemini_auto_inline'): DMessageToolInvocationPart {
function _create_CodeExecutionInvocation_Part(id: string, language: string, code: string, author: DMessageToolCodeExecutor): DMessageToolInvocationPart {
return { pt: 'tool_invocation', id, invocation: { type: 'code_execution', language, code, author } };
}
@@ -561,7 +561,7 @@ function _create_FunctionCallResponse_Part(id: string, error: boolean | string,
return { pt: 'tool_response', id, error, response: { type: 'function_call', name, result }, environment };
}
function _create_CodeExecutionResponse_Part(id: string, error: boolean | string, result: string, executor: 'gemini_auto_inline', environment: DMessageToolEnvironment): DMessageToolResponsePart {
function _create_CodeExecutionResponse_Part(id: string, error: boolean | string, result: string, executor: DMessageToolCodeExecutor, environment: DMessageToolEnvironment): DMessageToolResponsePart {
return { pt: 'tool_response', id, error, response: { type: 'code_execution', result, executor }, environment };
}
+58 -47
View File
@@ -1,65 +1,76 @@
import { DBlobAssetId, gcDBImageAssets } from '~/common/stores/blob/dblobs-portability';
import type { Immutable } from '~/common/types/immutable.types';
import type { DConversation } from './chat.conversation';
import type { DMessageFragment } from './chat.fragments';
import { isContentOrAttachmentFragment, isImageRefPart, isZyncAssetReferencePart } from './chat.fragments';
import { useChatStore } from './store-chats';
// --- Asset collector registration ---
/**
* Allows external systems (Beam, scratch chat, etc.), to protect their DBlob assets from GC without creating circular dependencies.
*/
const _assetCollectors: AssetCollectorFn[] = [];
type AssetCollectorFn = () => DBlobAssetId[];
/**
* Register a callback that returns additional DBlob asset IDs to keep during GC.
* Uses inversion of control to avoid circular dependency (chat/ -> chat-overlay/).
* @returns unregister function
*/
export function gcRegisterAssetCollector(collector: AssetCollectorFn): () => void {
_assetCollectors.push(collector);
return () => {
const idx = _assetCollectors.indexOf(collector);
if (idx >= 0) _assetCollectors.splice(idx, 1);
};
}
/**
* Collect DBlob asset IDs referenced in message fragments.
*/
export function collectFragmentAssetIds(fragments: Immutable<DMessageFragment[]>, assetIds: Set<DBlobAssetId>): void {
for (const fragment of fragments) {
if (!isContentOrAttachmentFragment(fragment)) continue;
// New References to Zync Assets (dblob refs for compatibility/migration)
if (isZyncAssetReferencePart(fragment.part) && fragment.part._legacyImageRefPart?.dataRef?.reftype === 'dblob')
assetIds.add(fragment.part._legacyImageRefPart.dataRef.dblobAssetId);
// Legacy 'image_ref' parts (direct dblob refs)
if (isImageRefPart(fragment.part) && fragment.part.dataRef?.reftype === 'dblob')
assetIds.add(fragment.part.dataRef.dblobAssetId);
}
}
/**
* Garbage collect unreferenced dblobs in global chats
* - This is ran as a side effect of the chat store rehydration
* - This is also ran when a conversation or message is deleted, or when a conversation messages history is replaced
*/
export async function gcChatImageAssets(conversations?: DConversation[]) {
export async function gcChatImageAssets(conversations?: Immutable<DConversation[]>) {
// find all the dblob references in all chats
const chatsAssetIDs: Set<DBlobAssetId> = new Set();
const _conversations = conversations || useChatStore.getState().conversations;
for (const chat of _conversations) {
for (const message of chat.messages) {
for (const fragment of message.fragments) {
for (const chat of _conversations)
for (const message of chat.messages)
collectFragmentAssetIds(message.fragments, chatsAssetIDs);
// only operate on content or attachment fragments
if (!isContentOrAttachmentFragment(fragment)) continue;
// New References to Zync Assets (dblob refs for compatibility/migration)
if (isZyncAssetReferencePart(fragment.part) && fragment.part._legacyImageRefPart?.dataRef?.reftype === 'dblob')
chatsAssetIDs.add(fragment.part._legacyImageRefPart.dataRef.dblobAssetId);
// Legacy 'image_ref' parts (direct dblob refs)
if (isImageRefPart(fragment.part) && fragment.part.dataRef?.reftype === 'dblob')
chatsAssetIDs.add(fragment.part.dataRef.dblobAssetId);
}
}
}
// FIXME: [ASSET-GC-BEAM] GC deletes assets still referenced in Beam rays, causing images to disappear
// Bug occurs when: (1) Beam is open with imported rays containing images, (2) user regenerates/deletes
// those messages in the chat pane, (3) GC only scans main conversation store, not Beam vanilla stores,
// (4) assets are deleted while still displayed in Beam rays.
// Fix: Uncomment code below to scan all Beam stores for asset references before GC.
// Note: Also add import: import { ConversationsManager } from '~/common/chat-overlay/ConversationsManager';
// Reproduction: Open Beam on right with images → regenerate (Ctrl+Shift+Z) on left -> images disappear.
//
// // Scan Beam rays for each conversation
// for (const conversation of _conversations) {
// const handler = ConversationsManager.getHandler(conversation.id);
// if (!handler.isValid()) continue;
//
// const rays = handler.beamStore.getState().rays;
// for (const ray of rays) {
// for (const fragment of ray.message.fragments) {
// if (!isContentOrAttachmentFragment(fragment)) continue;
//
// // New References to Zync Assets (dblob refs for compatibility/migration)
// if (isZyncAssetReferencePart(fragment.part) && fragment.part._legacyImageRefPart?.dataRef?.reftype === 'dblob')
// chatsAssetIDs.add(fragment.part._legacyImageRefPart.dataRef.dblobAssetId);
//
// // Legacy 'image_ref' parts (direct dblob refs)
// if (isImageRefPart(fragment.part) && fragment.part.dataRef?.reftype === 'dblob')
// chatsAssetIDs.add(fragment.part.dataRef.dblobAssetId);
// }
// }
// }
// [ASSET-GC-BEAM] Collect additional asset IDs from registered collectors (Beam, scratch chat, etc.)
// to prevent GC from deleting assets still displayed in ephemeral overlay stores (e.g. Beam rays/fusions).
// Bug: Beam images disappeared when regenerating/deleting chat messages while Beam was open, because
// GC only scanned conversation messages and not the vanilla Beam stores. Registration pattern avoids
// the circular dependency (chat/ -> chat-overlay/).
for (const collector of _assetCollectors)
for (const assetId of collector())
chatsAssetIDs.add(assetId);
// sanity check: if no blobs are referenced, do nothing; in case we have a state bug and we don't wipe the db
if (!chatsAssetIDs.size)
@@ -71,4 +82,4 @@ export async function gcChatImageAssets(conversations?: DConversation[]) {
// FIXME: [ASSET] will only be able to GC local assets that haven't been uploaded to the cloud - otherwise they could be used,
// in which case only the cloud can centralized-GC, or user will have to manually delete them
}
}
+6 -2
View File
@@ -141,6 +141,7 @@ export type DMessageGenerator = ({
},
}) & {
metrics?: DMetricsChatGenerate_Md; // medium-sized metrics stored in the message
providerInfraLabel?: string; // upstream provider that served the request (e.g., OpenRouter provider routing)
upstreamHandle?: {
uht: 'vnd.oai.responses',
responseId: string,
@@ -244,6 +245,7 @@ export function duplicateDMessageGenerator(generator: Readonly<DMessageGenerator
name: generator.name,
// ...(generator.xeOpCode ? { xeOpCode: generator.xeOpCode } : {}),
...(generator.metrics ? { metrics: { ...generator.metrics } } : {}),
...(generator.providerInfraLabel ? { providerInfraLabel: generator.providerInfraLabel } : {}),
...(generator.upstreamHandle ? { upstreamHandle: { ...generator.upstreamHandle } } : {}),
...(generator.tokenStopReason ? { tokenStopReason: generator.tokenStopReason } : {}),
};
@@ -253,6 +255,7 @@ export function duplicateDMessageGenerator(generator: Readonly<DMessageGenerator
name: generator.name,
aix: { ...generator.aix },
...(generator.metrics ? { metrics: { ...generator.metrics } } : {}),
...(generator.providerInfraLabel ? { providerInfraLabel: generator.providerInfraLabel } : {}),
...(generator.upstreamHandle ? { upstreamHandle: { ...generator.upstreamHandle } } : {}),
...(generator.tokenStopReason ? { tokenStopReason: generator.tokenStopReason } : {}),
};
@@ -263,6 +266,7 @@ export function duplicateDMessageGenerator(generator: Readonly<DMessageGenerator
// helpers - status checks
export function messageWasInterruptedAtStart(message: Pick<DMessage, 'generator' | 'fragments'>): boolean {
// FIXME: placeholder-check (see below) too here?
return message.generator?.tokenStopReason === 'client-abort' && !message.fragments?.length;
}
@@ -300,8 +304,8 @@ function _messageSetGeneratorAIX(message: Pick<DMessage, 'generator'>, modelLabe
export function messageSetGeneratorAIX_AutoLabel(message: Pick<DMessage, 'generator'>, modelVendorId: ModelVendorId, modelId: DLLMId): void {
// Simply strip the first part of the modelId, which is the serviceId, before the dash.
const heuristicLabel = modelId.includes('-') ? modelId.replace(/^[^-]+-/, '') : modelId;
// Strip the serviceId prefix: 'vendor-' or 'vendor-N-' (when multiple providers of same vendor)
const heuristicLabel = modelId.includes('-') ? modelId.replace(/^[^-]+-(\d-)?/, '') : modelId;
_messageSetGeneratorAIX(message, heuristicLabel, modelVendorId, modelId);
}
+10 -14
View File
@@ -41,7 +41,7 @@ export interface ChatActions {
abortConversationTemp: (cId: DConversationId) => void;
historyReplace: (cId: DConversationId, messages: DMessage[]) => void;
historyTruncateToIncluded: (cId: DConversationId, mId: DMessageId, offset: number) => void;
historyKeepLastThinkingOnly: (cId: DConversationId) => void;
historyStripThinking: (cId: DConversationId, keepCount: number /* 0 = discard all, 1 = keep last */) => void;
historyView: (cId: DConversationId) => Readonly<DMessage[]> | undefined;
appendMessage: (cId: DConversationId, message: DMessage) => void;
deleteMessage: (cId: DConversationId, mId: DMessageId) => void;
@@ -247,30 +247,26 @@ export const useChatStore = create<ConversationsStore>()(/*devtools(*/
};
}),
historyKeepLastThinkingOnly: (conversationId: DConversationId) =>
historyStripThinking: (conversationId: DConversationId, keepCount: number) =>
_get()._editConversation(conversationId, ({ messages: _currentMessages }) => {
let madeChanges = false;
const updatedMessages = [..._currentMessages];
let foundLastAssistant = false;
let assistantsSeen = 0;
// reverse iterate
// reverse iterate to find and skip `keepCount` most recent assistant messages
for (let i = updatedMessages.length - 1; i >= 0; i--) {
const message = updatedMessages[i];
// skip non-assistant messages
if (message.role !== 'assistant') continue;
// skip the last assistant message
if (!foundLastAssistant) {
foundLastAssistant = true;
continue;
}
// preserve the N most recent assistant messages
if (assistantsSeen++ < keepCount) continue;
// skip if doesn't have thinking blocks
const hasThinkingBlocks = message.fragments.some(isVoidThinkingFragment);
if (!hasThinkingBlocks) continue;
// strip thinking blocks from older messages
if (!message.fragments.some(isVoidThinkingFragment)) continue;
// Filter out thinking blocks
// filter out thinking blocks
updatedMessages[i] = {
...message,
fragments: message.fragments.filter(fragment => !isVoidThinkingFragment(fragment)),
@@ -283,7 +279,7 @@ export const useChatStore = create<ConversationsStore>()(/*devtools(*/
return {
messages: updatedMessages,
// No need to update the following as void fragments don't contribute
// tokenCount: updateMessagesTokenCounts(updatedMessages, true, 'historyKeepLastThinkingOnly'),
// tokenCount: updateMessagesTokenCounts(updatedMessages, true, 'historyStripThinking'),
// updated: Date.now(),
};
}),
@@ -0,0 +1,51 @@
import * as React from 'react';
import type { SxProps } from '@mui/joy/styles/types';
import { IconButton } from '@mui/joy';
import { DebouncedInputMemo } from '~/common/components/DebouncedInput';
import { StarredState } from '~/common/components/StarIcons';
const _styles = {
filterBox: {
m: 1.5,
mb: 1,
backgroundColor: 'background.level1',
'&:focus-within': { backgroundColor: 'background.popup' },
},
} as const satisfies Record<string, SxProps>;
/**
* Model Selection Dropdowns: Shared search input with starred filter toggle
*/
export const LLMSearchFilterInput = React.memo(function LLMSearchFilterInput(props: {
size?: 'sm' | 'md' | 'lg',
llmsCount: number,
onSearch: (search: string | null) => void,
onStarredToggle?: () => void, // if provided, shows the starred filter button
showStarredOnly?: boolean,
}) {
return (
<DebouncedInputMemo
size={props.size}
aggressiveRefocus
debounceTimeout={300}
onDebounce={props.onSearch}
placeholder={`Search ${props.llmsCount} models...`}
startDecorator={props.onStarredToggle ? (
<IconButton
size='sm'
variant='plain'
aria-label='Filter starred models'
onClick={props.onStarredToggle}
>
<StarredState isStarred={!!props.showStarredOnly} />
{/*<StarIconUnstyled isStarred={showStarredOnly} />*/}
</IconButton>
) : undefined}
sx={_styles.filterBox}
/>
);
});

Some files were not shown because too many files have changed in this diff Show More