Merge branch 'release-1.8.0'

Maintainers Release
1.8.0: Readme and Changelog
2026-05-11 14:10:15 -07:00 · 2023-12-20 02:38:51 -08:00 · 2023-12-20 02:32:47 -08:00 · 2023-12-20 02:11:13 -08:00 · 2023-12-20 01:54:23 -08:00 · 2023-12-20 01:11:08 -08:00
157 changed files with 6265 additions and 2184 deletions
@@ -15,12 +15,14 @@ assignees: enricoros
  - [ ] Assign all the shipped roadmap Issues
  - [ ] Assign the relevant [recently closed Isssues](https://github.com/enricoros/big-agi/issues?q=is%3Aclosed+sort%3Aupdated-desc)
 - Code changes:
-  - [ ] Create a release branch 'release-x.y.z', and commit:
+  - [ ] Create a release branch 'release-x.y.z': `git checkout -b release-1.2.3`
+  - [ ] Create a temporary tag `git tag v1.2.3 && git push opensource --tags`
+  - [ ] Create a [New Draft GitHub Release](https://github.com/enricoros/big-agi/releases/new), and generate the automated changelog (for new contributors)
  - [ ] Update the release version in package.json, and `npm i`
-  - [ ] Update in-app News [src/apps/news/news.data.tsx](src/apps/news/news.data.tsx)
+  - [ ] Update in-app News [src/apps/news/news.data.tsx](/src/apps/news/news.data.tsx)
  - [ ] Update the in-app News version number
  - [ ] Update the readme with the new release
-  - [ ] Copy the highlights to the [changelog](docs/changelog.md)
+  - [ ] Copy the highlights to the [docs/changelog.md](/docs/changelog.md)
 - Release:
  - [ ] merge onto main
  - [ ] verify deployment on Vercel
@@ -32,26 +34,42 @@ assignees: enricoros
  - [ ] Discord announcement
  - [ ] Twitter announcement

-## Artifacts

-1) first copy and paste the former release `discord announcement`, `news.data.ts`, `changelog.md`, `README.md`
-2) then copy and paste the milestone and each indivdual issue (content will be downloaded)
-3) then paste the git changelog 1.2.2...1.2.3
+## Links
+Milestone: 
+Former release task: 
+GitHub release: 

-### news.data.tsx
+
+## Artifacts Generation
+
+1) The following is my opensource application
+   - paste README.md
+2) I am announcing a new version, 1.7.0. The following were the announcements for 1.6.0. Discord announcement, GitHub Release, in-app news.data.tsx, changelog.md.
+   - paste the former: `discord announcement`, `GitHub release`, `news.data.tsx`, `changelog.md`
+3) The following is the new data I have for 1.7.0
+   - paste the link to the milestone (closed) and each individual issue (content will be downloaded)
+   - paste the git changelog  `git log v1.6.0..v1.7.0 | clip`
+   
+
+### news.data.TSX

 ```markdown
 I need the following from you:

-1. a table summarizing all the new features in 1.2.3, which will be used for the artifacts later
+1. a table summarizing all the new features in 1.2.3 (description, significance, usefulness, do not link the commit, but have the issue number), which will be used for the artifacts later
 2. after the table score each feature from a user impact and magnitude point of view
-3. Improve the table, in decreasing order of importance for features, fixing any detail that's missing
+3. Improve the table, in decreasing order of importance for features, fixing any detail that's missing, in particular check if there are commits of significance from a user or developer point of view, which are not contained in the table
 4. I want you then to update the news.data.tsx for the new release
 ```

 ### GitHub release

-Now paste the former release (or 1.5.0 which was accurate and great), including the new contributors and
+```markdown
+Please create the 1.2.3 Release Notes for GitHub. The following were the Release Notes for 1.1.0. Use a truthful and honest tone, undestanding that people's time and attention span is short. Today is 2023-12-20.
+```
+
+Now paste-attachment the former release notes (or 1.5.0 which was accurate and great), including the new contributors and
 some stats (# of commits, etc.), and roll it for the new release.

 ### Discord announcement
@@ -7,11 +7,15 @@
 # To get a newer version, you will need to update the SHA.
 # You can also reference a tag or branch, but the action may change without warning.

-name: Create and publish a Docker image
+name: Create and publish Docker images

 on:
  push:
-    branches: ['main']
+    branches:
+      - main
+      #- main-stable  # Disabled as the v* tag is used for stable releases
+    tags:
+      - 'v*'  # Trigger on version tags (e.g., v1.7.0)

 env:
  REGISTRY: ghcr.io
@@ -26,7 +30,7 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4

      - name: Log in to the Container registry
        uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
@@ -40,11 +44,17 @@ jobs:
        uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=raw,value=development,enable=${{ github.ref == 'refs/heads/main' }}
+            type=raw,value=stable,enable=${{ github.ref == 'refs/heads/main-stable' }}
+            type=ref,event=tag  # Use the tag name as a tag for tag builds
+            type=semver,pattern={{version}}  # Generate semantic versioning tags for tag builds

      - name: Build and push Docker image
        uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
        with:
          context: .
+          file: Dockerfile
          push: true
          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
+          labels: ${{ steps.meta.outputs.labels }}
@@ -1,8 +1,8 @@
 # BIG-AGI 🧠✨

-Welcome to big-AGI 👋, the GPT application for Pro users that combines utility,
-simplicity, and speed. Powered by the latest models from 7 vendors, including
-open-source, `big-AGI` offers best-in-class Voice and Chat with AI Personas,
+Welcome to big-AGI 👋, the GPT application for professionals that need function, form,
+simplicity, and speed. Powered by the latest models from 8 vendors and
+open-source model servers, `big-AGI` offers best-in-class Voice and Chat with AI Personas,
 visualizations, coding, drawing, calling, and quite more -- all in a polished UX.

 Pros use big-AGI. 🚀 Developers love big-AGI. 🤖
@@ -11,7 +11,7 @@ Pros use big-AGI. 🚀 Developers love big-AGI. 🤖

 Or fork & run on Vercel

-[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-agi&env=OPENAI_API_KEY,OPENAI_API_HOST&envDescription=OpenAI%20KEY%20for%20your%20deployment.%20Set%20HOST%20only%20if%20non-default.)
+[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-agi&env=OPENAI_API_KEY&envDescription=Backend%20API%20keys%2C%20optional%20and%20may%20be%20overridden%20by%20the%20UI.&envLink=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-AGI%2Fblob%2Fmain%2Fdocs%2Fenvironment-variables.md&project-name=big-agi)

 ## 👉 [roadmap](https://github.com/users/enricoros/projects/4/views/2)

@@ -21,7 +21,30 @@ shows the current developments and future ideas.
 - Got a suggestion? [_Add your roadmap ideas_](https://github.com/enricoros/big-agi/issues/new?&template=roadmap-request.md)
 - Want to contribute? [_Pick up a task!_](https://github.com/users/enricoros/projects/4/views/4) - _easy_ to _pro_

-### What's New in 1.6.0 - Nov 28, 2023 🌟
+### What's New in 1.8.0 · Dec 20, 2023 · To The Moon And Back · 🚀🌕🔙
+
+- **Google Gemini Support**: Use the newest Google models. [#275](https://github.com/enricoros/big-agi/issues/275)
+- **Mistral Platform**: Mixtral and future models support. [#273](https://github.com/enricoros/big-agi/issues/273)
+- **Diagram Instructions**. Thanks to @joriskalz! [#280](https://github.com/enricoros/big-agi/pull/280)
+- Ollama Chats: Enhanced chatting experience. [#270](https://github.com/enricoros/big-agi/issues/270)
+- Mac Shortcuts Fix: Improved UX on Mac
+- **Single-Tab Mode**: Data integrity with single window. [#268](https://github.com/enricoros/big-agi/issues/268)
+- **Updated Models**: Latest Ollama (v0.1.17) and OpenRouter models
+- Official Downloads: Easy access to the latest big-AGI on [big-AGI.com](https://big-agi.com)
+- For developers: [troubleshot networking](https://github.com/enricoros/big-AGI/issues/276#issuecomment-1858591483), fixed Vercel deployment, cleaned up the LLMs/Streaming framework
+
+### What's New in 1.7.0 · Dec 11, 2023
+
+- **Attachments System Overhaul**: Drag, paste, link, snap, text, images, PDFs and more. [#251](https://github.com/enricoros/big-agi/issues/251)
+- **Desktop Webcam Capture**: Image capture now available as Labs feature. [#253](https://github.com/enricoros/big-agi/issues/253)
+- **Independent Browsing**: Full browsing support with Browserless. [Learn More](https://github.com/enricoros/big-agi/blob/main/docs/config-browse.md)
+- **Overheat LLMs**: Push the creativity with higher LLM temperatures. [#256](https://github.com/enricoros/big-agi/issues/256)
+- **Model Options Shortcut**: Quick adjust with `Ctrl+Shift+O`
+- Optimized Voice Input and Performance
+- Latest Ollama and Oobabooga models
+- For developers: **Password Protection**: HTTP Basic Auth. [Learn How](https://github.com/enricoros/big-agi/blob/main/docs/deploy-authentication.md)
+
+### What's New in 1.6.0 - Nov 28, 2023

 - **Web Browsing**: Download web pages within chats - [browsing guide](https://github.com/enricoros/big-agi/blob/main/docs/config-browse.md)
 - **Branching Discussions**: Create new conversations from any message
@@ -82,7 +105,8 @@ the [past releases changelog](docs/changelog.md).
 ![React](https://img.shields.io/badge/React-61DAFB?style=&logo=react&logoColor=black)
 ![Next.js](https://img.shields.io/badge/Next.js-000000?style=&logo=vercel&logoColor=white)

-Clone this repo, install the dependencies, and run the development server:
+Clone this repo, install the dependencies (all locally), and run the development server (which auto-watches the
+files for changes):

 ```bash
 git clone https://github.com/enricoros/big-agi.git
@@ -91,15 +115,23 @@ npm install
 npm run dev
 ```

-The app will be running on `http://localhost:3000`
+The development app will be running on `http://localhost:3000`. Development builds have the advantage of not requiring
+a build step, but can be slower than production builds. Also, development builds won't have timeout on edge functions.

-Integrations:
+## 🌐 Deploy manually

-* Local models: Ollama, Oobabooga, LocalAi, etc.
-* [ElevenLabs](https://elevenlabs.io/) Voice Synthesis (bring your own voice too) - Settings > Text To Speech
-* [Helicone](https://www.helicone.ai/) LLM Observability Platform - Models > OpenAI > Advanced > API Host: 'oai.hconeai.com'
-* [Paste.gg](https://paste.gg/) Paste Sharing - Chat Menu > Share via paste.gg
-* [Prodia](https://prodia.com/) Image Generation - Settings > Image Generation > Api Key & Model
+The _production_ build of the application is optimized for performance and is performed by the `npm run build` command,
+after installing the required dependencies.
+
+```bash
+# .. repeat the steps above up to `npm install`, then:
+npm run build
+npm run start --port 3000
+```
+
+The app will be running on the specified port, e.g. `http://localhost:3000`.
+
+Want to deploy with username/password? See the [Authentication](docs/deploy-authentication.md) guide.

 ## 🐳 Deploy with Docker

@@ -115,7 +147,7 @@ docker run -d -p 3000:3000 big-agi
 Or run the official container:

 - manually: `docker run -d -p 3000:3000 ghcr.io/enricoros/big-agi`
- or, with docker-compose: `docker-compose up`
+- or, with docker-compose: `docker-compose up` or see [the documentation](docs/deploy-docker.md) for a composer file with integrated browsing

 ## ☁️ Deploy on Cloudflare Pages

@@ -125,9 +157,15 @@ Please refer to the [Cloudflare deployment documentation](docs/deploy-cloudflare

 Create your GitHub fork, create a Vercel project over that fork, and deploy it. Or press the button below for convenience.

-[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-agi&env=OPENAI_API_KEY,OPENAI_API_HOST&envDescription=OpenAI%20KEY%20for%20your%20deployment.%20Set%20HOST%20only%20if%20non-default.)
+[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-agi&env=OPENAI_API_KEY&envDescription=Backend%20API%20keys%2C%20optional%20and%20may%20be%20overridden%20by%20the%20UI.&envLink=https%3A%2F%2Fgithub.com%2Fenricoros%2Fbig-AGI%2Fblob%2Fmain%2Fdocs%2Fenvironment-variables.md&project-name=big-agi)

+## Integrations:

+* Local models: Ollama, Oobabooga, LocalAi, etc.
+* [ElevenLabs](https://elevenlabs.io/) Voice Synthesis (bring your own voice too) - Settings > Text To Speech
+* [Helicone](https://www.helicone.ai/) LLM Observability Platform - Models > OpenAI > Advanced > API Host: 'oai.hconeai.com'
+* [Paste.gg](https://paste.gg/) Paste Sharing - Chat Menu > Share via paste.gg
+* [Prodia](https://prodia.com/) Image Generation - Settings > Image Generation > Api Key & Model

 <br/>

@@ -1,2 +1,2 @@
 export const runtime = 'edge';
-export { openaiStreamingRelayHandler as POST } from '~/modules/llms/transports/server/openai/openai.streaming';
+export { llmStreamingRelayHandler as POST } from '~/modules/llms/server/llm.server.streaming';
@@ -1,8 +1,12 @@
+# Very simple docker-compose file to run the app on http://localhost:3000 (or http://127.0.0.1:3000).
+#
+# For more examples, such runnin big-AGI alongside a web browsing service, see the `docs/docker` folder.
+
 version: '3.9'

 services:
  big-agi:
-    image: ghcr.io/enricoros/big-agi:main
+    image: ghcr.io/enricoros/big-agi:latest
    ports:
      - "3000:3000"
    env_file:
@@ -5,12 +5,35 @@ by release.

 - For the live roadmap, please see [the GitHub project](https://github.com/users/enricoros/projects/4/views/2)

-### 1.7.0 - Dec 2023
+### 1.9.0 - Dec 2023

 - work in progress: [big-AGI open roadmap](https://github.com/users/enricoros/projects/4/views/2), [help here](https://github.com/users/enricoros/projects/4/views/4)
- milestone: [1.7.0](https://github.com/enricoros/big-agi/milestone/7)
+- milestone: [1.9.0](https://github.com/enricoros/big-agi/milestone/9)

-### ✨ What's New in 1.6.0 👊 - Nov 28, 2023
+### What's New in 1.8.0 · Dec 20, 2023 · To The Moon And Back · 🚀🌕🔙
+
+- **Google Gemini Support**: Use the newest Google models. [#275](https://github.com/enricoros/big-agi/issues/275)
+- **Mistral Platform**: Mixtral and future models support. [#273](https://github.com/enricoros/big-agi/issues/273)
+- **Diagram Instructions**. Thanks to @joriskalz! [#280](https://github.com/enricoros/big-agi/pull/280)
+- Ollama Chats: Enhanced chatting experience. [#270](https://github.com/enricoros/big-agi/issues/270)
+- Mac Shortcuts Fix: Improved UX on Mac
+- **Single-Tab Mode**: Data integrity with single window. [#268](https://github.com/enricoros/big-agi/issues/268)
+- **Updated Models**: Latest Ollama (v0.1.17) and OpenRouter models
+- Official Downloads: Easy access to the latest big-AGI on [big-AGI.com](https://big-agi.com)
+- For developers: [troubleshot networking](https://github.com/enricoros/big-AGI/issues/276#issuecomment-1858591483), fixed Vercel deployment, cleaned up the LLMs/Streaming framework
+
+### What's New in 1.7.0 · Dec 11, 2023 · Attachment Theory
+
+- **Attachments System Overhaul**: Drag, paste, link, snap, text, images, PDFs and more. [#251](https://github.com/enricoros/big-agi/issues/251)
+- **Desktop Webcam Capture**: Image capture now available as Labs feature. [#253](https://github.com/enricoros/big-agi/issues/253)
+- **Independent Browsing**: Full browsing support with Browserless. [Learn More](https://github.com/enricoros/big-agi/blob/main/docs/config-browse.md)
+- **Overheat LLMs**: Push the creativity with higher LLM temperatures. [#256](https://github.com/enricoros/big-agi/issues/256)
+- **Model Options Shortcut**: Quick adjust with `Ctrl+Shift+O`
+- Optimized Voice Input and Performance
+- Latest Ollama and Oobabooga models
+- For developers: **Password Protection**: HTTP Basic Auth. [Learn How](https://github.com/enricoros/big-agi/blob/main/docs/deploy-authentication.md)
+
+### What's New in 1.6.0 - Nov 28, 2023 · Surf's Up

 - **Web Browsing**: Download web pages within chats - [browsing guide](https://github.com/enricoros/big-agi/blob/main/docs/config-browse.md)
 - **Branching Discussions**: Create new conversations from any message
@@ -20,7 +43,7 @@ by release.
 - **New Features**: Anthropic Claude 2.1, `/help` command, and Flattener tool
 - **For Developers**: Code quality upgrades and snackbar notifications

-### What's New in 1.5.0 - Nov 19, 2023
+### What's New in 1.5.0 - Nov 19, 2023 · Loaded

 - **Continued Voice**: Engage with hands-free interaction for a seamless experience
 - **Visualization Tool**: Create data representations with our new visualization capabilities
@@ -3,25 +3,68 @@
 Allows users to load web pages across various components of `big-AGI`. This feature is supported by Puppeteer-based
 browsing services, which are the most common way to render web pages in a headless environment.

+Once configured, the Browsing service provides this functionality:
+
+- **Paste a URL**: Simply paste/drag a URL into the chat, and `big-AGI` will load and attach the page (very effective)
+- **Use /browse**: Type `/browse [URL]` in the chat to command `big-AGI` to load the specified web page
+- **ReAct**: ReAct will automatically use the `loadURL()` function whenever a URL is encountered
+
 First of all, you need to procure a Puppteer web browsing service endpoint. `big-AGI` supports services like:

- [BrightData](https://brightdata.com/products/scraping-browser) Scraping Browser
- [Cloudflare](https://developers.cloudflare.com/browser-rendering/) Browser Rendering, or
- any other Puppeteer-based service that provides a WebSocket endpoint (WSS)
- **including [your own browser](#your-own-chrome-browser)**
+| Service                                                                              | Working | Type        | Location       | Special Features                            |
+|--------------------------------------------------------------------------------------|---------|-------------|----------------|---------------------------------------------|
+| [BrightData Scraping Browser](https://brightdata.com/products/scraping-browser)      | Yes     | Proprietary | Cloud          | Advanced scraping tools, global IP pool     |
+| [Cloudflare Browser Rendering](https://developers.cloudflare.com/browser-rendering/) | ?       | Proprietary | Cloud          | Integrated CDN, optimized browser rendering |
+| ⬇️ [Browserless 2.0](#-browserless-20)                                               | Okay    | OpenSource  | Local (Docker) | Parallelism, debug viewer, advanced APIs    |
+| ⬇️ [Your Chrome Browser (ALPHA)](#-your-own-chrome-browser)                          | Alpha   | Proprietary | Local (Chrome) | Personal, experimental use (ALPHA!)         |
+| other Puppeteer-based WSS Services                                                   | ?       | Varied      | Cloud/Local    | Service-specific features                   |

 ## Configuration

-1. **Procure an Endpoint**: Ensure that your browsing service is running and has a WebSocket endpoint available:
-    - this mustbe in the form: `wss://${auth}@{some host}:{port}`
+1. **Procure an Endpoint**
+   - Ensure that your browsing service is running (remote or local) and has a WebSocket endpoint available
+   - Write down the address: `wss://${auth}@{some host}:{port}`, or ws:// for local services on your machine

-2. **Configure `big-AGI`**:  navigate to **Preferences** > **Tools** > **Browse** and enter the 'wss://...' connection
-   string provided by your browsing service
+2. **Configure `big-AGI`**
+   - navigate to **Preferences** > **Tools** > **Browse**
+   - Enter the 'wss://...' connection string provided by your browsing service

 3. **Enable Features**: Choose which browse-related features you want to enable:
-    - **Attach URLs**: Automatically load and attach a page when pasting a URL into the composer
-    - **/browse Command**: Use the `/browse` command in the chat to load a web page
-    - **ReAct**: Enable the `loadURL()` function in ReAct for advanced interactions
+   - **Attach URLs**: Automatically load and attach a page when pasting a URL into the composer
+   - **/browse Command**: Use the `/browse` command in the chat to load a web page
+   - **ReAct**: Enable the `loadURL()` function in ReAct for advanced interactions
+
+### 🌐 Browserless 2.0
+
+[Browserless 2.0](https://github.com/browserless/browserless) is a Docker-based service that provides a headless
+browsing experience compatible with `big-AGI`. An open-source solution that simplifies web automation tasks,
+in a scalable manner.
+
+Launch Browserless with:
+
+```bash
+docker run -p 9222:3000 browserless/chrome:latest
+```
+
+Now you can use the following connection string in `big-AGI`: `ws://127.0.0.1:9222`.
+You can also browse to [http://127.0.0.1:9222](http://127.0.0.1:9222) to see the Browserless debug viewer
+and configure some options.
+
+Note: if you are using `docker-compose`, please see the
+[docker/docker-compose-browserless.yaml](docker/docker-compose-browserless.yaml) file for an example
+on how to run `big-AGI` and Browserless simultaneously in a single application.
+
+### 🌐 Your own Chrome browser
+
+***EXPERIMENTAL - UNTESTED*** - You can use your own Chrome browser as a browsing service, by configuring it to expose
+a WebSocket endpoint.
+
+- close all the Chrome instances (on Windows, check the Task Manager if still running)
+- start Chrome with the following command line options (on Windows, you can edit the shortcut properties):
+  - `--remote-debugging-port=9222`
+- go to http://localhost:9222/json/version and copy the `webSocketDebuggerUrl` value
+  - it should be something like: `ws://localhost:9222/...`
+- paste the value into the Endpoint configuration (see point 2 in the configuration)

 ### Server-Side Configuration

@@ -33,26 +76,6 @@ Always deploy your own user authentication, authorization and security solution.
 route that provides browsing service, shall be secured with a user authentication and authorization solution,
 to prevent unauthorized access to the browsing service.

-### Your own Chrome browser
-
-***EXPERIMENTAL - UNTESTED*** - You can use your own Chrome browser as a browsing service, by configuring it to expose
-a WebSocket endpoint.
-
- close all the Chrome instances (on Windows, check the Task Manager if still running)
- start Chrome with the following command line options (on Windows, you can edit the shortcut properties):
-    - `--remote-debugging-port=9222`
- go to http://localhost:9222/json/version and copy the `webSocketDebuggerUrl` value
-    - it should be something like: `ws://localhost:9222/...`
- paste the value into the Endpoint configuration (see point 2 above)
-
-## Usage
-
-Once configured, you can start using the browse functionality:
-
- **Paste a URL**: Simply paste a URL into the chat, and `big-AGI` will load the page if the Attach URLs feature is enabled
- **Use /browse**: Type `/browse [URL]` in the chat to command `big-AGI` to load the specified web page
- **ReAct**: ReAct will automatically use the `loadURL()` function whenever a URL is encountered
-
 ## Support

 If you encounter any issues or have questions about configuring the browse functionality, join our community on Discord for support and discussions.
@@ -30,5 +30,5 @@ For instance with [Use luna-ai-llama2 with docker compose](https://localai.io/ba

 > NOTE: LocalAI does not list details about the mdoels. Every model is assumed to be
 > capable of chatting, and with a context window of 4096 tokens.
-> Please update the [src/modules/llms/transports/server/openai/models.data.ts](../src/modules/llms/transports/server/openai/models.data.ts)
+> Please update the [src/modules/llms/transports/server/openai/models.data.ts](../src/modules/llms/server/openai/models.data.ts)
 > file with the mapping information between LocalAI model IDs and names/descriptions/tokens, etc.
@@ -4,7 +4,7 @@ Integrate local Large Language Models (LLMs) with
 [oobabooga/text-generation-webui](https://github.com/oobabooga/text-generation-webui),
 a specialized interface that includes a custom variant of the OpenAI API for a smooth integration process.

-_Last updated on Nov 7, 2023_
+_Last updated on Dec 7, 2023_

 ### Components

@@ -20,26 +20,31 @@ This guide assumes that **big-AGI** is already installed on your system. Note th

 ### Text-web-ui Installation & Configuration:

-1. Install [text-generation-webui](https://github.com/oobabooga/text-generation-webui#Installation).
-    - Download the one-click installer, extract it, and double-click on "start" - ~10 minutes
-    - Close it afterwards as we need to modify the startup flags
+1. Install [text-generation-webui](https://github.com/oobabooga/text-generation-webui#Installation):
+    - Follow the instructions in the official page (basicall clone the repo and run a script) [~10 minutes]
+    - Stop the Web UI as we need to modify the startup flags to enable the OpenAI API
 2. Enable the **openai extension**
    - Edit `CMD_FLAGS.txt`
-    - Make sure that `--listen --extensions openai` is present and uncommented 
+    - Make sure that `--listen --api` is present and uncommented 
 3. Restart text-generation-webui
    - Double-click on "start"
    - You should see something like: 
      ```
-      2023-11-07 21:24:26 INFO:Loading the extension "openai"...
-      2023-11-07 21:24:27 INFO:OpenAI compatible API URL:
+      2023-12-07 21:51:21 INFO:Loading the extension "openai"...
+      2023-12-07 21:51:21 INFO:OpenAI-compatible API URL:
      
-      http://0.0.0.0:5000/v1
+      http://0.0.0.0:5000 
+      ...
+      INFO:     Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit)
+      Running on local URL:  http://0.0.0.0:7860
      ```
-    - The OpenAI API is now running on port 5000, on both localhost (127.0.0.1) and your network IP address
+    - This shows that:
+      - The Web UI is running on port 7860: http://127.0.0.1:7860
+      - **The OpenAI API is running on port 5000: http://127.0.0.1:5000**
 4. Load your first model
    - Open the text-generation-webui at [127.0.0.1:7860](http://127.0.0.1:7860/)
    - Switch to the **Model** tab
-    - Download, for instance, `TheBloke/Llama-2-7b-Chat-GPTQ:gptq-4bit-32g-actorder_True` - 4.3 GB
+    - Download, for instance, `TheBloke/Llama-2-7B-Chat-GPTQ`
    - Select the model once it's loaded

 ### Integrating text-web-ui with big-AGI:
@@ -51,4 +56,6 @@ This guide assumes that **big-AGI** is already installed on your system. Note th
        - The active model must be selected and LOADED on the text-generation-webui as it doesn't support model switching or parallel requests.
    - Select model & Chat

+![config-oobabooga-0.png](pixels/config-oobabooga-0.png)
+
 Enjoy the privacy and flexibility of local LLMs with `big-AGI` and `text-generation-webui`!
@@ -5,31 +5,46 @@ This guide helps you connect [Ollama](https://ollama.ai) [models](https://ollama
 experience. The integration brings the popular big-AGI features to Ollama, including: voice chats,
 editing tools, models switching, personas, and more.

+_Last updated Dec 16, 2023_
+
 ![config-local-ollama-0-example.png](pixels/config-ollama-0-example.png)

 ## Quick Integration Guide

-1. **Ensure Ollama API Server is Running**: Before starting, make sure your Ollama API server is up and running.
-2. **Add Ollama as a Model Source**: In `big-AGI`, navigate to the **Models** section, select **Add a model source**, and choose **Ollama**.
-3. **Enter Ollama Host URL**: Provide the Ollama Host URL where the API server is accessible (e.g., `http://localhost:11434`).
-4. **Refresh Model List**: Once connected, refresh the list of available models to include the Ollama models.
-5. **Start Using AI Personas**: Select an Ollama model and begin interacting with AI personas tailored to your needs.
+1. **Ensure Ollama API Server is Running**: Follow the official instructions to get Ollama up and running on your machine
+   - For detailed instructions on setting up the Ollama API server, please refer to the
+   [Ollama download page](https://ollama.ai/download) and [instructions for linux](https://github.com/jmorganca/ollama/blob/main/docs/linux.md). 
+2. **Add Ollama as a Model Source**: In `big-AGI`, navigate to the **Models** section, select **Add a model source**, and choose **Ollama**
+3. **Enter Ollama Host URL**: Provide the Ollama Host URL where the API server is accessible (e.g., `http://localhost:11434`)
+4. **Refresh Model List**: Once connected, refresh the list of available models to include the Ollama models
+   > Optional: use the Ollama Admin interface to see which models are available and 'Pull' them in your local machine. Note
+   that this operation will likely timeout due to Edge Functions timeout on the big-AGI server while pulling, and
+   you'll have to press the 'Pull' button again, until a green message appears.
+5. **Chat with Ollama models**: select an Ollama model and begin chatting with AI personas

-### Ollama: installation and Setup
+**Visual Configuration Guide**:

-For detailed instructions on setting up the Ollama API server, please refer to the
-[Ollama download page](https://ollama.ai/download) and [instructions for linux](https://github.com/jmorganca/ollama/blob/main/docs/linux.md).
+* After adding the `Ollama` model vendor, entering the IP address of an Ollama server, and refreshing models:<br/>
+  <img src="pixels/config-ollama-1-models.png" alt="config-local-ollama-1-models.png" width="320">

-### Visual Guide
+* The `Ollama` admin panel, with the `Pull` button highlighted, after pulling the "Yi" model:<br/>
+  <img src="pixels/config-ollama-2-admin-pull.png" alt="config-local-ollama-2-admin-pull.png" width="320">

-* After adding the `Ollama` model vendor, entering the IP address of an Ollama server, and refreshing models:
-  <img src="pixels/config-ollama-1-models.png" alt="config-local-ollama-1-models.png" style="max-width: 320px;">
+* You can now switch model/persona dynamically and text/voice chat with the models:<br/>
+  <img src="pixels/config-ollama-3-chat.png" alt="config-local-ollama-3-chat.png" width="320">

-* The `Ollama` admin panel, with the `Pull` button highlighted, after pulling the "Yi" model:
-  <img src="pixels/config-ollama-2-admin-pull.png" alt="config-local-ollama-2-admin-pull.png" style="max-width: 320px;">
+<br/>

-* You can now switch model/persona dynamically and text/voice chat with the models:
-  <img src="pixels/config-ollama-3-chat.png" alt="config-local-ollama-3-chat.png" style="max-width: 320px;">
+### ⚠️ Network Troubleshooting
+
+If you get errors about the server having trouble connecting with Ollama, please see 
+[this message](https://github.com/enricoros/big-AGI/issues/276#issuecomment-1858591483) on Issue #276.
+
+And in brief, make sure the Ollama endpoint is accessible from the servers where you run big-AGI (which could
+be localhost or cloud servers).
+![Ollama Networking Chart](pixels/config-ollama-network.png)
+
+<br/>

 ### Advanced: Model parameters

@@ -68,6 +83,8 @@ Then, edit the nginx configuration file `/etc/nginx/sites-enabled/default` and a

 Reach out to our community if you need help with this.

+<br/>
+
 ### Community and Support

 Join our community to share your experiences, get help, and discuss best practices:
@@ -78,4 +95,4 @@ Join our community to share your experiences, get help, and discuss best practic
 ---

 `big-AGI` is committed to providing a powerful, intuitive, and privacy-respecting AI experience.
-We are excited for you to explore the possibilities with Ollama models. Happy creating!
+We are excited for you to explore the possibilities with Ollama models. Happy creating!
@@ -0,0 +1,45 @@
+# Authentication
+
+`big-AGI` does not come with built-in authentication. To secure your deployment, you can implement authentication
+in one of the following ways:
+
+1. Build `big-AGI` with support for ⬇️ [HTTP Authentication](#http-authentication)
+2. Utilize user authentication features provided by your ⬇️ [cloud deployment platform](#cloud-deployments-authentication)
+3. Develop a custom authentication solution
+
+<br/>
+
+### HTTP Authentication
+
+[HTTP Basic Authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication) is a simple method
+to secure your application.
+
+To enable it in `big-AGI`, you **must manually build the application**:
+
+- Build `big-AGI` with HTTP authentication enabled:
+  - Clone the repository
+  - Rename `middleware_BASIC_AUTH.ts` to `middleware.ts`
+  - Build: usual simple build procedure (e.g. [Deploy manually](../README.md#-deploy-manually) or [Deploying with Docker](deploy-docker.md))
+
+- Configure the following [environment variables](environment-variables.md) before launching `big-AGI`:
+```dotenv
+HTTP_BASIC_AUTH_USERNAME=<your username>
+HTTP_BASIC_AUTH_PASSWORD=<your password>
+```
+
+- Start the application 🔒
+
+<br/>
+
+### Cloud Deployments Authentication
+
+> This approach allows you to enable authentication without rebuilding the application by using the features
+> provided by your cloud platform to manage user accounts and access.
+
+Many cloud deployment platforms offer built-in authentication mechanisms. Refer to the platform's documentation
+for setup instructions:
+
+1. [CloudFlare Access / Zero Trust](https://www.cloudflare.com/zero-trust/products/access/)
+2. [Vercel Authentication](https://vercel.com/docs/security/deployment-protection/methods-to-protect-deployments/vercel-authentication)
+3. [Vercel Password Protection](https://vercel.com/docs/security/deployment-protection/methods-to-protect-deployments/password-protection)
+4. Let us know when you test more solutions (Heroku, AWS IAM, Google IAP, etc.)
@@ -21,33 +21,23 @@ Docker ensures faster development cycles, easier collaboration, and seamless env
   ```
 4. Browse to [http://localhost:3000](http://localhost:3000)

-## Documentation
+<br/>

-The big-AGI repository includes a Dockerfile and a GitHub Actions workflow for building and publishing a
-Docker image of the application.
+## Run Official Containers 📦

-### Dockerfile
+`big-AGI` is pre-built from source code and published as a Docker image on the GitHub Container Registry (ghcr).
+The build process is transparent, and happens via GitHub Actions, as described in the
+file.

-The [`Dockerfile`](../Dockerfile) describes how to create a Docker image. It establishes a Node.js environment,
-installs dependencies, and creates a production-ready version of the application as a local container.
+### Official Images: [ghcr.io/enricoros/big-agi](https://github.com/enricoros/big-agi/pkgs/container/big-agi)

-### Official container images
+#### Run using *docker* 🚀

-The [`.github/workflows/docker-image.yml`](../.github/workflows/docker-image.yml) file automates the
-building and publishing of the Docker images to the GitHub Container Registry (ghcr) when changes are
-pushed to the `main` branch.
-
-Official pre-built containers: [ghcr.io/enricoros/big-agi](https://github.com/enricoros/big-agi/pkgs/container/big-agi)
-
-Run official pre-built containers:
 ```bash
-docker run -d -p 3000:3000 ghcr.io/enricoros/big-agi
+docker run -d -p 3000:3000 ghcr.io/enricoros/big-agi:latest
 ```

-### Run official containers
-
-In addition, the repository also includes a `docker-compose.yaml` file, configured to run the pre-built
-'ghcr image'. This file is used to define the `big-agi` service, the ports to expose, and the command to run.
+#### Run using *docker-compose* 🚀

 If you have Docker Compose installed, you can run the Docker container with `docker-compose up`
 to pull the Docker image (if it hasn't been pulled already) and start a Docker container. If you want to
@@ -57,4 +47,31 @@ update the image to the latest version, you can run `docker-compose pull` before
 docker-compose up -d
 ```

-Leverage Docker's capabilities for a reliable and efficient big-AGI deployment.
+### Make Local Services Visible to Docker 🌐
+
+To make local services running on your host machine accessible to a Docker container, such as a
+[Browseless](./config-browse.md) service or a local API, you can follow this simplified guide:
+
+| Operating System  | Steps to Make Local Services Visible to Docker                                                                                                                                                                                                                                                                                                                                               |
+|:------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Windows and macOS | Use the special DNS name `host.docker.internal` to refer to the host machine from within the Docker container. No additional network configuration is required. Access local services using `host.docker.internal:<PORT>`.                                                                                                                                                                   |
+| Linux             | Two options: *A*. Use <ins>--network="host"</ins> (`docker run --network="host" -d big-agi`) when running the Docker container to merge the container within the host network stack; however, this reduces container isolation. Alternatively: *B*. Connect to local services <ins>using the host's IP address</ins> directly, as host.docker.internal is not available by default on Linux. |
+
+<br/>
+
+### More Information
+
+The [`Dockerfile`](../Dockerfile) describes how to create a Docker image. It establishes a Node.js environment,
+installs dependencies, and creates a production-ready version of the application as a local container.
+
+The [`docker-compose.yaml`](../docker-compose.yaml) file is configured to run the
+official image (big-agi:latest). This file is used to define the `big-agi` service, to expose
+port 3000 on the host, and launch big-AGI within the container (startup command).
+
+The [`.github/workflows/docker-image.yml`](../.github/workflows/docker-image.yml) file is used
+to build the Official Docker images and publish them to the GitHub Container Registry (ghcr).
+The build process is transparent and happens via GitHub Actions.
+
+<br/>
+
+Leverage Docker's capabilities for a reliable and efficient big-AGI deployment!
@@ -0,0 +1,31 @@
+# This file is used to run `big-AGI` and `browserless` with Docker Compose.
+#
+# The two containers are linked together and `big-AGI` is configured to use `browserless`
+# as its Puppeteer endpoint (from the containers intranet, it is available browserless:3000).
+#
+# From your host, you can access big-AGI on http://127.0.0.1:3000 and browserless on http://127.0.0.1:9222.
+#
+# To start the containers, run:
+#   docker-compose -f docs/docker/docker-compose-browserless.yaml up
+
+version: '3.9'
+
+services:
+  big-agi:
+    image: ghcr.io/enricoros/big-agi:latest
+    ports:
+      - "3000:3000"
+    env_file:
+      - .env
+    environment:
+      - PUPPETEER_WSS_ENDPOINT=ws://browserless:3000
+    command: [ "next", "start", "-p", "3000" ]
+    depends_on:
+      - browserless
+
+  browserless:
+    image: browserless/chrome:latest
+    ports:
+      - "9222:3000"  # Map host's port 9222 to container's port 3000
+    environment:
+      - MAX_CONCURRENT_SESSIONS=10
@@ -9,10 +9,6 @@ which take place over _defaults_. This file is kept in sync with [`../src/server

 Environment variables can be set by creating a `.env` file in the root directory of the project.

-> For Docker deployment, ensure all necessary environment variables are set **both during build and run**.
-> If the Docker container is built without setting environment variables, the frontend UI will be unaware
-> of them, despite the backend being able to use them at runtime.
-
 The following is an example `.env` for copy-paste convenience:

 ```bash
@@ -28,6 +24,8 @@ AZURE_OPENAI_API_ENDPOINT=
 AZURE_OPENAI_API_KEY=
 ANTHROPIC_API_KEY=
 ANTHROPIC_API_HOST=
+GEMINI_API_KEY=
+MISTRAL_API_KEY=
 OLLAMA_API_HOST=
 OPENROUTER_API_KEY=

@@ -45,6 +43,13 @@ GOOGLE_CLOUD_API_KEY=
 GOOGLE_CSE_ID=
 # Browse
 PUPPETEER_WSS_ENDPOINT=
+
+# Backend Analytics
+BACKEND_ANALYTICS=
+
+# Backend HTTP Basic Authentication (see `deploy-authentication.md` for turning on authentication)
+HTTP_BASIC_AUTH_USERNAME=
+HTTP_BASIC_AUTH_PASSWORD=
 ```

 ## Variables Documentation
@@ -76,6 +81,8 @@ requiring the user to enter an API key
 | `AZURE_OPENAI_API_KEY`      | Azure OpenAI API key, see [config-azure-openai.md](config-azure-openai.md)                                                    | Optional, but if set `AZURE_OPENAI_API_ENDPOINT` must also be set |
 | `ANTHROPIC_API_KEY`         | The API key for Anthropic                                                                                                     | Optional                                                          |
 | `ANTHROPIC_API_HOST`        | Changes the backend host for the Anthropic vendor, to enable platforms such as [config-aws-bedrock.md](config-aws-bedrock.md) | Optional                                                          |
+| `GEMINI_API_KEY`            | The API key for Google AI's Gemini                                                                                            | Optional                                                          |
+| `MISTRAL_API_KEY`           | The API key for Mistral                                                                                                       | Optional                                                          |
 | `OLLAMA_API_HOST`           | Changes the backend host for the Ollama vendor. See [config-ollama.md](config-ollama.md)                                      |                                                                   |
 | `OPENROUTER_API_KEY`        | The API key for OpenRouter                                                                                                    | Optional                                                          |

@@ -95,21 +102,22 @@ It is currently supported for:

 Enable the app to Talk, Draw, and Google things up.

-| Variable                 | Description                                                                                                             |
-|:-------------------------|:------------------------------------------------------------------------------------------------------------------------|
-| **Text-To-Speech**       | [ElevenLabs](https://elevenlabs.io/) is a high quality speech synthesis service                                         |
-| `ELEVENLABS_API_KEY`     | ElevenLabs API Key - used for calls, etc.                                                                               |
-| `ELEVENLABS_API_HOST`    | Custom host for ElevenLabs                                                                                              |
-| `ELEVENLABS_VOICE_ID`    | Default voice ID for ElevenLabs                                                                                         |
-| **Google Custom Search** | [Google Programmable Search Engine](https://programmablesearchengine.google.com/about/)  produces links to pages        |
-| `GOOGLE_CLOUD_API_KEY`   | Google Cloud API Key, used with the '/react' command - [Link to GCP](https://console.cloud.google.com/apis/credentials) |
-| `GOOGLE_CSE_ID`          | Google Custom/Programmable Search Engine ID - [Link to PSE](https://programmablesearchengine.google.com/)               |
-| **Text-To-Image**        | [Prodia](https://prodia.com/) is a reliable image generation service                                                    |
-| `PRODIA_API_KEY`         | Prodia API Key - used with '/imagine ...'                                                                               |
-| **Browse**               |                                                                                                                         |
-| `PUPPETEER_WSS_ENDPOINT` | Puppeteer WebSocket endpoint - used for browsing, etc.                                                                  |
+| Variable                   | Description                                                                                                             |
+|:---------------------------|:------------------------------------------------------------------------------------------------------------------------|
+| **Text-To-Speech**         | [ElevenLabs](https://elevenlabs.io/) is a high quality speech synthesis service                                         |
+| `ELEVENLABS_API_KEY`       | ElevenLabs API Key - used for calls, etc.                                                                               |
+| `ELEVENLABS_API_HOST`      | Custom host for ElevenLabs                                                                                              |
+| `ELEVENLABS_VOICE_ID`      | Default voice ID for ElevenLabs                                                                                         |
+| **Google Custom Search**   | [Google Programmable Search Engine](https://programmablesearchengine.google.com/about/)  produces links to pages        |
+| `GOOGLE_CLOUD_API_KEY`     | Google Cloud API Key, used with the '/react' command - [Link to GCP](https://console.cloud.google.com/apis/credentials) |
+| `GOOGLE_CSE_ID`            | Google Custom/Programmable Search Engine ID - [Link to PSE](https://programmablesearchengine.google.com/)               |
+| **Text-To-Image**          | [Prodia](https://prodia.com/) is a reliable image generation service                                                    |
+| `PRODIA_API_KEY`           | Prodia API Key - used with '/imagine ...'                                                                               |
+| **Browse**                 |                                                                                                                         |
+| `PUPPETEER_WSS_ENDPOINT`   | Puppeteer WebSocket endpoint - used for browsing, etc.                                                                  |
+| **Backend**                |                                                                                                                         | 
+| `BACKEND_ANALYTICS`        | Semicolon-separated list of analytics flags (see backend.analytics.ts). Flags: `domain` logs the responding domain.     |
+| `HTTP_BASIC_AUTH_USERNAME` | See the [Authentication](deploy-authentication.md) guide. Username for HTTP Basic Authentication.                       |
+| `HTTP_BASIC_AUTH_PASSWORD` | Password for HTTP Basic Authentication.                                                                                 |

 ---
-
-
-
@@ -0,0 +1,59 @@
+/**
+ * Middleware to protect `big-AGI` with HTTP Basic Authentication
+ *
+ * For more information on how to deploy with HTTP Basic Authentication, see:
+ *  - [deploy-authentication.md](docs/deploy-authentication.md)
+ */
+
+import type { NextRequest } from 'next/server';
+import { NextResponse } from 'next/server';
+
+
+// noinspection JSUnusedGlobalSymbols
+export function middleware(request: NextRequest) {
+
+  // Validate deployment configuration
+  if (!process.env.HTTP_BASIC_AUTH_USERNAME || !process.env.HTTP_BASIC_AUTH_PASSWORD) {
+    console.warn('HTTP Basic Authentication is enabled but not configured');
+    return new Response('Unauthorized/Unconfigured', unauthResponse);
+  }
+
+  // Request client authentication if no credentials are provided
+  const authHeader = request.headers.get('authorization');
+  if (!authHeader?.startsWith('Basic '))
+    return new Response('Unauthorized', unauthResponse);
+
+  // Request authentication if credentials are invalid
+  const base64Credentials = authHeader.split(' ')[1];
+  const credentials = Buffer.from(base64Credentials, 'base64').toString('ascii');
+  const [username, password] = credentials.split(':');
+  if (
+    !username || !password ||
+    username !== process.env.HTTP_BASIC_AUTH_USERNAME ||
+    password !== process.env.HTTP_BASIC_AUTH_PASSWORD
+  )
+    return new Response('Unauthorized', unauthResponse);
+
+  return NextResponse.next();
+}
+
+
+// Response to send when authentication is required
+const unauthResponse: ResponseInit = {
+  status: 401,
+  headers: {
+    'WWW-Authenticate': 'Basic realm="Secure big-AGI"',
+  },
+};
+
+export const config = {
+  matcher: [
+    // Include root
+    '/',
+    // Include pages
+    '/(call|index|news|personas|link)(.*)',
+    // Include API routes
+    '/api(.*)',
+    // Note: this excludes _next, /images etc..
+  ],
+};
@@ -1,6 +1,6 @@
 {
  "name": "big-agi",
-  "version": "1.6.0",
+  "version": "1.8.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -18,13 +18,13 @@
    "@emotion/react": "^11.11.1",
    "@emotion/server": "^11.11.0",
    "@emotion/styled": "^11.11.0",
-    "@mui/icons-material": "^5.14.18",
-    "@mui/joy": "^5.0.0-beta.15",
-    "@next/bundle-analyzer": "^14.0.3",
-    "@prisma/client": "^5.6.0",
+    "@mui/icons-material": "^5.15.0",
+    "@mui/joy": "^5.0.0-beta.18",
+    "@next/bundle-analyzer": "^14.0.4",
+    "@prisma/client": "^5.7.0",
    "@sanity/diff-match-patch": "^3.1.1",
    "@t3-oss/env-nextjs": "^0.7.1",
-    "@tanstack/react-query": "^4.36.1",
+    "@tanstack/react-query": "~4.36.1",
    "@trpc/client": "^10.44.1",
    "@trpc/next": "^10.44.1",
    "@trpc/react-query": "^10.44.1",
@@ -33,8 +33,8 @@
    "browser-fs-access": "^0.35.0",
    "eventsource-parser": "^1.1.1",
    "idb-keyval": "^6.2.1",
-    "next": "^14.0.3",
-    "pdfjs-dist": "4.0.189",
+    "next": "^14.0.4",
+    "pdfjs-dist": "4.0.269",
    "plantuml-encoder": "^1.4.0",
    "prismjs": "^1.29.0",
    "react": "^18.2.0",
@@ -47,23 +47,23 @@
    "tesseract.js": "^5.0.3",
    "uuid": "^9.0.1",
    "zod": "^3.22.4",
-    "zustand": "~4.3.9"
+    "zustand": "^4.4.7"
  },
  "devDependencies": {
    "@cloudflare/puppeteer": "^0.0.5",
-    "@types/node": "^20.10.0",
+    "@types/node": "^20.10.4",
    "@types/plantuml-encoder": "^1.4.2",
    "@types/prismjs": "^1.26.3",
-    "@types/react": "^18.2.38",
+    "@types/react": "^18.2.45",
    "@types/react-dom": "^18.2.17",
-    "@types/react-katex": "^3.0.3",
+    "@types/react-katex": "^3.0.4",
    "@types/react-timeago": "^4.1.6",
    "@types/uuid": "^9.0.7",
-    "eslint": "^8.54.0",
-    "eslint-config-next": "^14.0.3",
-    "prettier": "^3.1.0",
-    "prisma": "^5.6.0",
-    "typescript": "^5.3.2"
+    "eslint": "^8.55.0",
+    "eslint-config-next": "^14.0.4",
+    "prettier": "^3.1.1",
+    "prisma": "^5.7.0",
+    "typescript": "^5.3.3"
  },
  "engines": {
    "node": "^20.0.0 || ^18.0.0"
@@ -11,6 +11,7 @@ import '~/common/styles/CodePrism.css';
 import '~/common/styles/GithubMarkdown.css';

 import { ProviderBackend } from '~/common/state/ProviderBackend';
+import { ProviderSingleTab } from '~/common/state/ProviderSingleTab';
 import { ProviderSnacks } from '~/common/state/ProviderSnacks';
 import { ProviderTRPCQueryClient } from '~/common/state/ProviderTRPCQueryClient';
 import { ProviderTheming } from '~/common/state/ProviderTheming';
@@ -25,13 +26,15 @@ const MyApp = ({ Component, emotionCache, pageProps }: MyAppProps) =>
    </Head>

    <ProviderTheming emotionCache={emotionCache}>
-      <ProviderTRPCQueryClient>
-        <ProviderSnacks>
-          <ProviderBackend>
-            <Component {...pageProps} />
-          </ProviderBackend>
-        </ProviderSnacks>
-      </ProviderTRPCQueryClient>
+      <ProviderSingleTab>
+        <ProviderTRPCQueryClient>
+          <ProviderSnacks>
+            <ProviderBackend>
+              <Component {...pageProps} />
+            </ProviderBackend>
+          </ProviderSnacks>
+        </ProviderTRPCQueryClient>
+      </ProviderSingleTab>
    </ProviderTheming>

    <VercelAnalytics debug={false} />
@@ -0,0 +1,98 @@
+import * as React from 'react';
+import { useRouter } from 'next/router';
+
+import { Box, Typography } from '@mui/joy';
+
+import { useModelsStore } from '~/modules/llms/store-llms';
+
+import { AppLayout } from '~/common/layout/AppLayout';
+import { InlineError } from '~/common/components/InlineError';
+import { apiQuery } from '~/common/util/trpc.client';
+import { navigateToIndex } from '~/common/app.routes';
+import { openLayoutModelsSetup } from '~/common/layout/store-applayout';
+
+
+function CallbackOpenRouterPage(props: { openRouterCode: string | undefined }) {
+
+  // external state
+  const { data, isError, error, isLoading } = apiQuery.backend.exchangeOpenRouterKey.useQuery({ code: props.openRouterCode || '' }, {
+    enabled: !!props.openRouterCode,
+    refetchOnWindowFocus: false,
+    staleTime: Infinity,
+  });
+
+  // derived state
+  const isErrorInput = !props.openRouterCode;
+  const openRouterKey = data?.key ?? undefined;
+  const isSuccess = !!openRouterKey;
+
+
+  // Success: save the key and redirect to the chat app
+  React.useEffect(() => {
+    if (!isSuccess)
+      return;
+
+    // 1. Save the key as the client key
+    useModelsStore.getState().setOpenRoutersKey(openRouterKey);
+
+    // 2. Navigate to the chat app
+    navigateToIndex(true).then(() => openLayoutModelsSetup());
+
+  }, [isSuccess, openRouterKey]);
+
+  return (
+    <Box sx={{
+      flexGrow: 1,
+      backgroundColor: 'background.level1',
+      overflowY: 'auto',
+      display: 'flex', justifyContent: 'center',
+      p: { xs: 3, md: 6 },
+    }}>
+
+      <Box sx={{
+        // my: 'auto',
+        display: 'flex', flexDirection: 'column', alignItems: 'center',
+        gap: 4,
+      }}>
+
+        <Typography level='title-lg'>
+          Welcome Back
+        </Typography>
+
+        {isLoading && <Typography level='body-sm'>Loading...</Typography>}
+
+        {isErrorInput && <InlineError error='There was an issue retrieving the code from OpenRouter.' />}
+
+        {isError && <InlineError error={error} />}
+
+        {data && (
+          <Typography level='body-md'>
+            Success! You can now close this window.
+          </Typography>
+        )}
+
+      </Box>
+
+    </Box>
+  );
+}
+
+
+/**
+ * This page will be invoked by OpenRouter as a Callback
+ *
+ * Docs: https://openrouter.ai/docs#oauth
+ * Example URL: https://localhost:3000/link/callback_openrouter?code=SomeCode
+ */
+export default function Page() {
+
+  // get the 'code=...' from the URL
+  const { query } = useRouter();
+  const { code: openRouterCode } = query;
+
+  return (
+    <AppLayout suspendAutoModelsSetup>
+      <CallbackOpenRouterPage openRouterCode={openRouterCode as (string | undefined)} />
+    </AppLayout>
+  );
+}
@@ -75,11 +75,11 @@ function AppShareTarget() {
    if (intentURL) {
      setIsDownloading(true);
      callBrowseFetchPage(intentURL)
-        .then(pageContent => {
-          if (pageContent)
-            queueComposerTextAndLaunchApp('\n\n```' + intentURL + '\n' + pageContent + '\n```\n');
+        .then(page => {
+          if (page.stopReason !== 'error')
+            queueComposerTextAndLaunchApp('\n\n```' + intentURL + '\n' + page.content + '\n```\n');
          else
-            setErrorMessage('Could not read any data');
+            setErrorMessage('Could not read any data' + page.error ? ': ' + page.error : '');
        })
        .catch(error => setErrorMessage(error?.message || error || 'Unknown error'))
        .finally(() => setIsDownloading(false));
@@ -15,8 +15,7 @@ import { useChatLLMDropdown } from '../chat/components/applayout/useLLMDropdown'

 import { EXPERIMENTAL_speakTextStream } from '~/modules/elevenlabs/elevenlabs.client';
 import { SystemPurposeId, SystemPurposes } from '../../data';
-import { VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
-import { streamChat } from '~/modules/llms/transports/streamChat';
+import { llmStreamingChatGenerate, VChatMessageIn } from '~/modules/llms/llm.client';
 import { useElevenLabsVoiceDropdown } from '~/modules/elevenlabs/useElevenLabsVoiceDropdown';

 import { Link } from '~/common/components/Link';
@@ -113,7 +112,7 @@ export function CallUI(props: {
        setCallMessages(messages => [...messages, createDMessage('user', transcribed)]);
    }
  }, []);
-  const { isSpeechEnabled, isRecording, isRecordingAudio, isRecordingSpeech, startRecording, stopRecording, toggleRecording } = useSpeechRecognition(onSpeechResultCallback, 1000, false);
+  const { isSpeechEnabled, isRecording, isRecordingAudio, isRecordingSpeech, startRecording, stopRecording, toggleRecording } = useSpeechRecognition(onSpeechResultCallback, 1000);

  // derived state
  const isRinging = stage === 'ring';
@@ -216,7 +215,7 @@ export function CallUI(props: {
    responseAbortController.current = new AbortController();
    let finalText = '';
    let error: any | null = null;
-    streamChat(chatLLMId, callPrompt, responseAbortController.current.signal, (updatedMessage: Partial<DMessage>) => {
+    llmStreamingChatGenerate(chatLLMId, callPrompt, null, null, responseAbortController.current.signal, (updatedMessage: Partial<DMessage>) => {
      const text = updatedMessage.text?.trim();
      if (text) {
        finalText = text;
@@ -3,7 +3,7 @@ import * as React from 'react';
 import { Chip, ColorPaletteProp, VariantProp } from '@mui/joy';
 import { SxProps } from '@mui/joy/styles/types';

-import { VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
+import type { VChatMessageIn } from '~/modules/llms/llm.client';


 export function CallMessage(props: {
@@ -12,15 +12,16 @@ import { TradeConfig, TradeModal } from '~/modules/trade/TradeModal';
 import { imaginePromptFromText } from '~/modules/aifn/imagine/imaginePromptFromText';
 import { speakText } from '~/modules/elevenlabs/elevenlabs.client';
 import { useBrowseStore } from '~/modules/browse/store-module-browsing';
-import { useModelsStore } from '~/modules/llms/store-llms';
+import { useChatLLM, useModelsStore } from '~/modules/llms/store-llms';

 import { ConfirmationModal } from '~/common/components/ConfirmationModal';
+import { GlobalShortcutItem, ShortcutKeyName, useGlobalShortcuts } from '~/common/components/useGlobalShortcut';
 import { addSnackbar, removeSnackbar } from '~/common/components/useSnackbarsStore';
 import { createDMessage, DConversationId, DMessage, getConversation, useConversation } from '~/common/state/store-chats';
-import { GlobalShortcutItem, ShortcutKeyName, useGlobalShortcuts } from '~/common/components/useGlobalShortcut';
-import { useLayoutPluggable } from '~/common/layout/store-applayout';
+import { openLayoutLLMOptions, useLayoutPluggable } from '~/common/layout/store-applayout';
 import { useUXLabsStore } from '~/common/state/store-ux-labs';

+import type { ComposerOutputMultiPart } from './components/composer/composer.types';
 import { ChatDrawerItemsMemo } from './components/applayout/ChatDrawerItems';
 import { ChatDropdowns } from './components/applayout/ChatDropdowns';
 import { ChatMenuItems } from './components/applayout/ChatMenuItems';
@@ -57,6 +58,8 @@ export function AppChat() {
  const composerTextAreaRef = React.useRef<HTMLTextAreaElement>(null);

  // external state
+  const { chatLLM } = useChatLLM();
+
  const {
    chatPanes,
    focusedConversationId,
@@ -180,13 +183,33 @@ export function AppChat() {
    setMessages(conversationId, history);
  }, [focusedSystemPurposeId, setMessages]);

-  const handleComposerNewMessage = async (chatModeId: ChatModeId, conversationId: DConversationId, userText: string) => {
+  const handleComposerAction = (chatModeId: ChatModeId, conversationId: DConversationId, multiPartMessage: ComposerOutputMultiPart): boolean => {
+
+    // validate inputs
+    if (multiPartMessage.length !== 1 || multiPartMessage[0].type !== 'text-block') {
+      addSnackbar({
+        key: 'chat-composer-action-invalid',
+        message: 'Only a single text part is supported for now.',
+        type: 'issue',
+        overrides: {
+          autoHideDuration: 2000,
+        },
+      });
+      return false;
+    }
+    const userText = multiPartMessage[0].text;
+
+    // find conversation
    const conversation = getConversation(conversationId);
-    if (conversation)
-      return await _handleExecute(chatModeId, conversationId, [
-        ...conversation.messages,
-        createDMessage('user', userText),
-      ]);
+    if (!conversation)
+      return false;
+
+    // start execution (async)
+    void _handleExecute(chatModeId, conversationId, [
+      ...conversation.messages,
+      createDMessage('user', userText),
+    ]);
+    return true;
  };

  const handleConversationExecuteHistory = async (conversationId: DConversationId, history: DMessage[]) =>
@@ -291,7 +314,14 @@ export function AppChat() {

  // Shortcuts

+  const handleOpenChatLlmOptions = React.useCallback(() => {
+    const { chatLLMId } = useModelsStore.getState();
+    if (!chatLLMId) return;
+    openLayoutLLMOptions(chatLLMId);
+  }, []);
+
  const shortcuts = React.useMemo((): GlobalShortcutItem[] => [
+    ['o', true, true, false, handleOpenChatLlmOptions],
    ['r', true, true, false, handleMessageRegenerateLast],
    ['n', true, false, true, handleConversationNew],
    ['b', true, false, true, () => isFocusedChatEmpty || focusedConversationId && handleConversationBranch(focusedConversationId, null)],
@@ -299,7 +329,7 @@ export function AppChat() {
    ['d', true, false, true, () => focusedConversationId && handleConversationDelete(focusedConversationId, false)],
    [ShortcutKeyName.Left, true, false, true, () => handleNavigateHistory('back')],
    [ShortcutKeyName.Right, true, false, true, () => handleNavigateHistory('forward')],
-  ], [focusedConversationId, handleConversationBranch, handleConversationDelete, handleConversationNew, handleMessageRegenerateLast, handleNavigateHistory, isFocusedChatEmpty]);
+  ], [focusedConversationId, handleConversationBranch, handleConversationDelete, handleConversationNew, handleMessageRegenerateLast, handleNavigateHistory, handleOpenChatLlmOptions, isFocusedChatEmpty]);
  useGlobalShortcuts(shortcuts);


@@ -357,6 +387,7 @@ export function AppChat() {

          <ChatMessageList
            conversationId={_conversationId}
+            chatLLMContextTokens={chatLLM?.contextTokens}
            isMessageSelectionMode={isMessageSelectionMode}
            setIsMessageSelectionMode={setIsMessageSelectionMode}
            onConversationBranch={handleConversationBranch}
@@ -395,10 +426,11 @@ export function AppChat() {
    </Box>

    <Composer
+      chatLLM={chatLLM}
+      composerTextAreaRef={composerTextAreaRef}
      conversationId={focusedConversationId}
      isDeveloperMode={focusedSystemPurposeId === 'Developer'}
-      composerTextAreaRef={composerTextAreaRef}
-      onNewMessage={handleComposerNewMessage}
+      onAction={handleComposerAction}
      sx={{
        zIndex: 21, // position: 'sticky', bottom: 0,
        backgroundColor: 'background.surface',
@@ -427,7 +459,7 @@ export function AppChat() {
    {/* [confirmation] Reset Conversation */}
    {!!clearConversationId && <ConfirmationModal
      open onClose={() => setClearConversationId(null)} onPositive={handleConfirmedClearConversation}
-      confirmationText={'Are you sure you want to discard all the messages?'} positiveActionText={'Clear conversation'}
+      confirmationText={'Are you sure you want to discard all messages?'} positiveActionText={'Clear conversation'}
    />}

    {/* [confirmation] Delete All */}
@@ -4,8 +4,7 @@ import { shallow } from 'zustand/shallow';
 import { Box, List } from '@mui/joy';
 import { SxProps } from '@mui/joy/styles/types';

-import { DiagramConfig } from '~/modules/aifn/digrams/DiagramsModal';
-import { useChatLLM } from '~/modules/llms/store-llms';
+import type { DiagramConfig } from '~/modules/aifn/digrams/DiagramsModal';

 import { ShortcutKeyName, useGlobalShortcut } from '~/common/components/useGlobalShortcut';
 import { InlineError } from '~/common/components/InlineError';
@@ -24,6 +23,7 @@ import { useChatShowSystemMessages } from '../store-app-chat';
 */
 export function ChatMessageList(props: {
  conversationId: DConversationId | null,
+  chatLLMContextTokens?: number,
  isMessageSelectionMode: boolean, setIsMessageSelectionMode: (isMessageSelectionMode: boolean) => void,
  onConversationBranch: (conversationId: DConversationId, messageId: string) => void,
  onConversationExecuteHistory: (conversationId: DConversationId, history: DMessage[]) => void,
@@ -40,20 +40,21 @@ export function ChatMessageList(props: {

  // external state
  const [showSystemMessages] = useChatShowSystemMessages();
-  const { conversationMessages, editMessage, deleteMessage, historyTokenCount } = useChatStore(state => {
+  const { conversationMessages, historyTokenCount, editMessage, deleteMessage, setMessages } = useChatStore(state => {
    const conversation = state.conversations.find(conversation => conversation.id === props.conversationId);
    return {
      conversationMessages: conversation ? conversation.messages : [],
-      editMessage: state.editMessage, deleteMessage: state.deleteMessage,
      historyTokenCount: conversation ? conversation.tokenCount : 0,
+      deleteMessage: state.deleteMessage,
+      editMessage: state.editMessage,
+      setMessages: state.setMessages,
    };
  }, shallow);
-  const { chatLLM } = useChatLLM();
  const { mayWork: isImaginable } = useCapabilityProdia();
  const { mayWork: isSpeakable } = useCapabilityElevenLabs();

  // derived state
-  const { conversationId, onConversationExecuteHistory, onConversationBranch, onTextDiagram, onTextImagine, onTextSpeak } = props;
+  const { conversationId, onConversationBranch, onConversationExecuteHistory, onTextDiagram, onTextImagine, onTextSpeak } = props;


  // text actions
@@ -76,6 +77,14 @@ export function ChatMessageList(props: {
    }
  }, [conversationId, onConversationExecuteHistory]);

+  const handleConversationTruncate = React.useCallback((messageId: string) => {
+    const messages = getConversation(conversationId)?.messages;
+    if (conversationId && messages) {
+      const truncatedHistory = messages.slice(0, messages.findIndex(m => m.id === messageId) + 1);
+      setMessages(conversationId, truncatedHistory);
+    }
+  }, [conversationId, setMessages]);
+
  const handleMessageDelete = React.useCallback((messageId: string) => {
    conversationId && deleteMessage(conversationId, messageId);
  }, [conversationId, deleteMessage]);
@@ -178,7 +187,7 @@ export function ChatMessageList(props: {
          <CleanerMessage
            key={'sel-' + message.id}
            message={message}
-            isBottom={idx === 0} remainingTokens={(chatLLM ? chatLLM.contextTokens : 0) - historyTokenCount}
+            isBottom={idx === 0} remainingTokens={(props.chatLLMContextTokens || 0) - historyTokenCount}
            selected={selectedMessages.has(message.id)} onToggleSelected={handleSelectMessage}
          />

@@ -192,6 +201,7 @@ export function ChatMessageList(props: {
            isImagining={isImagining} isSpeaking={isSpeaking}
            onConversationBranch={handleConversationBranch}
            onConversationRestartFrom={handleConversationRestartFrom}
+            onConversationTruncate={handleConversationTruncate}
            onMessageDelete={handleMessageDelete}
            onMessageEdit={handleMessageEdit}
            onTextDiagram={handleTextDiagram}
@@ -14,8 +14,8 @@ import { openLayoutLLMOptions, openLayoutModelsSetup } from '~/common/layout/sto

 function AppBarLLMDropdown(props: {
  llms: DLLM[],
-  llmId: DLLMId | null,
-  setLlmId: (llmId: DLLMId | null) => void,
+  chatLlmId: DLLMId | null,
+  setChatLlmId: (llmId: DLLMId | null) => void,
  placeholder?: string,
 }) {

@@ -23,32 +23,47 @@ function AppBarLLMDropdown(props: {
  const llmItems: DropdownItems = {};
  let prevSourceId: DModelSourceId | null = null;
  for (const llm of props.llms) {
-    if (!llm.hidden || llm.id === props.llmId) {
-      if (!prevSourceId || llm.sId !== prevSourceId) {
-        if (prevSourceId)
-          llmItems[`sep-${llm.id}`] = { type: 'separator', title: llm.sId };
-        prevSourceId = llm.sId;
-      }
-      llmItems[llm.id] = { title: llm.label };
+
+    // filter-out hidden models
+    if (!(!llm.hidden || llm.id === props.chatLlmId))
+      continue;
+
+    // add separators when changing sources
+    if (!prevSourceId || llm.sId !== prevSourceId) {
+      if (prevSourceId)
+        llmItems[`sep-${llm.id}`] = {
+          type: 'separator',
+          title: llm.sId,
+        };
+      prevSourceId = llm.sId;
    }
+
+    // add the model item
+    llmItems[llm.id] = {
+      title: llm.label,
+      // icon: llm.id.startsWith('some vendor') ? <VendorIcon /> : undefined,
+    };
  }

-  const handleChatLLMChange = (_event: any, value: DLLMId | null) => value && props.setLlmId(value);
+  const handleChatLLMChange = (_event: any, value: DLLMId | null) => value && props.setChatLlmId(value);

-  const handleOpenLLMOptions = () => props.llmId && openLayoutLLMOptions(props.llmId);
+  const handleOpenLLMOptions = () => props.chatLlmId && openLayoutLLMOptions(props.chatLlmId);


  return (
    <AppBarDropdown
      items={llmItems}
-      value={props.llmId} onChange={handleChatLLMChange}
+      value={props.chatLlmId} onChange={handleChatLLMChange}
      placeholder={props.placeholder || 'Models …'}
      appendOption={<>

-        {props.llmId && (
+        {props.chatLlmId && (
          <ListItemButton key='menu-opt' onClick={handleOpenLLMOptions}>
            <ListItemDecorator><SettingsIcon color='success' /></ListItemDecorator>
-            Options
+            <Box sx={{ flexGrow: 1, display: 'flex', justifyContent: 'space-between', gap: 1 }}>
+              Options
+              <KeyStroke combo='Ctrl + Shift + O' />
+            </Box>
          </ListItemButton>
        )}

@@ -74,7 +89,7 @@ export function useChatLLMDropdown() {
  }), shallow);

  const chatLLMDropdown = React.useMemo(
-    () => <AppBarLLMDropdown llms={llms} llmId={chatLLMId} setLlmId={setChatLLMId} />,
+    () => <AppBarLLMDropdown llms={llms} chatLlmId={chatLLMId} setChatLlmId={setChatLLMId} />,
    [llms, chatLLMId, setChatLLMId],
  );

@@ -0,0 +1,47 @@
+import * as React from 'react';
+
+import { Box, Button, IconButton, Tooltip } from '@mui/joy';
+import AddAPhotoIcon from '@mui/icons-material/AddAPhoto';
+
+import { CameraCaptureModal } from './CameraCaptureModal';
+
+
+const attachCameraLegend = (isMobile: boolean) =>
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    <b>Attach photo</b><br />
+    {isMobile ? 'Auto-OCR to read text' : 'See the world, on the go'}
+  </Box>;
+
+
+export const ButtonAttachCameraMemo = React.memo(ButtonAttachCamera);
+
+function ButtonAttachCamera(props: { isMobile?: boolean, onAttachImage: (file: File) => void }) {
+  // state
+  const [open, setOpen] = React.useState(false);
+
+  return <>
+
+    {/* The Button */}
+    {props.isMobile ? (
+      <IconButton variant='plain' color='neutral' onClick={() => setOpen(true)}>
+        <AddAPhotoIcon />
+      </IconButton>
+    ) : (
+      <Tooltip variant='solid' placement='top-start' title={attachCameraLegend(!!props.isMobile)}>
+        <Button fullWidth variant='plain' color='neutral' onClick={() => setOpen(true)} startDecorator={<AddAPhotoIcon />}
+                sx={{ justifyContent: 'flex-start' }}>
+          Camera
+        </Button>
+      </Tooltip>
+    )}
+
+    {/* The actual capture dialog, which will stream the video */}
+    {open && (
+      <CameraCaptureModal
+        onCloseModal={() => setOpen(false)}
+        onAttachImage={props.onAttachImage}
+      />
+    )}
+
+  </>;
+}
@@ -0,0 +1,32 @@
+import * as React from 'react';
+
+import { Box, Button, IconButton, Tooltip } from '@mui/joy';
+import ContentPasteGoIcon from '@mui/icons-material/ContentPasteGo';
+
+import { KeyStroke } from '~/common/components/KeyStroke';
+
+
+const pasteClipboardLegend =
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    <b>Attach clipboard 📚</b><br />
+    Auto-converts to the best types<br />
+    <KeyStroke combo='Ctrl + Shift + V' sx={{ mt: 1, mb: 0.5 }} />
+  </Box>;
+
+
+export const ButtonAttachClipboardMemo = React.memo(ButtonAttachClipboard);
+
+function ButtonAttachClipboard(props: { isMobile?: boolean, onClick: () => void }) {
+  return props.isMobile ? (
+    <IconButton onClick={props.onClick}>
+      <ContentPasteGoIcon />
+    </IconButton>
+  ) : (
+    <Tooltip variant='solid' placement='top-start' title={pasteClipboardLegend}>
+      <Button fullWidth variant='plain' color='neutral' startDecorator={<ContentPasteGoIcon />} onClick={props.onClick}
+              sx={{ justifyContent: 'flex-start' }}>
+        Paste
+      </Button>
+    </Tooltip>
+  );
+}
@@ -0,0 +1,29 @@
+import * as React from 'react';
+
+import { Box, Button, IconButton, Tooltip } from '@mui/joy';
+import AttachFileOutlinedIcon from '@mui/icons-material/AttachFileOutlined';
+
+
+const attachFileLegend =
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    <b>Attach files</b><br />
+    Drag & drop in chat for faster loads ⚡
+  </Box>;
+
+
+export const ButtonAttachFileMemo = React.memo(ButtonAttachFile);
+
+function ButtonAttachFile(props: { isMobile?: boolean, onAttachFilePicker: () => void }) {
+  return props.isMobile ? (
+    <IconButton onClick={props.onAttachFilePicker}>
+      <AttachFileOutlinedIcon />
+    </IconButton>
+  ) : (
+    <Tooltip variant='solid' placement='top-start' title={attachFileLegend}>
+      <Button fullWidth variant='plain' color='neutral' onClick={props.onAttachFilePicker} startDecorator={<AttachFileOutlinedIcon />}
+              sx={{ justifyContent: 'flex-start' }}>
+        File
+      </Button>
+    </Tooltip>
+  );
+}
@@ -0,0 +1,25 @@
+import * as React from 'react';
+
+import { Box, Button, IconButton, Tooltip } from '@mui/joy';
+import { SxProps } from '@mui/joy/styles/types';
+import CallIcon from '@mui/icons-material/Call';
+
+
+const callConversationLegend =
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    Quick call regarding this chat
+  </Box>;
+
+export function ButtonCall(props: { isMobile?: boolean, disabled?: boolean, onClick: () => void, sx?: SxProps }) {
+  return props.isMobile ? (
+    <IconButton variant='soft' color='primary' disabled={props.disabled} onClick={props.onClick} sx={props.sx}>
+      <CallIcon />
+    </IconButton>
+  ) : (
+    <Tooltip variant='solid' arrow placement='right' title={callConversationLegend}>
+      <Button variant='soft' color='primary' disabled={props.disabled} onClick={props.onClick} endDecorator={<CallIcon />} sx={props.sx}>
+        Call
+      </Button>
+    </Tooltip>
+  );
+}
@@ -1,34 +0,0 @@
-import * as React from 'react';
-
-import { Button, IconButton } from '@mui/joy';
-import AddAPhotoIcon from '@mui/icons-material/AddAPhoto';
-
-import { CameraCaptureModal } from './CameraCaptureModal';
-
-const CAMERA_ENABLE_ON_DESKTOP = false;
-
-
-export function ButtonCameraCapture(props: { isMobile: boolean, onOCR: (ocrText: string) => void }) {
-  // state
-  const [open, setOpen] = React.useState(false);
-
-  return <>
-
-    {/* The Button */}
-    {props.isMobile ? (
-      <IconButton variant='plain' color='neutral' onClick={() => setOpen(true)}>
-        <AddAPhotoIcon />
-      </IconButton>
-    ) : CAMERA_ENABLE_ON_DESKTOP ? (
-      <Button
-        fullWidth variant='plain' color='neutral' onClick={() => setOpen(true)} startDecorator={<AddAPhotoIcon />}
-        sx={{ justifyContent: 'flex-start' }}>
-        OCR
-      </Button>
-    ) : undefined}
-
-    {/* The actual capture dialog, which will stream the video */}
-    {open && <CameraCaptureModal onCloseModal={() => setOpen(false)} onOCR={props.onOCR} />}
-
-  </>;
-}
@@ -1,31 +0,0 @@
-import * as React from 'react';
-
-import { Box, Button, IconButton, Tooltip } from '@mui/joy';
-import ContentPasteGoIcon from '@mui/icons-material/ContentPasteGo';
-
-import { KeyStroke } from '~/common/components/KeyStroke';
-
-
-const pasteClipboardLegend =
-  <Box sx={{ p: 1, lineHeight: 2 }}>
-    <b>Paste as 📚 Markdown attachment</b><br />
-    Also converts Code and Tables<br />
-    <KeyStroke combo='Ctrl + Shift + V' />
-  </Box>;
-
-export function ButtonClipboardPaste(props: { isMobile: boolean, isDeveloperMode: boolean, onPaste: () => void }) {
-  return props.isMobile ? (
-    <IconButton onClick={props.onPaste}>
-      <ContentPasteGoIcon />
-    </IconButton>
-  ) : (
-    <Tooltip
-      variant='solid' placement='top-start'
-      title={pasteClipboardLegend}>
-      <Button fullWidth variant='plain' color='neutral' startDecorator={<ContentPasteGoIcon />} onClick={props.onPaste}
-              sx={{ justifyContent: 'flex-start' }}>
-        {props.isDeveloperMode ? 'Paste code' : 'Paste'}
-      </Button>
-    </Tooltip>
-  );
-}
@@ -1,70 +0,0 @@
-import { Box, Button, IconButton, Stack, Tooltip } from '@mui/joy';
-import * as React from 'react';
-import AttachFileOutlinedIcon from '@mui/icons-material/AttachFileOutlined';
-
-const attachFileLegend =
-  <Stack sx={{ p: 1, gap: 1 }}>
-    <Box sx={{ mb: 1 }}>
-      <b>Attach a file</b>
-    </Box>
-    <table>
-      <tbody>
-      <tr>
-        <td><b>Text</b></td>
-        <td align='center' style={{ opacity: 0.5 }}>→</td>
-        <td>📝 As-is</td>
-      </tr>
-      <tr>
-        <td><b>Code</b></td>
-        <td align='center' style={{ opacity: 0.5 }}>→</td>
-        <td>📚 Markdown</td>
-      </tr>
-      <tr>
-        <td><b>PDF</b></td>
-        <td width={36} align='center' style={{ opacity: 0.5 }}>→</td>
-        <td>📝 Text (summarized)</td>
-      </tr>
-      </tbody>
-    </table>
-    <Box sx={{ mt: 1, fontSize: '14px' }}>
-      Drag & drop in chat for faster loads ⚡
-    </Box>
-  </Stack>;
-
-
-export function ButtonFileAttach(props: { isMobile: boolean, onAttachFiles: (files: FileList) => Promise<void> }) {
-
-  // state
-  const attachmentFileInputRef = React.useRef<HTMLInputElement>(null);
-
-  const handleShowFilePicker = () => attachmentFileInputRef.current?.click();
-
-  const handleLoadAttachment = (event: React.ChangeEvent<HTMLInputElement>) => {
-    // NOTE: resetting the target value allows for the selector dialog to pop-up again
-    const files = event.target?.files;
-    if (files && files.length >= 1)
-      props.onAttachFiles(files).finally(() => event.target.value = '');
-    else
-      event.target.value = '';
-  };
-
-  return <>
-
-    {/* Mobile icon or Desktop button */}
-    {props.isMobile ? (
-      <IconButton onClick={handleShowFilePicker}>
-        <AttachFileOutlinedIcon />
-      </IconButton>
-    ) : (
-      <Tooltip variant='solid' placement='top-start' title={attachFileLegend}>
-        <Button fullWidth variant='plain' color='neutral' onClick={handleShowFilePicker} startDecorator={<AttachFileOutlinedIcon />}
-                sx={{ justifyContent: 'flex-start' }}>
-          Attach
-        </Button>
-      </Tooltip>
-    )}
-
-    <input type='file' multiple hidden ref={attachmentFileInputRef} onChange={handleLoadAttachment} />
-
-  </>;
-}
@@ -0,0 +1,26 @@
+import * as React from 'react';
+
+import { Box, IconButton } from '@mui/joy';
+import { ColorPaletteProp, VariantProp } from '@mui/joy/styles/types';
+import MicIcon from '@mui/icons-material/Mic';
+
+import { GoodTooltip } from '~/common/components/GoodTooltip';
+import { KeyStroke } from '~/common/components/KeyStroke';
+
+
+const micLegend =
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    Voice input<br />
+    <KeyStroke combo='Ctrl + M' sx={{ mt: 1, mb: 0.5 }} />
+  </Box>;
+
+
+export const ButtonMicMemo = React.memo(ButtonMic);
+
+function ButtonMic(props: { variant: VariantProp, color: ColorPaletteProp, noBackground?: boolean, onClick: () => void }) {
+  return <GoodTooltip placement='top' title={micLegend}>
+    <IconButton variant={props.variant} color={props.color} onClick={props.onClick} sx={props.noBackground ? { background: 'none' } : {}}>
+      <MicIcon />
+    </IconButton>
+  </GoodTooltip>;
+}
@@ -0,0 +1,22 @@
+import * as React from 'react';
+
+import { Box, IconButton, Tooltip } from '@mui/joy';
+import { ColorPaletteProp, SxProps, VariantProp } from '@mui/joy/styles/types';
+import AutoModeIcon from '@mui/icons-material/AutoMode';
+
+
+const micContinuationLegend =
+  <Box sx={{ px: 1, py: 0.75, lineHeight: '1.5rem' }}>
+    Voice Continuation
+  </Box>;
+
+
+export const ButtonMicContinuationMemo = React.memo(ButtonMicContinuation);
+
+function ButtonMicContinuation(props: { variant: VariantProp, color: ColorPaletteProp, onClick: () => void, sx?: SxProps }) {
+  return <Tooltip placement='bottom' title={micContinuationLegend}>
+    <IconButton variant={props.variant} color={props.color} onClick={props.onClick} sx={props.sx}>
+      <AutoModeIcon />
+    </IconButton>
+  </Tooltip>;
+}
@@ -0,0 +1,18 @@
+import * as React from 'react';
+
+import { Button, IconButton } from '@mui/joy';
+import { SxProps } from '@mui/joy/styles/types';
+import FormatPaintIcon from '@mui/icons-material/FormatPaint';
+
+
+export function ButtonOptionsDraw(props: { isMobile?: boolean, onClick: () => void, sx?: SxProps }) {
+  return props.isMobile ? (
+    <IconButton variant='soft' color='warning' onClick={props.onClick} sx={props.sx}>
+      <FormatPaintIcon />
+    </IconButton>
+  ) : (
+    <Button variant='soft' color='warning' onClick={props.onClick} endDecorator={<FormatPaintIcon />} sx={props.sx}>
+      Options
+    </Button>
+  );
+}
@@ -1,6 +1,7 @@
 import * as React from 'react';

-import { Box, Button, CircularProgress, IconButton, LinearProgress, Modal, ModalClose, Option, Select, Sheet, Typography } from '@mui/joy';
+import { Box, Button, IconButton, Modal, ModalClose, Option, Select, Sheet, Typography } from '@mui/joy';
+import CameraAltIcon from '@mui/icons-material/CameraAlt';
 import DownloadIcon from '@mui/icons-material/Download';
 import InfoIcon from '@mui/icons-material/Info';
 import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown';
@@ -9,6 +10,12 @@ import { InlineError } from '~/common/components/InlineError';
 import { useCameraCapture } from '~/common/components/useCameraCapture';


+function prettyFileName(renderedFrame: HTMLCanvasElement) {
+  const prettyDate = new Date().toISOString().replace(/[:-]/g, '').replace('T', '-').replace('Z', '');
+  const prettyResolution = `${renderedFrame.width}x${renderedFrame.height}`;
+  return `camera-${prettyDate}-${prettyResolution}.png`;
+}
+
 function renderVideoFrameToCanvas(videoElement: HTMLVideoElement): HTMLCanvasElement {
  // paint the video on a canvas, to save it
  const canvas = document.createElement('canvas');
@@ -19,6 +26,19 @@ function renderVideoFrameToCanvas(videoElement: HTMLVideoElement): HTMLCanvasEle
  return canvas;
 }

+function renderVideoFrameToFile(videoElement: HTMLVideoElement, callback: (file: File) => void) {
+  // video to canvas
+  const renderedFrame = renderVideoFrameToCanvas(videoElement);
+
+  // canvas to blob to file to callback
+  renderedFrame.toBlob((blob) => {
+    if (blob) {
+      const file = new File([blob], prettyFileName(renderedFrame), { type: blob.type });
+      callback(file);
+    }
+  }, 'image/png');
+}
+
 function downloadVideoFrameAsPNG(videoElement: HTMLVideoElement) {
  // video to canvas to png
  const renderedFrame = renderVideoFrameToCanvas(videoElement);
@@ -26,15 +46,19 @@ function downloadVideoFrameAsPNG(videoElement: HTMLVideoElement) {

  // auto-download
  const link = document.createElement('a');
-  link.download = 'image.png';
+  link.download = prettyFileName(renderedFrame);
  link.href = imageDataURL;
  link.click();
 }


-export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (ocrText: string) => void }) {
+export function CameraCaptureModal(props: {
+  onCloseModal: () => void,
+  onAttachImage: (file: File) => void
+  // onOCR: (ocrText: string) => void }
+}) {
  // state
-  const [ocrProgress, setOCRProgress] = React.useState<number | null>(null);
+  // const [ocrProgress/*, setOCRProgress*/] = React.useState<number | null>(null);
  const [showInfo, setShowInfo] = React.useState(false);

  // camera operations
@@ -51,7 +75,7 @@ export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (oc
    props.onCloseModal();
  };

-  const handleVideoOCRClicked = async () => {
+  /*const handleVideoOCRClicked = async () => {
    if (!videoRef.current) return;
    const renderedFrame = renderVideoFrameToCanvas(videoRef.current);

@@ -68,6 +92,14 @@ export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (oc
    setOCRProgress(null);
    stopAndClose();
    props.onOCR(result.data.text);
+  };*/
+
+  const handleVideoSnapClicked = () => {
+    if (!videoRef.current) return;
+    renderVideoFrameToFile(videoRef.current, (file) => {
+      props.onAttachImage(file);
+      stopAndClose();
+    });
  };

  const handleVideoDownloadClicked = () => {
@@ -111,7 +143,7 @@ export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (oc
            ref={videoRef} autoPlay playsInline
            style={{
              display: 'block', width: '100%', maxHeight: 'calc(100vh - 200px)',
-              background: '#8888', opacity: ocrProgress !== null ? 0.5 : 1,
+              background: '#8888', //opacity: ocrProgress !== null ? 0.5 : 1,
            }}
          />

@@ -124,7 +156,7 @@ export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (oc
            {info}
          </Typography>}

-          {ocrProgress !== null && <CircularProgress sx={{ position: 'absolute', top: 'calc(50% - 34px / 2)', left: 'calc(50% - 34px / 2)', zIndex: 2 }} />}
+          {/*{ocrProgress !== null && <CircularProgress sx={{ position: 'absolute', top: 'calc(50% - 34px / 2)', left: 'calc(50% - 34px / 2)', zIndex: 2 }} />}*/}
        </Box>

        {/* Bottom controls (zoom, ocr, download) & progress */}
@@ -134,16 +166,30 @@ export function CameraCaptureModal(props: { onCloseModal: () => void, onOCR: (oc

          {zoomControl}

-          {ocrProgress !== null && <LinearProgress color='primary' determinate value={100 * ocrProgress} sx={{ px: 2 }} />}
+          {/*{ocrProgress !== null && <LinearProgress color='primary' determinate value={100 * ocrProgress} sx={{ px: 2 }} />}*/}

          <Box sx={{ display: 'flex', gap: 1, justifyContent: 'space-between' }}>
-            <IconButton disabled={!info} variant='soft' color='neutral' size='lg' onClick={() => setShowInfo(info => !info)} sx={{ zIndex: 30 }}>
+            {/* Info */}
+            <IconButton disabled={!info} variant='soft' color='neutral' onClick={() => setShowInfo(info => !info)} sx={{ zIndex: 30 }}>
              <InfoIcon />
            </IconButton>
-            <Button disabled={ocrProgress !== null} fullWidth variant='solid' size='lg' onClick={handleVideoOCRClicked} sx={{ flex: 1, maxWidth: 260 }}>
-              Extract Text
+            {/*<Button disabled={ocrProgress !== null} fullWidth variant='solid' size='lg' onClick={handleVideoOCRClicked} sx={{ flex: 1, maxWidth: 260 }}>*/}
+            {/*  Extract Text*/}
+            {/*</Button>*/}
+
+            {/* Capture */}
+            <Button
+              fullWidth
+              variant='solid' color='neutral'
+              onClick={handleVideoSnapClicked}
+              endDecorator={<CameraAltIcon />}
+              sx={{ flex: 1, maxWidth: 200, py: 2, borderRadius: '3rem' }}
+            >
+              Capture
            </Button>
-            <IconButton variant='soft' color='neutral' size='lg' onClick={handleVideoDownloadClicked}>
+
+            {/* Download */}
+            <IconButton variant='soft' color='neutral' onClick={handleVideoDownloadClicked}>
              <DownloadIcon />
            </IconButton>
          </Box>
@@ -1,11 +1,18 @@
 import * as React from 'react';

-import { Badge, ColorPaletteProp, Tooltip } from '@mui/joy';
-import { SxProps } from '@mui/joy/styles/types';
+import { Badge, Box, ColorPaletteProp, Tooltip } from '@mui/joy';


-export function tokensPrettyMath(tokenLimit: number | 0, directTokens: number, indirectTokens?: number) {
-  const usedTokens = directTokens + (indirectTokens || 0);
+function alignRight(value: number, columnSize: number = 7) {
+  const str = value.toLocaleString();
+  return str.padStart(columnSize);
+}
+
+
+export function tokensPrettyMath(tokenLimit: number | 0, directTokens: number, historyTokens?: number, responseMaxTokens?: number): {
+  color: ColorPaletteProp, message: string, remainingTokens: number
+} {
+  const usedTokens = directTokens + (historyTokens || 0) + (responseMaxTokens || 0);
  const remainingTokens = tokenLimit - usedTokens;
  const gteLimit = (remainingTokens <= 0 && tokenLimit > 0);

@@ -17,23 +24,24 @@ export function tokensPrettyMath(tokenLimit: number | 0, directTokens: number, i
    message += `Requested: ${usedTokens.toLocaleString()} tokens`;
  }
  // has full information (d + i < l)
-  else if (indirectTokens) {
+  else if (historyTokens || responseMaxTokens) {
    message +=
-      `${Math.abs(remainingTokens).toLocaleString()} ${remainingTokens > 0 ? 'available' : 'excess'} tokens\n\n` +
-      `  = Model max tokens:  ${tokenLimit.toLocaleString()}\n` +
-      `      - Chat Message:  ${directTokens.toLocaleString()}` +
-      (indirectTokens ? `\n- History + Response:  ${indirectTokens?.toLocaleString()}` : '');
+      `${Math.abs(remainingTokens).toLocaleString()} ${remainingTokens >= 0 ? 'available' : 'excess'} message tokens\n\n` +
+      ` = Model max tokens: ${alignRight(tokenLimit)}\n` +
+      `     - This message: ${alignRight(directTokens)}\n` +
+      `          - History: ${alignRight(historyTokens || 0)}\n` +
+      `     - Max response: ${alignRight(responseMaxTokens || 0)}`;
  }
  // Cleaner mode: d + ? < R (total is the remaining in this case)
  else {
    message +=
      `${(tokenLimit + usedTokens).toLocaleString()} available tokens after deleting this\n\n` +
-      ` = Currently free: ${tokenLimit.toLocaleString()}\n` +
-      `   + This message: ${usedTokens.toLocaleString()}`;
+      ` = Currently free: ${alignRight(tokenLimit)}\n` +
+      `   + This message: ${alignRight(usedTokens)}`;
  }

  const color: ColorPaletteProp =
-    (tokenLimit && remainingTokens < 1)
+    (tokenLimit && remainingTokens < 0)
      ? 'danger'
      : remainingTokens < tokenLimit / 4
        ? 'warning'
@@ -43,35 +51,61 @@ export function tokensPrettyMath(tokenLimit: number | 0, directTokens: number, i
 }


+export const TokenTooltip = (props: { message: string | null, color: ColorPaletteProp, placement?: 'top' | 'top-end', children: React.JSX.Element }) =>
+  <Tooltip
+    placement={props.placement}
+    variant={props.color !== 'primary' ? 'solid' : 'soft'} color={props.color}
+    title={props.message
+      ? <Box sx={{ p: 2, whiteSpace: 'pre' }}>
+        {props.message}
+      </Box>
+      : null
+    }
+    sx={{
+      fontFamily: 'code',
+      boxShadow: 'xl',
+    }}
+  >
+    {props.children}
+  </Tooltip>;
+
+
 /**
 * Simple little component to show the token count (and a tooltip on hover)
 */
-export function TokenBadge({ directTokens, indirectTokens, tokenLimit, showExcess, absoluteBottomRight, inline, sx }: { directTokens: number, indirectTokens?: number, tokenLimit: number, showExcess?: boolean, absoluteBottomRight?: boolean, inline?: boolean, sx?: SxProps }) {
+export const TokenBadgeMemo = React.memo(TokenBadge);

-  const fontSx: SxProps = { fontFamily: 'code', ...(sx || {}) };
-  const outerSx: SxProps = absoluteBottomRight ? { position: 'absolute', bottom: 8, right: 8 } : {};
-  const innerSx: SxProps = (absoluteBottomRight || inline) ? { position: 'static', transform: 'none', ...fontSx } : fontSx;
+function TokenBadge(props: {
+  direct: number, history?: number, responseMax?: number, limit: number,
+  showExcess?: boolean, absoluteBottomRight?: boolean, inline?: boolean,
+}) {

-  const { message, color, remainingTokens } = tokensPrettyMath(tokenLimit, directTokens, indirectTokens);
+  const { message, color, remainingTokens } = tokensPrettyMath(props.limit, props.direct, props.history, props.responseMax);

  // show the direct tokens, unless we exceed the limit and 'showExcess' is enabled
-  const value = (showExcess && (tokenLimit && remainingTokens <= 0))
+  const value = (props.showExcess && (props.limit && remainingTokens <= 0))
    ? Math.abs(remainingTokens)
-    : directTokens;
+    : props.direct;

  return (
    <Badge
      variant='solid' color={color} max={100000}
-      invisible={!directTokens && remainingTokens >= 0}
+      invisible={!props.direct && remainingTokens >= 0}
      badgeContent={
-        <Tooltip title={<span style={{ whiteSpace: 'pre' }}>{message}</span>} color={color} sx={fontSx}>
+        <TokenTooltip color={color} message={message}>
          <span>{value.toLocaleString()}</span>
-        </Tooltip>
+        </TokenTooltip>
      }
-      sx={outerSx}
+      sx={{
+        ...((props.absoluteBottomRight) && { position: 'absolute', bottom: 8, right: 8 }),
+        cursor: 'help',
+      }}
      slotProps={{
        badge: {
-          sx: innerSx,
+          sx: {
+            fontFamily: 'code',
+            ...((props.absoluteBottomRight || props.inline) && { position: 'static', transform: 'none' }),
+          },
        },
      }}
    />
@@ -1,8 +1,8 @@
 import * as React from 'react';

-import { Box, Tooltip, useTheme } from '@mui/joy';
+import { Box, useTheme } from '@mui/joy';

-import { tokensPrettyMath } from './TokenBadge';
+import { tokensPrettyMath, TokenTooltip } from './TokenBadge';


 /**
@@ -10,15 +10,17 @@ import { tokensPrettyMath } from './TokenBadge';
 *
 * The Textarea contains it within the Composer (at least).
 */
-export function TokenProgressbar(props: { history: number, response: number, direct: number, limit: number }) {
+export const TokenProgressbarMemo = React.memo(TokenProgressbar);
+
+function TokenProgressbar(props: { direct: number, history: number, responseMax: number, limit: number }) {
  // external state
  const theme = useTheme();

-  if (!(props.limit > 0) || (!props.direct && !props.history && !props.response)) return null;
+  if (!(props.limit > 0) || (!props.direct && !props.history && !props.responseMax)) return null;

  // compute percentages
  let historyPct = 100 * props.history / props.limit;
-  let responsePct = 100 * props.response / props.limit;
+  let responsePct = 100 * props.responseMax / props.limit;
  let directPct = 100 * props.direct / props.limit;
  const totalPct = historyPct + responsePct + directPct;
  const isOverflow = totalPct >= 100;
@@ -38,7 +40,7 @@ export function TokenProgressbar(props: { history: number, response: number, dir
  const overflowColor = theme.palette.danger.softColor;

  // tooltip message/color
-  const { message, color } = tokensPrettyMath(props.limit, props.direct, props.history + props.response);
+  const { message, color } = tokensPrettyMath(props.limit, props.direct, props.history, props.responseMax);

  // sizes
  const containerHeight = 8;
@@ -46,11 +48,11 @@ export function TokenProgressbar(props: { history: number, response: number, dir

  return (

-    <Tooltip title={<span style={{ whiteSpace: 'pre' }}>{message}</span>} color={color} sx={{ fontFamily: 'code' }}>
+    <TokenTooltip color={color} message={props.direct ? null : message}>

      <Box sx={{
        position: 'absolute', left: 1, right: 1, bottom: 1, height: containerHeight,
-        overflow: 'hidden', borderBottomLeftRadius: 7, borderBottomRightRadius: 7,
+        overflow: 'hidden', borderBottomLeftRadius: 5, borderBottomRightRadius: 5,
      }}>

        {/* History */}
@@ -79,6 +81,6 @@ export function TokenProgressbar(props: { history: number, response: number, dir

      </Box>

-    </Tooltip>
+    </TokenTooltip>
  );
 }
@@ -0,0 +1,201 @@
+import * as React from 'react';
+
+import { Box, Button, CircularProgress, ColorPaletteProp, Sheet, Typography } from '@mui/joy';
+import AbcIcon from '@mui/icons-material/Abc';
+import CodeIcon from '@mui/icons-material/Code';
+import ImageOutlinedIcon from '@mui/icons-material/ImageOutlined';
+import PictureAsPdfIcon from '@mui/icons-material/PictureAsPdf';
+import PivotTableChartIcon from '@mui/icons-material/PivotTableChart';
+import TextFieldsIcon from '@mui/icons-material/TextFields';
+import TextureIcon from '@mui/icons-material/Texture';
+import WarningRoundedIcon from '@mui/icons-material/WarningRounded';
+
+import { GoodTooltip } from '~/common/components/GoodTooltip';
+import { ellipsizeFront, ellipsizeMiddle } from '~/common/util/textUtils';
+
+import type { Attachment, AttachmentConverterType, AttachmentId } from './store-attachments';
+import type { LLMAttachment } from './useLLMAttachments';
+
+
+// default attachment width
+const ATTACHMENT_MIN_STYLE = {
+  height: '100%',
+  minHeight: '40px',
+  minWidth: '64px',
+};
+
+
+const ellipsizeLabel = (label?: string) => {
+  if (!label)
+    return '';
+  return ellipsizeMiddle((label || '')
+    .replace(/https?:\/\/(?:www\.)?/, ''), 30)
+    .replace(/\/$/, '')
+    .replace('…', '…\n…');
+};
+
+
+/**
+ * Displayed while a source is loading
+ */
+const LoadingIndicator = React.forwardRef((props: { label: string }, _ref) =>
+  <Sheet
+    color='success' variant='soft'
+    sx={{
+      border: '1px solid',
+      borderColor: 'success.solidBg',
+      borderRadius: 'sm',
+      display: 'flex', alignItems: 'center', justifyContent: 'center', gap: 1,
+      ...ATTACHMENT_MIN_STYLE,
+      boxSizing: 'border-box',
+      px: 1,
+      py: 0.5,
+    }}
+  >
+    <CircularProgress color='success' size='sm' />
+    <Typography level='title-sm' sx={{ whiteSpace: 'nowrap' }}>
+      {ellipsizeLabel(props.label)}
+    </Typography>
+  </Sheet>,
+);
+LoadingIndicator.displayName = 'LoadingIndicator';
+
+
+const InputErrorIndicator = () =>
+  <WarningRoundedIcon sx={{ color: 'danger.solidBg' }} />;
+
+
+const converterTypeToIconMap: { [key in AttachmentConverterType]: React.ComponentType<any> } = {
+  'text': TextFieldsIcon,
+  'rich-text': CodeIcon,
+  'rich-text-table': PivotTableChartIcon,
+  'pdf-text': PictureAsPdfIcon,
+  'pdf-images': PictureAsPdfIcon,
+  'image': ImageOutlinedIcon,
+  'image-ocr': AbcIcon,
+  'unhandled': TextureIcon,
+};
+
+function attachmentConverterIcon(attachment: Attachment) {
+  const converter = attachment.converterIdx !== null ? attachment.converters[attachment.converterIdx] ?? null : null;
+  if (converter && converter.id) {
+    const Icon = converterTypeToIconMap[converter.id] ?? null;
+    if (Icon)
+      return <Icon sx={{ width: 24, height: 24 }} />;
+  }
+  return null;
+}
+
+function attachmentLabelText(attachment: Attachment): string {
+  return ellipsizeFront(attachment.label, 24);
+}
+
+
+export function AttachmentItem(props: {
+  llmAttachment: LLMAttachment,
+  menuShown: boolean,
+  onItemMenuToggle: (attachmentId: AttachmentId, anchor: HTMLAnchorElement) => void,
+}) {
+
+  // derived state
+
+  const { onItemMenuToggle } = props;
+
+  const {
+    attachment,
+    isUnconvertible,
+    isOutputMissing,
+    isOutputAttachable,
+  } = props.llmAttachment;
+
+  const {
+    inputError,
+    inputLoading: isInputLoading,
+    outputsConverting: isOutputLoading,
+  } = attachment;
+
+  const isInputError = !!inputError;
+  const showWarning = isUnconvertible || isOutputMissing || !isOutputAttachable;
+
+
+  const handleToggleMenu = React.useCallback((event: React.MouseEvent<HTMLAnchorElement>) => {
+    event.stopPropagation();
+    onItemMenuToggle(attachment.id, event.currentTarget);
+  }, [attachment, onItemMenuToggle]);
+
+
+  // compose tooltip
+  let tooltip: string | null = '';
+  if (attachment.source.media !== 'text')
+    tooltip += attachment.source.media + ': ';
+  tooltip += attachment.label;
+  // if (hasInput)
+  //   tooltip += `\n(${aInput.mimeType}: ${aInput.dataSize.toLocaleString()} bytes)`;
+  // if (aOutputs && aOutputs.length >= 1)
+  //   tooltip += `\n\n${JSON.stringify(aOutputs)}`;
+
+  // choose variants and color
+  let color: ColorPaletteProp;
+  let variant: 'soft' | 'outlined' | 'contained' = 'soft';
+  if (isInputLoading || isOutputLoading) {
+    color = 'success';
+  } else if (isInputError) {
+    tooltip = `Issue loading the attachment: ${attachment.inputError}\n\n${tooltip}`;
+    color = 'danger';
+  } else if (showWarning) {
+    tooltip = props.menuShown
+      ? null
+      : isUnconvertible
+        ? `Attachments of type '${attachment.input?.mimeType}' are not supported yet. You can open a feature request on GitHub.\n\n${tooltip}`
+        : `Not compatible with the selected LLM or not supported. Please select another format.\n\n${tooltip}`;
+    color = 'warning';
+  } else {
+    // all good
+    tooltip = null;
+    color = /*props.menuShown ? 'primary' :*/ 'neutral';
+    variant = 'outlined';
+  }
+
+
+  return <Box>
+
+    <GoodTooltip
+      title={tooltip}
+      isError={isInputError}
+      isWarning={showWarning}
+      sx={{ p: 1, whiteSpace: 'break-spaces' }}
+    >
+      {isInputLoading
+        ? <LoadingIndicator label={attachment.label} />
+        : (
+          <Button
+            size='sm'
+            variant={variant} color={color}
+            onClick={handleToggleMenu}
+            sx={{
+              backgroundColor: props.menuShown ? `${color}.softActiveBg` : variant === 'outlined' ? 'background.popup' : undefined,
+              border: variant === 'soft' ? '1px solid' : undefined,
+              borderColor: variant === 'soft' ? `${color}.solidBg` : undefined,
+              borderRadius: 'sm',
+              fontWeight: 'normal',
+              ...ATTACHMENT_MIN_STYLE,
+              px: 1, py: 0.5,
+              display: 'flex', flexDirection: 'row', gap: 1,
+            }}
+          >
+            {isInputError
+              ? <InputErrorIndicator />
+              : <>
+                {attachmentConverterIcon(attachment)}
+                {isOutputLoading
+                  ? <>Converting <CircularProgress color='success' size='sm' /></>
+                  : <Typography level='title-sm' sx={{ whiteSpace: 'nowrap' }}>
+                    {attachmentLabelText(attachment)}
+                  </Typography>}
+              </>}
+          </Button>
+        )}
+    </GoodTooltip>
+
+  </Box>;
+}
@@ -0,0 +1,186 @@
+import * as React from 'react';
+
+import { Box, ListDivider, ListItemDecorator, MenuItem, Radio, Typography } from '@mui/joy';
+import ClearIcon from '@mui/icons-material/Clear';
+import ContentCopyIcon from '@mui/icons-material/ContentCopy';
+import KeyboardArrowLeftIcon from '@mui/icons-material/KeyboardArrowLeft';
+import KeyboardArrowRightIcon from '@mui/icons-material/KeyboardArrowRight';
+import VerticalAlignBottomIcon from '@mui/icons-material/VerticalAlignBottom';
+
+import { CloseableMenu } from '~/common/components/CloseableMenu';
+import { copyToClipboard } from '~/common/util/clipboardUtils';
+
+import type { LLMAttachment } from './useLLMAttachments';
+import { useAttachmentsStore } from './store-attachments';
+
+
+// enable for debugging
+export const DEBUG_ATTACHMENTS = true;
+
+
+export function AttachmentMenu(props: {
+  llmAttachment: LLMAttachment,
+  menuAnchor: HTMLAnchorElement,
+  isPositionFirst: boolean,
+  isPositionLast: boolean,
+  onAttachmentInlineText: (attachmentId: string) => void,
+  onClose: () => void,
+}) {
+
+  // derived state
+
+  const isPositionFixed = props.isPositionFirst && props.isPositionLast;
+
+  const {
+    attachment,
+    attachmentOutputs,
+    isUnconvertible,
+    isOutputMissing,
+    isOutputTextInlineable,
+    tokenCountApprox,
+  } = props.llmAttachment;
+
+  const {
+    id: aId,
+    input: aInput,
+    converters: aConverters,
+    converterIdx: aConverterIdx,
+    outputs: aOutputs,
+  } = attachment;
+
+
+  // operations
+
+  const { onClose, onAttachmentInlineText } = props;
+
+  const handleInlineText = React.useCallback(() => {
+    onClose();
+    onAttachmentInlineText(aId);
+  }, [aId, onAttachmentInlineText, onClose]);
+
+  const handleMoveUp = React.useCallback(() => {
+    useAttachmentsStore.getState().moveAttachment(aId, -1);
+  }, [aId]);
+
+  const handleMoveDown = React.useCallback(() => {
+    useAttachmentsStore.getState().moveAttachment(aId, 1);
+  }, [aId]);
+
+  const handleRemove = React.useCallback(() => {
+    onClose();
+    useAttachmentsStore.getState().removeAttachment(aId);
+  }, [aId, onClose]);
+
+  const handleSetConverterIdx = React.useCallback(async (converterIdx: number | null) => {
+    return useAttachmentsStore.getState().setConverterIdx(aId, converterIdx);
+  }, [aId]);
+
+  // const handleSummarizeText = React.useCallback(() => {
+  //   onAttachmentSummarizeText(aId);
+  // }, [aId, onAttachmentSummarizeText]);
+
+  const handleCopyOutputToClipboard = React.useCallback(() => {
+    if (attachmentOutputs.length >= 1) {
+      const concat = attachmentOutputs.map(output => {
+        if (output.type === 'text-block')
+          return output.text;
+        else if (output.type === 'image-part')
+          return output.base64Url;
+        else
+          return null;
+      }).join('\n\n---\n\n');
+      copyToClipboard(concat.trim(), 'Converted attachment');
+    }
+  }, [attachmentOutputs]);
+
+
+  return (
+    <CloseableMenu
+      dense placement='top' sx={{ minWidth: 200 }}
+      open anchorEl={props.menuAnchor} onClose={props.onClose}
+      noTopPadding noBottomPadding
+    >
+
+      {/* Move Arrows */}
+      {!isPositionFixed && <Box sx={{ display: 'flex', alignItems: 'center' }}>
+        <MenuItem
+          disabled={props.isPositionFirst}
+          onClick={handleMoveUp}
+          sx={{ flex: 1, display: 'flex', justifyContent: 'center' }}
+        >
+          <KeyboardArrowLeftIcon />
+        </MenuItem>
+        <MenuItem
+          disabled={props.isPositionLast}
+          onClick={handleMoveDown}
+          sx={{ flex: 1, display: 'flex', justifyContent: 'center' }}
+        >
+          <KeyboardArrowRightIcon />
+        </MenuItem>
+      </Box>}
+      {!isPositionFixed && <ListDivider sx={{ mt: 0 }} />}
+
+      {/* Render Converters as menu items */}
+      {/*{!isUnconvertible && <ListItem>*/}
+      {/*  <Typography level='body-md'>*/}
+      {/*    Attach as:*/}
+      {/*  </Typography>*/}
+      {/*</ListItem>}*/}
+      {!isUnconvertible && aConverters.map((c, idx) =>
+        <MenuItem
+          disabled={c.disabled}
+          key={'c-' + c.id}
+          onClick={async () => idx !== aConverterIdx && await handleSetConverterIdx(idx)}
+        >
+          <ListItemDecorator>
+            <Radio checked={idx === aConverterIdx} />
+          </ListItemDecorator>
+          {c.unsupported
+            ? <Box>Unsupported 🤔 <Typography level='body-xs'>{c.name}</Typography></Box>
+            : c.name}
+        </MenuItem>,
+      )}
+      {!isUnconvertible && <ListDivider />}
+
+      {DEBUG_ATTACHMENTS && !!aInput && (
+        <MenuItem onClick={handleCopyOutputToClipboard} disabled={!isOutputTextInlineable}>
+          <ListItemDecorator><ContentCopyIcon /></ListItemDecorator>
+          <Box>
+            {!!aInput && <Typography level='body-xs'>
+              🡐 {aInput.mimeType}, {aInput.dataSize.toLocaleString()} bytes
+            </Typography>}
+            {/*<Typography level='body-xs'>*/}
+            {/*  Converters: {aConverters.map(((converter, idx) => ` ${converter.id}${(idx === aConverterIdx) ? '*' : ''}`)).join(', ')}*/}
+            {/*</Typography>*/}
+            <Typography level='body-xs'>
+              🡒 {isOutputMissing ? 'empty' : aOutputs.map(output => `${output.type}, ${output.type === 'text-block' ? output.text.length.toLocaleString() : '(base64 image)'} bytes`).join(' · ')}
+            </Typography>
+            {!!tokenCountApprox && <Typography level='body-xs'>
+              🡒 {tokenCountApprox.toLocaleString()} tokens
+            </Typography>}
+          </Box>
+        </MenuItem>
+      )}
+      {DEBUG_ATTACHMENTS && !!aInput && <ListDivider />}
+
+      {/* Destructive Operations */}
+      {/*<MenuItem onClick={handleCopyOutputToClipboard} disabled={!isOutputTextInlineable}>*/}
+      {/*  <ListItemDecorator><ContentCopyIcon /></ListItemDecorator>*/}
+      {/*  Copy*/}
+      {/*</MenuItem>*/}
+      {/*<MenuItem onClick={handleSummarizeText} disabled={!isOutputTextInlineable}>*/}
+      {/*  <ListItemDecorator><CompressIcon color='success' /></ListItemDecorator>*/}
+      {/*  Shrink*/}
+      {/*</MenuItem>*/}
+      <MenuItem onClick={handleInlineText} disabled={!isOutputTextInlineable}>
+        <ListItemDecorator><VerticalAlignBottomIcon /></ListItemDecorator>
+        Inline text
+      </MenuItem>
+      <MenuItem onClick={handleRemove}>
+        <ListItemDecorator><ClearIcon /></ListItemDecorator>
+        Remove
+      </MenuItem>
+
+    </CloseableMenu>
+  );
+}
@@ -0,0 +1,170 @@
+import * as React from 'react';
+
+import { Box, IconButton, ListItemDecorator, MenuItem } from '@mui/joy';
+import ClearIcon from '@mui/icons-material/Clear';
+import ExpandLessIcon from '@mui/icons-material/ExpandLess';
+import VerticalAlignBottomIcon from '@mui/icons-material/VerticalAlignBottom';
+
+import { CloseableMenu } from '~/common/components/CloseableMenu';
+import { ConfirmationModal } from '~/common/components/ConfirmationModal';
+
+import type { AttachmentId } from './store-attachments';
+import type { LLMAttachments } from './useLLMAttachments';
+import { AttachmentItem } from './AttachmentItem';
+import { AttachmentMenu } from './AttachmentMenu';
+
+
+/**
+ * Renderer of attachments, with menus, etc.
+ */
+export function Attachments(props: {
+  llmAttachments: LLMAttachments,
+  onAttachmentInlineText: (attachmentId: AttachmentId) => void,
+  onAttachmentsClear: () => void,
+  onAttachmentsInlineText: () => void,
+}) {
+
+  // state
+  const [confirmClearAttachments, setConfirmClearAttachments] = React.useState<boolean>(false);
+  const [itemMenu, setItemMenu] = React.useState<{ anchor: HTMLAnchorElement, attachmentId: AttachmentId } | null>(null);
+  const [overallMenuAnchor, setOverallMenuAnchor] = React.useState<HTMLAnchorElement | null>(null);
+
+  // derived state
+  const { llmAttachments, onAttachmentsClear, onAttachmentInlineText, onAttachmentsInlineText } = props;
+
+  const { attachments, isOutputTextInlineable } = llmAttachments;
+
+  const hasAttachments = attachments.length >= 1;
+
+  // derived item menu state
+
+  const itemMenuAnchor = itemMenu?.anchor;
+  const itemMenuAttachmentId = itemMenu?.attachmentId;
+  const itemMenuAttachment = itemMenuAttachmentId ? attachments.find(la => la.attachment.id === itemMenu.attachmentId) : undefined;
+  const itemMenuIndex = itemMenuAttachment ? attachments.indexOf(itemMenuAttachment) : -1;
+
+
+  // item menu
+
+  const handleItemMenuToggle = React.useCallback((attachmentId: AttachmentId, anchor: HTMLAnchorElement) => {
+    handleOverallMenuHide();
+    setItemMenu(prev => prev?.attachmentId === attachmentId ? null : { anchor, attachmentId });
+  }, []);
+
+  const handleItemMenuHide = React.useCallback(() => {
+    setItemMenu(null);
+  }, []);
+
+
+  // item menu operations
+
+  const handleAttachmentInlineText = React.useCallback((attachmentId: string) => {
+    handleItemMenuHide();
+    onAttachmentInlineText(attachmentId);
+  }, [handleItemMenuHide, onAttachmentInlineText]);
+
+
+  // menu
+
+  const handleOverallMenuHide = () => setOverallMenuAnchor(null);
+
+  const handleOverallMenuToggle = (event: React.MouseEvent<HTMLAnchorElement>) =>
+    setOverallMenuAnchor(anchor => anchor ? null : event.currentTarget);
+
+
+  // overall operations
+
+  const handleAttachmentsInlineText = React.useCallback(() => {
+    handleOverallMenuHide();
+    onAttachmentsInlineText();
+  }, [onAttachmentsInlineText]);
+
+  const handleClearAttachments = () => setConfirmClearAttachments(true);
+
+  const handleClearAttachmentsConfirmed = React.useCallback(() => {
+    handleOverallMenuHide();
+    setConfirmClearAttachments(false);
+    onAttachmentsClear();
+  }, [onAttachmentsClear]);
+
+
+  // no components without attachments
+  if (!hasAttachments)
+    return null;
+
+  return <>
+
+    {/* Attachments bar */}
+    <Box sx={{ position: 'relative' }}>
+
+      {/* Horizontally scrollable Attachments */}
+      <Box sx={{ display: 'flex', overflowX: 'auto', gap: 1, height: '100%', pr: 5 }}>
+        {attachments.map((llmAttachment) =>
+          <AttachmentItem
+            key={llmAttachment.attachment.id}
+            llmAttachment={llmAttachment}
+            menuShown={llmAttachment.attachment.id === itemMenuAttachmentId}
+            onItemMenuToggle={handleItemMenuToggle}
+          />,
+        )}
+      </Box>
+
+      {/* Overall Menu button */}
+      <IconButton
+        variant='plain' onClick={handleOverallMenuToggle}
+        sx={{
+          // borderRadius: 'sm',
+          borderRadius: 0,
+          position: 'absolute', right: 0, top: 0,
+          backgroundColor: 'neutral.softDisabledBg',
+        }}
+      >
+        <ExpandLessIcon />
+      </IconButton>
+
+    </Box>
+
+
+    {/* Attachment Menu */}
+    {!!itemMenuAnchor && !!itemMenuAttachment && (
+      <AttachmentMenu
+        llmAttachment={itemMenuAttachment}
+        menuAnchor={itemMenuAnchor}
+        isPositionFirst={itemMenuIndex === 0}
+        isPositionLast={itemMenuIndex === attachments.length - 1}
+        onAttachmentInlineText={handleAttachmentInlineText}
+        onClose={handleItemMenuHide}
+      />
+    )}
+
+
+    {/* Overall Menu */}
+    {!!overallMenuAnchor && (
+      <CloseableMenu
+        placement='top-start'
+        open anchorEl={overallMenuAnchor} onClose={handleOverallMenuHide}
+        noTopPadding noBottomPadding
+      >
+        <MenuItem onClick={handleAttachmentsInlineText} disabled={!isOutputTextInlineable}>
+          <ListItemDecorator><VerticalAlignBottomIcon /></ListItemDecorator>
+          Inline <span style={{ opacity: 0.5 }}>text attachments</span>
+        </MenuItem>
+        <MenuItem onClick={handleClearAttachments}>
+          <ListItemDecorator><ClearIcon /></ListItemDecorator>
+          Clear
+        </MenuItem>
+      </CloseableMenu>
+    )}
+
+    {/* 'Clear' Confirmation */}
+    {confirmClearAttachments && (
+      <ConfirmationModal
+        open onClose={() => setConfirmClearAttachments(false)} onPositive={handleClearAttachmentsConfirmed}
+        title='Confirm Removal'
+        positiveActionText='Remove All'
+        confirmationText={`This action will remove all (${attachments.length}) attachments. Do you want to proceed?`}
+      />
+    )}
+
+  </>;
+}
@@ -0,0 +1,335 @@
+import { callBrowseFetchPage } from '~/modules/browse/browse.client';
+
+import { createBase36Uid } from '~/common/util/textUtils';
+import { htmlTableToMarkdown } from '~/common/util/htmlTableToMarkdown';
+import { pdfToText } from '~/common/util/pdfUtils';
+
+import type { Attachment, AttachmentConverter, AttachmentId, AttachmentInput, AttachmentSource } from './store-attachments';
+import type { ComposerOutputMultiPart } from '../composer.types';
+
+
+// extensions to treat as plain text
+const PLAIN_TEXT_EXTENSIONS: string[] = ['.ts', '.tsx'];
+
+/**
+ * Creates a new Attachment object.
+ */
+export function attachmentCreate(source: AttachmentSource, checkDuplicates: AttachmentId[]): Attachment {
+  return {
+    id: createBase36Uid(checkDuplicates),
+    source: source,
+    label: 'Loading...',
+    ref: '',
+    inputLoading: false,
+    inputError: null,
+    input: undefined,
+    converters: [],
+    converterIdx: null,
+    outputsConverting: false,
+    outputs: [],
+    // metadata: {},
+  };
+}
+
+/**
+ * Asynchronously loads the input for an Attachment object.
+ *
+ * @param {Readonly<AttachmentSource>} source - The source of the attachment.
+ * @param {(changes: Partial<Attachment>) => void} edit - A function to edit the Attachment object.
+ */
+export async function attachmentLoadInputAsync(source: Readonly<AttachmentSource>, edit: (changes: Partial<Attachment>) => void) {
+  edit({ inputLoading: true });
+
+  switch (source.media) {
+
+    // Download URL (page, file, ..) and attach as input
+    case 'url':
+      edit({ label: source.refUrl, ref: source.refUrl });
+      try {
+        const page = await callBrowseFetchPage(source.url);
+        if (page.content) {
+          edit({
+            input: {
+              mimeType: 'text/plain',
+              data: page.content,
+              dataSize: page.content.length,
+            },
+          });
+        } else
+          edit({ inputError: 'No content found at this link' });
+      } catch (error: any) {
+        edit({ inputError: `Issue downloading page: ${error?.message || (typeof error === 'string' ? error : JSON.stringify(error))}` });
+      }
+      break;
+
+    // Attach file as input
+    case 'file':
+      edit({ label: source.refPath, ref: source.refPath });
+
+      // fix missing/wrong mimetypes
+      let mimeType = source.fileWithHandle.type;
+      if (!mimeType) {
+        // see note on 'attachAppendDataTransfer'; this is a fallback for drag/drop missing Mimes sometimes
+        console.warn('Assuming the attachment is text/plain. From:', source.origin, ', name:', source.refPath);
+        mimeType = 'text/plain';
+      } else {
+        // possibly fix wrongly assigned mimetypes (from the extension alone)
+        if (!mimeType.startsWith('text/') && PLAIN_TEXT_EXTENSIONS.some(ext => source.refPath.endsWith(ext)))
+          mimeType = 'text/plain';
+      }
+
+      // UX: just a hint of a loading state
+      await new Promise(resolve => setTimeout(resolve, 100));
+
+      try {
+        const fileArrayBuffer = await source.fileWithHandle.arrayBuffer();
+        edit({
+          input: {
+            mimeType,
+            data: fileArrayBuffer,
+            dataSize: fileArrayBuffer.byteLength,
+          },
+        });
+      } catch (error: any) {
+        edit({ inputError: `Issue loading file: ${error?.message || (typeof error === 'string' ? error : JSON.stringify(error))}` });
+      }
+      break;
+
+    case 'text':
+      if (source.textHtml && source.textPlain) {
+        edit({
+          label: 'Rich Text',
+          ref: '',
+          input: {
+            mimeType: 'text/plain',
+            data: source.textPlain,
+            dataSize: source.textPlain!.length,
+            altMimeType: 'text/html',
+            altData: source.textHtml,
+          },
+        });
+      } else {
+        const text = source.textHtml || source.textPlain || '';
+        edit({
+          label: 'Text',
+          ref: '',
+          input: {
+            mimeType: 'text/plain',
+            data: text,
+            dataSize: text.length,
+          },
+        });
+      }
+      break;
+  }
+
+  edit({ inputLoading: false });
+}
+
+/**
+ * Defines the possible converters for an Attachment object based on its input type.
+ *
+ * @param {AttachmentSource['media']} sourceType - The media type of the attachment source.
+ * @param {Readonly<AttachmentInput>} input - The input of the attachment.
+ * @param {(changes: Partial<Attachment>) => void} edit - A function to edit the Attachment object.
+ */
+export function attachmentDefineConverters(sourceType: AttachmentSource['media'], input: Readonly<AttachmentInput>, edit: (changes: Partial<Attachment>) => void) {
+
+  // return all the possible converters for the input
+  const converters: AttachmentConverter[] = [];
+
+  switch (true) {
+
+    // plain text types
+    case ['text/plain', 'text/html', 'text/markdown', 'text/csv', 'application/json'].includes(input.mimeType):
+      // handle a secondary layer of HTML 'text' origins: drop, paste, and clipboard-read
+      const textOriginHtml = sourceType === 'text' && input.altMimeType === 'text/html' && !!input.altData;
+      const isHtmlTable = !!input.altData?.startsWith('<table');
+
+      // p1: Tables
+      if (textOriginHtml && isHtmlTable) {
+        converters.push({
+          id: 'rich-text-table',
+          name: 'Markdown Table',
+        });
+      }
+
+      // p2: Text
+      converters.push({
+        id: 'text',
+        name: 'Text',
+      });
+
+      // p3: Html
+      if (textOriginHtml) {
+        converters.push({
+          id: 'rich-text',
+          name: 'HTML',
+        });
+      }
+      break;
+
+    // PDF
+    case ['application/pdf', 'application/x-pdf', 'application/acrobat'].includes(input.mimeType):
+      converters.push({ id: 'pdf-text', name: `PDF To Text` });
+      converters.push({ id: 'pdf-images', name: `PDF To Images`, disabled: true });
+      break;
+
+    // images
+    case input.mimeType.startsWith('image/'):
+      converters.push({ id: 'image', name: `Image (coming soon)` });
+      converters.push({ id: 'image-ocr', name: 'As Text (OCR)' });
+      break;
+
+    // catch-all
+    default:
+      converters.push({ id: 'unhandled', name: `${input.mimeType}`, unsupported: true });
+      converters.push({ id: 'text', name: 'As Text' });
+      break;
+  }
+
+  edit({ converters });
+}
+
+/**
+ * Converts the input of an Attachment object based on the selected converter.
+ *
+ * @param {Readonly<Attachment>} attachment - The Attachment object to convert.
+ * @param {number | null} converterIdx - The index of the selected conversion in the Attachment object's converters array.
+ * @param {(changes: Partial<Attachment>) => void} edit - A function to edit the Attachment object.
+ */
+export async function attachmentPerformConversion(attachment: Readonly<Attachment>, converterIdx: number | null, edit: (changes: Partial<Attachment>) => void) {
+
+  // set converter index
+  converterIdx = (converterIdx !== null && converterIdx >= 0 && converterIdx < attachment.converters.length) ? converterIdx : null;
+  edit({
+    converterIdx: converterIdx,
+    outputs: [],
+  });
+
+  // get converter
+  const { ref, input } = attachment;
+  const converter = converterIdx !== null ? attachment.converters[converterIdx] : null;
+  if (!converter || !input)
+    return;
+
+  edit({
+    outputsConverting: true,
+  });
+
+  // input datacould be a string or an ArrayBuffer
+  function inputDataToString(data: string | ArrayBuffer | null | undefined): string {
+    if (typeof data === 'string')
+      return data;
+    if (data instanceof ArrayBuffer)
+      return new TextDecoder().decode(data);
+    return '';
+  }
+
+  // apply converter to the input
+  const outputs: ComposerOutputMultiPart = [];
+  switch (converter.id) {
+
+    // text as-is
+    case 'text':
+      outputs.push({
+        type: 'text-block',
+        text: inputDataToString(input.data),
+        title: ref,
+        collapsible: true,
+      });
+      break;
+
+    // html as-is
+    case 'rich-text':
+      outputs.push({
+        type: 'text-block',
+        text: input.altData!,
+        title: ref,
+        collapsible: true,
+      });
+      break;
+
+    // html to markdown table
+    case 'rich-text-table':
+      let mdTable: string;
+      try {
+        mdTable = htmlTableToMarkdown(input.altData!, false);
+      } catch (error) {
+        // fallback to text/plain
+        mdTable = inputDataToString(input.data);
+      }
+      outputs.push({
+        type: 'text-block',
+        text: mdTable,
+        title: ref,
+        collapsible: true,
+      });
+      break;
+
+    case 'pdf-text':
+      if (!(input.data instanceof ArrayBuffer)) {
+        console.log('Expected ArrayBuffer for PDF converter, got:', typeof input.data);
+        break;
+      }
+      // duplicate the ArrayBuffer to avoid mutation
+      const pdfData = new Uint8Array(input.data.slice(0));
+      const pdfText = await pdfToText(pdfData);
+      outputs.push({
+        type: 'text-block',
+        text: pdfText,
+        title: ref,
+        collapsible: true,
+      });
+      break;
+
+    case 'pdf-images':
+      // TODO: extract all pages as individual images
+      break;
+
+    case 'image':
+      // TODO: continue here
+      /*outputs.push({
+        type: 'image-part',
+        base64Url: `data:notImplemented.yet:)`,
+        collapsible: false,
+      });*/
+      break;
+
+    case 'image-ocr':
+      if (!(input.data instanceof ArrayBuffer)) {
+        console.log('Expected ArrayBuffer for Image OCR converter, got:', typeof input.data);
+        break;
+      }
+      try {
+        const { recognize } = await import('tesseract.js');
+        const buffer = Buffer.from(input.data);
+        const result = await recognize(buffer, undefined, {
+          errorHandler: e => console.error(e),
+          logger: (message) => {
+            if (message.status === 'recognizing text')
+              console.log('OCR progress:', message.progress);
+          },
+        });
+        outputs.push({
+          type: 'text-block',
+          text: result.data.text,
+          title: ref,
+          collapsible: true,
+        });
+      } catch (error) {
+        console.error(error);
+      }
+      break;
+
+    case 'unhandled':
+      // force the user to explicitly select 'as text' if they want to proceed
+      break;
+  }
+
+  // update
+  edit({
+    outputsConverting: false,
+    outputs,
+  });
+}
@@ -0,0 +1,42 @@
+/*
+
+/// REDUCER
+
+import { ContentReducer } from '~/modules/aifn/summarize/ContentReducer';
+
+  const [reducerText, setReducerText] = React.useState('');
+  const [reducerTextTokens, setReducerTextTokens] = React.useState(0);
+
+{reducerText?.length >= 1 &&
+<ContentReducer
+  initialText={reducerText} initialTokens={reducerTextTokens} tokenLimit={remainingTokens}
+  onReducedText={handleReducedText} onClose={handleReducerClose}
+  />
+}
+  const handleReducerClose = () => setReducerText('');
+
+  const handleReducedText = (text: string) => {
+    handleReducerClose();
+    setComposeText(_t => _t + text);
+  };
+
+  const handleAttachFiles = async (files: FileList, overrideFileNames?: string[]): Promise<void> => {
+
+    // see how we fare on budget
+    if (chatLLMId) {
+      const newTextTokens = countModelTokens(newText, chatLLMId, 'reducer trigger');
+
+      // simple trigger for the reduction dialog
+      if (newTextTokens > remainingTokens) {
+        setReducerTextTokens(newTextTokens);
+        setReducerText(newText);
+        return;
+      }
+    }
+
+    // within the budget, so just append
+    setComposeText(text => expandPromptTemplate(PromptTemplates.Concatenate, { text: newText })(text));
+
+
+
+*/
@@ -0,0 +1,201 @@
+import { create } from 'zustand';
+import type { FileWithHandle } from 'browser-fs-access';
+
+import type { ComposerOutputMultiPart } from '../composer.types';
+import { attachmentPerformConversion, attachmentCreate, attachmentDefineConverters, attachmentLoadInputAsync } from './pipeline';
+
+
+// Attachment Types
+
+export type AttachmentSourceOriginDTO = 'drop' | 'paste';
+export type AttachmentSourceOriginFile = 'camera' | 'file-open' | 'clipboard-read' | AttachmentSourceOriginDTO;
+
+export type AttachmentSource = {
+  media: 'url';
+  url: string;
+  refUrl: string;
+} | {
+  media: 'file';
+  origin: AttachmentSourceOriginFile,
+  fileWithHandle: FileWithHandle;
+  refPath: string;
+} | {
+  media: 'text';
+  method: 'clipboard-read' | AttachmentSourceOriginDTO;
+  textPlain?: string;
+  textHtml?: string;
+};
+
+
+export type AttachmentInput = {
+  mimeType: string; // Original MIME type of the file
+  data: string | ArrayBuffer; // The original data of the attachment
+  dataSize: number; // Size of the original data in bytes
+  altMimeType?: string; // Alternative MIME type for the input
+  altData?: string; // Alternative data for the input
+  // preview?: AttachmentPreview; // Preview of the input
+};
+
+
+export type AttachmentConverterType =
+  | 'text' | 'rich-text' | 'rich-text-table'
+  | 'pdf-text' | 'pdf-images'
+  | 'image' | 'image-ocr'
+  | 'unhandled';
+
+export type AttachmentConverter = {
+  id: AttachmentConverterType;
+  name: string;
+  disabled?: boolean;
+  unsupported?: boolean;
+  // outputType: ComposerOutputPartType; // The type of the output after conversion
+  // isAutonomous: boolean; // Whether the conversion does not require user input
+  // isAsync: boolean; // Whether the conversion is asynchronous
+  // progress: number; // Conversion progress percentage (0..1)
+  // errorMessage?: string; // Error message if the conversion failed
+}
+
+
+export type AttachmentId = string;
+
+export type Attachment = {
+  readonly id: AttachmentId;
+  readonly source: AttachmentSource,
+  label: string;
+  ref: string;
+
+  inputLoading: boolean;
+  inputError: string | null;
+  input?: AttachmentInput;
+
+  // options to convert the input
+  converters: AttachmentConverter[]; // List of available converters for this attachment
+  converterIdx: number | null; // Index of the selected converter
+
+  outputsConverting: boolean;
+  outputs: ComposerOutputMultiPart; // undefined: not yet converted, []: conversion failed, [ {}+ ]: conversion succeeded
+
+  // metadata: {
+  //   size?: number; // Size of the attachment in bytes
+  //   creationDate?: Date; // Creation date of the file
+  //   modifiedDate?: Date; // Last modified date of the file
+  //   altText?: string; // Alternative text for images for screen readers
+  // };
+};
+
+
+/*export type AttachmentPreview = {
+  renderer: 'noPreview',
+  title: string; // A title for the preview
+} | {
+  renderer: 'textPreview'
+  fileName: string; // The name of the file
+  snippet: string; // A text snippet for documents
+  tooltip?: string; // A tooltip for the preview
+} | {
+  renderer: 'imagePreview'
+  thumbnail: string; // A thumbnail preview for images, videos, etc.
+  tooltip?: string; // A tooltip for the preview
+};*/
+
+
+/// Store
+
+interface AttachmentsStore {
+
+  attachments: Attachment[];
+
+  createAttachment: (source: AttachmentSource) => Promise<void>;
+  clearAttachments: () => void;
+  removeAttachment: (attachmentId: AttachmentId) => void;
+  moveAttachment: (attachmentId: AttachmentId, delta: 1 | -1) => void;
+  setConverterIdx: (attachmentId: AttachmentId, converterIdx: number | null) => Promise<void>;
+
+  _editAttachment: (attachmentId: AttachmentId, update: Partial<Attachment> | ((attachment: Attachment) => Partial<Attachment>)) => void;
+  _getAttachment: (attachmentId: AttachmentId) => Attachment | undefined;
+
+}
+
+export const useAttachmentsStore = create<AttachmentsStore>()(
+  (_set, _get) => ({
+
+    attachments: [],
+
+    createAttachment: async (source: AttachmentSource) => {
+      const { attachments, _getAttachment, _editAttachment, setConverterIdx } = _get();
+
+      const attachment = attachmentCreate(source, attachments.map(a => a.id));
+
+      _set({
+        attachments: [...attachments, attachment],
+      });
+
+      const editFn = (changes: Partial<Attachment>) => _editAttachment(attachment.id, changes);
+
+      // 1.Resolve the Input
+      await attachmentLoadInputAsync(source, editFn);
+      const loaded = _getAttachment(attachment.id);
+      if (!loaded || !loaded.input)
+        return;
+
+      // 2. Define the I->O Converters
+      attachmentDefineConverters(source.media, loaded.input, editFn);
+      const defined = _getAttachment(attachment.id);
+      if (!defined || !defined.converters.length || defined.converterIdx !== null)
+        return;
+
+      // 3. Select the first Converter
+      const firstEnabledIndex = defined.converters.findIndex(_c => !_c.disabled);
+      await setConverterIdx(attachment.id, firstEnabledIndex > -1 ? firstEnabledIndex : 0);
+    },
+
+    clearAttachments: () => _set({
+      attachments: [],
+    }),
+
+    removeAttachment: (attachmentId: AttachmentId) =>
+      _set(state => ({
+        attachments: state.attachments.filter(attachment => attachment.id !== attachmentId),
+      })),
+
+    moveAttachment: (attachmentId: AttachmentId, delta: 1 | -1) =>
+      _set(state => {
+        const attachments = [...state.attachments];
+        const currentIdx = attachments.findIndex(a => a.id === attachmentId);
+
+        // If the attachment is not found, or if trying to move beyond the array boundaries, no move is needed
+        if (currentIdx === -1 || (currentIdx === 0 && delta === -1) || (currentIdx === attachments.length - 1 && delta === 1))
+          return state;
+
+        // Swap the attachment with the adjacent one in the direction of delta
+        const targetIdx = currentIdx + delta;
+        [attachments[currentIdx], attachments[targetIdx]] = [attachments[targetIdx], attachments[currentIdx]];
+
+        return { attachments };
+      }),
+
+    setConverterIdx: async (attachmentId: AttachmentId, converterIdx: number | null) => {
+      const { _getAttachment, _editAttachment } = _get();
+      const attachment = _getAttachment(attachmentId);
+      if (!attachment || attachment.converterIdx === converterIdx)
+        return;
+
+      const editFn = (changes: Partial<Attachment>) => _editAttachment(attachmentId, changes);
+
+      await attachmentPerformConversion(attachment, converterIdx, editFn);
+    },
+
+    _editAttachment: (attachmentId: AttachmentId, update: Partial<Attachment> | ((attachment: Attachment) => Partial<Attachment>)) =>
+      _set(state => ({
+        attachments: state.attachments.map((attachment: Attachment): Attachment =>
+          attachment.id === attachmentId
+            ? { ...attachment, ...(typeof update === 'function' ? update(attachment) : update) }
+            : attachment,
+        ),
+      })),
+
+    _getAttachment: (attachmentId: AttachmentId) =>
+      _get().attachments.find(a => a.id === attachmentId),
+
+  }),
+);
@@ -0,0 +1,165 @@
+import * as React from 'react';
+import { shallow } from 'zustand/shallow';
+import type { FileWithHandle } from 'browser-fs-access';
+
+import { addSnackbar } from '~/common/components/useSnackbarsStore';
+import { asValidURL } from '~/common/util/urlUtils';
+import { extractFilePathsWithCommonRadix } from '~/common/util/dropTextUtils';
+import { getClipboardItems } from '~/common/util/clipboardUtils';
+
+import { AttachmentSourceOriginDTO, AttachmentSourceOriginFile, useAttachmentsStore } from './store-attachments';
+
+
+export const useAttachments = (enableLoadURLs: boolean) => {
+
+  // state
+
+  const { attachments, clearAttachments, createAttachment, removeAttachment } = useAttachmentsStore(state => ({
+    attachments: state.attachments,
+    clearAttachments: state.clearAttachments,
+    createAttachment: state.createAttachment,
+    removeAttachment: state.removeAttachment,
+  }), shallow);
+
+
+  // Creation helpers
+
+  const attachAppendFile = React.useCallback((origin: AttachmentSourceOriginFile, fileWithHandle: FileWithHandle, overrideFileName?: string) =>
+      createAttachment({
+        media: 'file', origin, fileWithHandle, refPath: overrideFileName || fileWithHandle.name,
+      })
+    , [createAttachment]);
+
+
+  const attachAppendDataTransfer = React.useCallback((dt: DataTransfer, method: AttachmentSourceOriginDTO, attachText: boolean): 'as_files' | 'as_url' | 'as_text' | false => {
+
+    // attach File(s)
+    if (dt.files.length >= 1) {
+      // rename files from a common prefix, to better relate them (if the transfer contains a list of paths)
+      let overrideFileNames: string[] = [];
+      if (dt.types.includes('text/plain')) {
+        const plainText = dt.getData('text/plain');
+        overrideFileNames = extractFilePathsWithCommonRadix(plainText);
+      }
+      const overrideNames = overrideFileNames.length === dt.files.length;
+
+      // attach as Files (paste and drop keep the original filename)
+      for (let i = 0; i < dt.files.length; i++) {
+        const file = dt.files[i];
+        // drag/drop of folders (or .tsx from IntelliJ) will have no type
+        if (!file.type) {
+          // NOTE: we are fixing it in attachmentLoadInputAsync, but would be better to do it here
+        }
+        void attachAppendFile(method, file, overrideNames ? overrideFileNames[i] || undefined : undefined);
+      }
+      return 'as_files';
+    }
+
+    // attach as URL
+    const textPlain = dt.getData('text/plain') || '';
+    if (textPlain && enableLoadURLs) {
+      const textPlainUrl = asValidURL(textPlain);
+      if (textPlainUrl && textPlainUrl) {
+        void createAttachment({
+          media: 'url', url: textPlainUrl, refUrl: textPlain,
+        });
+        return 'as_url';
+      }
+    }
+
+    // attach as Text/Html (further conversion, e.g. to markdown is done later)
+    const textHtml = dt.getData('text/html') || '';
+    if (attachText && (textHtml || textPlain)) {
+      void createAttachment({
+        media: 'text', method, textPlain, textHtml,
+      });
+      return 'as_text';
+    }
+
+    if (attachText)
+      console.warn(`Unhandled '${method}' attachment: `, dt.types?.map(t => `${t}: ${dt.getData(t)}`));
+
+    // did not attach anything from this data transfer
+    return false;
+  }, [attachAppendFile, createAttachment, enableLoadURLs]);
+
+
+  const attachAppendClipboardItems = React.useCallback(async () => {
+
+    // if there's an issue accessing the clipboard, show it passively
+    const clipboardItems = await getClipboardItems();
+    if (clipboardItems === null) {
+      addSnackbar({
+        key: 'clipboard-issue',
+        type: 'issue',
+        message: 'Clipboard empty or access denied',
+        overrides: {
+          autoHideDuration: 2000,
+        },
+      });
+      return;
+    }
+
+    // loop on all the possible attachments
+    for (const clipboardItem of clipboardItems) {
+
+      // attach as image
+      let imageAttached = false;
+      for (const mimeType of clipboardItem.types) {
+        if (mimeType.startsWith('image/')) {
+          try {
+            const imageBlob = await clipboardItem.getType(mimeType);
+            const imageName = mimeType.replace('image/', 'clipboard.').replaceAll('/', '.') || 'clipboard.png';
+            const imageFile = new File([imageBlob], imageName, { type: mimeType });
+            void attachAppendFile('clipboard-read', imageFile);
+            imageAttached = true;
+          } catch (error) {
+            // ignore getType error..
+          }
+        }
+      }
+      if (imageAttached)
+        continue;
+
+      // get the Plain text
+      const textPlain = clipboardItem.types.includes('text/plain') ? await clipboardItem.getType('text/plain').then(blob => blob.text()) : '';
+
+      // attach as URL
+      if (textPlain && enableLoadURLs) {
+        const textPlainUrl = asValidURL(textPlain);
+        if (textPlainUrl && textPlainUrl.trim()) {
+          void createAttachment({
+            media: 'url', url: textPlainUrl.trim(), refUrl: textPlain,
+          });
+          continue;
+        }
+      }
+
+      // attach as Text
+      const textHtml = clipboardItem.types.includes('text/html') ? await clipboardItem.getType('text/html').then(blob => blob.text()) : '';
+      if (textHtml || textPlain) {
+        void createAttachment({
+          media: 'text', method: 'clipboard-read', textPlain, textHtml,
+        });
+        continue;
+      }
+
+      console.warn('Clipboard item has no text/html or text/plain item.', clipboardItem.types, clipboardItem);
+    }
+  }, [attachAppendFile, createAttachment, enableLoadURLs]);
+
+
+  return {
+    // state
+    attachments,
+
+    // create attachments
+    attachAppendClipboardItems,
+    attachAppendDataTransfer,
+    attachAppendFile,
+
+    // manage attachments
+    clearAttachments,
+    removeAttachment,
+  };
+};
@@ -0,0 +1,147 @@
+import * as React from 'react';
+
+import type { DLLMId } from '~/modules/llms/store-llms';
+
+import { countModelTokens } from '~/common/util/token-counter';
+
+import type { Attachment, AttachmentId } from './store-attachments';
+import type { ComposerOutputMultiPart, ComposerOutputPartType } from '../composer.types';
+
+
+export interface LLMAttachments {
+  attachments: LLMAttachment[];
+  getAttachmentOutputs: (initialTextBlockText: string | null, attachmentId: AttachmentId) => ComposerOutputMultiPart;
+  getAttachmentsOutputs: (initialTextBlockText: string | null) => ComposerOutputMultiPart;
+  isOutputAttacheable: boolean;
+  isOutputTextInlineable: boolean;
+  tokenCountApprox: number;
+}
+
+export interface LLMAttachment {
+  attachment: Attachment;
+  attachmentOutputs: ComposerOutputMultiPart;
+  isUnconvertible: boolean;
+  isOutputMissing: boolean;
+  isOutputAttachable: boolean;
+  isOutputTextInlineable: boolean;
+  tokenCountApprox: number | null;
+}
+
+
+export function useLLMAttachments(attachments: Attachment[], chatLLMId: DLLMId | null): LLMAttachments {
+  return React.useMemo(() => {
+
+    // HACK: in the future, switch to LLM capabilities (LLM_IF_OAI_Chat, LLM_IF_OAI_Vision, etc.)
+    const supportsImages = !!chatLLMId?.endsWith('-vision-preview');
+    const supportedOutputPartTypes: ComposerOutputPartType[] = supportsImages ? ['text-block', 'image-part'] : ['text-block'];
+
+    const llmAttachments = attachments.map(attachment => toLLMAttachment(attachment, supportedOutputPartTypes, chatLLMId));
+
+    const getAttachmentOutputs = (initialTextBlockText: string | null, attachmentId: AttachmentId): ComposerOutputMultiPart => {
+      // get outputs of a specific attachment
+      const outputs = attachments.find(a => a.id === attachmentId)?.outputs || [];
+      return attachmentCollapseOutputs(initialTextBlockText, outputs);
+    };
+
+    const getAttachmentsOutputs = (initialTextBlockText: string | null): ComposerOutputMultiPart => {
+      // accumulate all outputs of all attachments
+      const allOutputs = llmAttachments.reduce((acc, a) => acc.concat(a.attachment.outputs), [] as ComposerOutputMultiPart);
+      return attachmentCollapseOutputs(initialTextBlockText, allOutputs);
+    };
+
+    return {
+      attachments: llmAttachments,
+      getAttachmentOutputs,
+      getAttachmentsOutputs,
+      isOutputAttacheable: llmAttachments.every(a => a.isOutputAttachable),
+      isOutputTextInlineable: llmAttachments.every(a => a.isOutputTextInlineable),
+      tokenCountApprox: llmAttachments.reduce((acc, a) => acc + (a.tokenCountApprox || 0), 0),
+    };
+  }, [attachments, chatLLMId]);
+}
+
+export function getTextBlockText(outputs: ComposerOutputMultiPart): string | null {
+  const textOutputs = outputs.filter(part => part.type === 'text-block');
+  return (textOutputs.length === 1 && textOutputs[0].type === 'text-block') ? textOutputs[0].text : null;
+}
+
+
+function toLLMAttachment(attachment: Attachment, supportedOutputPartTypes: ComposerOutputPartType[], llmForTokenCount: DLLMId | null): LLMAttachment {
+  const { converters, outputs } = attachment;
+
+  const isUnconvertible = converters.length === 0;
+  const isOutputMissing = outputs.length === 0;
+  const isOutputAttachable = areAllOutputsSupported(outputs, supportedOutputPartTypes);
+  const isOutputTextInlineable = areAllOutputsSupported(outputs, supportedOutputPartTypes.filter(pt => pt === 'text-block'));
+
+  const attachmentOutputs = attachmentCollapseOutputs(null, outputs);
+  const tokenCountApprox = llmForTokenCount
+    ? attachmentOutputs.reduce((acc, output) => {
+      if (output.type === 'text-block')
+        return acc + countModelTokens(output.text, llmForTokenCount, 'attachments tokens count');
+      console.warn('Unhandled token preview for output type:', output.type);
+      return acc;
+    }, 0)
+    : null;
+
+  return {
+    attachment,
+    attachmentOutputs,
+    isUnconvertible,
+    isOutputMissing,
+    isOutputAttachable,
+    isOutputTextInlineable,
+    tokenCountApprox,
+  };
+}
+
+function areAllOutputsSupported(outputs: ComposerOutputMultiPart, supportedOutputPartTypes: ComposerOutputPartType[]) {
+  return outputs.length
+    ? outputs.every(output => supportedOutputPartTypes.includes(output.type))
+    : false;
+}
+
+function attachmentCollapseOutputs(initialTextBlockText: string | null, outputs: ComposerOutputMultiPart): ComposerOutputMultiPart {
+  const accumulatedOutputs: ComposerOutputMultiPart = [];
+
+  // if there's initial text, make it a collapsible default (unquited) text block
+  if (initialTextBlockText !== null) {
+    accumulatedOutputs.push({
+      type: 'text-block',
+      text: initialTextBlockText,
+      title: null,
+      collapsible: true,
+    });
+  }
+
+  // Accumulate attachment outputs of the same type and 'collapsible' into a single object of that type.
+  for (const output of outputs) {
+    const last = accumulatedOutputs[accumulatedOutputs.length - 1];
+
+    // accumulationg over an existing part of the same type
+    if (last && last.type === output.type && output.collapsible) {
+      switch (last.type) {
+        case 'text-block':
+          last.text += `\n\n\`\`\`${output.title}\n${output.text}\n\`\`\``;
+          break;
+        default:
+          console.warn('Unhandled collapsing for output type:', output.type);
+      }
+    }
+    // start a new part
+    else {
+      if (output.type === 'text-block') {
+        accumulatedOutputs.push({
+          type: 'text-block',
+          text: `\n\n\`\`\`${output.title}\n${output.text}\n\`\`\``,
+          title: null,
+          collapsible: false,
+        });
+      } else {
+        accumulatedOutputs.push(output);
+      }
+    }
+  }
+
+  return accumulatedOutputs;
+}
@@ -0,0 +1,15 @@
+export type ComposerOutputPartType = 'text-block' | 'image-part';
+
+export type ComposerOutputPart = {
+  type: 'text-block',
+  text: string,
+  title: string | null,
+  collapsible: boolean,
+} | {
+  // TODO: not implemented yet
+  type: 'image-part',
+  base64Url: string,
+  collapsible: false,
+};
+
+export type ComposerOutputMultiPart = ComposerOutputPart[];
@@ -20,6 +20,7 @@ import ReplayIcon from '@mui/icons-material/Replay';
 import SettingsSuggestIcon from '@mui/icons-material/SettingsSuggest';
 import SmartToyOutlinedIcon from '@mui/icons-material/SmartToyOutlined';
 import TelegramIcon from '@mui/icons-material/Telegram';
+import VerticalAlignBottomIcon from '@mui/icons-material/VerticalAlignBottom';

 import { CloseableMenu } from '~/common/components/CloseableMenu';
 import { DMessage } from '~/common/state/store-chats';
@@ -166,6 +167,8 @@ function explainErrorInMessage(text: string, isAssistant: boolean, modelId?: str
      make sure the usage is under <Link noLinkStyle href='https://platform.openai.com/account/billing/limits' target='_blank'>the limits</Link>.
    </>;
  }
+  // else
+  //  errorMessage = <>{text || 'Unknown error'}</>;

  return { errorMessage, isAssistantError };
 }
@@ -205,6 +208,7 @@ export function ChatMessage(props: {
  isImagining?: boolean, isSpeaking?: boolean,
  onConversationBranch?: (messageId: string) => void,
  onConversationRestartFrom?: (messageId: string, offset: number) => void,
+  onConversationTruncate?: (messageId: string) => void,
  onMessageDelete?: (messageId: string) => void,
  onMessageEdit?: (messageId: string, text: string) => void,
  onTextDiagram?: (messageId: string, text: string) => Promise<void>
@@ -326,7 +330,12 @@ export function ChatMessage(props: {
    }
  };

-  const handleOpsDelete = (e: React.MouseEvent) => {
+  const handleOpsTruncate = (_e: React.MouseEvent) => {
+    props.onConversationTruncate && props.onConversationTruncate(messageId);
+    closeOperationsMenu();
+  };
+
+  const handleOpsDelete = (_e: React.MouseEvent) => {
    props.onMessageDelete && props.onMessageDelete(messageId);
  };

@@ -518,7 +527,7 @@ export function ChatMessage(props: {
                        ? <RenderLatex key={'latex-' + index} latexBlock={block} />
                        : block.type === 'diff'
                          ? <RenderTextDiff key={'latex-' + index} diffBlock={block} />
-                          : (renderMarkdown && props.noMarkdown !== true && !fromSystem)
+                          : (renderMarkdown && props.noMarkdown !== true && !fromSystem && !(fromUser && block.content.startsWith('/')))
                            ? <RenderMarkdown key={'text-md-' + index} textBlock={block} />
                            : <RenderText key={'text-' + index} textBlock={block} />)}

@@ -554,7 +563,7 @@ export function ChatMessage(props: {
      {/* Operations Menu (3 dots) */}
      {!!opsMenuAnchor && (
        <CloseableMenu
-          placement='bottom-end' sx={{ minWidth: 280 }}
+          dense placement='bottom-end' sx={{ minWidth: 280 }}
          open anchorEl={opsMenuAnchor} onClose={closeOperationsMenu}
        >
          <Box sx={{ display: 'flex', alignItems: 'center' }}>
@@ -615,10 +624,16 @@ export function ChatMessage(props: {
            Speak
          </MenuItem>}
          {!!props.onConversationRestartFrom && <ListDivider />}
+          {!!props.onConversationTruncate && (
+            <MenuItem onClick={handleOpsTruncate} disabled={props.isBottom}>
+              <ListItemDecorator><VerticalAlignBottomIcon /></ListItemDecorator>
+              Truncate <span style={{ opacity: 0.5 }}>after</span>
+            </MenuItem>
+          )}
          {!!props.onMessageDelete && (
            <MenuItem onClick={handleOpsDelete} disabled={false /*fromSystem*/}>
              <ListItemDecorator><ClearIcon /></ListItemDecorator>
-              Delete
+              Delete <span style={{ opacity: 0.5 }}>message</span>
            </MenuItem>
          )}
        </CloseableMenu>
@@ -627,12 +642,12 @@ export function ChatMessage(props: {
      {/* Selection (Contextual) Menu */}
      {!!selMenuAnchor && (
        <CloseableMenu
-          placement='bottom-start' sx={{ minWidth: 220 }}
+          dense placement='bottom-start' sx={{ minWidth: 220 }}
          open anchorEl={selMenuAnchor} onClose={closeSelectionMenu}
        >
          <MenuItem onClick={handleOpsCopy} sx={{ flex: 1 }}>
            <ListItemDecorator><ContentCopyIcon /></ListItemDecorator>
-            Copy selection
+            Copy <span style={{ opacity: 0.5 }}>selection</span>
          </MenuItem>
          {!!props.onTextDiagram && <MenuItem onClick={handleOpsDiagram} disabled={!couldDiagram || props.isImagining}>
            <ListItemDecorator><AccountTreeIcon color='success' /></ListItemDecorator>
@@ -6,7 +6,7 @@ import DeleteOutlineIcon from '@mui/icons-material/DeleteOutline';

 import { DMessage } from '~/common/state/store-chats';

-import { TokenBadge } from '../composer/TokenBadge';
+import { TokenBadgeMemo } from '../composer/TokenBadge';
 import { makeAvatar, messageBackground } from './ChatMessage';


@@ -94,7 +94,7 @@ export function CleanerMessage(props: { message: DMessage, isBottom: boolean, se
      </Typography>

      {props.remainingTokens !== undefined && <Box sx={{ display: 'flex', minWidth: { xs: 32, sm: 45 }, justifyContent: 'flex-end' }}>
-        <TokenBadge directTokens={messageTokenCount} tokenLimit={props.remainingTokens} inline />
+        <TokenBadgeMemo direct={messageTokenCount} limit={props.remainingTokens} inline />
      </Box>}

      <Typography level='body-md' sx={{
@@ -18,13 +18,13 @@ export const runBrowseUpdatingState = async (conversationId: string, url: string

  try {

-    const text = await callBrowseFetchPage(url);
-    if (!text) {
+    const page = await callBrowseFetchPage(url);
+    if (!page.content) {
      // noinspection ExceptionCaughtLocallyJS
      throw new Error('No text found.');
    }
    updateAssistantMessage({
-      text: text,
+      text: page.content,
      typing: false,
    });

@@ -2,8 +2,8 @@ import { DLLMId } from '~/modules/llms/store-llms';
 import { SystemPurposeId } from '../../../data';
 import { autoSuggestions } from '~/modules/aifn/autosuggestions/autoSuggestions';
 import { autoTitle } from '~/modules/aifn/autotitle/autoTitle';
+import { llmStreamingChatGenerate } from '~/modules/llms/llm.client';
 import { speakText } from '~/modules/elevenlabs/elevenlabs.client';
-import { streamChat } from '~/modules/llms/transports/streamChat';

 import { DMessage, useChatStore } from '~/common/state/store-chats';

@@ -63,7 +63,7 @@ async function streamAssistantMessage(
  const messages = history.map(({ role, text }) => ({ role, content: text }));

  try {
-    await streamChat(llmId, messages, abortSignal,
+    await llmStreamingChatGenerate(llmId, messages, null, null, abortSignal,
      (updatedMessage: Partial<DMessage>) => {
        // update the message in the store (and thus schedule a re-render)
        editMessage(updatedMessage);
@@ -22,6 +22,9 @@ interface AppChatStore {
  autoTitleChat: boolean;
  setAutoTitleChat: (autoTitleChat: boolean) => void;

+  micTimeoutMs: number;
+  setMicTimeoutMs: (micTimeoutMs: number) => void;
+
  showTextDiff: boolean;
  setShowTextDiff: (showTextDiff: boolean) => void;

@@ -46,6 +49,9 @@ const useAppChatStore = create<AppChatStore>()(persist(
    autoTitleChat: true,
    setAutoTitleChat: (autoTitleChat: boolean) => _set({ autoTitleChat }),

+    micTimeoutMs: 2000,
+    setMicTimeoutMs: (micTimeoutMs: number) => _set({ micTimeoutMs }),
+
    showTextDiff: false,
    setShowTextDiff: (showTextDiff: boolean) => _set({ showTextDiff }),

@@ -91,6 +97,12 @@ export const getChatAutoAI = (): {
  autoTitleChat: boolean,
 } => useAppChatStore.getState();

+export const useChatMicTimeoutMsValue = (): number =>
+  useAppChatStore(state => state.micTimeoutMs);
+
+export const useChatMicTimeoutMs = (): [number, (micTimeoutMs: number) => void] =>
+  useAppChatStore(state => [state.micTimeoutMs, state.setMicTimeoutMs], shallow);
+
 export const useChatShowTextDiff = (): [boolean, (showDiff: boolean) => void] =>
  useAppChatStore(state => [state.showTextDiff, state.setShowTextDiff], shallow);

@@ -1,19 +1,39 @@
 import * as React from 'react';
+import { keyframes } from '@emotion/react';
+import TimeAgo from 'react-timeago';

 import { Box, Button, Card, CardContent, Container, IconButton, Typography } from '@mui/joy';
 import ExpandMoreIcon from '@mui/icons-material/ExpandMore';

 import { Brand } from '~/common/app.config';
+import { GoodTooltip } from '~/common/components/GoodTooltip';
 import { Link } from '~/common/components/Link';
 import { ROUTE_INDEX } from '~/common/app.routes';
 import { capitalizeFirstLetter } from '~/common/util/textUtils';

 import { newsCallout, NewsItems } from './news.data';

+// number of news items to show by default, before the expander
+const DEFAULT_NEWS_COUNT = 2;
+
+export const cssColorKeyframes = keyframes`
+    0%, 100% {
+        color: #636B74; /* Neutral main color (500) */
+    }
+    25% {
+        color: #12467B; /* Primary darker shade (700) */
+    }
+    50% {
+        color: #0B6BCB; /* Primary main color (500) */
+    }
+    75% {
+        color: #97C3F0; /* Primary lighter shade (300) */
+    }`;
+

 export function AppNews() {
  // state
-  const [lastNewsIdx, setLastNewsIdx] = React.useState<number>(0);
+  const [lastNewsIdx, setLastNewsIdx] = React.useState<number>(DEFAULT_NEWS_COUNT - 1);

  // news selection
  const news = NewsItems.filter((_, idx) => idx <= lastNewsIdx);
@@ -36,11 +56,11 @@ export function AppNews() {
      }}>

        <Typography level='h1' sx={{ fontSize: '3rem' }}>
-          Welcome to {Brand.Title.Base} {firstNews?.versionName}!
+          Welcome to {Brand.Title.Base} <Box component='span' sx={{ animation: `${cssColorKeyframes} 10s infinite` }}>{firstNews?.versionCode}</Box>!
        </Typography>

        <Typography>
-          {capitalizeFirstLetter(Brand.Title.Base)} has been updated to version {firstNews?.versionName}.
+          {capitalizeFirstLetter(Brand.Title.Base)} has been updated to version {firstNews?.versionCode}
        </Typography>

        <Box>
@@ -58,30 +78,33 @@ export function AppNews() {

        {!!news && <Container disableGutters maxWidth='sm'>
          {news?.map((ni, idx) => {
-            const firstCard = idx === 0;
+            // const firstCard = idx === 0;
            const hasCardAfter = news.length < NewsItems.length;
            const showExpander = hasCardAfter && (idx === news.length - 1);
-            const addPadding = !firstCard; // || showExpander;
+            const addPadding = false; //!firstCard; // || showExpander;
            return <Card key={'news-' + idx} sx={{ mb: 2, minHeight: 32 }}>
              <CardContent sx={{ position: 'relative', pr: addPadding ? 4 : 0 }}>
-                {!!ni.text && <Typography level='title-lg' component='div'>
-                  {ni.text}
-                </Typography>}
+                <Box sx={{ display: 'flex', alignItems: 'center', justifyContent: 'space-between', gap: 0 }}>
+                  <GoodTooltip title={ni.versionName ? `${ni.versionName} ${ni.versionMoji || ''}` : null} placement='top-start'>
+                    <Typography level='title-sm' component='div' sx={{ flexGrow: 1 }}>
+                      {ni.text ? ni.text : ni.versionName ? `${ni.versionCode} · ${ni.versionName}` : `Version ${ni.versionCode}:`}
+                    </Typography>
+                  </GoodTooltip>
+                  {/*!firstCard &&*/ (
+                    <Typography level='body-sm'>
+                      {!!ni.versionDate && <TimeAgo date={ni.versionDate} />}
+                    </Typography>
+                  )}
+                </Box>

                {!!ni.items && (ni.items.length > 0) && <ul style={{ marginTop: 8, marginBottom: 8, paddingInlineStart: 24 }}>
                  {ni.items.filter(item => item.dev !== true).map((item, idx) => <li key={idx}>
-                    <Typography component='div'>
+                    <Typography component='div' level='body-sm'>
                      {item.text}
                    </Typography>
                  </li>)}
                </ul>}

-                {/*!firstCard &&*/ (
-                  <Typography level='body-sm' sx={{ position: 'absolute', right: 0, top: 0 }}>
-                    {ni.versionName}
-                  </Typography>
-                )}
-
                {showExpander && (
                  <IconButton
                    variant='plain' size='sm'
@@ -6,13 +6,14 @@ import LaunchIcon from '@mui/icons-material/Launch';
 import { Brand } from '~/common/app.config';
 import { Link } from '~/common/components/Link';
 import { clientUtmSource } from '~/common/util/pwaUtils';
+import { platformAwareKeystrokes } from '~/common/components/KeyStroke';


 // update this variable every time you want to broadcast a new version to clients
-export const incrementalVersion: number = 7;
+export const incrementalVersion: number = 9;

 const B = (props: { href?: string, children: React.ReactNode }) => {
-  const boldText = <Typography color={!!props.href ? 'primary' : 'warning'} sx={{ fontWeight: 600 }}>{props.children}</Typography>;
+  const boldText = <Typography color={!!props.href ? 'primary' : 'neutral'} sx={{ fontWeight: 600 }}>{props.children}</Typography>;
  return props.href ?
    <Link href={props.href + clientUtmSource()} target='_blank' sx={{ /*textDecoration: 'underline'*/ }}>{boldText} <LaunchIcon sx={{ ml: 1 }} /></Link> :
    boldText;
@@ -26,11 +27,12 @@ const RIssues = `${OpenRepo}/issues`;
 export const newsCallout =
  <Card>
    <CardContent sx={{ gap: 2 }}>
-      <Typography level='h3'>
+      <Typography level='title-lg'>
        Open Roadmap
      </Typography>
-      <Typography>
-        The roadmap is officially out. For the first time you get a look at what&apos;s brewing, up and coming, and get a chance to pick up cool features!
+      <Typography level='body-md'>
+        Take a peek at our roadmap to see what&apos;s in the pipeline.
+        Discover upcoming features and let us know what excites you the most!
      </Typography>
      <Grid container spacing={1}>
        <Grid xs={12} sm={7}>
@@ -38,7 +40,7 @@ export const newsCallout =
            fullWidth variant='soft' color='primary' endDecorator={<LaunchIcon />}
            component={Link} href={OpenProject} noLinkStyle target='_blank'
          >
-            Explore the Roadmap
+            Explore
          </Button>
        </Grid>
        <Grid xs={12} sm={5} sx={{ display: 'flex', flexAlign: 'center', justifyContent: 'center' }}>
@@ -66,12 +68,46 @@ export const NewsItems: NewsItem[] = [
    ],
  },*/
  {
-    versionName: '1.6.0',
-    text: 'Surf\'s Up in Chat Waves:',
+    versionCode: '1.8.0',
+    versionName: 'To The Moon And Back',
+    versionMoji: '🚀🌕🔙❤️',
+    versionDate: new Date('2023-12-20T09:30:00Z'),
+    items: [
+      { text: <><B href={RIssues + '/275'}>Google Gemini</B> models support</> },
+      { text: <><B href={RIssues + '/273'}>Mistral Platform</B> support</> },
+      { text: <><B href={RIssues + '/270'}>Ollama chats</B> perfection</> },
+      { text: <>Custom <B href={RIssues + '/280'}>diagrams instructions</B> (@joriskalz)</> },
+      { text: <><B>Single-Tab</B> mode, enhances data integrity and prevents DB corruption</> },
+      { text: <>Updated Ollama (v0.1.17) and OpenRouter models</> },
+      { text: <>More: fixed ⌘ shortcuts on Mac</> },
+      { text: <><Link href='https://big-agi.com'>Website</Link>: official downloads</> },
+      { text: <>Easier Vercel deployment, documented <Link href='https://github.com/enricoros/big-AGI/issues/276#issuecomment-1858591483'>network troubleshooting</Link></>, dev: true },
+    ],
+  },
+  {
+    versionCode: '1.7.0',
+    versionName: 'Attachment Theory',
+    // versionDate: new Date('2023-12-11T06:00:00Z'), // 1.7.3
+    versionDate: new Date('2023-12-10T12:00:00Z'), // 1.7.0
+    items: [
+      { text: <>Redesigned <B href={RIssues + '/251'}>attachments system</B>: drag, paste, link, snap, images, text, pdfs</> },
+      { text: <>Desktop <B href={RIssues + '/253'}>webcam access</B> for direct image capture (Labs option)</> },
+      { text: <>Independent browsing with <B href={RCode + '/docs/config-browse.md'}>Browserless</B> support</> },
+      { text: <><B href={RIssues + '/256'}>Overheat</B> LLMs with higher temperature limits</> },
+      { text: <>Enhanced security via <B href={RCode + '/docs/deploy-authentication.md'}>password protection</B></> },
+      { text: <>{platformAwareKeystrokes('Ctrl+Shift+O')}: quick access to model options</> },
+      { text: <>Optimized voice input and performance</> },
+      { text: <>Latest Ollama and Oobabooga models</> },
+    ],
+  },
+  {
+    versionCode: '1.6.0',
+    versionName: 'Surf\'s Up',
+    versionDate: new Date('2023-11-28T21:00:00Z'),
    items: [
      { text: <><B href={RIssues + '/237'}>Web Browsing</B> support, see the <B href={RCode + '/docs/config-browse.md'}>browsing user guide</B></> },
      { text: <><B href={RIssues + '/235'}>Branching Discussions</B> at any message</> },
-      { text: <><B href={RIssues + '/207'}>Keyboard Navigation</B>: use Ctrl+Shift+Left/Right to navigate chats</> },
+      { text: <><B href={RIssues + '/207'}>Keyboard Navigation</B>: use {platformAwareKeystrokes('Ctrl+Shift+Left/Right')} to navigate chats</> },
      { text: <><B href={RIssues + '/236'}>UI fixes</B> (thanks to the first sponsor)</> },
      { text: <>Added support for Anthropic Claude 2.1</> },
      { text: <>Large rendering performance optimization</> },
@@ -80,8 +116,9 @@ export const NewsItems: NewsItem[] = [
    ],
  },
  {
-    versionName: '1.5.0',
-    text: 'Enjoy what\'s new:',
+    versionCode: '1.5.0',
+    versionName: 'Loaded!',
+    versionDate: new Date('2023-11-19T21:00:00Z'),
    items: [
      { text: <><B href={RIssues + '/190'}>Continued Voice</B> for hands-free interaction</> },
      { text: <><B href={RIssues + '/192'}>Visualization</B> Tool for data representations</> },
@@ -95,7 +132,7 @@ export const NewsItems: NewsItem[] = [
    ],
  },
  {
-    versionName: '1.4.0',
+    versionCode: '1.4.0',
    items: [
      { text: <><B>Share and clone</B> conversations, with public links</> },
      { text: <><B href={RCode + '/docs/config-azure-openai.md'}>Azure</B> models, incl. gpt-4-32k</> },
@@ -105,7 +142,7 @@ export const NewsItems: NewsItem[] = [
    ],
  },
  {
-    versionName: '1.3.5',
+    versionCode: '1.3.5',
    items: [
      { text: <>AI in the real world with <B>Camera OCR</B> - MOBILE-ONLY</> },
      { text: <><B>Anthropic</B> models full support</> },
@@ -116,7 +153,7 @@ export const NewsItems: NewsItem[] = [
    ],
  },
  {
-    versionName: '1.3.1',
+    versionCode: '1.3.1',
    items: [
      { text: <><B>Flattener</B> - 4-mode conversations summarizer</> },
      { text: <><B>Forking</B> - branch your conversations</> },
@@ -126,7 +163,7 @@ export const NewsItems: NewsItem[] = [
    ],
  },
  {
-    versionName: '1.2.1',
+    versionCode: '1.2.1',
    // text: '',
    items: [
      { text: <>New home page: <b><Link href={Brand.URIs.Home + clientUtmSource()} target='_blank'>{Brand.URIs.Home.replace('https://', '')}</Link></b></> },
@@ -138,7 +175,10 @@ export const NewsItems: NewsItem[] = [


 interface NewsItem {
-  versionName: string;
+  versionCode: string;
+  versionName?: string;
+  versionMoji?: string;
+  versionDate?: Date;
  text?: string | React.JSX.Element;
  items?: {
    text: string | React.JSX.Element;
@@ -1,14 +1,13 @@
 import * as React from 'react';
 import { shallow } from 'zustand/shallow';
-import { useRouter } from 'next/router';

+import { navigateToNews } from '~/common/app.routes';
 import { useAppStateStore } from '~/common/state/store-appstate';

 import { incrementalVersion } from './news.data';


 export function useShowNewsOnUpdate() {
-  const { push: routerPush } = useRouter();
  const { usageCount, lastSeenNewsVersion } = useAppStateStore(state => ({
    usageCount: state.usageCount,
    lastSeenNewsVersion: state.lastSeenNewsVersion,
@@ -17,9 +16,9 @@ export function useShowNewsOnUpdate() {
    const isNewsOutdated = (lastSeenNewsVersion || 0) < incrementalVersion;
    if (isNewsOutdated && usageCount > 2) {
      // Disable for now
-      void routerPush('/news');
+      void navigateToNews();
    }
-  }, [lastSeenNewsVersion, routerPush, usageCount]);
+  }, [lastSeenNewsVersion, usageCount]);
 }

 export function useMarkNewsAsSeen() {
@@ -1,7 +1,7 @@
 import * as React from 'react';

 import { DLLMId, useModelsStore } from '~/modules/llms/store-llms';
-import { callChatGenerate, VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow, VChatMessageIn } from '~/modules/llms/llm.client';


 export interface LLMChainStep {
@@ -80,7 +80,7 @@ export function useLLMChain(steps: LLMChainStep[], llmId: DLLMId | undefined, ch
    _chainAbortController.signal.addEventListener('abort', globalToStepListener);

    // LLM call
-    callChatGenerate(llmId, llmChatInput, chain.overrideResponseTokens)
+    llmChatGenerateOrThrow(llmId, llmChatInput, null, null, chain.overrideResponseTokens)
      .then(({ content }) => {
        stepDone = true;
        if (!stepAbortController.signal.aborted)
@@ -1,12 +1,13 @@
 import * as React from 'react';
 import { shallow } from 'zustand/shallow';

-import { Button, FormControl, Radio, RadioGroup, Switch } from '@mui/joy';
+import { Button, FormControl, Switch } from '@mui/joy';
 import BuildCircleIcon from '@mui/icons-material/BuildCircle';
 import WidthNormalIcon from '@mui/icons-material/WidthNormal';
 import WidthWideIcon from '@mui/icons-material/WidthWide';

 import { FormLabelStart } from '~/common/components/forms/FormLabelStart';
+import { FormRadioControl } from '~/common/components/forms/FormRadioControl';
 import { isPwa } from '~/common/util/pwaUtils';
 import { openLayoutModelsSetup } from '~/common/layout/store-applayout';
 import { useIsMobile } from '~/common/components/useMatchMedia';
@@ -50,14 +51,10 @@ export function AppChatSettingsUI() {
    zenMode: state.zenMode, setZenMode: state.setZenMode,
  }), shallow);

-  const handleCenterModeChange = (event: React.ChangeEvent<HTMLInputElement>) => setCenterMode(event.target.value as 'narrow' | 'wide' | 'full' || 'wide');
-
  const handleEnterIsNewlineChange = (event: React.ChangeEvent<HTMLInputElement>) => setEnterIsNewline(!event.target.checked);

  const handleDoubleClickToEditChange = (event: React.ChangeEvent<HTMLInputElement>) => setDoubleClickToEdit(event.target.checked);

-  const handleZenModeChange = (event: React.ChangeEvent<HTMLInputElement>) => setZenMode(event.target.value as 'clean' | 'cleaner');
-
  const handleRenderMarkdownChange = (event: React.ChangeEvent<HTMLInputElement>) => setRenderMarkdown(event.target.checked);

  const handleShowSearchBarChange = (event: React.ChangeEvent<HTMLInputElement>) => setShowPurposeFinder(event.target.checked);
@@ -102,25 +99,27 @@ export function AppChatSettingsUI() {
              slotProps={{ endDecorator: { sx: { minWidth: 26 } } }} />
    </FormControl>}

-    <FormControl orientation='horizontal' sx={{ alignItems: 'center', justifyContent: 'space-between' }}>
-      <FormLabelStart title='Appearance'
-                      description={zenMode === 'clean' ? 'Show senders' : 'Minimal UI'} />
-      <RadioGroup orientation='horizontal' value={zenMode} onChange={handleZenModeChange}>
-        {/*<Radio value='clean' label={<Face6Icon sx={{ width: 24, height: 24, mt: -0.25 }} />} />*/}
-        <Radio value='clean' label='Clean' />
-        <Radio value='cleaner' label='Zen' />
-      </RadioGroup>
-    </FormControl>
+    <FormRadioControl
+      title='Appearance'
+      description={zenMode === 'clean' ? 'Show senders' : 'Minimal UI'}
+      options={[
+        { label: 'Clean', value: 'clean' },
+        { label: 'Zen', value: 'cleaner' },
+      ]}
+      value={zenMode} onChange={setZenMode} />

-    {!isPwa() && !isMobile && <FormControl orientation='horizontal' sx={{ alignItems: 'center', justifyContent: 'space-between' }}>
-      <FormLabelStart title='Page Size'
-                      description={centerMode === 'full' ? 'Full screen chat' : centerMode === 'narrow' ? 'Narrow chat' : 'Wide'} />
-      <RadioGroup orientation='horizontal' value={centerMode} onChange={handleCenterModeChange}>
-        <Radio value='narrow' label={<WidthNormalIcon sx={{ width: 25, height: 24, mt: -0.25 }} />} />
-        <Radio value='wide' label={<WidthWideIcon sx={{ width: 25, height: 24, mt: -0.25 }} />} />
-        <Radio value='full' label='Full' />
-      </RadioGroup>
-    </FormControl>}
+    {!isPwa() && !isMobile && (
+      <FormRadioControl
+        title='Page Size'
+        description={centerMode === 'full' ? 'Full screen chat' : centerMode === 'narrow' ? 'Narrow chat' : 'Wide'}
+        options={[
+          { value: 'narrow', label: <WidthNormalIcon sx={{ width: 25, height: 24, mt: -0.25 }} /> },
+          { value: 'wide', label: <WidthWideIcon sx={{ width: 25, height: 24, mt: -0.25 }} /> },
+          { value: 'full', label: 'Full' },
+        ]}
+        value={centerMode} onChange={setCenterMode}
+      />
+    )}

  </>;
 }
@@ -26,8 +26,9 @@ const shortcutsMd = `
 | Ctrl + Alt + D      | **Delete** chat                                 |
 | Ctrl + Alt + B      | **Branch** chat                                 |
 | **Settings**        |                                                 |
-| Ctrl + Shift + M    | 🧠 Models                                       |
 | Ctrl + Shift + P    | ⚙️ Preferences                                  |
+| Ctrl + Shift + M    | 🧠 Models                                       |
+| Ctrl + Shift + O    | Options (current Chat Model)                    |
 | Ctrl + Shift + ?    | Shortcuts                                       |

 `.trim();
@@ -1,6 +1,7 @@
 import * as React from 'react';

 import { FormControl, Typography } from '@mui/joy';
+import AddAPhotoIcon from '@mui/icons-material/AddAPhoto';
 import CallIcon from '@mui/icons-material/Call';
 import FormatPaintIcon from '@mui/icons-material/FormatPaint';
 import VerticalSplitIcon from '@mui/icons-material/VerticalSplit';
@@ -9,36 +10,43 @@ import YouTubeIcon from '@mui/icons-material/YouTube';
 import { FormLabelStart } from '~/common/components/forms/FormLabelStart';
 import { FormSwitchControl } from '~/common/components/forms/FormSwitchControl';
 import { Link } from '~/common/components/Link';
+import { useIsMobile } from '~/common/components/useMatchMedia';
 import { useUXLabsStore } from '~/common/state/store-ux-labs';


 export function UxLabsSettings() {

  // external state
+  const isMobile = useIsMobile();
  const {
-    labsCalling, /*labsEnhancedUI,*/ labsMagicDraw, labsPersonaYTCreator, labsSplitBranching,
-    setLabsCalling, /*setLabsEnhancedUI,*/ setLabsMagicDraw, setLabsPersonaYTCreator, setLabsSplitBranching,
+    labsCalling, labsCameraDesktop, /*labsEnhancedUI,*/ labsMagicDraw, labsPersonaYTCreator, labsSplitBranching,
+    setLabsCalling, setLabsCameraDesktop, /*setLabsEnhancedUI,*/ setLabsMagicDraw, setLabsPersonaYTCreator, setLabsSplitBranching,
  } = useUXLabsStore();

  return <>

    <FormSwitchControl
-      title={<><YouTubeIcon /> YouTube Personas</>} description={labsPersonaYTCreator ? 'Creator Enabled' : 'Disabled'}
+      title={<><YouTubeIcon color={labsPersonaYTCreator ? 'primary' : undefined} sx={{ mr: 0.25 }} /> YouTube Personas</>} description={labsPersonaYTCreator ? 'Creator Enabled' : 'Disabled'}
      checked={labsPersonaYTCreator} onChange={setLabsPersonaYTCreator}
    />

    <FormSwitchControl
-      title={<><FormatPaintIcon />Assisted Draw</>} description={labsMagicDraw ? 'Enabled' : 'Disabled'}
+      title={<><FormatPaintIcon color={labsMagicDraw ? 'primary' : undefined} sx={{ mr: 0.25 }} />Assisted Draw</>} description={labsMagicDraw ? 'Enabled' : 'Disabled'}
      checked={labsMagicDraw} onChange={setLabsMagicDraw}
    />

    <FormSwitchControl
-      title={<><CallIcon /> Voice Calls</>} description={labsCalling ? 'Call AGI' : 'Disabled'}
+      title={<><CallIcon color={labsCalling ? 'primary' : undefined} sx={{ mr: 0.25 }} /> Voice Calls</>} description={labsCalling ? 'Call AGI' : 'Disabled'}
      checked={labsCalling} onChange={setLabsCalling}
    />

+    {!isMobile && <FormSwitchControl
+      title={<><AddAPhotoIcon color={labsCameraDesktop ? 'primary' : undefined} sx={{ mr: 0.25 }} /> Webcam</>} description={labsCameraDesktop ? 'Enabled' : 'Disabled'}
+      checked={labsCameraDesktop} onChange={setLabsCameraDesktop}
+    />}
+
    <FormSwitchControl
-      title={<><VerticalSplitIcon /> Split Branching</>} description={labsSplitBranching ? 'Enabled' : 'Disabled'} disabled
+      title={<><VerticalSplitIcon color={labsSplitBranching ? 'primary' : undefined} sx={{ mr: 0.25 }} /> Split Branching</>} description={labsSplitBranching ? 'Enabled' : 'Disabled'} disabled
      checked={labsSplitBranching} onChange={setLabsSplitBranching}
    />

@@ -50,7 +58,7 @@ export function UxLabsSettings() {
    <FormControl orientation='horizontal' sx={{ justifyContent: 'space-between', alignItems: 'center' }}>
      <FormLabelStart title='Graduated' />
      <Typography level='body-xs'>
-        <Link href='https://github.com/enricoros/big-agi/issues/192' target='_blank'>Auto Diagrams</Link> · Relative chat size · Text Tools
+        <Link href='https://github.com/enricoros/big-agi/issues/192' target='_blank'>Auto Diagrams</Link> · Relative chat size · Text Tools · LLM Overheat
      </Typography>
    </FormControl>

@@ -1,23 +1,29 @@
 import * as React from 'react';

-import { FormControl, Radio, RadioGroup } from '@mui/joy';
+import { FormControl } from '@mui/joy';

-import { ChatAutoSpeakType, useChatAutoAI } from '../chat/store-app-chat';
+import { useChatAutoAI, useChatMicTimeoutMs } from '../chat/store-app-chat';

 import { useElevenLabsVoices } from '~/modules/elevenlabs/useElevenLabsVoiceDropdown';

 import { FormLabelStart } from '~/common/components/forms/FormLabelStart';
+import { FormRadioControl } from '~/common/components/forms/FormRadioControl';
 import { LanguageSelect } from '~/common/components/LanguageSelect';
+import { useIsMobile } from '~/common/components/useMatchMedia';


 export function VoiceSettings() {

  // external state
+  const isMobile = useIsMobile();
  const { autoSpeak, setAutoSpeak } = useChatAutoAI();
  const { hasVoices } = useElevenLabsVoices();
+  const [chatTimeoutMs, setChatTimeoutMs] = useChatMicTimeoutMs();


-  const handleAutoSpeakChange = (e: React.ChangeEvent<HTMLInputElement>) => setAutoSpeak((e.target.value || 'off') as ChatAutoSpeakType);
+  // this converts from string keys to numbers and vice versa
+  const chatTimeoutValue: string = '' + chatTimeoutMs;
+  const setChatTimeoutValue = (value: string) => value && setChatTimeoutMs(parseInt(value));

  return <>

@@ -29,16 +35,29 @@ export function VoiceSettings() {
      <LanguageSelect />
    </FormControl>

-    <FormControl orientation='horizontal' sx={{ alignItems: 'center', justifyContent: 'space-between' }}>
-      <FormLabelStart title='Speak Responses'
-                      description={autoSpeak === 'off' ? 'Off' : 'First paragraph'}
-                      tooltip={!hasVoices ? 'No voices available, please configure a voice synthesis service' : undefined} />
-      <RadioGroup orientation='horizontal' value={autoSpeak} onChange={handleAutoSpeakChange}>
-        <Radio disabled={!hasVoices} value='off' label='Off' />
-        <Radio disabled={!hasVoices} value='firstLine' label='Start' />
-        <Radio disabled={!hasVoices} value='all' label='Full' />
-      </RadioGroup>
-    </FormControl>
+    {!isMobile && <FormRadioControl
+      title='Mic Timeout'
+      description={chatTimeoutMs < 1000 ? 'Best for quick calls' : chatTimeoutMs > 5000 ? 'Best for thinking' : 'Standard'}
+      options={[
+        { value: '600', label: '.6s' },
+        { value: '2000', label: '2s' },
+        { value: '15000', label: '15s' },
+      ]}
+      value={chatTimeoutValue} onChange={setChatTimeoutValue}
+    />}
+
+    <FormRadioControl
+      title='Speak Responses'
+      description={autoSpeak === 'off' ? 'Off' : 'First paragraph'}
+      tooltip={!hasVoices ? 'No voices available, please configure a voice synthesis service' : undefined}
+      disabled={!hasVoices}
+      options={[
+        { value: 'off', label: 'Off' },
+        { value: 'firstLine', label: 'Start' },
+        { value: 'all', label: 'Full' },
+      ]}
+      value={autoSpeak} onChange={setAutoSpeak}
+    />

  </>;
 }
@@ -7,21 +7,37 @@
 import Router from 'next/router';

 import type { DConversationId } from '~/common/state/store-chats';
+import { isBrowser } from './util/pwaUtils';


 export const ROUTE_INDEX = '/';
 export const ROUTE_APP_CHAT = '/';
 export const ROUTE_APP_LINK_CHAT = '/link/chat/:linkId';
 export const ROUTE_APP_NEWS = '/news';
+const ROUTE_CALLBACK_OPENROUTER = '/link/callback_openrouter';

-export const getIndexLink = () => ROUTE_INDEX;
+
+// Get Paths
+
+export const getCallbackUrl = (source: 'openrouter') => {
+  const callbackUrl = new URL(window.location.href);
+  switch (source) {
+    case 'openrouter':
+      callbackUrl.pathname = ROUTE_CALLBACK_OPENROUTER;
+      break;
+    default:
+      throw new Error(`Unknown source: ${source}`);
+  }
+  return callbackUrl.toString();
+};

 export const getChatLinkRelativePath = (chatLinkId: string) => ROUTE_APP_LINK_CHAT.replace(':linkId', chatLinkId);

-const navigateFn = (path: string) => (replace?: boolean): Promise<boolean> =>
-  Router[replace ? 'replace' : 'push'](path);
+
+/// Simple Navigation

 export const navigateToIndex = navigateFn(ROUTE_INDEX);
+
 export const navigateToChat = async (conversationId?: DConversationId) => {
  if (conversationId) {
    await Router.push(
@@ -41,6 +57,15 @@ export const navigateToNews = navigateFn(ROUTE_APP_NEWS);

 export const navigateBack = Router.back;

+export const reloadPage = () => isBrowser && window.location.reload();
+
+function navigateFn(path: string) {
+  return (replace?: boolean): Promise<boolean> => Router[replace ? 'replace' : 'push'](path);
+}
+
+
+/// Launch Apps
+
 export interface AppCallQueryParams {
  conversationId: string;
  personaId: string;
@@ -46,6 +46,7 @@ export const appTheme = extendTheme({
        text: {
          icon: 'var(--joy-palette-neutral-700)',           // <IconButton color='neutral' /> icon color
          secondary: 'var(--joy-palette-neutral-800)',      // increase contrast a bit
+          // tertiary: 'var(--joy-palette-neutral-700)',       // increase contrast a bit
        },
        // popup [white] > surface [50] > level1 [100] > level2 [200] > level3 [300] > body [white -> 400]
        background: {
@@ -2,7 +2,7 @@ import * as React from 'react';
 import { KeyboardEvent } from 'react';

 import { ClickAwayListener, Popper, PopperPlacementType } from '@mui/base';
-import { MenuList, styled, VariantProp } from '@mui/joy';
+import { MenuList, styled } from '@mui/joy';
 import { SxProps } from '@mui/joy/styles/types';


@@ -23,7 +23,8 @@ const Popup = styled(Popper)({
 */
 export function CloseableMenu(props: {
  open: boolean, anchorEl: HTMLElement | null, onClose: () => void,
-  variant?: VariantProp,
+  dense?: boolean,
+  // variant?: VariantProp,
  // color?: ColorPaletteProp,
  // size?: 'sm' | 'md' | 'lg',
  placement?: PopperPlacementType,
@@ -71,13 +72,12 @@ export function CloseableMenu(props: {
    >
      <ClickAwayListener onClickAway={handleClose}>
        <MenuList
-          variant={props.variant}
-          // color={props.color}
+          // variant={props.variant} color={props.color}
          onKeyDown={handleListKeyDown}
          sx={{
            '--Icon-fontSize': 'var(--joy-fontSize-xl2)',
-            '--ListItem-minHeight': '3rem',
-            '--ListItemDecorator-size': '2.75rem',
+            '--ListItem-minHeight': props.dense ? '2.5rem' : '3rem',
+            '--ListItemDecorator-size': '2.75rem', // icon width
            backgroundColor: 'background.popup',
            boxShadow: 'md',
            ...(props.maxHeightGapPx !== undefined ? { maxHeight: `calc(100dvh - ${props.maxHeightGapPx}px)`, overflowY: 'auto' } : {}),
@@ -1,39 +1,41 @@
 import * as React from 'react';

-import { Box, Button, Divider, Modal, ModalDialog, Typography } from '@mui/joy';
+import { Box, Button, Divider, Typography } from '@mui/joy';
 import WarningRoundedIcon from '@mui/icons-material/WarningRounded';

+import { GoodModal } from '~/common/components/GoodModal';
+

 /**
 * A confirmation dialog (Joy Modal)
 * Pass the question and the positive answer, and get called when it's time to close the dialog, or when the positive action is taken
 */
 export function ConfirmationModal(props: {
-  open: boolean, onClose: () => void, onPositive: () => void,
+  open?: boolean, onClose: () => void, onPositive: () => void,
  title?: string | React.JSX.Element,
  confirmationText: string | React.JSX.Element,
  positiveActionText: string
 }) {
  return (
-    <Modal open={props.open} onClose={props.onClose}>
-      <ModalDialog variant='outlined' color='neutral'>
-        <Typography component='h2' startDecorator={<WarningRoundedIcon />}>
-          {props.title || 'Confirmation'}
-        </Typography>
-        {/*<ModalClose/>*/}
-        <Divider sx={{ my: 2 }} />
-        <Typography level='body-md'>
-          {props.confirmationText}
-        </Typography>
-        <Box sx={{ display: 'flex', gap: 1, justifyContent: 'flex-end', mt: 2 }}>
-          <Button variant='plain' color='neutral' onClick={props.onClose}>
-            Cancel
-          </Button>
-          <Button variant='solid' color='danger' onClick={props.onPositive} sx={{ lineHeight: '1.5em' }}>
-            {props.positiveActionText}
-          </Button>
-        </Box>
-      </ModalDialog>
-    </Modal>
+    <GoodModal
+      open={props.open === undefined ? true : props.open}
+      title={props.title || 'Confirmation'}
+      titleStartDecorator={<WarningRoundedIcon sx={{ color: 'danger.solidBg' }} />}
+      onClose={props.onClose}
+      hideBottomClose
+    >
+      <Divider />
+      <Typography level='body-md'>
+        {props.confirmationText}
+      </Typography>
+      <Box sx={{ display: 'flex', gap: 1, justifyContent: 'flex-end', mt: 2 }}>
+        <Button autoFocus variant='plain' color='neutral' onClick={props.onClose}>
+          Cancel
+        </Button>
+        <Button variant='solid' color='danger' onClick={props.onPositive} sx={{ lineHeight: '1.5em' }}>
+          {props.positiveActionText}
+        </Button>
+      </Box>
+    </GoodModal>
  );
 }
@@ -9,17 +9,21 @@ import { SxProps } from '@mui/joy/styles/types';
 */
 export function GoodModal(props: {
  title?: string | React.JSX.Element,
-  strongerTitle?: boolean, noTitleBar?: boolean,
+  titleStartDecorator?: React.JSX.Element,
+  strongerTitle?: boolean,
+  noTitleBar?: boolean,
  dividers?: boolean,
  open: boolean,
  onClose?: () => void,
+  hideBottomClose?: boolean,
  startButton?: React.JSX.Element,
  sx?: SxProps,
  children: React.ReactNode,
 }) {
+  const showBottomClose = !!props.onClose && props.hideBottomClose !== true;
  return (
    <Modal open={props.open} onClose={props.onClose}>
-      <ModalOverflow>
+      <ModalOverflow sx={{p:1}}>
        <ModalDialog
          sx={{
            minWidth: { xs: 360, sm: 500, md: 600, lg: 700 },
@@ -29,7 +33,7 @@ export function GoodModal(props: {
          }}>

          {!props.noTitleBar && <Box sx={{ mb: -1, display: 'flex', flexDirection: 'row', alignItems: 'center', justifyContent: 'space-between' }}>
-            <Typography level={props.strongerTitle !== true ? 'title-md' : 'title-lg'}>
+            <Typography level={props.strongerTitle !== true ? 'title-md' : 'title-lg'} startDecorator={props.titleStartDecorator}>
              {props.title || ''}
            </Typography>
            {!!props.onClose && <ModalClose sx={{ position: 'static', mr: -1 }} />}
@@ -41,9 +45,9 @@ export function GoodModal(props: {

          {props.dividers === true && <Divider />}

-          {(!!props.startButton || !!props.onClose) && <Box sx={{ mt: 'auto', display: 'flex', flexWrap: 'wrap', gap: 1, justifyContent: 'space-between' }}>
+          {(!!props.startButton || showBottomClose) && <Box sx={{ mt: 'auto', display: 'flex', flexWrap: 'wrap', gap: 1, justifyContent: 'space-between' }}>
            {props.startButton}
-            {!!props.onClose && <Button variant='solid' color='neutral' onClick={props.onClose} sx={{ ml: 'auto', minWidth: 100 }}>
+            {showBottomClose && <Button variant='solid' color='neutral' onClick={props.onClose} sx={{ ml: 'auto', minWidth: 100 }}>
              Close
            </Button>}
          </Box>}
@@ -1,12 +1,25 @@
 import * as React from 'react';

 import { Tooltip } from '@mui/joy';
+import { SxProps } from '@mui/joy/styles/types';


 /**
 * Tooltip with text that wraps to multiple lines (doesn't go too long)
 */
-export const GoodTooltip = (props: { title: string | React.JSX.Element, children: React.JSX.Element }) =>
-  <Tooltip title={props.title} sx={{ maxWidth: { sm: '50vw', md: '25vw' } }}>
+export const GoodTooltip = (props: {
+  title: string | React.JSX.Element | null,
+  placement?: 'top' | 'bottom' | 'top-start',
+  isError?: boolean, isWarning?: boolean,
+  children: React.JSX.Element,
+  sx?: SxProps
+}) =>
+  <Tooltip
+    title={props.title}
+    placement={props.placement}
+    variant={(props.isError || props.isWarning) ? 'soft' : undefined}
+    color={props.isError ? 'danger' : props.isWarning ? 'warning' : undefined}
+    sx={{ maxWidth: { sm: '50vw', md: '25vw' }, ...props.sx }}
+  >
    {props.children}
  </Tooltip>;
@@ -23,7 +23,7 @@ export const FormLabelStart = (props: {
      <FormLabel
        onClick={props.onClick}
        sx={{
-          width: settingsCol1Width,
+          minWidth: settingsCol1Width,
          ...(!!props.onClick && { cursor: 'pointer', textDecoration: 'underline' }),
          ...props.sx,
        }}
@@ -0,0 +1,36 @@
+import * as React from 'react';
+
+import { FormControl, Radio, RadioGroup } from '@mui/joy';
+
+
+import { FormLabelStart } from './FormLabelStart';
+
+
+export type FormRadioOption<T extends string> = {
+  value: T,
+  label: string | React.JSX.Element,
+  disabled?: boolean
+};
+
+
+export const FormRadioControl = <TValue extends string>(props: {
+  title: string | React.JSX.Element,
+  description?: string | React.JSX.Element,
+  tooltip?: string | React.JSX.Element,
+  disabled?: boolean;
+  options: FormRadioOption<TValue>[];
+  value: TValue;
+  onChange: (value: TValue) => void;
+}) =>
+  <FormControl orientation='horizontal' disabled={props.disabled} sx={{ justifyContent: 'space-between', alignItems: 'center' }}>
+    {(!!props.title || !!props.description) && <FormLabelStart title={props.title} description={props.description} tooltip={props.tooltip} />}
+    <RadioGroup
+      orientation='horizontal'
+      value={props.value}
+      onChange={(event: React.ChangeEvent<HTMLInputElement>) => event.target.value && props.onChange(event.target.value as TValue)}
+    >
+      {props.options.map((option) =>
+        <Radio key={'opt-' + option.value} value={option.value} label={option.label} disabled={option.disabled || props.disabled} />,
+      )}
+    </RadioGroup>
+  </FormControl>;
@@ -13,6 +13,7 @@ export function FormSliderControl(props: {
  min?: number, max?: number, step?: number, defaultValue?: number,
  valueLabelDisplay?: 'on' | 'auto' | 'off',
  value: number, onChange: (value: number) => void,
+  endAdornment?: React.ReactNode,
 }) {
  return (
    <FormControl orientation='horizontal' sx={{ justifyContent: 'space-between', alignItems: 'center' }}>
@@ -24,6 +25,7 @@ export function FormSliderControl(props: {
        valueLabelDisplay={props.valueLabelDisplay}
        // sx={{ py: 1, mt: 1.1 }}
      />
+      {props.endAdornment}
    </FormControl>
  );
 }
@@ -2,8 +2,7 @@ import * as React from 'react';

 import { FormControl, FormLabel, Radio, RadioGroup } from '@mui/joy';

-
-export type FormRadioOption<T extends string> = { label: string, value: T, disabled?: boolean };
+import { FormRadioOption } from './FormRadioControl';


 /**
@@ -3,7 +3,8 @@ import { shallow } from 'zustand/shallow';

 import { DLLM, useModelsStore } from '~/modules/llms/store-llms';

-import { FormRadioOption, useFormRadio } from '~/common/components/forms/useFormRadio';
+import { FormRadioOption } from './FormRadioControl';
+import { useFormRadio } from './useFormRadio';


 type LlmType = 'chat' | 'fast';
@@ -0,0 +1,10 @@
+import * as React from 'react';
+
+import { SvgIcon } from '@mui/joy';
+import { SxProps } from '@mui/joy/styles/types';
+
+export function MistralIcon(props: { sx?: SxProps }) {
+  return <SvgIcon viewBox='0 0 24 24' width='24' height='24' strokeWidth={0} stroke='none' fill='currentColor' strokeLinecap='butt' strokeLinejoin='miter' {...props}>
+    <path d='m 2,2 v 4 4 V 14 v 4 4 h 4 v -4 -4 h 4 v 4 h 4 v -4 h 4 v 4 4 h 4 v -4 -4 -4 -4 V 2 h -4 v 4 h -4 v 4 h -4 v -4 H 6 V 2 Z' />
+  </SvgIcon>;
+}
@@ -21,8 +21,13 @@ export const useGlobalShortcut = (shortcutKey: string | false, useCtrl: boolean,
    if (!shortcutKey) return;
    const lcShortcut = shortcutKey.toLowerCase();
    const handleKeyDown = (event: KeyboardEvent) => {
-      if ((useCtrl === event.ctrlKey) && (useShift === event.shiftKey) && (useAlt === event.altKey)
-        && event.key.toLowerCase() === lcShortcut) {
+      const isCtrlOrCmd = (event.ctrlKey && !event.metaKey) || (event.metaKey && !event.ctrlKey);
+      if (
+        (useCtrl === isCtrlOrCmd) &&
+        (useShift === event.shiftKey) &&
+        (useAlt === event.altKey) &&
+        event.key.toLowerCase() === lcShortcut
+      ) {
        event.preventDefault();
        event.stopPropagation();
        callback();
@@ -46,9 +51,10 @@ export const useGlobalShortcuts = (shortcuts: GlobalShortcutItem[]) => {
  React.useEffect(() => {
    const handleKeyDown = (event: KeyboardEvent) => {
      for (const [key, useCtrl, useShift, useAlt, action] of shortcuts) {
+        const isCtrlOrCmd = (event.ctrlKey && !event.metaKey) || (event.metaKey && !event.ctrlKey);
        if (
          key &&
-          (useCtrl === event.ctrlKey) &&
+          (useCtrl === isCtrlOrCmd) &&
          (useShift === event.shiftKey) &&
          (useAlt === event.altKey) &&
          event.key.toLowerCase() === key.toLowerCase()
@@ -0,0 +1,95 @@
+import * as React from 'react';
+
+/**
+ * The AloneDetector class checks if the current client is the only one present for a given app. It uses
+ * BroadcastChannel to talk to other clients. If no other clients reply within a short time, it assumes it's
+ * the only one and tells the caller.
+ */
+class AloneDetector {
+  private readonly clientId: string;
+  private readonly broadcastChannel: BroadcastChannel;
+
+  private aloneCallback: ((isAlone: boolean) => void) | null;
+  private aloneTimerId: number | undefined;
+
+  constructor(channelName: string, onAlone: (isAlone: boolean) => void) {
+
+    this.clientId = Math.random().toString(36).substring(2, 10);
+    this.aloneCallback = onAlone;
+
+    this.broadcastChannel = new BroadcastChannel(channelName);
+    this.broadcastChannel.onmessage = this.handleIncomingMessage;
+
+  }
+
+  public onUnmount(): void {
+    // close channel
+    this.broadcastChannel.onmessage = null;
+    this.broadcastChannel.close();
+
+    // clear timeout
+    if (this.aloneTimerId)
+      clearTimeout(this.aloneTimerId);
+
+    this.aloneTimerId = undefined;
+    this.aloneCallback = null;
+  }
+
+  public checkIfAlone(): void {
+
+    // triggers other clients
+    this.broadcastChannel.postMessage({ type: 'CHECK', sender: this.clientId });
+
+    // if no response within 500ms, assume this client is alone
+    this.aloneTimerId = window.setTimeout(() => {
+      this.aloneTimerId = undefined;
+      this.aloneCallback?.(true);
+    }, 500);
+
+  }
+
+  private handleIncomingMessage = (event: MessageEvent): void => {
+
+    // ignore self messages
+    if (event.data.sender === this.clientId) return;
+
+    switch (event.data.type) {
+
+      case 'CHECK':
+        this.broadcastChannel.postMessage({ type: 'ALIVE', sender: this.clientId });
+        break;
+
+      case 'ALIVE':
+        // received an ALIVE message, tell the client they're not alone
+        if (this.aloneTimerId) {
+          clearTimeout(this.aloneTimerId);
+          this.aloneTimerId = undefined;
+        }
+        this.aloneCallback?.(false);
+        this.aloneCallback = null;
+        break;
+
+    }
+  };
+}
+
+
+/**
+ * React hook that checks whether the current tab is the only one open for a specific channel.
+ *
+ * @param {string} channelName - The name of the BroadcastChannel to communicate on.
+ * @returns {boolean | null} - True if the current tab is alone, false if not, or null before the check completes.
+ */
+export function useSingleTabEnforcer(channelName: string): boolean | null {
+  const [isAlone, setIsAlone] = React.useState<boolean | null>(null);
+
+  React.useEffect(() => {
+    const tabManager = new AloneDetector(channelName, setIsAlone);
+    tabManager.checkIfAlone();
+    return () => {
+      tabManager.onUnmount();
+    };
+  }, [channelName]);
+
+  return isAlone;
+}
@@ -3,7 +3,6 @@ import * as React from 'react';
 import { isBrowser, isChromeDesktop, isIPhoneUser } from '~/common/util/pwaUtils';

 import { CapabilityBrowserSpeechRecognition } from './useCapabilities';
-import { useGlobalShortcut } from './useGlobalShortcut';
 import { useUIPreferencesStore } from '../state/store-ui';


@@ -13,7 +12,8 @@ type DoneReason =
  | 'continuous-deadline' // we hit our `softStopTimeout` while listening continuously
  | 'api-unknown-timeout' // a timeout has occurred
  | 'api-error'           // underlying .onerror
-  | 'api-no-speech';      // underlying .onerror, user did not speak
+  | 'api-no-speech'       // underlying .onerror, user did not speak
+  | 'react-unmount';      // the component is unmounting - the App shall never see this (set on unmount and not transmitted)

 export interface SpeechResult {
  transcript: string;         // the portion of the transcript that is finalized (or all the transcript if done)
@@ -43,36 +43,51 @@ export const browserSpeechRecognitionCapability = (): CapabilityBrowserSpeechRec
 * We use a hook to default to 'false/null' and dynamically create the engine and update the UI.
 * @param onResultCallback - the callback to invoke when a result is received
 * @param softStopTimeout - FOR INTERIM LISTENING, on desktop: delay since the last word before sending the final result
- * @param useShortcutCtrlKey - the key to use as a shortcut to start/stop the speech recognition (e.g. 'm' for "Ctrl + M")
 */
-export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) => void, softStopTimeout: number, useShortcutCtrlKey: string | false) => {
+export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) => void, softStopTimeout: number) => {
+
+  // external state (will update this function when changed)
+  const preferredLanguage = useUIPreferencesStore(state => state.preferredLanguage);
+
  // enablers
-  const refRecognition = React.useRef<SpeechRecoControls | null>(null);
  const onResultCallbackRef = React.useRef(onResultCallback);
+  const softStopTimeoutRef = React.useRef<number>(softStopTimeout);
+  const speechControlsRef = React.useRef<SpeechRecoControls | null>(null);
+  const preferredLanguageRef = React.useRef<string>(preferredLanguage);

  // session
-  const [isSpeechEnabled, setIsSpeechEnabled] = React.useState<boolean>(false);
  const refStarted = React.useRef<boolean>(false);
+  const [isSpeechEnabled, setIsSpeechEnabled] = React.useState<boolean>(false);
  const [isRecording, setIsRecording] = React.useState<boolean>(false);
  const [isRecordingAudio, setIsRecordingAudio] = React.useState<boolean>(false);
  const [isRecordingSpeech, setIsRecordingSpeech] = React.useState<boolean>(false);
  const [isSpeechError, setIsSpeechError] = React.useState<boolean>(false);

-  // external state (will update this function when changed)
-  const preferredLanguage = useUIPreferencesStore(state => state.preferredLanguage);

-  // Update the ref each time the component calling the hook re-renders with a new callback
+  // Sync refs with external state (callback, soft timeout, language)
  React.useEffect(() => {
+    // update callback
    onResultCallbackRef.current = onResultCallback;
-  }, [onResultCallback]);
+
+    // update soft stop timeout
+    softStopTimeoutRef.current = softStopTimeout;
+
+    // update language on the running instance (requires an instance method invocation)
+    if (preferredLanguageRef.current !== preferredLanguage) {
+      preferredLanguageRef.current = preferredLanguage;
+      if (speechControlsRef.current)
+        speechControlsRef.current.setLang(preferredLanguage);
+    }
+  }, [onResultCallback, softStopTimeout, preferredLanguage]);
+

  // create the Recognition engine
  React.useEffect(() => {
    if (!isBrowser) return;

    // do not re-initialize, just update the language (if we're here there's a high chance the language has changed)
-    if (refRecognition.current) {
-      refRecognition.current.setLang(preferredLanguage);
+    if (speechControlsRef.current) {
+      console.warn('useSpeechRecognition: [dev-warn]: Speech recognition is already initialized.');
      return;
    }

@@ -86,7 +101,15 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
    if (!webSpeechAPI)
      return;

-    // local memory within a session
+
+    // create the SpeechRecognition instance
+    const instance = new webSpeechAPI();
+    instance.lang = preferredLanguageRef.current;
+    instance.interimResults = isChromeDesktop && softStopTimeoutRef.current > 0;
+    instance.maxAlternatives = 1;
+    instance.continuous = true;
+
+    // result within closure
    const speechResult: SpeechResult = {
      transcript: '',
      interimTranscript: '',
@@ -94,13 +117,6 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
      doneReason: undefined,
    };

-    const instance = new webSpeechAPI();
-    instance.lang = preferredLanguage;
-    instance.interimResults = isChromeDesktop && softStopTimeout > 0;
-    instance.maxAlternatives = 1;
-    instance.continuous = true;
-
-    // soft inactivity timer
    let inactivityTimeoutId: any | null = null;

    const clearInactivityTimeout = () => {
@@ -119,6 +135,7 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
      }, timeoutMs);
    };

+
    instance.onaudiostart = () => setIsRecordingAudio(true);

    instance.onaudioend = () => setIsRecordingAudio(false);
@@ -137,7 +154,7 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
      onResultCallbackRef.current(speechResult);
      // let the system handle the first stop (as long as possible)
      // if (instance.interimResults)
-      //   reloadInactivityTimeout(2 * softStopTimeout);
+      //   reloadInactivityTimeout(2 * softStopTimeoutRef.current);
    };

    instance.onend = () => {
@@ -197,35 +214,66 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>

      // auto-stop
      if (instance.interimResults)
-        reloadInactivityTimeout(softStopTimeout, 'continuous-deadline');
+        reloadInactivityTimeout(softStopTimeoutRef.current, 'continuous-deadline');
    };

+
    // store the control interface
-    refRecognition.current = {
+    speechControlsRef.current = {
      setLang: (lang: string) => instance.lang = lang,
      start: () => instance.start(),
      stop: (reason: DoneReason) => {
        speechResult.doneReason = reason;
        instance.stop();
      },
+      unmount: () => {
+        // Clear any inactivity timeout to prevent it from running after unmount
+        clearInactivityTimeout();
+
+        // Explicitly remove event listeners
+        instance.onaudiostart = undefined;
+        instance.onaudioend = undefined;
+        instance.onspeechstart = undefined;
+        instance.onspeechend = undefined;
+        instance.onstart = undefined;
+        instance.onend = undefined;
+        instance.onerror = undefined;
+        instance.onresult = undefined;
+
+        // Stop the recognition if it's running
+        if (refStarted.current) {
+          speechResult.doneReason = 'react-unmount';
+          instance.stop();
+          refStarted.current = false;
+        }
+      },
    };
+
    refStarted.current = false;
    setIsSpeechEnabled(true);

-  }, [preferredLanguage, softStopTimeout]);
+    // Cleanup function to play well when the component unmounts
+    return () => {
+      if (speechControlsRef.current) {
+        speechControlsRef.current.unmount();
+        speechControlsRef.current = null;
+      }
+      setIsSpeechEnabled(false);
+    };
+  }, []);


  // ACTIONS: start/stop recording

  const startRecording = React.useCallback(() => {
-    if (!refRecognition.current)
+    if (!speechControlsRef.current)
      return console.error('startRecording: Speech recognition is not supported or not initialized.');
    if (refStarted.current)
      return console.error('startRecording: Start recording called while already recording.');

    setIsSpeechError(false);
    try {
-      refRecognition.current.start();
+      speechControlsRef.current.start();
    } catch (error: any) {
      setIsSpeechError(true);
      console.log('Speech recognition error - clicking too quickly?', error?.message);
@@ -233,12 +281,12 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
  }, []);

  const stopRecording = React.useCallback(() => {
-    if (!refRecognition.current)
+    if (!speechControlsRef.current)
      return console.error('stopRecording: Speech recognition is not supported or not initialized.');
    if (!refStarted.current)
      return console.error('stopRecording: Stop recording called while not recording.');

-    refRecognition.current.stop('manual');
+    speechControlsRef.current.stop('manual');
  }, []);

  const toggleRecording = React.useCallback(() => {
@@ -248,7 +296,6 @@ export const useSpeechRecognition = (onResultCallback: (result: SpeechResult) =>
      startRecording();
  }, [startRecording, stopRecording]);

-  useGlobalShortcut(useShortcutCtrlKey, true, false, false, toggleRecording);

  return {
    isRecording,
@@ -288,17 +335,17 @@ interface ISpeechRecognition extends EventTarget {
  stop: () => void;
  // abort: () => void;

-  onaudiostart: (event: any) => void;
-  // onsoundstart: (event: any) => void;
-  onspeechstart: (event: any) => void;
-  onspeechend: (event: any) => void;
-  // onsoundend: (event: any) => void;
-  onaudioend: (event: any) => void;
-  onresult: (event: ISpeechRecognitionEvent) => void;
-  // onnomatch: (event: any) => void;
-  onerror: (event: any) => void;
-  onstart: (event: any) => void;
-  onend: (event: any) => void;
+  onaudiostart?: (event: any) => void;
+  // onsoundstart?: (event: any) => void;
+  onspeechstart?: (event: any) => void;
+  onspeechend?: (event: any) => void;
+  // onsoundend?: (event: any) => void;
+  onaudioend?: (event: any) => void;
+  onresult?: (event: ISpeechRecognitionEvent) => void;
+  // onnomatch?: (event: any) => void;
+  onerror?: (event: any) => void;
+  onstart?: (event: any) => void;
+  onend?: (event: any) => void;
 }

 interface ISpeechRecognitionEvent extends Event {
@@ -310,4 +357,5 @@ interface SpeechRecoControls {
  setLang: (lang: string) => void;
  start: () => void;
  stop: (reason: DoneReason) => void;
+  unmount: () => void;
 }
@@ -9,6 +9,7 @@ export type DropdownItems = Record<string, {
  title: string,
  symbol?: string,
  type?: 'separator'
+  icon?: React.ReactNode,
 }>;


@@ -71,20 +72,25 @@ export function AppBarDropdown<TValue extends string>(props: {
    {!!props.prependOption && Object.keys(props.items).length >= 1 && <Divider />}

    <Box sx={{ overflowY: 'auto' }}>
-      {Object.keys(props.items).map((key: string, idx: number) => <React.Fragment key={'key-' + idx}>
-        {props.items[key].type === 'separator'
-          ? <ListDivider />
-          : <Option value={key} sx={{ whiteSpace: 'nowrap' }}>
-            {props.showSymbols && <ListItemDecorator sx={{ fontSize: 'xl' }}>{props.items[key]?.symbol + ' '}</ListItemDecorator>}
-            {props.items[key].title}
+      {Object.keys(props.items).map((key: string, idx: number) => {
+        const item = props.items[key];
+
+        if (item.type === 'separator')
+          return <ListDivider key={'key-' + idx} />;
+
+        return (
+          <Option key={'key-' + idx} value={key} sx={{ whiteSpace: 'nowrap' }}>
+            {props.showSymbols && <ListItemDecorator sx={{ fontSize: 'xl' }}>{item?.symbol + ' '}</ListItemDecorator>}
+            {props.showSymbols && !!item.icon && <ListItemDecorator>{item?.icon}</ListItemDecorator>}
+            {item.title}
            {/*{key === props.value && (*/}
            {/*  <IconButton variant='soft' onClick={() => alert('aa')} sx={{ ml: 'auto' }}>*/}
            {/*    <SettingsIcon color='success' />*/}
            {/*  </IconButton>*/}
            {/*)}*/}
          </Option>
-        }
-      </React.Fragment>)}
+        );
+      })}
    </Box>

    {!!props.appendOption && Object.keys(props.items).length >= 1 && <ListDivider />}
@@ -3,7 +3,7 @@ import { shallow } from 'zustand/shallow';

 import { Box, Container } from '@mui/joy';

-import { ModelsModal } from '../../apps/models-modal/ModelsModal';
+import { ModelsModal } from '~/modules/llms/models-modal/ModelsModal';
 import { SettingsModal } from '../../apps/settings-modal/SettingsModal';
 import { ShortcutsModal } from '../../apps/settings-modal/ShortcutsModal';

@@ -0,0 +1,42 @@
+import * as React from 'react';
+
+import { Button, Sheet, Typography } from '@mui/joy';
+
+import { Brand } from '../app.config';
+import { reloadPage } from '../app.routes';
+import { useSingleTabEnforcer } from '../components/useSingleTabEnforcer';
+
+
+export const ProviderSingleTab = (props: { children: React.ReactNode }) => {
+
+  // state
+  const isSingleTab = useSingleTabEnforcer('big-agi-tabs');
+
+  // pass-through until we know for sure that other tabs are open
+  if (isSingleTab === null || isSingleTab)
+    return props.children;
+
+
+  return (
+    <Sheet
+      variant='solid'
+      invertedColors
+      sx={{
+        flexGrow: 1,
+        display: 'flex', flexDirection: { xs: 'column', md: 'row' }, justifyContent: 'center', alignItems: 'center', gap: 2,
+        p: 3,
+      }}
+    >
+
+      <Typography>
+        It looks like {Brand.Title.Base} is already running in another tab or window.
+        To continue here, please close the other instance first.
+      </Typography>
+
+      <Button onClick={reloadPage}>
+        Reload
+      </Button>
+
+    </Sheet>
+  );
+};
@@ -15,6 +15,9 @@ interface UXLabsStore {
  labsCalling: boolean;
  setLabsCalling: (labsCalling: boolean) => void;

+  labsCameraDesktop: boolean;
+  setLabsCameraDesktop: (labsCameraDesktop: boolean) => void;
+
  labsEnhancedUI: boolean;
  setLabsEnhancedUI: (labsEnhancedUI: boolean) => void;

@@ -36,6 +39,9 @@ export const useUXLabsStore = create<UXLabsStore>()(
      labsCalling: false,
      setLabsCalling: (labsCalling: boolean) => set({ labsCalling }),

+      labsCameraDesktop: false,
+      setLabsCameraDesktop: (labsCameraDesktop: boolean) => set({ labsCameraDesktop }),
+
      labsEnhancedUI: false,
      setLabsEnhancedUI: (labsEnhancedUI: boolean) => set({ labsEnhancedUI }),

@@ -24,8 +24,13 @@ export function copyToClipboard(text: string, typeLabel: string) {
 // NOTE: this could be implemented in a platform-agnostic manner with !!.read, but we call it out here for clarity
 export const supportsClipboardRead = !isFirefox;

-export async function getClipboardItems(): Promise<ClipboardItem[]> {
+export async function getClipboardItems(): Promise<ClipboardItem[] | null> {
  if (!isBrowser || !window.navigator.clipboard?.read)
    return [];
-  return await window.navigator.clipboard.read();
+  try {
+    return await window.navigator.clipboard.read();
+  } catch (error: any) {
+    console.warn('Failed to read clipboard: ', error);
+    return null;
+  }
 }
@@ -2,11 +2,13 @@
 * @fileoverview Utility functions for Markdown.
 */

+import { isBrowser } from '~/common/util/pwaUtils';
+
 /**
 * Quick and dirty conversion of HTML tables to Markdown tables.
 * Big plus: doesn't require any dependencies.
 */
-export function htmlTableToMarkdown(html: string): string {
+export function htmlTableToMarkdown(html: string, includeInvisible: boolean): string {
  const parser = new DOMParser();
  const doc = parser.parseFromString(html, 'text/html');
  const table = doc.querySelector('table');
@@ -16,20 +18,53 @@ export function htmlTableToMarkdown(html: string): string {
  const headerCells = table.querySelectorAll('thead th');
  if (headerCells.length > 0) {
    const headerRow = '| ' + Array.from(headerCells)
-      .map(cell => cell.textContent?.trim() || '')
-      .join(' | ') + '| ';
+      .map(cell => getTextWithSpaces(cell, includeInvisible).trim())
+      .join(' | ') + ' |';
    markdownRows.push(headerRow);
-    markdownRows.push('|:' + Array(headerCells.length).fill('-').join('|:') + '|');
+    markdownRows.push('|:' + Array(headerCells.length).fill('---').join('|:') + '|');
  }

  const bodyRows = table.querySelectorAll('tbody tr');
  for (const row of Array.from(bodyRows)) {
    const rowCells = row.querySelectorAll('td');
    const markdownRow = '| ' + Array.from(rowCells)
-      .map(cell => cell.textContent?.trim() || '')
+      .map(cell => getTextWithSpaces(cell, includeInvisible).trim())
      .join(' | ') + ' |';
    markdownRows.push(markdownRow);
  }

  return markdownRows.join('\n');
+}
+
+// Helper function to get text with spaces, ignoring hidden elements
+function getTextWithSpaces(node: Node, includeInvisible: boolean): string {
+  let text = '';
+  node.childNodes.forEach(child => {
+    if (child.nodeType === Node.TEXT_NODE)
+      text += child.textContent;
+    else if (child.nodeType === Node.ELEMENT_NODE)
+      if (includeInvisible || isVisible(child as Element))
+        text += ' ' + getTextWithSpaces(child, includeInvisible) + ' ';
+  });
+  return text;
+}
+
+// Helper function to determine if an element is visible
+function isVisible(element: Element): boolean {
+  if (!isBrowser) return true;
+
+  // if the cell is hidden, don't include it
+  const style = window.getComputedStyle(element);
+  if (style.display === 'none' || style.visibility === 'hidden')
+    return false;
+
+  // Check for common classes used to hide content or indicate tooltip/popover content.
+  // You may need to add more classes here based on your actual HTML/CSS.
+  const ignoredClasses = ['hidden', 'group-hover', 'tooltip', 'pointer-events-none', 'opacity-0'];
+  for (const ignoredClass of ignoredClasses)
+    if (element.classList.contains(ignoredClass))
+      return false;
+
+  // Otherwise, the element is considered visible
+  return true;
 }
@@ -1,8 +1,3 @@
-// Type guard to check if an item has a 'str' property
-function isTextItem(item: any): item is { str: string } {
-  return 'str' in item && typeof item.str === 'string';
-}
-
 /**
 * Extracts text from a PDF file
 *
@@ -12,17 +7,16 @@ function isTextItem(item: any): item is { str: string } {
 * is called. This is useful because the 'pdfjs-dist' library is quite large,
 * and we don't want to load it unless we need to. [Faster startup time!]
 *
- * @param file - The PDF file to extract text from
+ * @param pdfBuffer The content of a PDF file
 */
-export const pdfToText = async (file: File): Promise<string> => {
+export async function pdfToText(pdfBuffer: ArrayBuffer): Promise<string> {
  // Dynamically import the 'pdfjs-dist' library [nextjs]
  const { getDocument, GlobalWorkerOptions } = await import('pdfjs-dist');

  // Set the worker script path
-  GlobalWorkerOptions.workerSrc = '/workers/pdf.worker.min.js';
+  GlobalWorkerOptions.workerSrc = '/workers/pdf.worker.min.mjs';

-  const arrayBuffer = await file.arrayBuffer();
-  const pdf = await getDocument(arrayBuffer).promise;
+  const pdf = await getDocument(pdfBuffer).promise;
  const textPages: string[] = []; // Initialize an array to hold text from all pages

  for (let i = 1; i <= pdf.numPages; i++) {
@@ -35,4 +29,9 @@ export const pdfToText = async (file: File): Promise<string> => {
  }

  return textPages.join(''); // Join all the page texts at the end
-};
+}
+
+// Type guard to check if an item has a 'str' property
+function isTextItem(item: any): item is { str: string } {
+  return 'str' in item && typeof item.str === 'string';
+}
@@ -1,3 +1,24 @@
 export function capitalizeFirstLetter(string: string) {
  return string.charAt(0).toUpperCase() + string.slice(1);
+}
+
+export function createBase36Uid(checkDuplicates: string[]): string {
+  let id = '';
+  do {
+    id = Math.random().toString(36).substring(2, 10);
+  } while (checkDuplicates.includes(id));
+  return id;
+}
+
+export function ellipsizeFront(text: string, maxLength: number) {
+  if (text.length <= maxLength)
+    return text;
+  return '…' + text.slice(-(maxLength - 1));
+}
+
+export function ellipsizeMiddle(text: string, maxLength: number) {
+  if (text.length <= maxLength)
+    return text;
+  const half = Math.floor(maxLength / 2);
+  return text.slice(0, half) + '…' + text.slice(-(maxLength - half - 1));
 }
@@ -1,4 +1,4 @@
-import { callChatGenerateWithFunctions, VChatFunctionIn } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow, VChatFunctionIn } from '~/modules/llms/llm.client';
 import { useModelsStore } from '~/modules/llms/store-llms';

 import { useChatStore } from '~/common/state/store-chats';
@@ -71,7 +71,7 @@ export function autoSuggestions(conversationId: string, assistantMessageId: stri

  // Follow-up: Question
  if (suggestQuestions) {
-    // callChatGenerateWithFunctions(funcLLMId, [
+    // llmChatGenerateOrThrow(funcLLMId, [
    //     { role: 'system', content: systemMessage.text },
    //     { role: 'user', content: userMessage.text },
    //     { role: 'assistant', content: assistantMessageText },
@@ -83,15 +83,18 @@ export function autoSuggestions(conversationId: string, assistantMessageId: stri

  // Follow-up: Auto-Diagrams
  if (suggestDiagrams) {
-    void callChatGenerateWithFunctions(funcLLMId, [
+    void llmChatGenerateOrThrow(funcLLMId, [
        { role: 'system', content: systemMessage.text },
        { role: 'user', content: userMessage.text },
        { role: 'assistant', content: assistantMessageText },
      ], [suggestPlantUMLFn], 'draw_plantuml_diagram',
    ).then(chatResponse => {

+      if (!('function_arguments' in chatResponse))
+        return;
+
      // parse the output PlantUML string, if any
-      const functionArguments = chatResponse?.function_arguments ?? null;
+      const functionArguments = chatResponse.function_arguments ?? null;
      if (functionArguments) {
        const { code, type }: { code: string, type: string } = functionArguments as any;
        if (code && type) {
@@ -105,6 +108,8 @@ export function autoSuggestions(conversationId: string, assistantMessageId: stri
          editMessage(conversationId, assistantMessageId, { text: assistantMessageText }, false);
        }
      }
+    }).catch(err => {
+      console.error('autoSuggestions::diagram:', err);
    });
  }

@@ -1,4 +1,4 @@
-import { callChatGenerate } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow } from '~/modules/llms/llm.client';
 import { useModelsStore } from '~/modules/llms/store-llms';

 import { useChatStore } from '~/common/state/store-chats';
@@ -27,7 +27,7 @@ export function autoTitle(conversationId: string) {
  });

  // LLM
-  void callChatGenerate(fastLLMId, [
+  void llmChatGenerateOrThrow(fastLLMId, [
    { role: 'system', content: `You are an AI conversation titles assistant who specializes in creating expressive yet few-words chat titles.` },
    {
      role: 'user', content:
@@ -39,7 +39,7 @@ export function autoTitle(conversationId: string) {
        historyLines.join('\n') +
        '```\n',
    },
-  ]).then(chatResponse => {
+  ], null, null).then(chatResponse => {

    const title = chatResponse?.content
      ?.trim()
@@ -1,6 +1,6 @@
 import * as React from 'react';

-import { Box, Button, ButtonGroup, CircularProgress, Divider, Grid, IconButton } from '@mui/joy';
+import { Box, Button, ButtonGroup, CircularProgress, Divider, Grid, IconButton, Input, FormControl, FormLabel } from '@mui/joy';
 import AccountTreeIcon from '@mui/icons-material/AccountTree';
 import ExpandLessIcon from '@mui/icons-material/ExpandLess';
 import ExpandMoreIcon from '@mui/icons-material/ExpandMore';
@@ -8,8 +8,9 @@ import ReplayIcon from '@mui/icons-material/Replay';
 import StopOutlinedIcon from '@mui/icons-material/StopOutlined';
 import TelegramIcon from '@mui/icons-material/Telegram';

+import { llmStreamingChatGenerate } from '~/modules/llms/llm.client';
+
 import { ChatMessage } from '../../../apps/chat/components/message/ChatMessage';
-import { streamChat } from '~/modules/llms/transports/streamChat';

 import { GoodModal } from '~/common/components/GoodModal';
 import { InlineError } from '~/common/components/InlineError';
@@ -48,6 +49,7 @@ export function DiagramsModal(props: { config: DiagramConfig, onClose: () => voi
  const [message, setMessage] = React.useState<DMessage | null>(null);
  const [diagramType, diagramComponent] = useFormRadio<DiagramType>('auto', diagramTypes, 'Visualize');
  const [diagramLanguage, languageComponent] = useFormRadio<DiagramLanguage>('plantuml', diagramLanguages, 'Style');
+  const [customInstruction, setCustomInstruction] = React.useState<string>('');
  const [errorMessage, setErrorMessage] = React.useState<string | null>(null);
  const [abortController, setAbortController] = React.useState<AbortController | null>(null);

@@ -81,10 +83,10 @@ export function DiagramsModal(props: { config: DiagramConfig, onClose: () => voi
    const stepAbortController = new AbortController();
    setAbortController(stepAbortController);

-    const diagramPrompt = bigDiagramPrompt(diagramType, diagramLanguage, systemMessage.text, subject);
+    const diagramPrompt = bigDiagramPrompt(diagramType, diagramLanguage, systemMessage.text, subject, customInstruction);

    try {
-      await streamChat(diagramLlm.id, diagramPrompt, stepAbortController.signal,
+      await llmStreamingChatGenerate(diagramLlm.id, diagramPrompt, null, null, stepAbortController.signal,
        (update: Partial<{ text: string, typing: boolean, originLLM: string }>) => {
          assistantMessage = { ...assistantMessage, ...update };
          setMessage(assistantMessage);
@@ -103,7 +105,7 @@ export function DiagramsModal(props: { config: DiagramConfig, onClose: () => voi
      setAbortController(null);
    }

-  }, [abortController, conversationId, diagramLanguage, diagramLlm, diagramType, subject]);
+  }, [abortController, conversationId, diagramLanguage, diagramLlm, diagramType, subject, customInstruction]);


  // [Effect] Auto-abort on unmount
@@ -149,6 +151,12 @@ export function DiagramsModal(props: { config: DiagramConfig, onClose: () => voi
        <Grid xs={12} xl={6}>
          {llmComponent}
        </Grid>
+        <Grid xs={12} md={6}>
+            <FormControl>
+              <FormLabel>Custom Instruction</FormLabel>
+              <Input title="Custom Instruction" placeholder='e.g. visualize as state' value={customInstruction}  onChange={(e) => setCustomInstruction(e.target.value)} />
+            </FormControl>
+          </Grid>
      </Grid>
    )}

@@ -1,6 +1,5 @@
-import type { VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
-
-import type { FormRadioOption } from '~/common/components/forms/useFormRadio';
+import type { FormRadioOption } from '~/common/components/forms/FormRadioControl';
+import type { VChatMessageIn } from '~/modules/llms/llm.client';


 export type DiagramType = 'auto' | 'mind';
@@ -60,12 +59,15 @@ function plantumlDiagramPrompt(diagramType: DiagramType): { sys: string, usr: st
  }
 }

-export function bigDiagramPrompt(diagramType: DiagramType, diagramLanguage: DiagramLanguage, chatSystemPrompt: string, subject: string): VChatMessageIn[] {
+export function bigDiagramPrompt(diagramType: DiagramType, diagramLanguage: DiagramLanguage, chatSystemPrompt: string, subject: string, customInstruction: string): VChatMessageIn[] {
  const { sys, usr } = diagramLanguage === 'mermaid' ? mermaidDiagramPrompt(diagramType) : plantumlDiagramPrompt(diagramType);
+  if (customInstruction) {
+    customInstruction = 'Also consider the following instructions: ' + customInstruction;
+  }
  return [
    { role: 'system', content: sys },
    { role: 'system', content: chatSystemPrompt },
    { role: 'assistant', content: subject },
-    { role: 'user', content: usr },
+    { role: 'user', content: `${usr} ${customInstruction}` },
  ];
 }
@@ -1,4 +1,4 @@
-import { callChatGenerate } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow } from '~/modules/llms/llm.client';
 import { useModelsStore } from '~/modules/llms/store-llms';


@@ -14,10 +14,10 @@ export async function imaginePromptFromText(messageText: string): Promise<string
  const { fastLLMId } = useModelsStore.getState();
  if (!fastLLMId) return null;
  try {
-    const chatResponse = await callChatGenerate(fastLLMId, [
+    const chatResponse = await llmChatGenerateOrThrow(fastLLMId, [
      { role: 'system', content: simpleImagineSystemPrompt },
      { role: 'user', content: 'Write a prompt, based on the following input.\n\n```\n' + messageText.slice(0, 1000) + '\n```\n' },
-    ]);
+    ], null, null);
    return chatResponse.content?.trim() ?? null;
  } catch (error: any) {
    console.error('imaginePromptFromText: fetch request error:', error);
@@ -5,7 +5,7 @@
 import { DLLMId } from '~/modules/llms/store-llms';
 import { callApiSearchGoogle } from '~/modules/google/search.client';
 import { callBrowseFetchPage } from '~/modules/browse/browse.client';
-import { callChatGenerate, VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow, VChatMessageIn } from '~/modules/llms/llm.client';


 // prompt to implement the ReAct paradigm: https://arxiv.org/abs/2210.03629
@@ -128,7 +128,7 @@ export class Agent {
    S.messages.push({ role: 'user', content: prompt });
    let content: string;
    try {
-      content = (await callChatGenerate(llmId, S.messages, 500)).content;
+      content = (await llmChatGenerateOrThrow(llmId, S.messages, null, null, 500)).content;
    } catch (error: any) {
      content = `Error in callChat: ${error}`;
    }
@@ -187,8 +187,8 @@ async function search(query: string): Promise<string> {

 async function browse(url: string): Promise<string> {
  try {
-    const data = await callBrowseFetchPage(url);
-    return JSON.stringify(data ? { text: data } : { error: 'Issue reading the page' });
+    const page = await callBrowseFetchPage(url);
+    return JSON.stringify(page.content ? { text: page.content } : { error: 'Issue reading the page' });
  } catch (error) {
    console.error('Error browsing:', (error as Error).message);
    return 'An error occurred while browsing to the URL. Missing WSS Key?';
@@ -5,7 +5,7 @@ import { Alert, Box, Button, CircularProgress, Divider, FormControl, Option, Sel

 import { DLLM, DLLMId, useModelsStore } from '~/modules/llms/store-llms';

-import { TokenBadge } from '../../../apps/chat/components/composer/TokenBadge';
+import { TokenBadgeMemo } from '../../../apps/chat/components/composer/TokenBadge';

 import { FormLabelStart } from '~/common/components/forms/FormLabelStart';
 import { GoodModal } from '~/common/components/GoodModal';
@@ -145,7 +145,7 @@ export function ContentReducer(props: {
              lineHeight: 1.75,
            }} />

-          <TokenBadge directTokens={reducedTokens} tokenLimit={props.tokenLimit} absoluteBottomRight />
+          <TokenBadgeMemo direct={reducedTokens} limit={props.tokenLimit} absoluteBottomRight />

          {/* indicator we're processing */}
          {processing && (
@@ -1,5 +1,5 @@
 import { DLLMId, findLLMOrThrow } from '~/modules/llms/store-llms';
-import { callChatGenerate } from '~/modules/llms/transports/chatGenerate';
+import { llmChatGenerateOrThrow } from '~/modules/llms/llm.client';


 // prompt to be tried when doing recursive summerization.
@@ -80,10 +80,10 @@ async function cleanUpContent(chunk: string, llmId: DLLMId, _ignored_was_targetW
  const autoResponseTokensSize = Math.floor(contextTokens * outputTokenShare);

  try {
-    const chatResponse = await callChatGenerate(llmId, [
+    const chatResponse = await llmChatGenerateOrThrow(llmId, [
      { role: 'system', content: cleanupPrompt },
      { role: 'user', content: chunk },
-    ], autoResponseTokensSize);
+    ], null, null, autoResponseTokensSize);
    return chatResponse?.content ?? '';
  } catch (error: any) {
    return '';
@@ -1,8 +1,7 @@
 import * as React from 'react';

 import type { DLLMId } from '~/modules/llms/store-llms';
-import type { VChatMessageIn } from '~/modules/llms/transports/chatGenerate';
-import { streamChat } from '~/modules/llms/transports/streamChat';
+import { llmStreamingChatGenerate, VChatMessageIn } from '~/modules/llms/llm.client';


 export function useStreamChatText() {
@@ -25,7 +24,7 @@ export function useStreamChatText() {

    try {
      let lastText = '';
-      await streamChat(llmId, prompt, abortControllerRef.current.signal, (update) => {
+      await llmStreamingChatGenerate(llmId, prompt, null, null, abortControllerRef.current.signal, (update) => {
        if (update.text) {
          lastText = update.text;
          setPartialText(lastText);
@@ -0,0 +1,23 @@
+import { track } from '@vercel/analytics/server';
+
+import { env } from '~/server/env.mjs';
+
+
+// all the backend analytics flags
+type BackendAnalyticsFlag =
+  | 'domain';   // logs which domain the initial (capabilities) request is sent to
+
+
+const checkAnalyticsFlag = (flag: BackendAnalyticsFlag): boolean =>
+  env.BACKEND_ANALYTICS?.includes(flag) || false;
+
+
+export function analyticsListCapabilities(backendHostName: string) {
+  if (checkAnalyticsFlag('domain')) {
+    // Note: fire-and-forget
+    void track('backend-domain', {
+      hostname: backendHostName,
+      vercel_url: process.env.VERCEL_URL || 'no-vercel',
+    });
+  }
+}
@@ -1,5 +1,12 @@
+import { z } from 'zod';
+
+import type { BackendCapabilities } from '~/modules/backend/state-backend';
+
 import { createTRPCRouter, publicProcedure } from '~/server/api/trpc.server';
 import { env } from '~/server/env.mjs';
+import { fetchJsonOrTRPCError } from '~/server/api/trpc.serverutils';
+
+import { analyticsListCapabilities } from './backend.analytics';


 /**
@@ -12,7 +19,8 @@ export const backendRouter = createTRPCRouter({

  /* List server-side capabilities (pre-configured by the deployer) */
  listCapabilities: publicProcedure
-    .query(async () => {
+    .query(async ({ ctx }) => {
+      analyticsListCapabilities(ctx.hostName);
      return {
        hasDB: !!env.POSTGRES_PRISMA_URL && !!env.POSTGRES_URL_NON_POOLING,
        hasBrowsing: !!env.PUPPETEER_WSS_ENDPOINT,
@@ -20,11 +28,26 @@ export const backendRouter = createTRPCRouter({
        hasImagingProdia: !!env.PRODIA_API_KEY,
        hasLlmAnthropic: !!env.ANTHROPIC_API_KEY,
        hasLlmAzureOpenAI: !!env.AZURE_OPENAI_API_KEY && !!env.AZURE_OPENAI_API_ENDPOINT,
+        hasLlmGemini: !!env.GEMINI_API_KEY,
+        hasLlmMistral: !!env.MISTRAL_API_KEY,
        hasLlmOllama: !!env.OLLAMA_API_HOST,
        hasLlmOpenAI: !!env.OPENAI_API_KEY || !!env.OPENAI_API_HOST,
        hasLlmOpenRouter: !!env.OPENROUTER_API_KEY,
        hasVoiceElevenLabs: !!env.ELEVENLABS_API_KEY,
-      };
+      } satisfies BackendCapabilities;
+    }),
+
+
+  // The following are used for various OAuth integrations
+
+  /* Exchange the OpenrRouter 'code' (from PKCS) for an OpenRouter API Key */
+  exchangeOpenRouterKey: publicProcedure
+    .input(z.object({ code: z.string() }))
+    .query(async ({ input }) => {
+      // Documented here: https://openrouter.ai/docs#oauth
+      return await fetchJsonOrTRPCError<{ key: string }, { code: string }>('https://openrouter.ai/api/v1/auth/keys', 'POST', {}, {
+        code: input.code,
+      }, 'Backend.exchangeOpenRouterKey');
    }),

 });
@@ -9,6 +9,8 @@ export interface BackendCapabilities {
  hasImagingProdia: boolean;
  hasLlmAnthropic: boolean;
  hasLlmAzureOpenAI: boolean;
+  hasLlmGemini: boolean;
+  hasLlmMistral: boolean;
  hasLlmOllama: boolean;
  hasLlmOpenAI: boolean;
  hasLlmOpenRouter: boolean;
@@ -30,6 +32,8 @@ const useBackendStore = create<BackendStore>()(
    hasImagingProdia: false,
    hasLlmAnthropic: false,
    hasLlmAzureOpenAI: false,
+    hasLlmGemini: false,
+    hasLlmMistral: false,
    hasLlmOllama: false,
    hasLlmOpenAI: false,
    hasLlmOpenRouter: false,
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Enrico Ros	0fc83cf6f5	Merge branch 'release-1.8.0'	2023-12-20 02:38:51 -08:00
Enrico Ros	2949feccd5	Maintainers Release	2023-12-20 02:32:47 -08:00
Enrico Ros	d6f1c2da81	1.8.0: Readme and Changelog	2023-12-20 02:11:13 -08:00
Enrico Ros	fabb433fde	1.8.0: news.data.tsx	2023-12-20 01:54:23 -08:00
Enrico Ros	b57445eb14	1.8.0: Version	2023-12-20 01:11:08 -08:00
Enrico Ros	5f8f4aba78	Ollama: update models	2023-12-20 00:59:14 -08:00
Enrico Ros	d693cdaeba	Ollama: update admin panel	2023-12-20 00:59:03 -08:00
Enrico Ros	39fbcfd97b	OpenRouter: update models	2023-12-20 00:55:27 -08:00
Enrico Ros	7694bc3d52	OpenRouter: update models	2023-12-20 00:53:16 -08:00
Enrico Ros	7f21b2ac3d	Merge branch 'feature-gemini' Fixes #275	2023-12-20 00:16:44 -08:00
Enrico Ros	fdb66da1a7	Gemini: choose a content filtering threshold	2023-12-20 00:14:53 -08:00
Enrico Ros	6b62a6733b	Gemini: show block reason	2023-12-20 00:14:53 -08:00
Enrico Ros	5d62056807	Streaming: muxing format	2023-12-20 00:14:53 -08:00
Enrico Ros	efff7126af	Gemini: final touches	2023-12-20 00:14:53 -08:00
Enrico Ros	45046c70ed	Gemini: stream on	2023-12-20 00:14:53 -08:00
Enrico Ros	7b5b852793	Gemini: trim key	2023-12-20 00:14:53 -08:00
Enrico Ros	9952b757b8	Gemini: client version	2023-12-20 00:14:53 -08:00
Enrico Ros	b08ecc9012	Models Modal: improve caps	2023-12-20 00:14:53 -08:00
Enrico Ros	bc5a38fa89	Models List: show a helpful message	2023-12-20 00:14:53 -08:00
Enrico Ros	bee49a4b1c	Llms: streaming as a vendor function (then all directed to the unified)	2023-12-20 00:14:53 -08:00
Enrico Ros	0ece1ce58c	Llms: vendor-specific RPC to ChatGenerate	2023-12-20 00:14:53 -08:00
Enrico Ros	fd897b55b2	Llms: improve list generics	2023-12-20 00:14:53 -08:00
Enrico Ros	dd41a402d0	Llms: move models modal	2023-12-20 00:14:53 -08:00
Enrico Ros	3f9defd18c	Llms: restructure	2023-12-20 00:14:53 -08:00
Enrico Ros	49c77f5a10	Llms: cleanup model lists (bits)	2023-12-20 00:14:52 -08:00
Enrico Ros	6b2bfa6060	Llms: cleanup model lists	2023-12-20 00:14:52 -08:00
Enrico Ros	8e3f247bfb	Gemini: cleaner	2023-12-20 00:14:52 -08:00
Enrico Ros	201e3a7252	Streaming: cleanup	2023-12-20 00:14:52 -08:00
Enrico Ros	044ed4df79	Bits for the future	2023-12-20 00:14:52 -08:00
Enrico Ros	0df7297cca	Gemini: configuration, list models, and immediate generation	2023-12-20 00:14:52 -08:00
Enrico Ros	453a3e5751	LLM Vendors: auto IDs	2023-12-20 00:14:52 -08:00
Enrico Ros	34c1c425b9	Gemini: backend env var	2023-12-20 00:14:52 -08:00
Enrico Ros	e0a010189f	LLMOptions Modal: fix display	2023-12-20 00:14:52 -08:00
Enrico Ros	7a07f10ed1	Move ModelVendor enum	2023-12-20 00:14:52 -08:00
Enrico Ros	33cb2b84b2	Anthropic: allow for 39 chars sks	2023-12-20 00:13:58 -08:00
Enrico Ros	3adec85e1f	Fix shortcuts on Mac.	2023-12-18 19:59:03 -08:00
Enrico Ros	18cfe5e296	DB: drop URL validation for POSTGRES_PRISMA_URL. #277	2023-12-18 15:16:02 -08:00
Enrico Ros	566ba366b4	Merge pull request #280 [Visualize] Add custom instruction #218	2023-12-18 12:19:03 -08:00
Enrico Ros	7ed653b315	Fix.	2023-12-18 04:54:04 -08:00
Enrico Ros	cb333c33d7	Better 1-click deployment, fixes #279	2023-12-18 03:22:18 -08:00
Joris Kalz	22ba37074b	[Visualize] Add custom instruction #218	2023-12-16 23:22:47 +01:00
Enrico Ros	84d7b7644a	Ollama: update models	2023-12-15 15:48:41 -08:00
Enrico Ros	71445dafc8	Ollama: improved diagram	2023-12-15 15:29:56 -08:00
Enrico Ros	66a5ad7f00	Ollama: update md	2023-12-15 15:27:11 -08:00
Enrico Ros	09f80adfaa	Ollama: update md	2023-12-15 15:26:38 -08:00
Enrico Ros	9febd97065	Ollama: update md	2023-12-15 15:24:48 -08:00
Enrico Ros	5219f9928d	Ollama: update md	2023-12-15 15:24:13 -08:00
Enrico Ros	aec9f4665f	Update config-ollama.md	2023-12-15 15:23:48 -08:00
Enrico Ros	db48465204	Ollama: document network issue resolution. #276	2023-12-15 15:20:33 -08:00
Enrico Ros	c2c858730a	Bite the bullet with Zustand	2023-12-13 14:57:06 -08:00
Enrico Ros	402bde9a81	Newpad	2023-12-13 02:06:19 -08:00
Enrico Ros	ba1c0ba0d9	Enforce a Single instance (Tab) of the app. Closes #268	2023-12-13 00:09:56 -08:00
Enrico Ros	084d77cd78	Linting	2023-12-12 18:24:59 -08:00
Enrico Ros	30c17a9b73	Roll Joy	2023-12-12 18:10:46 -08:00
Enrico Ros	2442463da3	deploy-docker.md: update Official guide	2023-12-12 17:52:28 -08:00
Enrico Ros	84a3e8cfdb	Fix docker-compose to point to the 'latest' (stable) version, instead of the no more existing 'main'	2023-12-12 17:17:30 -08:00
Enrico Ros	6ae440d252	1.7.3: Patch release for Mistral support	2023-12-12 17:01:40 -08:00
Enrico Ros	c0c724afc1	Mistral Platform: full support Closes #273.	2023-12-12 16:39:06 -08:00
Enrico Ros	a265112ce1	Mistral Platform: backend-configurable support (#273 )	2023-12-12 16:39:06 -08:00
Enrico Ros	75605ed408	Dropdown: support model vendor icons	2023-12-12 16:39:06 -08:00
Enrico Ros	ad38ff4157	LLMs: safer and smarter access	2023-12-12 16:39:06 -08:00
Enrico Ros	08c60e53b1	LLMs: reorder template params	2023-12-12 16:39:06 -08:00
Enrico Ros	d0dcb2ac02	LLMs: getTransportAccess	2023-12-12 16:39:06 -08:00
Enrico Ros	fbeb604b26	Update README.md	2023-12-12 03:42:05 -08:00
Enrico Ros	c4f3b1df77	Update README.md	2023-12-12 03:40:44 -08:00
Enrico Ros	5a1f9caaac	Roll rest	2023-12-12 03:16:35 -08:00
Enrico Ros	2fc70d5e95	Roll other dev deps	2023-12-12 03:12:43 -08:00
Enrico Ros	43adadef78	Roll Material/Joy/Next	2023-12-12 03:11:14 -08:00
Enrico Ros	96f6e7628b	Roll Prisma	2023-12-12 03:08:10 -08:00
Enrico Ros	32ad82bcee	Drag/Drop: do not remove the text from the source	2023-12-12 03:07:31 -08:00
Enrico Ros	3d72aec369	Roll pdfjs-dist	2023-12-12 02:58:06 -08:00
Enrico Ros	d244ee2cca	Update Docker image workflow. Assume the vX.Y.Z is the latest (and will have the latest tag). Removing this to remove the 'stable' tag, as latest is better. The 'main' branch keeps the development tag.	2023-12-12 01:38:57 -08:00
Enrico Ros	cc8a235ae3	Bits	2023-12-12 01:21:43 -08:00
Enrico Ros	ae348812de	OpenRouter: improve showing of discounted models	2023-12-12 01:14:33 -08:00
Enrico Ros	6053636f66	OpenRouter: OAuth login support	2023-12-11 22:35:40 -08:00
Enrico Ros	f2e2aee672	1.7.2: Stable Patch Version	2023-12-11 21:22:31 -08:00
Enrico Ros	11cbb2bbf0	OpenRouter: update models	2023-12-11 21:21:22 -08:00
Enrico Ros	30bd19d6ce	HTML Table to Markdown Table: improve reliability and ignore hidden data	2023-12-11 20:46:34 -08:00
Enrico Ros	d0b5c02062	Improve how Stream errors are shown	2023-12-11 18:22:15 -08:00
Enrico Ros	771192e406	Ollama: support ollama errors via API	2023-12-11 18:19:38 -08:00
Enrico Ros	13f502bd76	1.7.1: Release (Ollama chat). #270	2023-12-10 22:17:35 -08:00
Enrico Ros	11055b12ca	Ollama: use the new Chat endpoint. Closes #270	2023-12-10 22:12:51 -08:00
Enrico Ros	d0ea96eec0	Ollama: Admin: optional sort by Pulls, and UI link to the Model page	2023-12-10 22:03:55 -08:00
Enrico Ros	02eafc03f1	Ollama: update models, and sort by Featured	2023-12-10 22:01:50 -08:00
Enrico Ros	33d07a0313	Ollama: update documentation	2023-12-10 21:30:30 -08:00
Enrico Ros	763b852148	Ollama: administration: external link	2023-12-10 20:24:20 -08:00
Enrico Ros	d5b0617fd7	Comment for now	2023-12-10 06:14:49 -08:00
Enrico Ros	e3ce83674c	Update Ollama	2023-12-10 06:09:54 -08:00
Enrico Ros	5cc5df6909	1.7.0: Fix	2023-12-10 04:52:00 -08:00
Enrico Ros	11d8cf8996	Update GitHub docker action	2023-12-10 04:51:37 -08:00
Enrico Ros	eae578970e	1.7.0: UpDate	2023-12-10 04:12:05 -08:00
Enrico Ros	e076953c6a	Merge branch 'release-1.7.0'	2023-12-10 04:08:29 -08:00
Enrico Ros	5c455591ea	1.7.0: Readme and Changelog	2023-12-10 04:06:50 -08:00
Enrico Ros	19b3dcd927	Update maintainers-release.md	2023-12-10 03:43:42 -08:00
Enrico Ros	702e27edbf	Update deploy-authentication.md	2023-12-10 03:29:13 -08:00
Enrico Ros	7c872de9af	Update deploy-authentication.md	2023-12-10 03:28:38 -08:00
Enrico Ros	53b18143e7	Update deploy-authentication.md	2023-12-10 03:27:49 -08:00
Enrico Ros	d812813aac	Update deploy-authentication.md	2023-12-10 03:27:09 -08:00
Enrico Ros	9505b7fd7f	Update deploy-authentication.md	2023-12-10 03:26:27 -08:00
Enrico Ros	9e07822598	Update deploy-authentication.md	2023-12-10 03:26:02 -08:00
Enrico Ros	6d6604a043	Update maintainers-release.md	2023-12-10 03:10:29 -08:00
Enrico Ros	64d5071eb4	Update maintainers-release.md	2023-12-10 03:02:27 -08:00
Enrico Ros	4a29ff0b19	Update maintainers-release.md	2023-12-10 02:43:42 -08:00
Enrico Ros	6acab83ac5	1.7.0: Version	2023-12-10 02:28:54 -08:00
Enrico Ros	a3391b46ec	1.7.0: News	2023-12-10 02:28:54 -08:00
Enrico Ros	9d021a0ea9	News: improve page	2023-12-10 01:58:15 -08:00
Enrico Ros	5b35435136	Removed stray page. #177	2023-12-10 01:56:48 -08:00
Enrico Ros	38b1cd1e4b	Composer: premature optimizations	2023-12-10 01:47:37 -08:00
Enrico Ros	50e4bf30f2	Composer: more optimizations	2023-12-10 01:30:16 -08:00
Enrico Ros	6f8d6462b9	Composer: optimizations	2023-12-10 01:07:32 -08:00
Enrico Ros	596bb1ccc6	Readme: refer to http basic auth. #269	2023-12-10 00:20:21 -08:00
Enrico Ros	8023d4fd7e	Improve HTTP Basic Auth docs. Improves #269	2023-12-10 00:17:34 -08:00
Enrico Ros	5808c5ae27	Merge branch 'LennardSchwarz-add-basic-auth'	2023-12-10 00:11:04 -08:00
Enrico Ros	0945bc1e74	Documented HTTP basic Auth. Fixes #269	2023-12-10 00:10:01 -08:00
Enrico Ros	c82ea978da	Improve Build/Deploy instructions	2023-12-09 23:05:56 -08:00
Enrico Ros	9184e28691	Merge branch 'add-basic-auth' of https://github.com/LennardSchwarz/lenn-big-agi into LennardSchwarz-add-basic-auth	2023-12-09 22:26:59 -08:00
Enrico Ros	59784af72c	Browser: initial screenshot support	2023-12-08 04:45:43 -08:00
Enrico Ros	8feb1881b9	Merge branch 'feature-new-attachments' Fixes #251	2023-12-08 04:45:26 -08:00
Enrico Ros	62747e07f1	Mic: greatly improve, with unmounting	2023-12-08 04:37:11 -08:00
Enrico Ros	934511a21f	Mic: properly fix #221 . The timeout was not reapplied.	2023-12-08 04:37:11 -08:00
Enrico Ros	e36b71db9c	Mic: Fix back on Desktop	2023-12-08 04:37:11 -08:00
Enrico Ros	924cd7018f	Attachments: MultiPart-ready. Closes #251 for this stage.	2023-12-08 04:37:11 -08:00
Enrico Ros	d5e91f9ce7	Optimize	2023-12-08 04:37:11 -08:00
Enrico Ros	f1ad8cd55e	Attachments: cleanups	2023-12-08 04:37:11 -08:00
Enrico Ros	d177c73642	Attachments: Send!	2023-12-08 04:37:11 -08:00
Enrico Ros	011bcf8ccd	Misc smaller improvements	2023-12-08 04:37:11 -08:00
Enrico Ros	7d0e5809e1	Misc cleanups	2023-12-08 04:37:11 -08:00
Enrico Ros	b369148057	Attachments: Inlining: done. Use a hook that derives data from another hook.	2023-12-08 04:37:11 -08:00
Enrico Ros	2e0105b5ed	Attachments: improvements and cleanups (still not attaching)	2023-12-08 04:37:11 -08:00
Enrico Ros	3f24ade8e6	Attachments: expire older parts	2023-12-08 04:37:11 -08:00
Enrico Ros	9cdaf26174	Attachments: remove Camera OCR (now common image OCR)	2023-12-08 04:37:11 -08:00
Enrico Ros	3b2c604615	Attachments: first inlining	2023-12-08 04:37:11 -08:00
Enrico Ros	223689316b	Token Progress Bar: improve margins	2023-12-08 04:37:11 -08:00
Enrico Ros	6456a0de0c	Token Progress Bar: disable Tooltip	2023-12-08 04:37:10 -08:00
Enrico Ros	57458fb32f	Attachments: closer to ejection	2023-12-08 04:37:10 -08:00
Enrico Ros	b2521060cc	Attachments: cleanup Outputs	2023-12-08 04:37:10 -08:00
Enrico Ros	13b6a1ba7e	Attachments: use ComposerOutputPart and cleanups	2023-12-08 04:37:10 -08:00
Enrico Ros	ec81d802d5	Attachments: extract item menu	2023-12-08 04:37:10 -08:00
Enrico Ros	f6eca257d6	Cleanup action group, slightly improves #258	2023-12-08 04:37:10 -08:00
Enrico Ros	e744b1afcd	Attachments: bits	2023-12-08 04:37:10 -08:00
Enrico Ros	bfcae972f7	Attachments: cached token counting	2023-12-08 04:37:10 -08:00
Enrico Ros	360f886c37	Attachments: improve console log	2023-12-08 04:37:10 -08:00
Enrico Ros	305c278e1c	Beauty: right align	2023-12-08 04:37:10 -08:00
Enrico Ros	ccfcf6235f	Beauty: by 2 pixels	2023-12-08 04:37:10 -08:00
Enrico Ros	62f7d92bb2	Beauty: token reporting	2023-12-08 04:37:10 -08:00
Enrico Ros	f8915141c8	Attachments: major steps forward towards ejectability	2023-12-08 04:37:10 -08:00
Enrico Ros	7e1e4af19b	Beauty: highlight user commands	2023-12-08 04:37:10 -08:00
Enrico Ros	439c462a9b	Beauty: buttons	2023-12-08 04:37:10 -08:00
Enrico Ros	95aa71abd6	Beauty: mic buttons	2023-12-08 04:37:10 -08:00
Enrico Ros	3c829cbf97	Good Tooltip	2023-12-08 04:37:10 -08:00
Enrico Ros	29a31d5ca3	Beauty: main button	2023-12-08 04:37:10 -08:00
Enrico Ros	4a8bb24c0f	Attachments: move withing composer	2023-12-08 04:37:10 -08:00
Enrico Ros	6b6c3afe0c	Attachment: improve UX	2023-12-08 04:37:10 -08:00
Enrico Ros	fd41388584	Attachment: outputsLoading for the spinners	2023-12-08 04:37:10 -08:00
Enrico Ros	b418b69dc3	Attachment: improve Unsupported (without requiring user action to switch to the generic text-block)	2023-12-08 04:37:10 -08:00
Enrico Ros	e1e2962a02	Attachment: bits	2023-12-08 04:37:10 -08:00
Enrico Ros	f1662e174f	Attachment: PDF to text, sync conversion, and debug	2023-12-08 04:37:10 -08:00
Enrico Ros	a73c55fc1f	Attachment: fixes	2023-12-08 04:37:10 -08:00
Enrico Ros	0aa923a99d	Attachment: remove	2023-12-08 04:37:10 -08:00
Enrico Ros	b75160bb2b	Attachment: rename pipeline	2023-12-08 04:37:10 -08:00
Enrico Ros	3d515102a1	Attachment: initial image support	2023-12-08 04:37:10 -08:00
Enrico Ros	b857cc18d8	Attachment: many cleanups	2023-12-08 04:37:10 -08:00
Enrico Ros	4737d962db	Attachment: begin conversions	2023-12-08 04:37:10 -08:00
Enrico Ros	7ba71078a8	Attachment: conversion logic for text, finished popups	2023-12-08 04:37:10 -08:00
Enrico Ros	bee0fa8751	Attachment: group Logic	2023-12-08 04:37:10 -08:00
Enrico Ros	5916dfb08d	pdfUtils: move	2023-12-08 04:37:10 -08:00
Enrico Ros	9d13b03923	Enable Camera on desktop, #233	2023-12-08 04:37:10 -08:00
Enrico Ros	48e6385ac7	FormLabelStart: try with 'minWidth'	2023-12-08 04:37:10 -08:00
Enrico Ros	cf664ff486	Attachment: improve auto-mime	2023-12-08 04:37:09 -08:00
Enrico Ros	5ccf8ba128	Attachment: push forward flow	2023-12-08 04:37:09 -08:00
Enrico Ros	3cd5917207	Attachment: set tooltip on button only	2023-12-08 04:37:09 -08:00
Enrico Ros	e2dcca274f	Browser: close incognito context	2023-12-08 04:37:09 -08:00
Enrico Ros	7369e898af	Browser: make the wss endpoint always overridable	2023-12-08 04:37:09 -08:00
Enrico Ros	1e2c12fddb	New Attach System: downloads almost ok	2023-12-08 04:37:09 -08:00
Enrico Ros	4f7369b940	Browser: improve behavior when loading non-pages (files)	2023-12-08 04:37:09 -08:00
Enrico Ros	f566049890	Browser: further improve error handling	2023-12-08 04:37:09 -08:00
Enrico Ros	fbc2da8b09	Browser: further improve error handling	2023-12-08 04:37:09 -08:00
Enrico Ros	af70b39515	Browse: beginning to cleanup page load	2023-12-08 04:37:09 -08:00
Enrico Ros	e080d72e8a	New Attach System: Components	2023-12-08 04:37:09 -08:00
Enrico Ros	fd24e3676a	Confirmation Modals: prettier	2023-12-08 04:37:09 -08:00
Enrico Ros	942cd461f5	Drag & drop in Composer: exclude self-drags	2023-12-08 04:37:09 -08:00
Enrico Ros	9567e1cbaa	New Attach System: renames	2023-12-08 04:37:09 -08:00
Enrico Ros	2d5d31268e	New Attach System: transfer specialized functions to the hook	2023-12-08 04:37:09 -08:00
Enrico Ros	b376608709	Fix on-demand clipboard item read. Note: shall remove this and go for ctrl+v only?	2023-12-08 04:37:09 -08:00
Enrico Ros	551e502caf	New Attach System: porting	2023-12-08 04:37:09 -08:00
Enrico Ros	9fb7fcd22f	New Attach System: framework	2023-12-08 04:37:09 -08:00
Enrico Ros	1cda7d195b	Revert "Browser: initial screenshot support" This reverts commit `4a02923dda`.	2023-12-08 04:36:17 -08:00
Enrico Ros	4a02923dda	Browser: initial screenshot support	2023-12-08 04:13:44 -08:00
Enrico Ros	a8a45631c2	Browser: update the documentation - large #247 improvement (@stevenlafl)	2023-12-08 03:40:51 -08:00
Enrico Ros	eaa755d4ce	Browser: update the documentation - integrates #247	2023-12-08 03:17:02 -08:00
Enrico Ros	872396a90e	Browser: update Markdown, see #247	2023-12-08 02:08:00 -08:00
Enrico Ros	6b3a2772cc	Bits	2023-12-08 01:41:56 -08:00
Enrico Ros	f378733abe	Oobabooga: document the changes	2023-12-07 22:18:06 -08:00
Enrico Ros	0cf8f0439d	Oobabooga: fix with recent API changes	2023-12-07 22:09:28 -08:00
Enrico Ros	ab53087b3a	LLM Overheat: intuitive UX	2023-12-05 15:13:08 -08:00
Enrico Ros	b50923a3b7	Denser menus: Message context & Selection	2023-12-05 14:59:45 -08:00
Enrico Ros	1b4a8da313	Backend: add support for analytics (log which host name responds)	2023-12-05 02:48:14 -08:00
Enrico Ros	31684c2fee	[shortcuts] Ctrl+Shift+O: current Chat Model options (temperature, etc..)	2023-12-04 23:54:48 -08:00
Enrico Ros	fedd4b1fda	Fix setting reactivity on the new Voice Input Timeout. Closes #221	2023-12-04 23:36:18 -08:00
Enrico Ros	a41667f427	Overheat LLMs OpenAI LLMs can go up to 2 as far as temperature. We don't enable >1 by default, but we have a new labs setting to enable 'overheating' (max temperature raised from 1 to 2) for Really Well Done LLMs.	2023-12-04 23:15:40 -08:00
Enrico Ros	021fa3b313	Update README.md	2023-12-02 01:50:49 -08:00
Lennard Schwarz	b7ca69aa0e	Update realm info	2023-12-01 18:31:04 +01:00
Lennard Schwarz	1efcadbf46	Update readme	2023-12-01 18:29:06 +01:00
Lennard Schwarz	598a6a8e0b	Merge branch 'main' of github-ls:LennardSchwarz/lenn-big-agi into add-basic-auth	2023-12-01 18:25:58 +01:00
Enrico Ros	1cd441a2f5	Clipboard: intercept exception, e.g. when a jpeg/png file is copied to clipboard, chrome won't consider it valid on read (yes on ctrl+v)	2023-11-29 15:40:12 -08:00
Enrico Ros	783dc55d02	Ollama: pulling warning	2023-11-29 11:30:07 -08:00
Enrico Ros	88418d1ed0	Enable Toppy-M	2023-11-29 11:13:42 -08:00
Enrico Ros	6a74d1900f	History truncation	2023-11-29 11:06:52 -08:00
Enrico Ros	5566e29bcc	OpenRouter: update models	2023-11-29 10:43:10 -08:00
Enrico Ros	1f49195251	Ollama: update models, including a marker of the new models	2023-11-29 10:16:31 -08:00
Enrico Ros	c5e15ece14	Composer: bits	2023-11-28 14:10:41 -08:00
Enrico Ros	7ceb176d70	Composer: cleanup overlays	2023-11-28 14:08:32 -08:00
Enrico Ros	b93bd1bd0b	move pdfToText	2023-11-28 12:35:38 -08:00
Enrico Ros	088133ec37	Configurable Voice Input timeout. #221	2023-11-28 03:46:23 -08:00
Enrico Ros	784766442d	Extract FormRadioControl	2023-11-28 03:28:06 -08:00
Enrico Ros	e014a7c828	Clarityx	2023-11-28 02:45:11 -08:00
Enrico Ros	224e745a71	Cosmetix	2023-11-28 02:35:06 -08:00
Lennard Schwarz	89f3e6f955	Update readme	2023-10-30 14:57:51 +01:00
Lennard Schwarz	e79b429c5e	Update Readme	2023-10-30 14:57:45 +01:00
Lennard Schwarz	c240f6bd5b	Add deploy button	2023-10-30 14:55:53 +01:00
Lennard Schwarz	33312e0fd9	Add my middleware thing	2023-10-30 14:52:43 +01:00