CSF/DC: update docs

This commit is contained in:
Enrico Ros
2026-04-20 13:15:17 -07:00
parent 436a858cb0
commit 6c0a60e0d1
3 changed files with 58 additions and 1 deletions
+21
View File
@@ -80,6 +80,27 @@ and then are send to the upstream AI services.
![data_ownership_hosted.png](pixels/data_ownership_hosted.png)
### Direct Connection (Browser → AI Service)
Most AI services offer a **Direct Connection** toggle (under a service's Advanced settings). When enabled, the browser calls the AI provider's API directly, skipping the Big-AGI server entirely.
Benefits:
- **No 4.5 MB upload limit** - the Vercel body-size cap does not apply, so larger attachments and long prompts go through.
- **No 300-second timeout** - the Vercel function timeout does not apply, so long-running generations keep streaming.
- **More privacy** - connection metadata (IP, timestamp, edge region, Vercel telemetry) is not observable by the Big-AGI edge server.
Tradeoff:
- **Slightly more downlink bandwidth**: when traffic passes through the Big-AGI edge, repetitive streaming frames are compacted; direct streams arrive verbatim from the provider.
Availability requires both:
1. The API key is set in your browser (client-side), not via server environment variables. Server-key deployments cannot use Direct Connection because the browser has no credential to send.
2. The AI service allows CORS (browser-origin requests). Most major providers do; Big-AGI sets any extra headers they require.
Direct Connection is a net win on speed, limits, and privacy whenever the provider permits it.
## Security Best Practices
**Basic Security**:
+20
View File
@@ -2,6 +2,26 @@
Quick answers to common questions about Big-AGI. For detailed documentation, see our [Website Docs](https://big-agi.com/docs).
### Connectivity
<details open>
<summary><b>What is "Direct Connection" and should I enable it?</b></summary>
Direct Connection lets the browser call the AI provider's API directly, skipping the Big-AGI edge server. It appears as a toggle in each AI service's Advanced settings when your API key is set client-side.
**When available, it is a net win**: faster, fewer restrictions, more privacy.
- **No 4.5 MB upload limit** (Vercel body-size cap does not apply).
- **No 300-second timeout** (Vercel function timeout does not apply; call length is bound only by the AI service).
- **More privacy** - connection metadata (IP, timestamp, edge region, Vercel telemetry) is not observable by the Big-AGI edge server.
- **Slightly more downlink bandwidth** - when passing through the edge, Big-AGI sheds repetitive streaming frames; direct streams arrive verbatim.
**When it is unavailable**:
1. **Server-side keys** - if the deployment stores API keys in server environment variables, the browser has no credential to send directly.
2. **Provider does not allow CORS** - browsers cannot call APIs that block cross-origin requests. Most major providers permit it; Big-AGI sets any required headers.
</details>
### Versions
<details open>
+17 -1
View File
@@ -1,6 +1,22 @@
# CSF - Client-Side Fetch
Client-Side Fetch (CSF) enables direct browser-to-API communication, bypassing the server for LLM requests. When enabled, the browser makes requests directly to vendor APIs (e.g., `api.openai.com`, `api.groq.com`) instead of routing through the Next.js server. This reduces latency, decreases server load, and is particularly useful for local models where the browser can communicate directly with Ollama or LM Studio.
Client-Side Fetch (CSF), surfaced to users as **"Direct Connection"**, enables direct browser-to-API communication, bypassing the server for LLM requests. When enabled, the browser makes requests directly to vendor APIs (e.g., `api.openai.com`, `api.groq.com`) instead of routing through the Next.js server. This reduces latency, decreases server load, and is particularly useful for local models where the browser can communicate directly with Ollama or LM Studio.
## User-facing tradeoffs (Direct Connection vs via-server)
Wins when Direct Connection is on:
- **No 4.5MB upload limit** (Vercel body-size cap does not apply to direct browser-to-API requests).
- **No 300s function timeout** (Vercel serverless/edge timeout does not apply; call duration is bound only by the AI service).
- **More privacy**: connection metadata (IP, timestamp, edge region, Vercel telemetry) is not observable by the Big-AGI edge server.
Costs:
- **Slightly more downlink bandwidth**: when traffic passes through the Big-AGI server, repetitive streaming frames are shed/compacted; direct streams arrive verbatim.
Availability requires both:
1. The API key is on the **client** (localStorage), not a server-side env var. Server-key deployments cannot use CSF because the browser has no credential to send.
2. The AI service **allows CORS** from browsers. Most major providers do; some require specific headers which Big-AGI sets.
Net: Direct Connection is a win on speed, limits, and privacy whenever the provider permits it. It is unavailable when keys are server-side or the provider blocks browser-origin requests.
## Implementation