Cleaned up the code a bit (thx gpt4), with:
- dynamic module load: JS is chunked up and deferred to PDF loading,
which improves all the sessions where PDFs are not loaded
- unified path for drag/drop and 'load file' (shall call it "magic drop"
so PDFs are text'ified upon drag/dop as well
- fixed "not being able to load the same doc twice" (thx gpt4)
- using minified worker, as it's loaded dynamically, we save ~50% bandwidth
Chats are exported to paste.gg, are unlisted by default, and expire
in 30 days by default. The user is also provided with the deletion
key which will be only shown at the time of creation, and it's needed
to take down the paste.
Rendering looks quite great, including code and turns.
Closes#32. Enable Users / Deployments to change the host where
OpenAI API calls are directed to. This enables project like
[Helicone](https://www.helicone.ai/) (Observability of LLM ops)
for tracking prompt/responses quality in real-time.
Configuration:
- User: App > Settings > Advanced > API host (e.g. "oai.hconeai.com")
- Deployment: set the 'API_API_HOST=...' environment variable
User takes precedence over deployment over api.openai.com. Realtime
switching in chat apps works well.
Note: the Helicone team is fixing dashboard reporting for 'streaming'
over the /v1/chat/completions endpoint.
The UI side works well, including aborting the network
request to the api, and showing status and issues.
The API function (api/chat.ts) doesn't seem to work yet,
as the try/catch blocks are not triggered (both on the
'handler', and in the 'OpenAIStream').
In the meantime, having the Stop button on the UI is still
a net gain, despite tokens still running/being used from
the API call.
With this simple change, if the key is defined _at build time_,
the user will not be prompted to enter a key, and it will be
marked as 'not required' in the Settings dialog.
Note that this verifies a build-time key (next build), not that
the same key is set at runtime (next start).
The behavior of the UI is just altered slightly, but enough to
show that the key is not required when set on the server.
Basic implementation using Zustand to store values. For now just the
ChatModel and the SystemPurpose that are currently active. This enables
app-wide synchronization and automated persistence. Meaning we can now
have settings (and in future conversations) which are persisted and
synchronized.
API keys have not been migrated yet, because we need to import them from
the existing localStorage key, for a smooth transition.
Behind the scenes, we detect the model name from the OpenAI
streaming API (it's in all the chunks), and stream it back
to the UI as a serialized JSON object. This is more of a
workaround, required by virtue of the streaming API, and can
be cleaned up later.
Model names available when hovering 'gpt-4' / '3.5-turbo'.
Fix an uninitialized 'model' variable in case the user dismissed the
settings dialog. Fixed a missing variable expansion, '{{Today}}', for
Chat and Custom purposes.
The keys are stored within the localStorage of the browser.
The string is passed to the backend as a parameter in the REST
API call (to '/api/chat'), where it has priority over other
keys set in the OPENAI_API_KEY of the backend.
Enrico Ros