nai-degen
|
c8d8e2e58f
|
avoids instantiating new Claude tiktoken on every call
|
2023-08-08 17:38:03 -05:00 |
|
nai-degen
|
d1d83b41fa
|
uses accurate Claude tokenization
|
2023-08-08 17:29:36 -05:00 |
|
nai-degen
|
d29c304d5a
|
increases tokenizer failsafe to 500000 characters
|
2023-07-27 15:21:06 -05:00 |
|
khanon
|
56a4902599
|
Add tokenizers and configurable context size limits (khanon/oai-reverse-proxy!28)
|
2023-07-22 00:11:32 +00:00 |
|