OSSForks/localGPT
1
0
Fork 0
mirror of https://github.com/zebrajr/localGPT.git synced 2025-12-06 00:20:19 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Restore context windows size to 8096 tokens

Browse Source
This commit is contained in:
jxmai 2024-10-28 17:24:40 -05:00 committed by GitHub
parent 51eb664c43
commit c3620d972b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
constants.py
View File

@ -29,7 +29,7 @@ CHROMA_SETTINGS = Settings(
)
# Context Window and Max New Tokens
CONTEXT_WINDOW_SIZE = 2048
CONTEXT_WINDOW_SIZE = 8096
MAX_NEW_TOKENS = CONTEXT_WINDOW_SIZE # int(CONTEXT_WINDOW_SIZE/4)
#### If you get a "not enough space in the buffer" error, you should reduce the values below, start with half of the original values and keep halving the value until the error stops appearing