deepseekmirror/DeepSeek-LLM
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-LLM.git synced 2025-02-23 06:09:06 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source 
add notes on ChatLLM.cpp.
This commit is contained in:
Judd 2023-12-09 18:10:38 +08:00 committed by GitHub
parent 81cc42c689
commit 21ffed3f33
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@ -346,6 +346,9 @@ python convert-hf-to-gguf.py <MODEL_PATH> --outfile <GGUF_PATH> --model-name dee
./quantize <GGUF_PATH> <OUTPUT_PATH> q4_0
./main -m <OUTPUT_PATH> -n 128 -p <PROMPT>
```
You can also try out other [ggml](https://github.com/ggerganov/ggml)-based inferencers, such as [ChatLLM.cpp](https://github.com/foldl/chatllm.cpp), as well.
#### GPTQ(exllamav2)
`UPDATE:`[exllamav2](https://github.com/turboderp/exllamav2) has been able to support HuggingFace Tokenizer. Please pull the latest version and try out.