From 96a277d5c120363b1ebe381a9e0a2bb02b23ff94 Mon Sep 17 00:00:00 2001 From: Judd Date: Sat, 9 Dec 2023 18:05:57 +0800 Subject: [PATCH] Update README.md add notes on ChatLLM.cpp --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index ca7c966..28d564c 100644 --- a/README.md +++ b/README.md @@ -350,6 +350,9 @@ python convert-hf-to-gguf.py --outfile --model-name dee ./quantize q4_0 ./main -m -n 128 -p ``` + +You can also try out other [ggml](https://github.com/ggerganov/ggml)-based inferencers, such as [ChatLLM.cpp](https://github.com/foldl/chatllm.cpp), as well. + ##### GPTQ(exllamav2) `UPDATE:`[exllamav2](https://github.com/turboderp/exllamav2) has been able to support Huggingface Tokenizer. Please pull the latest version and try out.