From 8b216f7cef132a07de92c069c5645b1cdb32ec91 Mon Sep 17 00:00:00 2001 From: Fuli Luo Date: Mon, 4 Dec 2023 19:45:45 +0800 Subject: [PATCH] Update README.md (#16) --- README.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 959084e..05a4b99 100644 --- a/README.md +++ b/README.md @@ -212,11 +212,7 @@ On the basis of `Python >= 3.8` environment, install the necessary dependencies pip install -r requirements.txt ``` -### Inference - -Here are some examples of utilizing our models. - -#### Inference with Huggingface's Transformers +### Inference with Huggingface's Transformers You can directly employ [Huggingface's Transformers](https://github.com/huggingface/transformers) for model inference. @@ -274,7 +270,7 @@ Assistant: **Note:** By default (`add_special_tokens=True`), our tokenizer automatically adds a `bos_token` (`<|begin▁of▁sentence|>`) before the input text. Additionally, since the system prompt is not compatible with this version of our models, we DO NOT RECOMMEND including the system prompt in your input. -#### Inference with vLLM +### Inference with vLLM You can also employ [vLLM](https://github.com/vllm-project/vllm) for high-throughput inference.