Instructions to use Xwin-LM/Xwin-Math-7B-V1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Xwin-LM/Xwin-Math-7B-V1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Xwin-LM/Xwin-Math-7B-V1.0")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Xwin-LM/Xwin-Math-7B-V1.0")
model = AutoModelForCausalLM.from_pretrained("Xwin-LM/Xwin-Math-7B-V1.0")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Xwin-LM/Xwin-Math-7B-V1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Xwin-LM/Xwin-Math-7B-V1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xwin-LM/Xwin-Math-7B-V1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Xwin-LM/Xwin-Math-7B-V1.0

SGLang

How to use Xwin-LM/Xwin-Math-7B-V1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Xwin-LM/Xwin-Math-7B-V1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xwin-LM/Xwin-Math-7B-V1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Xwin-LM/Xwin-Math-7B-V1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xwin-LM/Xwin-Math-7B-V1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Xwin-LM/Xwin-Math-7B-V1.0 with Docker Model Runner:
```
docker model run hf.co/Xwin-LM/Xwin-Math-7B-V1.0
```

LC-Edward commited on Nov 23, 2023

Commit

709bb66

2 Parent(s): cd905b7 0fdbfb3

Merge branch 'main' of https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0 into main

Browse files

Files changed (1) hide show

README.md +4 -4

README.md CHANGED Viewed

@@ -8,12 +8,12 @@ license: llama2
   <a href="https://huggingface.co/Xwin-LM"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue"></a>
 </p>
-Xwin-Math is a series of powerful SFT LLMs for math problem based on LLaMA-2.
 ## 🔥 News
 - 💥 [Nov, 2023] The [Xwin-Math-70B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-70B-V1.0) model achieves **31.8 pass@1 on the MATH benchmark** and **87.0 pass@1 on the GSM8K benchmark**. This performance places it first amongst all open-source models!
-- 💥 [Nov, 2023] The [Xwin-Math-7B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0) and [Xwin-Math-13B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-13B-V1.0) models achieve **66.6 and 76.2 pass@1 on the GSM8K benchmark**, ranking as top-1 among all LLaMA-2 based 7B and 13B open-source models, respectively!
 ## ✨ Model Card
@@ -52,11 +52,11 @@ Xwin-Math-70B-V1.0 has achieved **31.8% on MATH** and **87.0% on GSM8K**. These
 |  LEMAv1-7B            |  10.0  |  54.7  |
 |**Xwin-Math-7B-V1.0**  |  17.4  |  66.6  |
-We obtain these results using our flexible evaluation strategy. Due to differences in environment and hardware, the numbers may be different from the reported results, but we ensure that the evaluation is as accurate and fair as possible.
 ### Xwin-Math performance on other math benchmarks.
-Our 70B model shows strong mathematical synthesis capabilities among all open-sourced models. Also note that our model even approaches or surpasses the performance of GPT-35-Turbo on some benchmarks.
 | **Model** | SVAMP | ASDiv | NumGlue | Algebra | MAWPS | **Average** |
 |:-:|:-:|:-:|:-:|:-:|:-:|:-:|

   <a href="https://huggingface.co/Xwin-LM"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue"></a>
 </p>
+Xwin-Math is a series of powerful SFT LLMs for math problems based on LLaMA-2.
 ## 🔥 News
 - 💥 [Nov, 2023] The [Xwin-Math-70B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-70B-V1.0) model achieves **31.8 pass@1 on the MATH benchmark** and **87.0 pass@1 on the GSM8K benchmark**. This performance places it first amongst all open-source models!
+- 💥 [Nov, 2023] The [Xwin-Math-7B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0) and [Xwin-Math-13B-V1.0](https://huggingface.co/Xwin-LM/Xwin-Math-13B-V1.0) models achieve **66.6 and 76.2 pass@1 on the GSM8K benchmark**, ranking as top-1 among all LLaMA-2 based 7B and 13B open-source models respectively!
 ## ✨ Model Card
 |  LEMAv1-7B            |  10.0  |  54.7  |
 |**Xwin-Math-7B-V1.0**  |  17.4  |  66.6  |
+We obtain these results using our flexible evaluation strategy. Due to differences in environment and hardware, the test results may be slightly different from the report, but we ensure that the evaluation is as accurate and fair as possible.
 ### Xwin-Math performance on other math benchmarks.
+Our 70B model shows strong mathematical reasoning capabilities among all open-sourced models. Also note that our model even approaches or surpasses the performance of GPT-35-Turbo on some benchmarks.
 | **Model** | SVAMP | ASDiv | NumGlue | Algebra | MAWPS | **Average** |
 |:-:|:-:|:-:|:-:|:-:|:-:|:-:|