Instructions to use DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer")
model = AutoModelForCausalLM.from_pretrained("DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer

SGLang

How to use DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer with Docker Model Runner:
```
docker model run hf.co/DoppelReflEx/QWQ-32B-Dawnwhisper-QWQTokenizer
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

QWQ-32B-Dawnwhisper-QwenTokenizer

This version is make with QWQ Tokenizer. It should be better than Origin version in Reasoning mode, that's all.

Like many people said: Tiny Deepseek R1 at home if you don't have too good specs. 16GB Vram card could run IQ3 variants very well.

~~After quick test, this merge perform very good result and strong capability in roleplay. Nothing more~~. Thank you for using my merge model.

After full testing, I can say this model is one of the best models I have created (merged). Dialogues likely vivid and very smart compare to its size. Thank mradermacher's team for quantizing this model, that help me available to do full testing.

The cons are: anything but not English and Chinese, will decrease the performance of this model, you should turn on reasoning mode to have better experience in non-English and non-Chinese language.

GGUF (Thank mradermacher and his team, especially nicoboss)

Static - Imatrix

EXL3 (Thank you, I didn't expect this)

EXL3

Setting

Please use ChatML template

Reasoning is not necessary to turn on, but a nice feature to enable if you want to 'boost' your experience when roleplaying and multitasking (but more time consuming :/).

Reasoning token is <thinking> </thinking>. You could search how to enable thinking mode in internet. Note that in silly tarven, you should turn off Always add character's name to prompt in Context Formatting and Include names Never in Instruct Template.

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: trashpanda-org/QwQ-32B-Snowdrop-v0
    parameters:
      density: 0.9
      weight: 1
  - model: ArliAI/QwQ-32B-ArliAI-RpR-v3
    parameters:
      density: 0.8
      weight: 0.8
merge_method: dare_ties
base_model: Qwen/QwQ-32B
parameters:
  normalize: true
  rescale: true
tokenizer_source: Qwen/QwQ-32B
dtype: bfloat16