Instructions to use ChuckMcSneed/ArcaneEntanglement-model64-70b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ChuckMcSneed/ArcaneEntanglement-model64-70b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ChuckMcSneed/ArcaneEntanglement-model64-70b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ChuckMcSneed/ArcaneEntanglement-model64-70b")
model = AutoModelForCausalLM.from_pretrained("ChuckMcSneed/ArcaneEntanglement-model64-70b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ChuckMcSneed/ArcaneEntanglement-model64-70b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ChuckMcSneed/ArcaneEntanglement-model64-70b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChuckMcSneed/ArcaneEntanglement-model64-70b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ChuckMcSneed/ArcaneEntanglement-model64-70b

SGLang

How to use ChuckMcSneed/ArcaneEntanglement-model64-70b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ChuckMcSneed/ArcaneEntanglement-model64-70b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChuckMcSneed/ArcaneEntanglement-model64-70b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ChuckMcSneed/ArcaneEntanglement-model64-70b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ChuckMcSneed/ArcaneEntanglement-model64-70b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ChuckMcSneed/ArcaneEntanglement-model64-70b with Docker Model Runner:
```
docker model run hf.co/ChuckMcSneed/ArcaneEntanglement-model64-70b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

What is this

My experiment. Continuation of Benchmaxxxer series (meme models), but a bit more serious. Performs high on my benchmark and on huggingface benchmark, moderately-high in practice. Worth trying? Yeah. It is on the gooder side.

Observations

GPTslop: medium-low. Avoid at all costs or it won't stop generating it though.
Writing style: difficult to describe. Not the usual stuff. A bit of an autopilot like thing, if you write your usual lazy "ahh ahh mistress" it can give you a whole page of good text in return. High.
Censorship: if you can handle Xwin, you can handle this model. Medium.
Optimism: medium-low.
Violence: medium-low.
Intelligence: medium.
Creativity: medium-high.
Doesn't like high temperature. Keep below 1.5.

Prompt format

Vicuna or Alpaca.

Merge Details

This is a merge of pre-trained language models created using mergekit.

This model was merged using the linear merge method.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: spicyboros
    parameters:
      weight: [0.093732305,0.403220342,0.055438423,0.043830778,0.054189303,0.081136828]
  - model: xwin
    parameters:
      weight: [0.398943486,0.042069007,0.161586088,0.470977297,0.389315704,0.416739102]
  - model: euryale
    parameters:
      weight: [0.061483013,0.079698633,0.043067724,0.00202751,0.132183868,0.36578003]
  - model: dolphin
    parameters:
      weight: [0.427942847,0.391488452,0.442164138,0,0,0.002174793]
  - model: wizard
    parameters:
      weight: [0.017898349,0.083523566,0.297743627,0.175345857,0.071770095,0.134169247]
  - model: WinterGoddess
    parameters:
      weight: [0,0,0,0.30781856,0.352541031,0]
merge_method: linear
dtype: float16
tokenizer_source: base

Benchmarks

NeoEvalPlusN_benchmark

My meme benchmark.

Name	B	C	D	S	P	total	BCD	SP
ChuckMcSneed/PMaxxxer-v1-70b	3	1	1	6.75	4.75	16.5	5	11.5
ChuckMcSneed/SMaxxxer-v1-70b	2	1	0	7.25	4.25	14.5	3	11.5
ChuckMcSneed/ArcaneEntanglement-model64-70b	3	2	1	7.25	6	19.25	6	13.25

Absurdly high. That's what happens when you optimize the merges for a benchmark.

Open LLM Leaderboard Evaluation Results

Leaderboard on Huggingface

Model	Average	ARC	HellaSwag	MMLU	TruthfulQA	Winogrande	GSM8K
ChuckMcSneed/ArcaneEntanglement-model64-70b	72.79	71.42	87.96	70.83	60.53	83.03	63
ChuckMcSneed/PMaxxxer-v1-70b	72.41	71.08	87.88	70.39	59.77	82.64	62.7
ChuckMcSneed/SMaxxxer-v1-70b	72.23	70.65	88.02	70.55	60.7	82.87	60.58

This model is simply superior to my other meme models here.

Downloads last month: 102

Safetensors

Model size

69B params

Tensor type

F16

Model tree for ChuckMcSneed/ArcaneEntanglement-model64-70b

Quantizations

2 models

Paper for ChuckMcSneed/ArcaneEntanglement-model64-70b

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Paper • 2203.05482 • Published Mar 10, 2022 • 8