Instructions to use allenai/Molmo-72B-0924 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use allenai/Molmo-72B-0924 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="allenai/Molmo-72B-0924", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("allenai/Molmo-72B-0924", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use allenai/Molmo-72B-0924 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "allenai/Molmo-72B-0924"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Molmo-72B-0924",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/allenai/Molmo-72B-0924

SGLang

How to use allenai/Molmo-72B-0924 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "allenai/Molmo-72B-0924" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Molmo-72B-0924",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "allenai/Molmo-72B-0924" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "allenai/Molmo-72B-0924",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use allenai/Molmo-72B-0924 with Docker Model Runner:
```
docker model run hf.co/allenai/Molmo-72B-0924
```

Fine Tuning Molmo

by chrishoertnagl - opened Sep 26, 2024

Discussion

chrishoertnagl

Sep 26, 2024

Really cool work!
Is it possible to futher fine-tune the model e.g. on an OCR dataset or something? Are there any notebooks or examples you could share?

chrisc36

Ai2 org Sep 26, 2024

Thank you! We will release full training code for these models but we do not have that ready yet. For now these models do support a forward pass and backpropagation so you can place them into a standard torch training loop yourself.

mavericklsd

Nov 14, 2024

Any update about releasing the fine tuning molmo script? thanks in advance

amanrangapur

Nov 14, 2024

Hey @mavericklsd , we are planning to release the code along with checkpoints and weights by the end of November.

delip

Nov 25, 2024

@amanrangapur don't want to be that guy, but can't help it. Is this released yet?

amanrangapur

Nov 25, 2024

Hey @delip , we're planning to release this week. Stay tuned.

delip

Nov 25, 2024

Thanks @amanrangapur and MolMo team!

scm-aiml

Nov 29, 2024

@amanrangapur I saw the Molmo dataset(s) were released. Is there still a plan to release the training code on GitHub as well this week?

amanrangapur

Nov 29, 2024

Hey @scm-aiml ! Since it’s the holidays here, we’ll be releasing next week!

amanrangapur

Dec 5, 2024

•

edited Dec 18, 2024

Hey @scm-aiml @delip , the wait is over: https://github.com/allenai/molmo

syazvinski

Dec 18, 2024

Any update on a fine-tuning script? Ive been patiently waiting quite some time. Updates would be appreciated. 🙃

amanrangapur

Dec 18, 2024

•

edited Dec 18, 2024

Hey @syazvinski , We are not releasing any custom fine-tuning script until next month. I will keep you posted if there is anything around.

chisarie

Jan 13, 2025

Hi @amanrangapur , thank you for the great work. Do you know if there is any timeline for the release of a fine-tuning script? Just to know whether I should wait, or find an alternative approach/model for my project. Thank you!

amanrangapur

Jan 13, 2025

Hey @chisarie , we have released bunch of training code and stuff on GitHub, and you can fine-tune the model on custom data, you probably need to work on Data Loaders and other things.. As of now we're not on releasing anything like custom fine-tuning script.

chisarie

Jan 14, 2025

All right, thank you for the prompt response!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment