Models for local deployment - a yjernite Collection

yjernite 's Collections

Text embedding models I use

super-smol to fine-tune

Models for local deployment

Inference-supported production models

Document processing

Privacy

Models for local deployment

updated 6 days ago

Qwen/Qwen3-4B-Instruct-2507

Text Generation • 4B • Updated Sep 17 • 6.26M • • 533
unsloth/Qwen3-4B-Instruct-2507-GGUF

4B • Updated Aug 20 • 63.7k • 103
HuggingFaceTB/SmolLM3-3B

Text Generation • 3B • Updated Sep 10 • 90.8k • • 834
unsloth/SmolLM3-3B-GGUF

3B • Updated Jul 8 • 2.28k • 17
Qwen/Qwen3-Embedding-4B-GGUF

4B • Updated Aug 3 • 50.1k • 74

Note ./build/bin/llama-server -hf Qwen/Qwen3-Embedding-4B-GGUF:Q6_K --port 8080 --pooling last --verbose-prompt --parallel 4 --batch-size 8 --embedding --ctx-size 8000 --cache-type-k q4_0 --gpu-layers 36
Qwen/Qwen3-Embedding-0.6B

Feature Extraction • 0.6B • Updated Jun 20 • 4.29M • • 770
google/gemma-3-270m

Text Generation • 0.3B • Updated Aug 14 • 70.8k • 920
ByteDance/Ouro-1.4B

Text Generation • Updated 23 days ago • 11k • 56