Hugging Face's logo

prithivMLmods
/
Gliese-OCR-7B-Post1.0-GGUF

Image-Text-to-Text
Transformers
GGUF
English
Chinese
qwen2_5_vl
ggml
llama.cpp
Document
VLM
KIE
OCR
VL
Camel
Openpdf
text-generation-inference
Extraction
Linking
Markdown
.Md
Document Digitization
Intelligent Document Processing (IDP)
Intelligent Word Recognition (IWR)
Optical Mark Recognition (OMR)
conversational
Model card Files
xet
Community
prithivMLmods commited on
Commit
8f1fe77
·
verified ·
1 Parent(s): 6d750fd

Upload folder using huggingface_hub

Browse files
Files changed (27) hide show
  1. .gitattributes +25 -14
  2. Gliese-OCR-7B-Post1.0.i1-IQ1_M.gguf +3 -0
  3. Gliese-OCR-7B-Post1.0.i1-IQ1_S.gguf +3 -0
  4. Gliese-OCR-7B-Post1.0.i1-IQ2_M.gguf +3 -0
  5. Gliese-OCR-7B-Post1.0.i1-IQ2_S.gguf +3 -0
  6. Gliese-OCR-7B-Post1.0.i1-IQ2_XS.gguf +3 -0
  7. Gliese-OCR-7B-Post1.0.i1-IQ2_XXS.gguf +3 -0
  8. Gliese-OCR-7B-Post1.0.i1-IQ3_M.gguf +3 -0
  9. Gliese-OCR-7B-Post1.0.i1-IQ3_S.gguf +3 -0
  10. Gliese-OCR-7B-Post1.0.i1-IQ3_XS.gguf +3 -0
  11. Gliese-OCR-7B-Post1.0.i1-IQ3_XXS.gguf +3 -0
  12. Gliese-OCR-7B-Post1.0.i1-IQ4_NL.gguf +3 -0
  13. Gliese-OCR-7B-Post1.0.i1-IQ4_XS.gguf +3 -0
  14. Gliese-OCR-7B-Post1.0.i1-Q2_K.gguf +3 -0
  15. Gliese-OCR-7B-Post1.0.i1-Q2_K_S.gguf +3 -0
  16. Gliese-OCR-7B-Post1.0.i1-Q3_K_L.gguf +3 -0
  17. Gliese-OCR-7B-Post1.0.i1-Q3_K_M.gguf +3 -0
  18. Gliese-OCR-7B-Post1.0.i1-Q3_K_S.gguf +3 -0
  19. Gliese-OCR-7B-Post1.0.i1-Q4_0.gguf +3 -0
  20. Gliese-OCR-7B-Post1.0.i1-Q4_1.gguf +3 -0
  21. Gliese-OCR-7B-Post1.0.i1-Q4_K_M.gguf +3 -0
  22. Gliese-OCR-7B-Post1.0.i1-Q4_K_S.gguf +3 -0
  23. Gliese-OCR-7B-Post1.0.i1-Q5_K_M.gguf +3 -0
  24. Gliese-OCR-7B-Post1.0.i1-Q5_K_S.gguf +3 -0
  25. Gliese-OCR-7B-Post1.0.i1-Q6_K.gguf +3 -0
  26. Gliese-OCR-7B-Post1.0.imatrix.gguf +3 -0
  27. README.md +106 -0
.gitattributes CHANGED
@@ -33,17 +33,28 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
- Gliese-OCR-7B-Post1.0.mmproj-f16.gguf filter=lfs diff=lfs merge=lfs -text
37
- Gliese-OCR-7B-Post1.0.mmproj-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
38
- Gliese-OCR-7B-Post1.0.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
39
- Gliese-OCR-7B-Post1.0.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
40
- Gliese-OCR-7B-Post1.0.f16.gguf filter=lfs diff=lfs merge=lfs -text
41
- Gliese-OCR-7B-Post1.0.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
42
- Gliese-OCR-7B-Post1.0.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
43
- Gliese-OCR-7B-Post1.0.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
- Gliese-OCR-7B-Post1.0.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
- Gliese-OCR-7B-Post1.0.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
46
- Gliese-OCR-7B-Post1.0.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
47
- Gliese-OCR-7B-Post1.0.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
48
- Gliese-OCR-7B-Post1.0.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
- Gliese-OCR-7B-Post1.0.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Gliese-OCR-7B-Post1.0.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Gliese-OCR-7B-Post1.0.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Gliese-OCR-7B-Post1.0.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Gliese-OCR-7B-Post1.0.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Gliese-OCR-7B-Post1.0.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Gliese-OCR-7B-Post1.0.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Gliese-OCR-7B-Post1.0.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Gliese-OCR-7B-Post1.0.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Gliese-OCR-7B-Post1.0.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Gliese-OCR-7B-Post1.0.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Gliese-OCR-7B-Post1.0.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Gliese-OCR-7B-Post1.0.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Gliese-OCR-7B-Post1.0.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Gliese-OCR-7B-Post1.0.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Gliese-OCR-7B-Post1.0.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Gliese-OCR-7B-Post1.0.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Gliese-OCR-7B-Post1.0.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Gliese-OCR-7B-Post1.0.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Gliese-OCR-7B-Post1.0.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Gliese-OCR-7B-Post1.0.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
56
+ Gliese-OCR-7B-Post1.0.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
57
+ Gliese-OCR-7B-Post1.0.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
58
+ Gliese-OCR-7B-Post1.0.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
59
+ Gliese-OCR-7B-Post1.0.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
60
+ Gliese-OCR-7B-Post1.0.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Gliese-OCR-7B-Post1.0.i1-IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b81d24d8ecaf4f943045d6db7d4d1c83d93fea71e4f9cc6418722de8322d1e99
3
+ size 2042195296
Gliese-OCR-7B-Post1.0.i1-IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f78e0705e6c8991f932544601c6de5c66d3b977627881b413f7795ef33f9db6
3
+ size 1903666528
Gliese-OCR-7B-Post1.0.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9a320049e9f1d6ffa6d130500c8f410288e4ccc4965c161b83cbade3906be99
3
+ size 2780341600
Gliese-OCR-7B-Post1.0.i1-IQ2_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bb02be850593dc24737253c0d678d7c3ab75e46656b3215320c9bb42a8465b0
3
+ size 2595636576
Gliese-OCR-7B-Post1.0.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f712d27cf0172c774979d08d493725e61d1fe0b32f10f1d2cfec7727164c019
3
+ size 2469021024
Gliese-OCR-7B-Post1.0.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:194b460ff2ba7d473993e96e12f35e723ba6291ed385961e32dbea45938d559f
3
+ size 2273076576
Gliese-OCR-7B-Post1.0.i1-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f0e34cc0fcfb62a4940f026c542c9cf1dcadec10a05a7247aae7902860ddc1b
3
+ size 3574011232
Gliese-OCR-7B-Post1.0.i1-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4be6a3e6b876c0176685a47ef45d3b7062a130ccfc4f3e4a1ff36c8579b66c24
3
+ size 3499191648
Gliese-OCR-7B-Post1.0.i1-IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ac150de25856ae82b67d73edd9c581737bd3ccfdf1252cccdaa8dd5ee321911
3
+ size 3346255200
Gliese-OCR-7B-Post1.0.i1-IQ3_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc11f59886662d335e16350768410a2ea5d722abca5c94a60ad710447f2f2747
3
+ size 3114513760
Gliese-OCR-7B-Post1.0.i1-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f5217188be1d953c0fa30045ba53e15e4c969fd1c9d3dfc95c7756698107aa9
3
+ size 4437812576
Gliese-OCR-7B-Post1.0.i1-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1eeca6a9c3c3606ebe205c8421b936d705e9df93aadf7e614b36b937e5ac2b72
3
+ size 4218471776
Gliese-OCR-7B-Post1.0.i1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28a188c8803287ac5862983fa0a8f42b67dbf078be14052cedc6758843e70046
3
+ size 3015939424
Gliese-OCR-7B-Post1.0.i1-Q2_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b1e60cb2f3920450b03650cfd42aa76e5a6da34d5b55b9b262f98365e314771
3
+ size 2834072928
Gliese-OCR-7B-Post1.0.i1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b70dc20fec9bd45bb9a16ee6fd88ca2178a8021f5350ea99108fa7511a8f07a
3
+ size 4088458592
Gliese-OCR-7B-Post1.0.i1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b9e9f4fadd69355381678ad7d531a5c14795ca955954bad493bee22bccb4e0a
3
+ size 3808390496
Gliese-OCR-7B-Post1.0.i1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f2e73ee5870bbfe19351a22857f661c4920adca4082a9a2500875541709aa84
3
+ size 3492367712
Gliese-OCR-7B-Post1.0.i1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:846d46b47b72467eafb07b3762b955c6c148363d121b22014e50dea9d9e8f2a8
3
+ size 4444120416
Gliese-OCR-7B-Post1.0.i1-Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:934781b457ae0711ec1351403a16b7caf992a2e0b901f53c40469d537802d2e5
3
+ size 4873282912
Gliese-OCR-7B-Post1.0.i1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22871707e0bf5116e1392ceeec84dc600918aae0d357f704f47a3e67f8feaa91
3
+ size 4683072864
Gliese-OCR-7B-Post1.0.i1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:735a17404d519f8e4c749049a7e72acc32a8f71e56675956118303037e41044b
3
+ size 4457768288
Gliese-OCR-7B-Post1.0.i1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f92298117fc9eb143a1ad7da3abd4aceab0ce79673c0aaaa2fbd3e2824bd3fb2
3
+ size 5444830560
Gliese-OCR-7B-Post1.0.i1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9e87beb84a8b15866cc8ddca1ef5d07348557b220efad29eb1f92624c32666d
3
+ size 5315175776
Gliese-OCR-7B-Post1.0.i1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9d99bf22787569b4255c34aba9cf9cdcd7eb6d17b361e2175683337688c816b
3
+ size 6254198112
Gliese-OCR-7B-Post1.0.imatrix.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:faa4ddd84a516cb3445688c6b25159923740aa5fceac2bb32fa9bd4728328f17
3
+ size 4560352
README.md ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: prithivMLmods/Gliese-OCR-7B-Post1.0
3
+ language:
4
+ - en
5
+ - zh
6
+ library_name: transformers
7
+ license: apache-2.0
8
+ mradermacher:
9
+ readme_rev: 1
10
+ quantized_by: mradermacher
11
+ tags:
12
+ - trl
13
+ - Document
14
+ - VLM
15
+ - KIE
16
+ - OCR
17
+ - VL
18
+ - Camel
19
+ - Openpdf
20
+ - text-generation-inference
21
+ - Extraction
22
+ - Linking
23
+ - Markdown
24
+ - .Md
25
+ - Document Digitization
26
+ - Intelligent Document Processing (IDP)
27
+ - Intelligent Word Recognition (IWR)
28
+ - Optical Mark Recognition (OMR)
29
+ ---
30
+ ## About
31
+
32
+ <!-- ### quantize_version: 2 -->
33
+ <!-- ### output_tensor_quantised: 1 -->
34
+ <!-- ### convert_type: hf -->
35
+ <!-- ### vocab_type: -->
36
+ <!-- ### tags: nicoboss -->
37
+ <!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
38
+ <!-- ### quants_skip: -->
39
+ <!-- ### skip_mmproj: -->
40
+ weighted/imatrix quants of https://huggingface.co/prithivMLmods/Gliese-OCR-7B-Post1.0
41
+
42
+ <!-- provided-files -->
43
+
44
+ ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Gliese-OCR-7B-Post1.0-i1-GGUF).***
45
+
46
+ static quants are available at https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-GGUF
47
+
48
+ **This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-GGUF).**
49
+ ## Usage
50
+
51
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
52
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
53
+ more details, including on how to concatenate multi-part files.
54
+
55
+ ## Provided Quants
56
+
57
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
58
+
59
+ | Link | Type | Size/GB | Notes |
60
+ |:-----|:-----|--------:|:------|
61
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own qwuants) |
62
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ1_S.gguf) | i1-IQ1_S | 2.0 | for the desperate |
63
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ1_M.gguf) | i1-IQ1_M | 2.1 | mostly desperate |
64
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.4 | |
65
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.6 | |
66
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ2_S.gguf) | i1-IQ2_S | 2.7 | |
67
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ2_M.gguf) | i1-IQ2_M | 2.9 | |
68
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.9 | very low quality |
69
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q2_K.gguf) | i1-Q2_K | 3.1 | IQ3_XXS probably better |
70
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.2 | lower quality |
71
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.4 | |
72
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.6 | IQ3_XS probably better |
73
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ3_S.gguf) | i1-IQ3_S | 3.6 | beats Q3_K* |
74
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ3_M.gguf) | i1-IQ3_M | 3.7 | |
75
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.9 | IQ3_S probably better |
76
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.2 | IQ3_M probably better |
77
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.3 | |
78
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.5 | prefer IQ4_XS |
79
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q4_0.gguf) | i1-Q4_0 | 4.5 | fast, low quality |
80
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.6 | optimal size/speed/quality |
81
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.8 | fast, recommended |
82
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q4_1.gguf) | i1-Q4_1 | 5.0 | |
83
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.4 | |
84
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.5 | |
85
+ | [GGUF](https://huggingface.co/mradermacher/Gliese-OCR-7B-Post1.0-i1-GGUF/resolve/main/Gliese-OCR-7B-Post1.0.i1-Q6_K.gguf) | i1-Q6_K | 6.4 | practically like static Q6_K |
86
+
87
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
88
+ types (lower is better):
89
+
90
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
91
+
92
+ And here are Artefact2's thoughts on the matter:
93
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
94
+
95
+ ## FAQ / Model Request
96
+
97
+ See https://huggingface.co/mradermacher/model_requests for some answers to
98
+ questions you might have and/or if you want some other model quantized.
99
+
100
+ ## Thanks
101
+
102
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
103
+ me use its servers and providing upgrades to my workstation to enable
104
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
105
+
106
+ <!-- end -->