Add PyLaia model trained on NorHand v3 (#1)

- Add PyLaia model trained on NorHand v3 (748a8a0a6aee6fce2eb298c3d4bfbd78dbdd2c0a)

README.md

@@ -1,3 +1,44 @@
 ---
+library_name: PyLaia
 license: mit
+tags:
+- PyLaia
+- PyTorch
+- Handwritten text recognition
+metrics:
+- CER
+- WER
+language:
+- 'no'
 ---
+
+# NorHand v3 handwritten text recognition
+
+This model performs Handwritten Text Recognition in Norwegian on historical documents.
+
+## Model description
+
+The model was trained using the PyLaia library on the [NorHand v3 dataset](https://zenodo.org/records/10255840).
+
+For training, text-lines were resized with a fixed height of 128 pixels, keeping the original aspect ratio. Vertical lines are discarded.
+
+| split | N lines | N horizontal lines |
+| ----- | ------: | -----------------: |
+| train | 224,173 |            223,971 |
+| val   |  22,828 |             22,811 |
+| test  |   1,573 |              1,573 |
+
+An external 6-gram character language model can be used to improve recognition. The language model is trained on the text from the NorHand v3 training set.
+
+## Evaluation results
+
+The model achieves the following results:
+
+| set   | Language model | CER (%)    | WER (%) | N lines   |
+|:------|:---------------|:----------:|:-------:|----------:|
+| test  | no             | 7.52       |   22.99 |     1,573 |
+| test  | yes            | 6.36       |   18.11 |     1,573 |
+
+## How to use
+
+Please refer to the [documentation](https://atr.pages.teklia.com/pylaia/).

language_model.arpa.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43b6a1f11ec090c604566d86058ced36507510b1ec74fb9d1e0c11c5d10a0bae
+size 54823896

lexicon.txt

@@ -0,0 +1,169 @@
+<ctc> <ctc>
+! !
+" "
+# #
+$ $
+% %
+& &
+' '
+( (
+) )
+* *
++ +
+, ,
+- -
+. .
+/ /
+0 0
+1 1
+2 2
+3 3
+4 4
+5 5
+6 6
+7 7
+8 8
+9 9
+: :
+; ;
+< <
+= =
+> >
+? ?
+A A
+B B
+C C
+D D
+E E
+F F
+G G
+H H
+I I
+J J
+K K
+L L
+M M
+N N
+O O
+P P
+Q Q
+R R
+S S
+T T
+U U
+V V
+W W
+X X
+Y Y
+Z Z
+[ [
+\ \
+] ]
+_ _
+` `
+a a
+b b
+c c
+d d
+e e
+f f
+g g
+h h
+i i
+j j
+k k
+l l
+m m
+n n
+o o
+p p
+q q
+r r
+s s
+t t
+u u
+v v
+w w
+x x
+y y
+z z
+{ {
+| |
+} }
+£ £
+§ §
+« «
+¬ ¬
+° °
+´ ´
+¹ ¹
+º º
+» »
+¼ ¼
+½ ½
+¾ ¾
+Á Á
+Ä Ä
+Å Å
+Æ Æ
+É É
+Ö Ö
+Ø Ø
+Ü Ü
+Þ Þ
+ß ß
+à à
+á á
+â â
+ä ä
+å å
+æ æ
+ç ç
+è è
+é é
+ê ê
+ë ë
+í í
+ï ï
+ð ð
+ñ ñ
+ò ò
+ó ó
+ô ô
+ö ö
+÷ ÷
+ø ø
+ù ù
+ú ú
+û û
+ü ü
+ý ý
+þ þ
+œ œ
+ɔ ɔ
+ː ː
+˚ ˚
+̄ ̄
+Ֆ Ֆ
+ẞ ẞ
+– –
+— —
+‘ ‘
+’ ’
+“ “
+” ”
+„ „
+… …
+⁄ ⁄
+⁰ ⁰
+⁴ ⁴
+⁶ ⁶
+₀ ₀
+₁ ₁
+₅ ₅
+⅓ ⅓
+⅛ ⅛
+♂ ♂
+ 
+<unk> <unk>
+<space> <space>

syms.txt

@@ -0,0 +1,169 @@
+<ctc> 0
+! 1
+" 2
+# 3
+$ 4
+% 5
+& 6
+' 7
+( 8
+) 9
+* 10
++ 11
+, 12
+- 13
+. 14
+/ 15
+0 16
+1 17
+2 18
+3 19
+4 20
+5 21
+6 22
+7 23
+8 24
+9 25
+: 26
+; 27
+< 28
+= 29
+> 30
+? 31
+A 32
+B 33
+C 34
+D 35
+E 36
+F 37
+G 38
+H 39
+I 40
+J 41
+K 42
+L 43
+M 44
+N 45
+O 46
+P 47
+Q 48
+R 49
+S 50
+T 51
+U 52
+V 53
+W 54
+X 55
+Y 56
+Z 57
+[ 58
+\ 59
+] 60
+_ 61
+` 62
+a 63
+b 64
+c 65
+d 66
+e 67
+f 68
+g 69
+h 70
+i 71
+j 72
+k 73
+l 74
+m 75
+n 76
+o 77
+p 78
+q 79
+r 80
+s 81
+t 82
+u 83
+v 84
+w 85
+x 86
+y 87
+z 88
+{ 89
+| 90
+} 91
+£ 92
+§ 93
+« 94
+¬ 95
+° 96
+´ 97
+¹ 98
+º 99
+» 100
+¼ 101
+½ 102
+¾ 103
+Á 104
+Ä 105
+Å 106
+Æ 107
+É 108
+Ö 109
+Ø 110
+Ü 111
+Þ 112
+ß 113
+à 114
+á 115
+â 116
+ä 117
+å 118
+æ 119
+ç 120
+è 121
+é 122
+ê 123
+ë 124
+í 125
+ï 126
+ð 127
+ñ 128
+ò 129
+ó 130
+ô 131
+ö 132
+÷ 133
+ø 134
+ù 135
+ú 136
+û 137
+ü 138
+ý 139
+þ 140
+œ 141
+ɔ 142
+ː 143
+˚ 144
+̄ 145
+Ֆ 146
+ẞ 147
+– 148
+— 149
+‘ 150
+’ 151
+“ 152
+” 153
+„ 154
+… 155
+⁄ 156
+⁰ 157
+⁴ 158
+⁶ 159
+₀ 160
+₁ 161
+₅ 162
+⅓ 163
+⅛ 164
+♂ 165
+ 166
+<unk> 167
+<space> 168

tokens.txt

@@ -0,0 +1,169 @@
+<ctc>
+!
+"
+#
+$
+%
+&
+'
+(
+)
+*
++
+,
+-
+.
+/
+0
+1
+2
+3
+4
+5
+6
+7
+8
+9
+:
+;
+<
+=
+>
+?
+A
+B
+C
+D
+E
+F
+G
+H
+I
+J
+K
+L
+M
+N
+O
+P
+Q
+R
+S
+T
+U
+V
+W
+X
+Y
+Z
+[
+\
+]
+_
+`
+a
+b
+c
+d
+e
+f
+g
+h
+i
+j
+k
+l
+m
+n
+o
+p
+q
+r
+s
+t
+u
+v
+w
+x
+y
+z
+{
+|
+}
+£
+§
+«
+¬
+°
+´
+¹
+º
+»
+¼
+½
+¾
+Á
+Ä
+Å
+Æ
+É
+Ö
+Ø
+Ü
+Þ
+ß
+à
+á
+â
+ä
+å
+æ
+ç
+è
+é
+ê
+ë
+í
+ï
+ð
+ñ
+ò
+ó
+ô
+ö
+÷
+ø
+ù
+ú
+û
+ü
+ý
+þ
+œ
+ɔ
+ː
+˚
+̄
+Ֆ
+ẞ
+–
+—
+‘
+’
+“
+”
+„
+…
+⁄
+⁰
+⁴
+⁶
+₀
+₁
+₅
+⅓
+⅛
+♂
+
+<unk>
+<space>

weights.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e49b4233c9bea4cfbe9a35e239e1c25421ceed9b194b07f6ec3d96e1b1a69fc
+size 43033372