Create README.md

**UnitronX** is a 32k byte-level BPE tokenizer optimized for English, multilingual (ru/ar/de/es/fr/it/cs/hr/sr), and code.
It enforces safe merge boundaries (script changes, ZWJ, letter↔digit), preserves code identifiers, and uses
placeholder tokens for URLs/emails/paths/hashes/UUIDs/handles/hashtags.

language:
- en
- zh
- ja
- ko
- ru
- ar
- de
- fr
- es
- it
- cs
- hr
- sr
license: mit
library_name: tokenizers
tags:
- tokenizer
- bpe
- byte-level
- multilingual
- code
- unitronx
---

# {DISPLAY_NAME}


## Files
- `tokenizer.json`, `merges.txt`, `vocab.json`
- `tokenizer_config.json`, `special_tokens_map.json`
- `meta.json`
- *(optional)* `unitronx.tiktoken.json` (tiktoken-compatible)

## Load with Transformers

```python
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("YOUR_ORG_OR_USER/REPO_NAME")
print(tok.encode("don't split hyphen-words or fooBar123_id in code!"))

README.md

@@ -0,0 +1,18 @@
+---
+license: mit
+language:
+- ro
+- hr
+- en
+- de
+- es
+- sr
+- zh
+- ja
+- ko
+- fr
+- is
+- cs
+tags:
+- Tokenizer
+---