ace-1
/

mgpt2-tokenizer

Model card Files Files and versions

mgpt2-tokenizer

Ctrl+K

Ctrl+K

1 contributor

History: 8 commits

ace-1's picture

Update tokenizer artifact (verified corpus retrain) + eval metrics

68abf7e verified 6 days ago

tokenizer
Fix transformers v5 auto_map + HF init about 1 month ago
.gitattributes

1.52 kB
initial commit about 2 months ago
README.md

1.8 kB
Publish mgpt2 tokenizer (GPT-2 exact merges) + eval metrics about 1 month ago
added_tokens.json

29 Bytes
Upload mgpt2 tokenizer about 1 month ago
evaluation.json

7.57 kB
Update tokenizer artifact (verified corpus retrain) + eval metrics 6 days ago
special_tokens_map.json

35 Bytes
Upload mgpt2 tokenizer about 2 months ago
tokenization_mgpt2.py

80 Bytes
Publish mgpt2 tokenizer (GPT-2 exact merges) + eval metrics about 1 month ago
tokenizer.model

464 kB
xet

Update tokenizer artifact (verified corpus retrain) + eval metrics 6 days ago
tokenizer.vocab

1.76 MB
Update tokenizer artifact (verified corpus retrain) + eval metrics 6 days ago
tokenizer_config.json

530 Bytes
Fix transformers v5 auto_map + HF init about 1 month ago