Why Tokenizer Vocabulary Size Is a Hardware Choice

Before an AI model reads a word, a pre-trained tokenizer chops the text into pieces using byte pair encoding. While vocabulary size is often treated as an arbitrary convention, its optimal dimension varies dramatically depending on whether a model runs on-device or across datacenter GPUs. Looking into merge counts, multilingual additions from Llama 3 to Gemini, and recent findings on Zipf's law, this analysis reveals why vocabulary tuning is fundamentally an infrastructure optimization. Key sources: • Sennrich, Haddow & Birch, Neural Machine Translation of Rare Words with Subword Units, ACL 2016, pp. 1715 — https://aclanthology.org/P16-1162.pdf • Llama Team, AI @ Meta, The Llama 3 Herd of Models, dated 2024-07-23 . Retrieved 2026-11-08. — https://arxiv.org/abs/2407.21783v3 • OpenAI, gpt-oss-120b model card, created 2025-08-04, last modified 2025-08-26 . Retrieved 2026-11-08. — https://huggingface.co/openai/gpt-oss-120b Episode 6503 — myweirdprompts.com/n/6503 (permanent short link: myweirdprompts.com/129808) Listen on Android: MWP FM Player — myweirdprompts.com/apk