That's a really useful breakdown ... the 1.36-1.48x token cost for Turkish output is exactly the kind of number that's hard to find in public benchmarks. The point about reasoning traces staying in English while only the final answer translates is smart ik that's where the real savings would be.
On the alignment side, we're seeing something similar: models aligned on Turkish domain documents handle the terminology natively without the translate-in/out overhead. Early days, but the token cost comparison would be interesting to measure head-to-head. If you're curious, check out nugen.in .... happy to share an invite code with free credits if you want to test it on Turkish tax content.
Thanks for the detailed response!
🏗️ Building on HF
Sayali Pinge
saayoo
·
AI & ML interests
None yet
Recent Activity
repliedto serkanpeker's post 9 minutes ago
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: https://huggingface.co/datasets/serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-tax repliedto serkanpeker's post about 20 hours ago
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: https://huggingface.co/datasets/serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-taxOrganizations
None yet
replied to serkanpeker's post 9 minutes ago
replied to serkanpeker's post about 20 hours ago
Solid analysis — the breakdown of tokenizer overhead vs. load-dependent cost is really useful, especially the counterintuitive result that scaling Turkish output actually dropped the cost ratio. The Kumru-2B outlier at 0.64x is interesting too — suggests purpose-built tokenizers can erase the tax entirely. Tho curious about a downstream effect: does this cost differential push teams toward English prompts even for Turkish domain tasks? We work on domain alignment for non-English models (disclosure: I'm at Nugen) and have seen teams default to "translate to English → run inference → translate back" to avoid the overhead — which trades cost for accuracy loss on domain-specific content.
Has that pattern come up in your benchmarking? Would be interesting to see whether that workaround's quality tradeoff is worse than paying the 1.45x.