Article
Türkçe vergisi: açık LLM'lerde Türkçenin servis maliyeti
serkanpeker
• • 1Serkan Peker PRO
serkanpeker
AI & ML interests
Sovereign & Air-Gapped AI Architectures, Large-Scale Distributed Pre-training & Distillation (Slurm / InfiniBand HPC), Turkish LLM & Custom Tokenizer Engineering, High-Throughput Inference & AI Gateways (vLLM), End-to-End MLOps & Active Learning Computer Vision Pipelines, and Autonomous Multi-Agent Reasoning Systems with Enterprise RAG.
Recent Activity
repliedto their post about 11 hours ago
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: https://huggingface.co/datasets/serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-tax repliedto their post about 16 hours ago
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: https://huggingface.co/datasets/serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-tax repliedto their post about 16 hours ago
Generating the same text in Turkish took an estimated 1.45 to 1.62 times the GPU time of English on four open models (one H200 each, vLLM 0.33, 256 concurrent requests, prefix caching off).
Token count explains most of it. On 2,009 parallel FLORES+ sentences, six of seven tokenizers need 1.36 to 1.62 times as many tokens for Turkish as for English; Kumru-2B needs 0.64 times as many.
Dynamic FP8 in vLLM raised perplexity on FLORES+ by 0.30% to 0.81% in Turkish and 0.27% to 0.39% in English; per token, only Turkish-Gemma-9b-T1 showed a clearly larger rise in Turkish. For GGUF at Q4_K_M on two models, using an importance matrix at all mattered far more than calibrating it on Turkish.
Article (English): https://huggingface.co/blog/serkanpeker/turkish-tax
Data: https://huggingface.co/datasets/serkanpeker/turkish-tax
Code: https://github.com/speker/turkish-taxOrganizations
published an article about 18 hours ago
published an article 2 days ago
Article
The Turkish tax: what serving Turkish costs on open LLMs
serkanpeker
•