MNN-Pocket-0.3B-RU — a Russian continuation model on a 0.3B base

params tokens context gguf license status

Status: v1 released. 1.2 B tokens, one epoch, weights in this repository, GGUF in gguf/. Training log, checkpoints and the evaluation script live in the private repo Rahspide/mnn-pocket-ru-run.

A continued-pretraining run of taobao-mnn/MNN-Pocket-0.3B-Base that teaches it Russian while keeping English and Chinese readable. The keyboard is the use case that set the size and latency budget.

It is trained as a plain causal LM on raw text, so it continues a typed prefix instead of answering it. For dialogue there is a separate checkpoint — Rahspide/MNN-Pocket-0.3B-RU-Chat.

Two models, two jobs

One base, two jobs

This model — MNN-Pocket-0.3B-RU Sister — MNN-Pocket-0.3B-RU-Chat
Job continue text follow instructions
Input a prefix, often mid-sentence a system + user message
Output the plausible continuation an answer
Trained with continued pretraining, 1.2 B tokens of raw text SFT on top of this model, 50 M tokens
Choose it for keyboard suggestions, autocomplete, prefix completion chat, Q&A, short rewrites
Do not choose it for assistant behaviour: it continues your question instead of answering autocomplete: it answers instead of continuing

The two tasks reward opposite behaviour. Trained on both jobs, one checkpoint does each badly: it answers when it should continue, and continues when it should answer.

Specs

Base taobao-mnn/MNN-Pocket-0.3B-Base (Qwen3 architecture). 302.8 M unique parameters here: 58.7 M embeddings + 244.1 M others. The base file lists 361.5 M because it stores lm_head as a separate copy of the tied matrix
Tokenizer 65,536 tokens, byte-level BPE, `<
Training tokens 1.2 B — 0.9 B ru + 0.204 B en + 0.096 B zh
Method full fine-tune, 1 epoch, no LoRA
Steps 36,621 × 32,768 tokens
Hardware one Colab T4 16 GB, 4.8 k tok/s, ~69 h
Precision fp32 master weights, fp16 autocast (T4 has no usable bf16)

Results

Bits per character: base vs v1

100 k held-out tokens per language, identical settings for both checkpoints:

Lang ppl base → v1 bits/char base → v1 top-1 base → v1 top-5 base → v1
ru 10.42 → 5.05 1.625 → 1.123 (−31%) 46.2% → 61.0% 69.3% → 80.6%
en 21.85 → 41.16 1.016 → 1.225 (+21%) 39.9% → 35.5% 64.1% → 58.7%
zh 23.95 → 51.86 3.274 → 4.071 (+24%) 41.9% → 37.7% 62.0% → 55.6%

The published weights were re-downloaded and re-evaluated independently (CPU, fp32, 20 k tokens per language) and reproduce the same ordering: ru 1.022 against base 1.519, en 1.337 against 1.124, zh 4.053 against 3.393.

Russian landed past the target (1.123 against a 1.40 gate). English and Chinese regressed 21–24% instead of the ≤10% the recipe aimed for. a 25% replay mix is too thin at 0.36 B when the target language is tokenised this badly. A second pass with a 50/25/25 mix at lr 5e-5 was started, got through ~50 M of a planned 200 M tokens, and changed nothing measurable on English (1.337 → 1.337 on the same eval), so it is not published. Treat v1 as Russian-first, with working but measurably weaker English and Chinese.

The base vocabulary is itself a pruned set: the technical report describes cutting Qwen3's ~151 k tokens down to ~64 k by frequency while keeping BPE dependencies and byte fallback. Poor Cyrillic coverage is what that pruning cost.

Why the tokenizer shapes the recipe

Russian characters per token, base vs extended vocabulary

The base vocabulary is a distilled 65 k set with almost no Cyrillic merges, and it is byte-level: the vocabulary stores UTF-8 bytes remapped to latin-1, so convert_ids_to_tokens prints С where the text says С. That is a display artifact; decode returns the word. The count is what matters:

Word Characters Tokens Pieces
Спасибо 7 6 С · п · ас · и · б · о
привет 6 3 пр · ив · ет
покупать 8 4 п · ок · уп · ать
Ёжик 4 4 Ё lands in two separate byte tokens

Because the vocabulary is byte-level, a token can end in the middle of a character: in Ёжик and ЩУКА the uppercase letters are cut into single bytes. Decoding the whole sequence is still exact, but a streaming client must hold an incomplete UTF-8 sequence back instead of printing it. llama.cpp and transformers buffer it for you; a hand-rolled decoder has to do the same or the output shows replacement characters.

Tokenizer RU chars/token RU tokens per word EN chars/token
MNN-Pocket / Qwen3-distilled (65,536) 1.75–1.85 (2.08 on dialogue text) ≈3.3 3.78
+8,192 Cyrillic merges (73,728) 3.10 ≈1.8 —
+16,384 Cyrillic merges (81,920) 3.39 ≈1.6 —
ruGPT3-small (50 k, Russian BPE) 4.2 ≈1.3 —

Consequences that shaped the plan:

  • the same 1.2 B-token budget buys 2.2 B Russian characters (340 M words), roughly 2.2× less text than a Russian-native tokenizer would fit;
  • on a phone one suggested Russian word costs ~3 decode steps instead of ~1, which matters for a keyboard latency budget;
  • vocabulary surgery was measured but kept out of v1, so v1 stays tokenizer-compatible with the base and drops into the existing MNN / GGUF toolchain. An extended-vocabulary artefact (model_ext, 73,728 rows, embeddings initialised as the mean of their child tokens) exists in the private run repo but is not part of this release.

Recipe

  • Full fine-tune from MNN-Pocket-0.3B-Base, fp32 master weights with fp16 autocast.
  • Sequence length 1024, random windows over a packed uint16 token stream, <|endoftext|> between documents.
  • 32,768 tokens per optimizer step (micro-batch 4 × 1024, gradient accumulation 8).
  • AdamW, β = (0.9, 0.95), weight decay 0.1, gradient clip 1.0, lr 2e-4 with 1% linear warmup and cosine decay.
  • One epoch over 1.2 B tokens = 36,621 steps.
  • Checkpoint every 500 steps to a private HF repo; training auto-resumes from the last checkpoint after a session restart.

Data

All sources are public Hugging Face datasets. * marks the validation split: the first 100 k tokens of each stream are held out before training starts, so validation text is never seen.

Language Dataset Config Tokens Share
ru HuggingFaceFW/fineweb-2 rus_Cyrl 0.90 B (100 k held out*) 75%
en HuggingFaceFW/fineweb sample-100BT 0.204 B (100 k held out*) 17%
zh HuggingFaceFW/fineweb-2 cmn_Hani 0.096 B (100 k held out*) 8%

Licences: FineWeb / FineWeb-2 are ODC-BY 1.0. No scraped Telegram dumps or other unlicensed chat corpora were used, so the resulting weights stay publishable.

Builds

GGUF sizes

Build Size Download Note
Q8_0 310 MB MNN-Pocket-0.3B-RU-Q8_0.gguf near-lossless, for desktop
Q4_K_M 223 MB MNN-Pocket-0.3B-RU-Q4_K_M.gguf the size to ship on a phone
Q4_1 208 MB MNN-Pocket-0.3B-RU-Q4_1.gguf fastest generation on CPU
MNN INT4 165 MB mnn/MNN-Pocket-0.3B-RU-INT4 MNN runtime, Android

Both files sit under gguf/ with the quant tag in the name, which is what llama.cpp matches on:

llama-cli -hf Rahspide/MNN-Pocket-0.3B-RU:Q4_K_M -p "Сегодня я" -n 40

Converted with llama.cpp convert_hf_to_gguf.py (f16) and quantised with llama-quantize. Measured on a 16-thread x86 CPU, no GPU involved:

Build Prompt processing Generation
Q4_K_M ~410 tok/s ~130 tok/s
Q8_0 ~525 tok/s ~99 tok/s

Phone figures will be several times lower; these are desktop-CPU numbers. Measured back to back on the same machine, Q4_1 generates 62.6 tok/s against 54.3 for Q4_K_M, but processes prompts slower (199 vs 251 tok/s).

The MNN build runs through MNN's own runtime. Runtime sources: alibaba/MNN, the LLM export and inference guide, and the official Android app apps/Android/MnnLlmChat. Build llm_demo from the MNN repository with -DMNN_BUILD_LLM=ON, then point it at the folder:

cd mnn/MNN-Pocket-0.3B-RU-INT4 && llm_demo config.json prompt.txt

165 MB on disk, ~354 tok/s prefill and ~77 tok/s decode on 12 CPU threads here. llm_config.json carries no chat template on purpose: this checkpoint continues the prompt, so the runtime should not wrap it in a dialogue format.

Two conversion quirks, both already fixed in this repository's tokenizer_config.json.

  • the base pre-tokenizer does not hash-match in current llama.cpp, so the Qwen pre-tokenizer has to be pinned by hand (qwen2);
  • extra_special_tokens must not be a list, or transformers v4 refuses to load the tokenizer.

Reproduce

Everything below runs on CPU. The numbers in this card came from these paths.

# bits/char, ppl and top-k against the base, 20k tokens per language
python bench/eval_bpc.py --model Rahspide/MNN-Pocket-0.3B-RU --base taobao-mnn/MNN-Pocket-0.3B-Base \
    --dataset HuggingFaceFW/fineweb-2 --config rus_Cyrl --tokens 20000

# throughput and a smoke generation on the quantised build
LLAMA_CLI=/path/to/llama-cli bash bench/bench_cpu.sh gguf/MNN-Pocket-0.3B-RU-Q4_K_M.gguf

bench/eval_bpc.py reports the same three metrics as the tables above (ppl, bits/char, top-1/top-5) and exits non-zero when the model loses more than a given --max-bpc-regression against the base, so it can run as a regression gate in CI. bench/bench_cpu.sh prints prompt and generation tok/s and checks that generation stops at <|endoftext|>.

Intended use

  • On-device next-word and short-phrase suggestion in a keyboard or IME, Russian first, English and Chinese from the same checkpoint (a keyboard can pick the language from the current word or the layout).
  • Text completion, prefix continuation and autocomplete for Russian anywhere a ~220 MB quantised model has to run locally: chat composers, note apps, search fields, terminal prompts.
  • Any pipeline that needs a small causal LM fluent in Russian that can still read English and Chinese, for example tagging, short rewriting suggestions, or generating Russian seed text.

Not intended for: instruction following, factual question answering, code, long-form reasoning, or translation. This is a 0.36 B model trained for 1.2 B tokens; its knowledge is narrow by construction. The language mix is a deliberate trade: Russian is the target, English and Chinese are kept from degrading, neither is improved.

Limitations

  • Russian is tokenised near character level (see above): slower decoding and a larger effective cost per Russian word than the same model pays for English.
  • Token boundaries do not follow character boundaries. Rare and uppercase Cyrillic letters are split into single bytes, so a streaming decoder must buffer partial UTF-8 sequences; llama.cpp and transformers already do this.
  • Web text only. Register is mostly written/informal blog Russian; no dialogue corpora, no post-training.
  • Capacity is 0.36 B. Expect weak facts, weak arithmetic, and drift on long generations.
  • English and Chinese regressed 21–24% against the base. The 25% replay mix limits the damage; it does not remove it.
  • The ChatML template from the base is inherited but untrained here. Do not use this checkpoint in chat mode.

Releases

Date Version What changed
2026-11-10 v1.0 1.2 B-token continued pretraining released: bf16 weights, GGUF Q8_0 and Q4_K_M, measured ru/en/zh bits-per-char
2026-11-11 v1.1 MNN INT4 build for the MNN runtime: 165 MB, ~77 tok/s decode on 12 CPU threads
2026-11-10 v2.0 (sister) Assistant checkpoint released: 50 M-token SFT on top of v1, weights and GGUF

Licence and attribution

Weights: Apache-2.0, inherited from the base model. Data: ODC-BY 1.0 (FineWeb, FineWeb-2). If you use this model, credit the base release:

@misc{mnn_pocket_03b,
  title  = {MNN-Pocket-0.3B},
  author = {Taobao MNN team},
  year   = {2026},
  url    = {https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B}
}
Downloads last month
1,311
Safetensors
Model size
0.3B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Rahspide/MNN-Pocket-0.3B-RU

Quantized
(1)
this model
Quantizations
2 models

Datasets used to train Rahspide/MNN-Pocket-0.3B-RU

Paper for Rahspide/MNN-Pocket-0.3B-RU