paperweight/tinyfix

Spell check can't tell that “form” was meant to be “from”,
since both are real words. This 2.1 MB model can.

Try the demo →   ·   Write with it in offline.page →

tinyfix reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with ONNX Runtime Web, so nothing you type is sent anywhere.

What it does

You type	It becomes
I will send it form home tomorrow.	I will send it from home tomorrow.
Let me now if that works.	Let me know if that works.
We saw there new house.	We saw their new house.
Please fill out the form and return it.	No change. “form” is right here.

How well it works

94% of its fixes are right

40% of wrong words fixed 24% before this version

~3 ms per check in Chrome

Measured by typing test text into the code offline.page runs, on text the model never trained on and that nothing was tuned on. The 40% is the average of five test sets:

Test set	Wrong words fixed	Fixes that were right
Classic mix-ups put into email, chat and encyclopedia text (their/there, to/too, its/it's…)	69%	99%
Hand-written sentences	58%	100%
Finger slips that make a real word	29%	87%
Real errors from children's writing	28%	91%
Real typos from GitHub commits	18%	96%

On about 110,000 words of correct email, chat, encyclopedia, news, documentation and fiction it wrongly changed about 1 word in 1,800, the same as the version before it.

The first two sets are built from the mix-up pairs the model was trained to watch for, so they flatter it. The last three are the fairer picture of everyday typing.

What it misses

• It still misses more wrong words than it catches, and about four in five real typos from GitHub commits.
• When both words make sense it stays quiet: “writing this form a plane” is left alone, because “writing this form” is something people say.
• Two-letter words are hit and miss. It fixes “I am to tired” but not every “to” that should be “too”.
• It can only judge the 20,000 words it knows, so a rarer word like “ware” typed for “were” is left alone.
• It needs the words that come after, so these fixes land a couple of words later.
• English only.

Using the model

The model scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making.

Inputs and outputs

Name	Shape	What it holds
left (input)	[batch, 16] int64	The words before, left-padded with 0, starting with <s> when there are fewer than 16
right (input)	[batch, 4] int64	The words after, nearest word last, left-padded with 0
candidates (input)	[batch, 9] int64	The typed word first, then up to 8 nearby words, padded with 0 at the end
scores (output)	[batch, 9] float	One score per candidate

IDs are line numbers in wrong-word-vocab.txt, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become <name>.

Files

File	What it is
wrong-word.onnx	The quantized model
wrong-word-vocab.txt	The 20,000 words it knows, one per line
wrong-word-confusions.txt	234 words people mix up, like “their” and “there”, each with the words they are mistaken for
train_realword.py	The training script

The model only scores words. The rules that decide when to make a swap (which words to check, the extra confidence needed at the start of a sentence, and so on) are in offline.page's src/tinyfix.

Training

A word-level GRU that reads 16 words before and up to 4 words after, trained for 120,000 steps on one GPU. It learned from FineWeb-Edu, SmolTalk and OpenAssistant text, by swapping real words for nearby real words and learning to swap them back. About one example in four is centered on a known mix-up, half of them used correctly, so it also learns when to leave a word alone.

_{tinyfix is one of two tiny models behind offline.page, a writing app that works in airplane mode.
Its partner, tinynext, suggests the next few words.
Both are made by paperweight, purpose-built small models.}