Papers
arxiv:2611.04434

Sketchpad: certified exact arithmetic with a small vision-language model

Published on Nov 3
· Submitted by
Neo Mohsenvand
on Nov 9

Abstract

A language model with a low-rank adapter and bounded external memory can be certified exact for arbitrary-length arithmetic by checking finitely many views, and it performs multiple exact arithmetic procedures on a laptop browser.

Language models are accurate on most long-arithmetic problems, but their accuracy is estimated on samples and carries no guarantee for an unseen input; tool use is exact but moves the computation outside the network. We show that exactness at all input lengths becomes a finite, checkable property of a network when each of its steps reads only a bounded view of an external working memory. In Sketchpad, Qwen3.5-0.8B with a low-rank adapter executes arithmetic procedures on a rendered sheet. Each forward pass reads a three-cell view and emits one step: a digit, a carry or borrow, and the next operation. A fixed runtime moves the view and writes no digit of a result. We prove in Lean that the model is then exact on all inputs of any length if and only if it is exact on the finite set of views the runtime presents, that no finite test suffices when steps also read earlier steps, and that the guarantee tolerates bounded numerical perturbation and extends to compositions of procedures. All 2,446 views pass on the deployment device, a laptop GPU running the model in a web browser. The certified model performs signed addition and subtraction, long multiplication and division, square roots and greatest common divisors at arbitrary length. It solved 200 of 200 pre-registered problems in the browser, and 220 of 220 sums of up to 1,000 digits and products of up to 20 digits, of which GPT-6.1 Sol, the strongest of ten other models, solved 217. A symbolic runtime built on these procedures solves, expands and factors polynomials with every number computed by the model, and a multiplicative gate on the adapter recovers the base model exactly when the sheet is not in use.

Community

Paper author Paper submitter

Author here. Can a small model be proven exact at arithmetic on inputs of every length?

🧮 The theory

If each step of a model reads only a bounded view of an external memory, exactness at all lengths reduces to exactness on a finite set of views. Proved in Lean, along with:

  • Necessity: once steps can also read earlier steps, no finite test suffices.
  • Robustness: the guarantee survives bounded numerical error.
  • Composition: certified procedures stay certified when combined.

⚙️ In practice

Qwen3.5-0.8B + a small adapter passes all 2,446 views on a laptop GPU, in the browser. It is certified for + − × ÷ √ gcd at any length.

220 pre-registered problems Score
Sketchpad (0.8B) 220
GPT-6.1 Sol · Claude Opus 5.5 · Gemini 4 Argon 217 · 216 · 207
Five open-weight models 149–199
Qwen3.5-27B · Qwen3.5-0.8B (base) 121 · 23

On 1,000-digit sums: Sketchpad 20/20; frontier models 19, 17, 9; every open-weight model ≤ 2.

🔗 Live demo · Code

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.04434
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.04434 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.04434 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.