EVAThink-Qwen3

EVAThink-Qwen3 is a vision-language model fine-tuned from Qwen3-VL-8B-Instruct with reinforcement learning to decide per question whether to think. When the visual evidence plainly supports an answer, it replies directly. When the evidence is sparse, discontinuous, or scattered across frames, it writes a short, evidence-grounded reasoning trace first.

You don't need to tell it which mode to use. Give it the plain question and options, and it picks the mode itself.

Highlights

• Adaptive, not always-on, reasoning. How often it thinks follows how hard the task is to reason about. On purely perceptual spatial questions (VSI-Bench) it thinks on 0% of questions. On knowledge-heavy lecture videos (Video-MMMU) it thinks on 86%. Within a benchmark, the rate rises with video length: on Video-MME it goes 14% → 41% → 48% from short to medium to long videos.
• Short outputs. Mean response length is 4 tokens on VSI-Bench, 14 on MVBench, 95 on LongVideoBench, 117 on Video-MME, 257 on MMVU, and 794 on Video-MMMU, where it thinks on 86% of questions.
• Better than the base model or on par on all six benchmarks, and ahead of VideoAuto-R1 (Qwen3-VL-8B) on five of six, all under the same frame budget.

Results

All models were evaluated by us under one shared protocol:

• 64 frames sampled uniformly (first and last frame included)
• each frame resized to a fixed 200,704 pixels (factor 28)
• temperature=0.0, greedy decoding

Each model uses its intended prompt:

• EVAThink: the bare question and options, with no instruction.
• Qwen3-VL-8B-Instruct: each benchmark's official prompt.
• VideoAuto-R1: its own answer-twice prompt, with confidence-based early exit.

Benchmark (split, #Q)	Qwen3-VL-8B-Instruct	VideoAuto-R1-Qwen3-VL-8B	EVAThink-Qwen3	EVAThink think-rate
Video-MME w/o subs (2,700)	64.9	66.0	66.3	34.4%
MVBench (4,000)	65.8	69.0	66.5	5.1%
MMVU, multiple-choice (625)	70.4	69.6	72.8	59.4%
Video-MMMU (900)	65.6	64.3	67.4	85.7%
LongVideoBench val (1,337)	60.4	54.6	60.7	46.1%
VSI-Bench (5,130)	58.3	54.6	59.9	0.0%

Notes:

• These numbers come from our own fixed-budget protocol and are not directly comparable with numbers in other papers, which use different frame counts, resolutions, and prompts.
• MVBench uses the original segment boundaries for the tasks that define them.
• VSI-Bench scores numerical questions with mean relative accuracy (MRA) and multiple-choice questions with accuracy; the table reports the average over all questions.
• Video-MMMU includes its 21 open-ended questions, scored with the official MMMU rules.
• A response counts as "thinking" (think-rate) when it doesn't start directly with the answer.

Citation

@misc{evathink2026,
  title  = {TODO},
  author = {TODO},
  year   = {2026},
  note   = {TODO: arXiv link}
}