EVAThink-Stage1

EVAThink-Stage1 is the intermediate checkpoint after the first of the two reinforcement learning stages behind EVAThink-Qwen3. It is fine-tuned from Qwen3-VL-8B-Instruct and already chooses per question whether to answer directly or to write an evidence-grounded reasoning trace first, but it still reasons on most questions.

For general use, prefer EVAThink-Qwen3. It reaches about the same accuracy with roughly a third fewer output tokens. This checkpoint is released for studying the training stages.

Highlights

• Accuracy is already close to the final model. Averaged over six benchmarks it scores 65.4, against 65.6 for EVAThink-Qwen3 and 64.3 for the base model.
• It thinks much more often than it needs to. It reasons on 77% of Video-MME questions and 83% of LongVideoBench questions (EVAThink-Qwen3: 34% and 46%). Stage 2 mainly teaches the model to skip reasoning when it isn't needed.
• It still adapts to the task. On purely perceptual spatial questions (VSI-Bench) it almost never reasons (0.2%), and on MVBench it reasons on about a third of questions.

Results

Same protocol as EVAThink-Qwen3, all models evaluated by us:

• 64 frames sampled uniformly (first and last frame included)
• each frame resized to a fixed 200,704 pixels (factor 28)
• greedy decoding with presence penalty 1.5, up to 4,096 new tokens
• vLLM

EVAThink models get the bare question and options with no instruction. Qwen3-VL-8B-Instruct gets each benchmark's official prompt.

Benchmark (split, #Q)	Qwen3-VL-8B-Instruct	EVAThink-Stage1	EVAThink-Qwen3	Stage1 think-rate	Stage1 mean tokens
Video-MME w/o subs (2,700)	64.9	65.4	66.3	76.7%	203
MVBench (4,000)	65.8	65.2	66.5	31.9%	90
MMVU, multiple-choice (625)	70.4	73.4	72.8	96.0%	413
Video-MMMU (900)	65.6	66.7	67.4	98.3%	1,038
LongVideoBench val (1,337)	60.7	61.3	60.7	83.2%	237
VSI-Bench (5,130)	58.3	60.3	59.9	0.2%	6

Notes:

• These numbers come from our own fixed-budget protocol and are not directly comparable with numbers in other papers.
• Mean output length over the six benchmarks is 331 tokens for EVAThink-Stage1 and 213 for EVAThink-Qwen3.
• A response counts as "thinking" (think-rate) when it doesn't start directly with the answer.
• See the EVAThink-Qwen3 card for per-benchmark scoring details.

Usage

EVAThink-Stage1 uses the standard Qwen3-VL architecture and chat template, so any Qwen3-VL-compatible stack works. Give it just the question and options.

Transformers

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "MaplesWCT/EVAThink-Stage1"
model = Qwen3VLForConditionalGeneration.from_pretrained(model_id, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

question = "What does the person do right after opening the fridge?\nA. Takes out milk\nB. Closes it\nC. Pours water\nD. Leaves the room"
messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": "path/to/video.mp4"},
        {"type": "text", "text": question},
    ],
}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
    fps=2,
).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

vLLM

vllm serve MaplesWCT/EVAThink-Stage1 --max-model-len 64000 --limit-mm-per-prompt '{"video": 1}'

Output format

Same as EVAThink-Qwen3: either the answer alone (e.g. B), or a short reasoning passage followed by the final answer. Take the letter at the start of the response; if there isn't one, take the last letter that follows "answer is".

Training

Stage 1 starts from Qwen3-VL-8B-Instruct and runs GRPO on EVAThink-Dataset-Stage1: 1,766 multiple-choice video questions, each shown in a dense view (the evidence fills the clip) and a sparse view (the evidence is diluted in a longer video), 32 frames per view.

• The reward depends on the view. On the dense view a correct direct answer earns the most; on the sparse view a correct reasoned answer does. Malformed responses are penalized.
• A regularizer on the first response token keeps the model from collapsing onto a single mode.
• Both views of a question go into the same update: 32 prompts (16 questions × 2 views) × 8 rollouts per step.
• One epoch (110 steps), learning rate 1e-6, no KL penalty.

EVAThink-Qwen3 continues from this checkpoint with a stage-2 reward that no longer looks at the view.

Limitations

• It reasons on most questions, so it is slower than EVAThink-Qwen3 at about the same accuracy.
• It was evaluated on multiple-choice video QA, plus numerical estimation in VSI-Bench. Behavior on open-ended tasks has not been studied.
• Evaluation uses a fixed 64-frame budget. Results under other frame budgets may differ.

Citation

@misc{evathink2026,
  title  = {TODO},
  author = {TODO},
  year   = {2026},
  note   = {TODO: arXiv link}
}

Acknowledgements

Built on Qwen3-VL.