sergiopaniego/sft-agent-traces-trackio-bucket
1.2 MB
Sergio Paniego PRO
sergiopaniego
AI & ML interests
None yet
Recent Activity
upvoted an article about 1 hour ago
Decision Models in Hugging Face posted an update about 1 hour ago
New post-training example: SFT on agent traces with TRL
→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks
About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest
Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces updated a dataset about 6 hours ago
trl-lib/documentation-imagesOrganizations
Posts 124
Post
13
New post-training example: SFT on agent traces with TRL
→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks
About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest
Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces
→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks
About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest
Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces
Articles 38
Article
25
MOPD: how frontier labs merge RL experts into one model and how to do it using TRL
-
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-500
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-1k
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-full
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/smoldataenvs-opencode-traces
Viewer • Updated • 3.8k
Experiment from the blog huggingface.co/blog/sergiopaniego/mopd-trl: Open-MOPD models and data, the 60-step checkpoint and its Trackio dashboard
-
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT
Text Generation • 3B • Updated • 1.01k • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-Math
Text Generation • 3B • Updated • 374 • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-Code
Text Generation • 3B • Updated • 368 • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-IF
Text Generation • 3B • Updated • 313 • 1
-
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-500
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-1k
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-full
Image-Text-to-Text • 2B • Updated • 7 -
sergiopaniego/smoldataenvs-opencode-traces
Viewer • Updated • 3.8k
Experiment from the blog huggingface.co/blog/sergiopaniego/mopd-trl: Open-MOPD models and data, the 60-step checkpoint and its Trackio dashboard
-
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT
Text Generation • 3B • Updated • 1.01k • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-Math
Text Generation • 3B • Updated • 374 • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-Code
Text Generation • 3B • Updated • 368 • 1 -
BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-IF
Text Generation • 3B • Updated • 313 • 1
spaces 200
pinned
Running on Zero
Agents
Featured
127
VLM Object Understanding
🦀
Explore object detection, visual grounding, keypoint Detecti
pinned
Runtime error
Agents
11
Qwen2-VL-7B
🔥
Ask questions about charts in images
pinned
Running on Zero
Agents
SmolVLM-trl-dpo-rlaif-v
🔥
Generate text from an image and question
pinned
Runtime error
Agents
SmolVLM-trl-sft-ChartQA
🔥
Ask questions about charts in images
Running
Agents
Sft Agent Traces Trackio
🎯
Show an interactive tracking dashboard
Running
Agents
Open Mopd Trl
🎯
Display interactive tracking dashboard
models 145
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-full
Image-Text-to-Text • 2B • Updated • 7
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-1k
Image-Text-to-Text • 2B • Updated • 7
sergiopaniego/Qwen3.5-2B-SmolDataEnvs-SFT-500
Image-Text-to-Text • 2B • Updated • 7
sergiopaniego/open-mopd-smollm3-trl-naive
Text Generation • 3B • Updated • 478
sergiopaniego/qwen3-1.7b-mbpp-grpo
Text Generation • 2B • Updated • 16
sergiopaniego/rl-envs-youtube-livestream-4-scripts
Updated
sergiopaniego/qwen3-1.7b-wordle-grpo
Text Generation • 2B • Updated • 14
sergiopaniego/Qwen3-4B-claude-code-local-grpo
Text Generation • 4B • Updated • 10
sergiopaniego/Qwen3-4B-claude-code-deepcoder-grpo
Text Generation • 4B • Updated • 8
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B-judged
Text Generation • 2B • Updated • 30
datasets 18
sergiopaniego/smoldataenvs-opencode-rollouts
Updated
sergiopaniego/smoldataenvs-opencode-traces
Viewer • Updated • 3.8k
sergiopaniego/opencode-rollout-trace
Updated • 990
sergiopaniego/pelican-svg-drawings
Viewer • Updated • 169 • 618 • 1
sergiopaniego/math-sdpo-hints-plain
Viewer • Updated • 600 • 37
sergiopaniego/math-sdpo-hints
Viewer • Updated • 600 • 31
sergiopaniego/gsm8k-sdpo-plain
Viewer • Updated • 700 • 36
sergiopaniego/gsm8k-sdpo-hints
Viewer • Updated • 700 • 29
sergiopaniego/pi-mono-chat
Viewer • Updated • 886 • 43
sergiopaniego/requests-pr-diff
Viewer • Updated • 1 • 256