README.md

metadata

title: Code4Scene
emoji: 🏗️
colorFrom: yellow
colorTo: gray
sdk: static
app_file: index.html
pinned: false
short_description: Benchmarking coding agents that build 3D scenes

Code4Scene

Project page for Code4Scene: the leaderboard, the paper's takeaways, and selected cases whose saved Unreal Engine scenes can be viewed in 3D. Each section has a Text-to-Scene tab (construction; selected by default, the 20-scene Research Track) and an Image-to-Scene tab (editing; the 75-case public evaluation). scripts/t2s_only.py removed Image-to-Scene on 2026-10-01; scripts/i2s_tabs.py brought it back as the second tab on 2026-10-04; scripts/i2s_more_cases.py adds four more Image-to-Scene cases (six in all) and scripts/harbor_card.py features GPT-6 Astra's Industrial Harbor, shown in the same live viewer as the other cards (scripts/build_harbor_glb.mjs builds its GLB from the viewer Space's library; scripts/harbor_live.py packs it) and linked to its full-resolution viewer Space.

Paper: arXiv:2609.36777 · Code: SimWorld-AI/Code4Scene

Part of the SimWorld project.

Text-to-Scene re-run (October 2026)

Since 2026-10-16 the Text-to-Scene tabs show the sandboxed re-run of the 20 Research Track scenes instead of the paper's numbers (user: replace the T2S leaderboard, use the latest cases, put the gallery up, and leave out runs that have not finished). The Image-to-Scene tabs, the Task Format example, the Takeaways and the case explorer (cases.html) keep the paper's evaluation; the Takeaways and the explorer say so.

• Leaderboard: bars, Pareto, ranking and tooltips for every configuration whose 20 cases are all scored. Case score = 0.2 × Detailed + 0.6 × Overview + 0.2 × Physical; a case without a valid scene scores 0; the model's score is the mean. As on the internal leaderboard, scores are never rounded up (bar labels ×100 rounded down, table values truncated to three decimals) and cost is the median USD per case, to the cent. An agent that records no token usage has no cost and no Pareto point.
• Muse Wonder Full 1.0 is sold by subscription and its harness stream carries no usage, so its cost is recomputed from the muse CLI's own session logs (scripts/t2s_muse_cost.py): every model call's token counts, main agent and subagents, matching the CLI's own cost ledger, priced at Muse Wonder's OpenRouter list rate ($4 input, $0.20 cached input, $20 output per million; the CLI's background model Muse Spark 1.3 at its own rate). The CLI's skill-reminder side calls are not logged in any file the run keeps readable, so they are not included.
• Cases: four agents (Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol, Gemini 4 Argon) building the same scene, as a build time-lapse and an orbit, with a scene picker (assets/compare-player.js, media/t2s/compare/). The Industrial Harbor showcase stays.
• Gallery (gallery/, linked from the menu): every shown configuration on all 20 scenes, by agent and by scene, as the evaluator's four renders (WebP), with case scores, components and costs.
• data/t2s-sandbox.json holds the public data behind all three.

Rebuild from a snapshot of the internal Text-to-Scene leaderboard (its served data.json, gallery/gallery.json, gallery/img/ and grids/<case>-compare.{mp4,jpg} in one folder) plus components.json, which scripts/t2s_components.py extracts with the run-status page's own selection of results:

ssh <host> python3 - <muse run dirs> < scripts/t2s_muse_cost.py > ../t2s-src/musewf_cost.json
python3 scripts/t2s_sandbox_round.py --src ../t2s-src [--videos blue-city,ruins,...]
python3 scripts/t2s_gallery.py --src ../t2s-src

Both are re-runnable: a later snapshot (a run that has since finished, a re-filmed comparison) just needs another run.

Website design

Layout, navigation, wireframe mark, typography and card geometry follow the user-provided BuildingBench design. The bar chart rendering and company palette are adapted from its Figure Kit, specifically bar-static/build.py. All scores, configurations, authors, media, models and evaluator content remain Code4Scene's. The paper's 20-case public Text-to-Scene evaluation is distinguished from the current 129 public construction cases and the 160-case full construction target, including private cases.

The reference toolkit's Inter, IBM Plex Mono and Source Serif 4 font files are bundled with their original SIL Open Font License notices in assets/fonts/. Provider marks are from the toolkit's Lobehub icons, used to identify the corresponding model providers.

Deployment remains this Hugging Face static Space: index.html and cases.html, with the existing media/model paths and headers. The pre-restyle version is recoverable at commit 9ff787d0f9d72905d92f780da56bd2229782b981.

The Pareto renderer (assets/pareto.js) ports the reference toolkit's reversed logarithmic cost axis, connected frontier, provider palette and greedy collision-aware label placement. data/pareto.json combines unrounded scores from the existing data/site.js with exact mean costs from the original published tables; each view recomputes non-dominance from those values. The table displays three decimals but sorts at source precision.

Homepage scenes (assets/scene-cards.js) adapt BuildingBench's shared WebGL renderer and per-view canvas engine. Existing public packed models are decoded with the same adapter as cases.html; geometry and the full case explorer remain unchanged. Visible scenes load automatically; repeated scenes share decoded assets. Drag or arrow keys orbit, wheel or +/- zoom, and the pause control stops rotation. Both Image-to-Scene cards switch between the saved repair, corrupted input and ground truth using the reference camera.

Interaction and loading reliability

Task Format switches the entire construction/editing example, including instructions, inputs, saved output and evaluator. Bar marks, labels and Pareto points share pointer, keyboard and tap tooltips with source-precision scores and costs in data/leaderboard-details.json. The ranking supports filtering and source-precision sorting.

Homepage model work uses a shared two-job download/decode queue. Hidden/offscreen views release their request; stale results cannot replace the selected state. Card textures are limited to 512px and render resolution is bounded, while original model files and the full case explorer stay intact. Idle decoded assets are evicted; the 128 MiB cache target is not a hard limit on currently visible geometry. Downloads show progress, abort after 20 seconds without data or 90 seconds total, retry once, then expose an explicit Retry button. Context loss pauses drawing and restoration rebuilds the environment; failures expose recovery instead of an indefinite loading overlay.

Queue tests: node --test tests/scene-assets.test.mjs. Browser QA includes public-model cold loads, quick scrolling, task/repair switching, desktop/mobile resize, forced HTTP failures, stalled downloads, retry and WebGL context loss.

Geometry-first scene cards

The homepage uses data/progressive-scenes.json to choose immutable packed derivatives for 18 of its public scene/state variants (the suburban Image-to-Scene scenes are almost all geometry, so their cards load the packed model directly). Geometry and meshopt bytes are unchanged; embedded images are moved into small packed batches. The original packed models remain available to the full case explorer. assets/progressive-gltf.js returns the geometry with neutral texture placeholders, then the visible card starts material work after its first rendered frame. Image updates preserve glTF samplers, color spaces, transforms and material parameters. At most two texture jobs run across the page; offscreen cards pause them, and a texture failure keeps the geometry visible with a separate retry control.

Three.js r160, GLTFLoader, MeshoptDecoder, Sky and OrbitControls are served from assets/vendor/three-r160/, retaining the official MIT license. Both page import maps use this pinned same-origin version. Original sources: https://github.com/mrdoob/three.js/tree/r160 .

The 11 original files total 73,341,132 bytes; their initial geometry payloads total 53,799,776 bytes (26.6% less before the first geometry frame). This is a payload comparison, not a promise of an equivalent wall-clock speedup. Card image batches add 19,821,674 bytes across variants before deduplication. Rebuild with Pillow and python scripts/build_progressive.py; generated texture batches keep images at a maximum of 512px for the cards. Original model/media/data files and the full-resolution case explorer are unchanged.