README.md

---
title: Code4Scene
emoji: 🏗️
colorFrom: yellow
colorTo: gray
sdk: static
app_file: index.html
pinned: false
short_description: Benchmarking coding agents that build 3D scenes
---

# Code4Scene

Project page for Code4Scene: the leaderboard, the paper's takeaways, and selected cases whose saved Unreal Engine scenes can
be viewed in 3D. Each section has a Text-to-Scene tab (construction; selected by default, the 20-scene Research Track) and an
Image-to-Scene tab (editing; the 75-case public evaluation). `scripts/t2s_only.py` removed Image-to-Scene on 2026-10-01;
`scripts/i2s_tabs.py` brought it back as the second tab on 2026-10-04; `scripts/i2s_more_cases.py` adds four more
Image-to-Scene cases (six in all) and `scripts/harbor_card.py` features GPT-6 Astra's Industrial Harbor, shown in the same live viewer as the other cards
(`scripts/build_harbor_glb.mjs` builds its GLB from the viewer Space's library; `scripts/harbor_live.py` packs it) and linked
to its full-resolution viewer Space.

Paper: [arXiv:2609.36777](https://arxiv.org/abs/2609.36777) · Code: [SimWorld-AI/Code4Scene](https://github.com/SimWorld-AI/Code4Scene)

Part of the [SimWorld](https://simworld.org) project.

## Text-to-Scene re-run (October 2026)

Since 2026-10-16 the Text-to-Scene tabs show the sandboxed re-run of the 20 Research Track scenes instead of the paper's
numbers (user: replace the T2S leaderboard, use the latest cases, put the gallery up, and leave out runs that have not
finished). The Image-to-Scene tabs, the Task Format example, the Takeaways and the case explorer (`cases.html`) keep the
paper's evaluation; the Takeaways and the explorer say so.

- Leaderboard: bars, Pareto, ranking and tooltips for every configuration whose 20 cases are all scored. Case score =
  0.2 × Detailed + 0.6 × Overview + 0.2 × Physical; a case without a valid scene scores 0; the model's score is the mean.
  As on the internal leaderboard, scores are never rounded up (bar labels ×100 rounded down, table values truncated to
  three decimals) and cost is the median USD per case, to the cent. An agent that records no token usage, or a run on
  self-hosted weights that is not priced, has no cost and no Pareto point; the table, tooltips, Pareto caption and
  gallery say which.
- Gemma 4 31B (thinking on) ran on self-hosted weights, so nothing was billed; its cost is the token counts its agent
  logged on every model call (`scripts/t2s_token_cost.py`, no cached input) at Gemma 4 31B's OpenRouter list price
  ($0.20 input, $0.40 output per million): median $0.056 per case (user, 2026-10-16).
- Snapshots: 2026-10-16 06:21 UTC (15 configurations); 2026-10-16 16:01 UTC (19: GPT-6 Luna, Muse Spark Open, MiMo V2.6
  Pro and Gemma 4 31B (thinking on) finished; Inkling had not started scoring).
- Xiaomi (MiMo V2.6 Pro) is drawn with Xiaomi's own mark from [Simple Icons](https://simpleicons.org/) (CC0; brand
  colour #FF6900), since Lobehub has only a MiMo wordmark that is unreadable at table size, and with the internal
  leaderboard's amber (#f2b418) for its bar and Pareto point. Open-weights tags follow whether the weights are published:
  OpenRouter's catalogue lists Hugging Face weights for Muse Spark Open and MiMo V2.6 Pro, so both are tagged (user,
  2026-10-16).
- Muse Wonder Full 1.0 is sold by subscription and its harness stream carries no usage, so its cost is recomputed from
  the muse CLI's own session logs (`scripts/t2s_muse_cost.py`): every model call's token counts, main agent and
  subagents, matching the CLI's own cost ledger, priced at Muse Wonder's OpenRouter list rate ($4 input, $0.20 cached
  input, $20 output per million; the CLI's background model Muse Spark 1.3 at its own rate). The CLI's skill-reminder
  side calls are not logged in any file the run keeps readable, so they are not included.
- Cases: four agents (Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol, Gemini 4 Argon) building the same scene, as a build
  time-lapse and an orbit, with a scene picker (`assets/compare-player.js`, `media/t2s/compare/`). The Industrial Harbor
  showcase stays.
- Gallery (`gallery/`, linked from the menu): every shown configuration on all 20 scenes, by agent and by scene, as the
  evaluator's four renders (WebP), with case scores, components and costs.
- `data/t2s-sandbox.json` holds the public data behind all three.

Rebuild from a snapshot of the internal Text-to-Scene leaderboard (its served `data.json`, `gallery/gallery.json`,
`gallery/img/` and `grids/<case>-compare.{mp4,jpg}` in one folder) plus `components.json`, which
`scripts/t2s_components.py` extracts with the run-status page's own selection of results:

    ssh <host> python3 - <muse run dirs> < scripts/t2s_muse_cost.py > ../t2s-src/musewf_cost.json
    ssh <host> python3 - 0.20 0.20 0.40 <gemma run dir> < scripts/t2s_token_cost.py > ../t2s-src/gemma4think_cost.json
    python3 scripts/t2s_sandbox_round.py --src ../t2s-src [--videos blue-city,ruins,...]
    python3 scripts/t2s_gallery.py --src ../t2s-src

Both are re-runnable: a later snapshot (a run that has since finished, a re-filmed comparison) just needs another run.

## Website design

Layout, navigation, wireframe mark, typography and card geometry follow the user-provided [BuildingBench](https://enactra.ai/buildingbench/) design. The bar chart rendering and company palette are adapted from its [Figure Kit](http://ds-serv12.ucsd.edu:8875/figure-kit/#bar), specifically `bar-static/build.py`. All scores, configurations, authors, media, models and evaluator content remain Code4Scene's. The paper's 20-case public Text-to-Scene evaluation is distinguished from the current 129 public construction cases and the 160-case full construction target, including private cases.

The reference toolkit's Inter, IBM Plex Mono and Source Serif 4 font files are bundled with their original SIL Open Font License notices in `assets/fonts/`. Provider marks are from the toolkit's Lobehub icons (Xiaomi's from Simple Icons), used to identify the corresponding model providers.

Deployment remains this Hugging Face static Space: `index.html` and `cases.html`, with the existing media/model paths and headers. The pre-restyle version is recoverable at commit `9ff787d0f9d72905d92f780da56bd2229782b981`.

The Pareto renderer (`assets/pareto.js`) ports the reference toolkit's reversed logarithmic cost axis, connected frontier, provider palette and greedy collision-aware label placement. `data/pareto.json` combines unrounded scores from the existing `data/site.js` with exact mean costs from the original published tables; each view recomputes non-dominance from those values. The table displays three decimals but sorts at source precision.

Homepage scenes (`assets/scene-cards.js`) adapt BuildingBench's shared WebGL renderer and per-view canvas engine. Existing public packed models are decoded with the same adapter as `cases.html`; geometry and the full case explorer remain unchanged. Visible scenes load automatically; repeated scenes share decoded assets. Drag or arrow keys orbit, wheel or +/- zoom, and the pause control stops rotation. Both Image-to-Scene cards switch between the saved repair, corrupted input and ground truth using the reference camera.

## Interaction and loading reliability

Task Format switches the entire construction/editing example, including instructions, inputs, saved output and evaluator. Bar marks, labels and Pareto points share pointer, keyboard and tap tooltips with source-precision scores and costs in `data/leaderboard-details.json`. The ranking supports filtering and source-precision sorting.

Homepage model work uses a shared two-job download/decode queue. Hidden/offscreen views release their request; stale results cannot replace the selected state. Card textures are limited to 512px and render resolution is bounded, while original model files and the full case explorer stay intact. Idle decoded assets are evicted; the 128 MiB cache target is not a hard limit on currently visible geometry. Downloads show progress, abort after 20 seconds without data or 90 seconds total, retry once, then expose an explicit Retry button. Context loss pauses drawing and restoration rebuilds the environment; failures expose recovery instead of an indefinite loading overlay.

Queue tests: `node --test tests/scene-assets.test.mjs`. Browser QA includes public-model cold loads, quick scrolling, task/repair switching, desktop/mobile resize, forced HTTP failures, stalled downloads, retry and WebGL context loss.

## Geometry-first scene cards

The homepage uses `data/progressive-scenes.json` to choose immutable packed derivatives for 18 of its public scene/state variants (the suburban Image-to-Scene scenes are almost all geometry, so their cards load the packed model directly). Geometry and meshopt bytes are unchanged; embedded images are moved into small packed batches. The original packed models remain available to the full case explorer. `assets/progressive-gltf.js` returns the geometry with neutral texture placeholders, then the visible card starts material work after its first rendered frame. Image updates preserve glTF samplers, color spaces, transforms and material parameters. At most two texture jobs run across the page; offscreen cards pause them, and a texture failure keeps the geometry visible with a separate retry control.

Three.js r160, GLTFLoader, MeshoptDecoder, Sky and OrbitControls are served from `assets/vendor/three-r160/`, retaining the official MIT license. Both page import maps use this pinned same-origin version. Original sources: https://github.com/mrdoob/three.js/tree/r160 .

The 11 original files total 73,341,132 bytes; their initial geometry payloads total 53,799,776 bytes (26.6% less before the first geometry frame). This is a payload comparison, not a promise of an equivalent wall-clock speedup. Card image batches add 19,821,674 bytes across variants before deduplication. Rebuild with Pillow and `python scripts/build_progressive.py`; generated texture batches keep images at a maximum of 512px for the cards. Original model/media/data files and the full-resolution case explorer are unchanged.