WanHand-E2E

Consolidated inference weights for WanHand-E2E: metric bimanual MANO hands, pinhole intrinsics, camera trajectories and anatomical contact/intensity from recorded egocentric RGB video.

This snapshot is the EMA of L4-prod-final at step 20,000, the final checkpoint of this training run. The training evaluator and deployed inference settings use different protocols; measurements for one must not be presented as measurements of the other.

Components

• dit/: blocks 0–15 of the Wan transformer, trained adapter updates merged into ordinary bf16 weights.
• projector/: fixed L4 hand decoder, intrinsics head, camera aggregator, hand-scale and tactile heads.
• vae/: default Wan video encoder.
• taehv/: optional approximate video encoder; using it changes predictions.
• mano/: geometry assets for both hands.
• detector/: optional WiLoR visibility-verification detector.
• caption_embedding.safetensors: fixed conditioning embedding.
• tactile_regions.json: MANO vertex-to-region assignments used for saved-result visualization.

No optimizer, training dependency, adapter loader, or separate calibrated/uncalibrated checkpoint is needed. The same model uses predicted intrinsics by default or accepts supplied calibration.

Use

from pathlib import Path
from wanhand import ModelConfig
from wanhand.inference import load_estimator

estimator = load_estimator(ModelConfig(repo_id="blanchon/WanHand-E2E"))
estimator.predict(Path("input.mp4"), Path("results/recording"))

The implementation uses overlapping windows for long recordings and preserves both camera-space geometry and camera-to-world poses. Optional loop closure is a separate CPU correction.

Interpretation

The world origin is the first camera, not a surveyed world frame. Monocular metric scale and motion can be wrong. Visibility scores are not pose-accuracy estimates. Tactile intensity is normalized model output rather than force in newtons. Six of the 16 tactile regions are not supervised by the JQ glove sources; anatomical granularity does not imply uniform reliability.

The merged architecture was checked against the frozen training implementation with the same merged weights, including both supplied and predicted calibration. Merging bf16 adapter paths changes arithmetic relative to live adapters; the resulting numerical difference must not be confused with the accuracy of the model against ground truth.

The training configuration enables encoder compilation. Published inference retains eager Wan encoding; export parity supplies identical input latents to both model implementations. The 20,000-step export exactly matched all 15 checked output fields of the merged training implementation on deterministic synthetic latents on CPU, with both predicted and supplied calibration. This checks the numerical model port, not video-encoder execution or ground-truth accuracy. A recorded-video GPU check is pending the shared evaluation queue. Adapter merging changes arithmetic relative to unmerged bf16 execution.

Checkpoint evaluation

The 20,000-step held-out evaluation is running. Its results will be added when complete; previous checkpoint measurements should not be attributed to these weights. Run-to-run variability at 832 pixels remains unmeasured. Known campaign weaknesses include phantom hands, unusual camera orientations, and dataset-dependent focal/depth error. Upstream ACE dataset exposure also means an adaptation-held-out split need not be dataset-zero-shot.

Attribution and license

Derived from ACE-Ego-Hand and Wan2.2. The original repackaged ACE and WiLoR detector weights are CC BY-NC 4.0. Wan weights are Apache 2.0 and TAEHV is MIT. MANO retains its own license from the Max Planck Institute. Adaptation uses research datasets including non-commercial sources; these assets are for non-commercial research. A repository-level license does not replace MANO's conditions or other source-asset terms.