Request access to the Stringman world model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Access is granted by the author on request. Tell me who you are and what you'd like to try.
Log in or Sign Up to review the conditions and access this model content.
Stringman world model
A small learned dynamics model of Neufangled's Stringman cable robot: give it the last second of states and commands, and the room's four mount points, and it predicts how the robot moves under any sequence of future commands. It knows nothing about tasks. In my work a sampling planner uses it to imagine a few hundred futures every control step, and each chore is written as costs on those futures.
- 28,401 parameters, 127 KB. Runs on a CPU: 128 two-second futures in about 15 ms on a laptop.
- One model for every room. The room enters only through its calibrated mount points, so a room it never saw is just another set of four points.
- Trained without a GPU, on task-free simulated motion in 12,000 random rooms, with the robot itself varied from one trajectory to the next.
By Edgar Moreau · huggingface.co/Ethgar
Try it
pip install torch numpy
python example.py
import numpy as np
from stringman_wm import LearnedCDPRModel
model = LearnedCDPRModel.load("stringman_wm_g1.pt", dt=0.2) # your control period, a multiple of 0.1 s
model.set_room(mounts) # (4, 3) metres, the firmware's line order
model.reset_history(x_now, a_now) # then model.observe(x, a) after every 0.1 s
future = model.rollout(None, None, commands) # (S, T, 5) commands -> (S, T, 14) states
Inputs and outputs
| units | ||
|---|---|---|
| state (14) | gantry position p (3), where the four lines meet, room frame | m |
| gantry velocity v (3) | m/s | |
| pole swing s (2): x, y of the pole axis, = sin(tilt) per axis | – | |
| swing rate (2) | 1/s | |
| wrist angle, 0 = neutral (firmware degrees 0..1080 = 540 + degrees) | rad | |
| wrist rate | rad/s | |
| finger closure, 0 open .. 1 closed (firmware -90..90 = -90 + 180 × closure) | – | |
| closure rate | 1/s | |
| command (5) | gantry velocity (3), room frame, as move_direction_speed takes it |
m/s |
| wrist rate | rad/s | |
| closure target | 0..1 | |
| room | mount points (4, 3): where each free line span leaves from | m |
Model step 0.1 s, history 1 s (10 steps). Line lengths and tensions are not inputs.
How good it is (simulation)
Open-loop prediction, 2 s ahead, on 500 held-out windows in 310 random rooms it never saw:
| grasp point error, median / p90 | with a payload on or changing | pole tilt error, median | |
|---|---|---|---|
| this model | 19.6 / 60 mm | 23.2 mm | 0.89° |
| MuJoCo itself, nominal physics (reference) | 21.0 / 56 mm | 24.3 mm | 0.90° |
In two real rooms imported from public Stringman recordings (neither in training): 20.5 mm and 20.2 mm median, tilt 0.88° and 0.80°. The grasp point is between the finger pads, 0.515 m down the pole.
Planning with it in closed loop (sim): clearing 12 cubes into a bin in those two real rooms, 33/36 and 34/36 over 3 episodes each; in three furnished 3D rooms, 22, 20 and 24 of 24 over 2 episodes each, with no furniture contact.
Limits
- Simulation only. Trained and tested on a MuJoCo twin of Stringman. The twin's masses, line stretch, spool acceleration and swing damping are estimates, not measurements of a real robot. It has not run on real hardware yet.
- Stringman's own two-finger gripper. Another gripper changes the swing and needs a new model.
- No contact. Furniture, the floor and the objects are outside the model: that is the planner's and the task's business.
- Payloads up to 0.5 kg. The gantry's heading is assumed held by the lines.
License
Apache-2.0.