Executable Environments rebuilt from agent traces by Kullback and graded by code. Each card carries its own replay fidelity and trusted Task count.
AI & ML interests
RL Envs and Post training models
Recent Activity
View all activity
Organization Card
leibler
Leibler rebuilds the system a working agent ran in from its traces, then grades other models in code on what they change there. Kullback is the open-source Builder and Runner behind it, under Apache-2.0.
Environments
| Environment | What the agent does | Source |
|---|---|---|
| leibler/retail | Retail customer support: orders, returns, exchanges | tau2-bench traces (MIT) |
| leibler/airline | Airline customer support: bookings, changes, cancellations | tau2-bench traces (MIT) |
| leibler/travel-agency | Travel agency support across airlines, rail, hotels, rides, cars and cruises | Real public travel complaints |
Words
- Trace: a recorded Run, an agent's conversation with the tools.
- Replay: re-driving a Trace; it is confirmed when every call agrees.
- Fidelity: the share of Tasks, or of calls, whose replay agrees.
- Verifier: a Task's End-state check, written by the Examiner, never by the Builder.
- Trusted: a Verifier that passed every gate; a release replays at least 90% of its Tasks.
Fetch and run
uv pip install git+https://github.com/leiblerdev/kullback.git
uv run kullback fetch leibler/<environment> --out env
uv run kullback run --workdir env --task <task id> --model provider/model
uv run kullback verdict --workdir env
uv run kullback report --workdir env
Travel Agency ships as Harbor tasks: hf download leibler/travel-agency --repo-type dataset --local-dir travel-agency, then harbor run -p travel-agency -a <agent>.
Links
- Harness: https://github.com/leiblerdev/kullback
- Site: https://leibler.dev
- Licence: Apache-2.0 for the harness and every package; each corpus keeps its own.
models 0
None public yet