Papers
arxiv:2611.16378

Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents

Published on Nov 12
· Submitted by
Subodh Lonkar
on Nov 16
Authors:

Abstract

A governance framework separates generation from adoption of self-modifying autonomous-agent changes, using fresh paired audits, non-inferiority checks, and calibrated statistical bounds to confirm improvements before bounded deployment.

Autonomous agents modify prompts, memories, tools, workflows, and policies using interaction histories. Yet a plausible modification is not necessarily suitable for persistent adoption: candidate selection is adaptive, evaluators may be imperfect, and local gains can hide degradation against an earlier baseline. We introduce Evidence-Governed Evolution (EGE), a framework for governing adoption of self-generated agent modifications. EGE separates candidate generation from confirmatory adoption, evaluates frozen candidates on fresh paired audits, requires meaningful improvement on a prespecified primary objective, protects declared capabilities through reference-relative non-inferiority tests, and applies governance constraints before bounded canary deployment. We establish an evaluator-calibrated simultaneous lower-bound construction for a countable sequence of adaptively generated candidates. It combines candidate-wise Hoeffding bounds with independent calibration samples comparing the operational evaluator with a prespecified reference procedure. We also formalise reference replacement through bridge tests, derive a cumulative degradation bound along a versioned reference lineage, and provide an audit-planning condition relating effect-size margin, sample sizes, and acceptance probability. A Monte Carlo stress test illustrates calibration under candidate-dependent evaluator optimism; it is a stylised statistical experiment, not an evaluation of a deployed agent. The guarantees are conditional on bounded paired scores, fresh sampling, a valid reference-evaluation construct, and declared evaluation distributions; they do not establish universal agent safety or robustness to arbitrary distribution shift.

Community

Paper author Paper submitter

What happens when an AI agent decides to improve itself? More importantly, how do we know whether those improvements are actually making it better?

As agents begin modifying their own prompts, memories, tools, and workflows, one question becomes increasingly important: Should every self-generated improvement be trusted?

This paper introduces Evidence-Governed Evolution (EGE), a framework for ensuring that an agent's proposed improvements earn the right to become permanent through independent evaluation, measurable evidence, and safeguards against capability degradation.

The central idea is simple: An AI agent should be free to propose changes, but evidence should decide which changes survive.

I'd be interested to hear your thoughts on how we should govern the next generation of self-evolving AI agents.

I'm broadly on board with the framing — gating self-modification behind evidence beats trusting the agent's own confidence — but the whole thing stands or falls on the statistical power of the paired audit, and that part worries me. If the fresh-sample audit is small, which is what you can afford when each rollout costs tokens and tool calls, then a prespecified improvement threshold is close to a coin flip: noisy tasks produce false adoptions, genuine fixes get rejected, and the gate becomes decoration. I'd want to see the audit size swept against task variance, and the adoption rate reported at each point, otherwise it's hard to tell whether the mechanism works or just adds latency. The other thing is that "adoption" is treated as one decision when it's really at least two — a candidate that only edits its own config is trivially reversible, but one that has already written to memory or fired external tool calls has side effects you can't roll back by flipping a pointer. Separating those classes before the gate fires would make this much closer to something I'd actually deploy. Curious whether the rollback story in the paper assumes a clean checkpoint every time.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.16378
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.16378 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.16378 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.16378 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.