Active exploration · Embodied self-improvement

Active Exploration for Self-Improvement
in Embodied Agents

A coding agent that explores the environment, develops the robot’s harness,
and carries useful skills into the next iteration.

Zhang Zhang^1,*Shengjun Zhang^1,*,†Simin Huang¹Min Chen¹Chong Xia¹Xin Fei²Yueqi Duan^1,‡

¹ Tsinghua University² National University of Singapore

* Equal contribution † Project Leader ‡ Corresponding author

Paper Code Watch video

Fixed model parametersExecutable skillsThree robotic benchmarks

VISTA in action

Project video · 1:29

01 Experience becomes executable capability

Improve the harness.
Keep the models fixed.

A robot’s harness organizes its instructions, skills, memory, and decision logic. VISTA makes this executable layer the object of improvement, combining learning-based policies, perception, motion planning, and feedback control.

A meta-agent starts with a harness version and its execution evidence. It can investigate the environment, develop new skills, and test candidate changes. Useful discoveries are written into the harness and validated through autonomous execution.

Versions and their supporting evidence remain available for later development. The result is a process in which embodied experience can accumulate as reusable code and procedural knowledge.

VISTA overview with a meta-agent inspecting, developing, interacting, and revising a robot harness, followed by the Goal-t09 progression from 0 to 16 successes. — The meta-agent develops the harness through code revision and interaction with the environment. The underlying models remain fixed. The example below the diagram follows the same Goal-t09 task shown in the case-study video.

02 The development loop

A meta-agent that can
run its own experiments.

Active exploration gives the meta-agent control over how it obtains new evidence: what to observe, which operations to try, and what to investigate next.

Start from a version and its evidence

Connect an outcome to the decisions that produced it.

Inspect the current harness together with observations, tool calls, robot actions, and task outcomes. Use this evidence to identify a behavior to improve and a hypothesis worth testing.

What carries forwardA concrete development hypothesis

Exploration informs development; autonomous evaluation checks the result. Submitted harnesses are frozen and evaluated without meta-agent intervention. Selection determines which version is retained.

03 Inside a harness revision

From a failed placement
to an inherited skill.

Goal-t09 asks the robot to place a cream-cheese carton on a rack. Across development, the harness acquires a transport-and-release procedure, bounded recovery, and improved lift-state interpretation.

The same scene. Three harness versions.

LIBERO · Goal-t09

Initial harness0/16

Round 19/16

Round 216/16

This rollout: unsuccessfulThis rollout: unsuccessfulThis rollout: successful

Autonomous executions from the same initial scene. Counts above summarize all 16 evaluations for each version; Round 1 fails on this particular instance. Playback is 2× simulated physical time, with inference waiting time omitted and final frames held for comparison.

Initial

0/16

The initial harness does not complete the task in the evaluation matrix.

Round 1

9/16

Add a placement skill, verify the lift before transport, and allow bounded re-localization and grasp retry.

Round 2

16/16

Refine the lift reference and release behavior. Re-evaluating the same version again yields 16/16.

Goal-t09 development: build a rack-placement skill in round one, refine lift verification and release behavior in round two. — Formal evaluation: 0/16 → 9/16 → 16/16. Values labeled “local” inside the figure refer to small tests during development.

How versions and discoveries accumulate

Harness DAG records code lineage alongside experiments and selection decisions. Development can continue from retained versions, revisit earlier candidates, and reuse useful mechanisms.

Harness development patterns: linear refinement, branching search, and recombination, with a Goal-t08 example. — The accompanying Goal-t08 example distinguishes internal tests from full formal evaluations.

04 Across three benchmarks

Different environments.
A shared path to improvement.

Compare initial harnesses with the evolved versions selected for each task. Every reported version is evaluated through autonomous execution with fixed model parameters.

LIBERO-Pro

17.595.0%

Eight T/S suites · 1,280 episodes per condition

RoboTwin

48.574.0%

50 tasks · 400 episodes per condition

RoboCasa

Seen16.4 → 24.2%

Unseen4.7 → 13.3%

32 tasks · 256 episodes per category and condition

Task success (%) · T: instruction redirection · S: position perturbation
Suite	Initial H₀	Evolved	Gain (pp)
Spatial-T	18.8	98.1	+79.4
Object-T	49.4	100.0	+50.6
Goal-T	16.3	98.8	+82.5
LIBERO-10-T	6.9	89.4	+82.5
Average-T	22.8	96.6	+73.8
Spatial-S	30.0	100.0	+70.0
Object-S	14.4	99.4	+85.0
Goal-S	4.4	97.5	+93.1
LIBERO-10-S	0.0	76.9	+76.9
Average-S	12.2	93.4	+81.3

Each suite contains 10 tasks with 16 evaluation episodes per task. The aggregate combines 224/1280 initial successes and 1216/1280 selected-version successes. Percentage-point gains are calculated before rounding.

Task success (%) · 16 tasks per category
Category	Initial H₀	Evolved	Gain (pp)
Composite Seen	16.4	24.2	+7.8
Composite Unseen	4.7	13.3	+8.6

Seen: 42/256 → 62/256. Unseen: 12/256 → 34/256. “Unseen” denotes the benchmark category; harnesses are selected separately for each task. Results use the adopted October 17, 2026 snapshot.

Task success (%) · 50 tasks × 4 scenes × 2 repetitions
Tasks	Initial H₀	Evolved	Gain (pp)
All 50 tasks	48.5	74.0	+25.5

194/400 → 296/400 successful autonomous episodes. Results correspond to the current task-specific selected harnesses.

Inspect the Goal-T development trajectory

Ten Goal-T tasks across development rounds; eight achieve a 16/16 candidate evaluation within the first three rounds. — Each heatmap cell reports candidate successes out of 16. The right-hand gains and lower curve use selected versions. Dotted cells carry a retained result after development ends; they are not additional evaluations.

05 Resources

Explore VISTA.

The method, experimental setup, and implementation.

Read the paper ↗View the code ↗

BibTeX

@misc{zhang2026vista,
  title  = {VISTA: Active Exploration for Self-Improvement
            in Embodied Agents},
  author = {Zhang Zhang and Shengjun Zhang and Simin Huang
            and Min Chen and Chong Xia and Xin Fei and Yueqi Duan},
  year   = {2026},
  url    = {https://agenticei.github.io/VISTA-page/}
}

Active Exploration for Self-Improvement in Embodied Agents