ReStruct: Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

Yiyuan Pan, Hanjiang Hu, Shangtao Li, Xusheng Luo*, Changliu Liu*

Robotics Institute, Carnegie Mellon University

Conference on Robot Learning (CoRL) 2026

Overview of ReStruct on PegInsert (Figure 3 of the paper). (a) Base policy 𝒫₁, a neural automaton policy. (b) The preference is compiled into a DFA; its synchronous product with the skeleton ℳ₁ yields the steered skeleton ℳ₂. (c) Demonstrations accepted by ℳ₂ are replayed to refill its action lists, producing 𝒫₂, which keeps the frozen low-level controller of 𝒫₁.

Code

The code release is in preparation and will be published in this repository. Watch the repository to be notified.

Overview

ReStruct is an inference-time policy steering method for robot manipulation. A preference stated in plain language is compiled into a temporal-logic (LTL_f) automaton and merged, through a synchronous product, into the learned state machine of a neural automaton policy (ENAP, RSS 2026: project page, code). The low-level controller stays frozen: the action priors on the state machine's edges are rebuilt from the demonstrations that satisfy the preference, so the policy follows it without retraining.

Results from the paper's tables, averaged over the preferences of each benchmark:

• ManiSkill: success with the preference kept in 70.9% of trials, against 33.7% for π_0.5 and 19.1% for OpenVLA steered by prompt injection.
• Real robot (Kinova Gen3): 92.5%, against 23.1% for π_0.5.
• CALVIN (four requested skill orders): all five skills completed in order in 86.0% of trials, against 50.8% for FLOWER and 67.6% for FLOWER fine-tuned on the requested order.

Citation

@article{pan2026inference,
  title={Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure},
  author={Pan, Yiyuan and Hu, Hanjiang and Li, Shangtao and Luo, Xusheng and Liu, Changliu},
  journal={arXiv preprint arXiv:2606.26588},
  year={2026}
}