Tune to Learn: How Controller Gains Affect Robot Policy Learning

Abstract

Position controllers have become the dominant interface for executing learned manipulation policies. Yet a critical design decision remains understudied: how should we choose controller gains for policy learning? We argue that gain selection should be guided by learnability: how amenable different gain settings are to the learning algorithm in use.

Behavior Cloning (BC) learning benefits from compliant and overdamped gain regimes,
Reinforcement Learning (RL) can succeed across all gain regimes given compatible hyperparameter tuning, and
Sim-to-Real transfer is harmed by stiff and overdamped gain regimes.

These findings reveal that optimal gain selection depends not on the desired task behavior, but on the learning paradigm employed.

Video Summary

Key Findings

Behavior Cloning

BC benefits from compliant, overdamped gain regimes. Swapping to the right gain setting can improve success rates by over 30% on the same task with the same data.

Reinforcement Learning

RL can succeed across all gain regimes given compatible hyperparameter tuning. The learning algorithm adapts to the dynamics imposed by different gains.

Sim-to-Real Transfer

Sim-to-real transfer is harmed by stiff, overdamped configurations. Compliant gains reduce the sim-to-real gap and improve transfer success.

BibTeX

@inproceedings{author2026method,
  title     = {Your Paper Title Goes Here},
  author    = {One, Author and Two, Author and Three, Author and Four, Author},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2026}
}