Empirical Evaluation of Trust Region Compensation for Graph Policies in Variable-Dimensional MuJoCo Control

Workspace

Problem Statement

This project tests the hypothesis that trust-region and clipping constraints in TRPO/PPO become systematically miscalibrated as action-space dimensionality changes in graph policies for variable morphologies, and that an explicit trust-region compensation method improves stability and performance across MuJoCo-style morphology benchmarks. The hypothesis would be supported if measured KL divergence, clipping fraction, policy update magnitude, and final return vary predictably with action dimensionality under standard PPO/TRPO, and if the compensated method reduces those dimension-dependent effects while maintaining or improving return. It would be refuted if action dimensionality does not produce consistent clipping/KL effects, or if compensation fails to improve stability or performance over carefully matched baselines.

Evaluation plan

Run controlled PPO/TRPO experiments across MuJoCo-style continuous-control benchmarks with multiple action-space dimensions and morphology variants. Compare standard graph policies against the proposed trust-region compensation method using identical seeds, training budgets, architectures, and optimizer settings. Record final return, sample efficiency, KL divergence per update, clipping fraction, entropy, gradient/update norms, and variance across seeds. Test whether clipping fraction and KL behavior scale with action dimension, and whether compensation reduces that scaling while preserving or improving performance. Include ablations over compensation strength and graph-policy components. Report confidence intervals and failure cases.

Datasets / Resources

Gymnasium MuJoCo-style continuous-control benchmarks
including the Swimmer morphology setting described in the submission context
with additional MuJoCo benchmark environments selected to test variable action-space dimensionality.

Empirical Evaluation of Trust Region Compensation for Graph Policies in Variable-Dimensional MuJoCo Control

Workspace

Problem Statement

Evaluation plan

Datasets / Resources

Tags

Workspace

Problem Statement

Evaluation plan

Datasets / Resources

Tags

Discussion