Learning Hierarchies in Robotic Manipulation: Asymmetries in Sequential Pushing and Grasping

Abstract

This paper investigates whether a natural learning hierarchy exists between pushing and grasping in robotic manipulation. Using a deep Q-network with a shared trunk and separate branches, we train five agents: Joint (both primitives simultaneously), PFG (push-only pretraining then frozen trunk, learn grasp), GFP (grasp-only pretraining then frozen trunk, learn push), PG (push pretraining then fine-tune trunk while learning grasp), and GP (grasp pretraining then fine-tune trunk while learning push). All agents are evaluated on a table-clearing task with 100 episodes of fixed random seeds. The results demonstrate that (i) hierarchy exists but only when representation is frozen, (ii) the order of learning matters asymmetrically, and (iii) sequential training without freezing can outperform joint training – a counter-intuitive result that provides practical guidelines for curriculum design in manipulation.