Learning Hierarchical Skill Policies with Offline Quality-Diversity Reinforcement Learning
QDOS (Quality-Diversity Offline Skill learning), a unified pipeline for robust offline-to-online learning, incorporates an Advantage-Weighted Quality-Diversity pretraining objective, which weights the skill extraction and diversity objectives by the estimated advantage of each trajectory segment.