GALA: Generative Aligned Learning for Adaptive Multimodal Representation in the Taobao Shangou Recommender System
TL;DR
GALA is a three-stage pipeline extending existing learning frameworks, whose core innovation lies in an intermediate “generative RL alignment” stage that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards, effectively bridging the pretraining-fine-tuning gap to align with downstream objectives.
Abstract
Modern recommender systems in food delivery increasingly leverage multimodal signals-including images, text, and user interaction histories-to enhance user experience, yet effective fusion of these heterogeneous modalities remains challenging due to discrepancies in semantics, scale, and update frequency, hindering both the joint modeling of multimodal signals and adaptation to evolving user intent. In mainstream two-stage approaches, the separation between content-semantic pretraining of image-text encoders and behavior-driven ranking models limits alignment between semantic understanding and user behavior patterns. To address these issues, we present GALA, a three-stage pipeline extending existing learning frameworks, whose core innovation lies in an intermediate “generative RL alignment” stage that constructs multimodal pretraining data from user behavior and refines it via conversion-based rewards, effectively bridging the pretraining-fine-tuning gap to align with downstream objectives. Specifically, GALA comprises three tightly coupled stages: first, behavior-aware triplet pretraining on query-image-text pairs from search logs to early capture user intent and content preferences; second, the novel intermediate stage, which refines multimodal embeddings through rewarddriven optimization (GRPO) to dynamically align them with user behavior and bridge the pretraining-fine-tuning gap; and finally, integration of multimodal and ID embeddings via adaptive gating with a hybrid loss, preserving multimodal contributions under long-term ID-dominant training. GALA has been deployed in the production environment at Taobao Shangou, serving over 200 million daily active users. Compared with state-of-the-art (SOTA) methods, it delivers consistent offline gains of +0.12/+0.20 AUC along with better PCOC metrics. Large-scale online A/B tests further report a 0.55% increase in order volume, confirming GALA's effectiveness at industrial scale and its robustness across diverse demand patterns.