Towards Industrial-Scale Parametric Query Optimization
TL;DR
A KL-divergence-driven model fine-tuning and plan updating strategy that dynamically adapts to workload changes is introduced, demonstrating improved scalability and robustness for industrial-scale PQO.
Abstract
Parametric Query Optimization (PQO) is crucial for efficiently executing parametrized queries (PQ) in modern industrial database systems. This paper addresses two key challenges overlooked by existing PQO techniques: large-scale template generalization and online adaptability. For large-scale workloads, maintaining one model per template is impractical, while sharing a single model across all templates leads to degraded performance. To overcome this issue, we propose a representation-based clustering strategy coupled with hierarchical model training, which significantly reduces model cost while preserving accuracy. For online adaptability, we observe that query parameter distributions shift over time, rendering fixed plan caches suboptimal. To address this, we introduce a KL-divergence-driven model fine-tuning and plan updating strategy that dynamically adapts to workload changes. Our approach is implemented on OceanBase and extensively evaluated on six workloads. Results show that it achieves up to 1.62× acceleration over the OceanBase optimizer and outperforms RankPQO, a state-of-the-art PQO method, by up to 1.23×, demonstrating improved scalability and robustness for industrial-scale PQO.