Federated Computation of ROC and PR Curves

Abstract

Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are fundamental analytical tools for evaluating classification models, providing a comprehensive view of performance across decision thresholds. In modern data management settings, such evaluation must often be performed over data that is distributed across multiple parties and subject to strict privacy constraints. This setting arises naturally in federated learning (FL) and collaborative data platforms, where raw prediction scores and labels cannot be centrally collected. We study the problem of computing ROC and PR curves over distributed data under formal privacy guarantees. The key challenge is that exact computation requires access to all prediction scores and labels, leading to linear communication costs and privacy risks. To address this, we propose a novel framework for approximating ROC and PR curves using quantiles of the score distribution, which can be computed efficiently under secure aggregation and distributed differential privacy. Our approach requires only $O(Q)$ communication, where $Q$ is the number of quantiles, and avoids sharing raw data entirely. We provide theoretical guarantees on the approximation quality by bounding the Area Error (AE) between the true and estimated curves, demonstrating principled trade-offs among approximation accuracy, privacy protection, and communication cost. Our method is robust to data heterogeneity and skew, making it suitable for real-world distributed data management scenarios. Experiments on real-world datasets show that our approach achieves high accuracy with low communication overhead under strong privacy constraints. Overall, this work provides a practical and theoretically grounded solution for privacy-preserving evaluation of classification models over distributed data.