Auditability for Federated Learning

SoK: Verifiable Integrity Claims for Privacy-Preserving Federated Learning

This SoK model federated learning as an append-only transcript of submissions, admissions, aggregation, and finalization events, and formalize verifiability as a collection of integrity claims issued by clients and the aggregator, and checked by different verifier classes.

Auditability for Secure Machine Learning in the Cloud

Trusted Credentials, Untrusted Behavior: Benchmarking LLM-Agent Security in High-Performance Computing

Large language model (LLM) agents are starting to take on routine work in high-performance computing (HPC), including monitoring Slurm jobs, diagnosing failed builds, inspecting simulation output, and coordinating scientific workflows. To do this work, an agent commonly acts under its user's credentials and inherits the user's access to files and the scheduler. This arrangement creates a failure mode that ordinary account-level controls do not capture. Adversarial instructions in a log, tool description, shared file, or peer-agent message may redirect the agent beyond the task the user assigned, even though every resulting command is authenticated and permitted for that account. We refer to this as the hijacked authorized agent problem. Existing agent-security studies explain relevant mechanisms, such as indirect prompt injection and tool misuse, but generally evaluate them in web, enterprise, or personal-assistant settings. HPC security, by contrast, has mature controls for identity and isolation but does not ordinarily represent the intent of a particular task. This paper defines the threat model in the HPC setting, identifies attack surfaces created by schedulers, shared storage, multi-project accounts, and scientific workflows, and examines where current controls fall short. It concludes with a research agenda and a plan for an empirical benchmark, TaskBound.

Manipulation-Proof Oblivious Audits against Deceptive Model Providers

Audits have emerged as a critical instrument for algorithmic governance, providing a mechanism for external scrutiny and governance of machine learning models. However, ensuring the integrity of such assessments remains a challenging issue. For instance in regulatory contexts, audits are typically declared or easily detected, thus enabling model providers to manipulate the process, whether intentionally or inadvertently. This vulnerability is particularly acute in the context of fairness evaluations, in which providers can often infer sensitive attributes and strategically equalize allocation rates between groups to satisfy fairness metrics. In this paper, we introduce a novel audit protocol designed to significantly increase the post-audit detectability of such manipulations by enabling the auditor to query the model in an oblivious manner. Our approach leverages a Private Information Retrieval mechanism to require the provider to label a large set of instances, while preventing it from knowing which subset will ultimately be used for the audit. The protocol is efficient, imposes minimal overhead on the auditor, and requires no modification to the audited model, its training procedure, or its inference pipeline. We provide theoretical guarantees showing that, under this protocol, a provider attempting to hide unfairness must falsify a significantly larger number of responses, thereby increasing both the difficulty and the likelihood of detection of manipulation. Experimental results across representative audit scenarios confirm the effectiveness and practicality of our approach.

Poison to Detect: Detection of Targeted Overfitting in Federated Learning

Federated Learning (FL) enables collaborative model training among clients without centralising data, making it a widely adopted privacy enhancing technology (PET). Despite its privacy benefits, FL remains vulnerable to orchestrator-driven privacy attacks. In this paper, we study an underexplored threat in which a dishonest orchestrator intentionally manipulates the aggregation process to induce targeted overfitting in local models of specific clients. Although prior work focuses on reducing information leakage during training, we emphasise early client-side detection of targeted overfitting, allowing clients to disengage before significant harm occurs. To this end, we propose three detection techniques—label flipping, backdoor trigger injection, and model fingerprinting—which enable clients to verify the integrity of the global aggregation. We evaluated our methods across multiple datasets and attack scenarios. In single-client attacks, all three methods detect orchestrator-induced overfitting within 1–2 training rounds with F1 scores exceeding 0.7. Scalability experiments further show that detection effectiveness is influenced by cohort composition and method parameters. These results demonstrate that client-side integrity testing can provide early, effective, and scalable detection, supporting safer deployment of FL systems.

Poisoning Attacks in Federated Learning: An Accountability- Oriented Survey with Centralized Learning as a Baseline

Artificial intelligence (AI) systems are increasingly deployed in high-stakes domains, where poisoning attacks can corrupt training data, manipulate model updates, or implant covert backdoors. This survey examines poisoning attacks in federated learning (FL), using centralized learning as a baseline to explain how distributed data, client heterogeneity, privacy-preserving aggregation, and untrusted coordination expand the threat surface. It positions prior surveys and synthesizes representative primary studies through an accountability-oriented lens focused on attribution, audit evidence, traceability, and forensic readiness. The review compares major attack classes, including data poisoning, model poisoning, backdoor insertion, server-side manipulation, Sybil behavior, collusion, and multi-round poisoning. It also evaluates countermeasures such as Byzantine-robust aggregation, anomaly detection, validation-based filtering, malicious-secure aggregation, authenticated update handling, provenance mechanisms, ledger-based evidence, and verifiable aggregation protocols. The analysis shows that robustness alone is insufficient for trustworthy FL unless defenses also preserve evidence that supports independent verification, post-incident reconstruction, and governance review. Persistent gaps remain in causal forensic attribution, privacy-preserving evidence governance, malicious-server threat modeling, scalable verifiability tooling, recovery after poisoning, and deployment-ready benchmarks. The survey concludes that accountable FL should be designed as an evidence-producing system, not merely as a privacy-preserving or attack-resistant training architecture, especially for regulated, cross-silo, and high-risk real-world deployments.