Frederik Pahde

ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on

This work reframe shortcut auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON), which separates learned shortcuts from merely correlated concepts, which it confirms with independent validation checks.

ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts

Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should be audited for reliance on a set of concepts, such as acquisition artifacts or demographics. Current methods, such as linear probes and concept activation vectors, measure reliance by asking...

ICON Decomposition : Multivariate Concept-Level Explanations of Deep Representations for Model Auditing

Safety Signals to Verify NetOps Agents with Action-Level Granularity

This work shows that agent verifiers leveraging internal signals predict both harm and progress more reliably than a baseline using observable signals only, and aims to use these signals as safety feedback to an agent harness to abstain from risky actions and protect the target system.

ICON decomposition : Auditing deep neural networks with multivariate, variance-based concept-level explanations

CON decomposition is introduced, which quantifies how much of a layer's variance each concept explains given all other concepts and the outcome, and how much none of them explains, yielding layer-comparable, calibrated scores that suppress false positives.