Wojciech Samek

ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on

This work reframe shortcut auditing as a decomposition of the variance of intermediary representations with Independent Canonical cONcepts (ICON), which separates learned shortcuts from merely correlated concepts, which it confirms with independent validation checks.

ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts

Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should be audited for reliance on a set of concepts, such as acquisition artifacts or demographics. Current methods, such as linear probes and concept activation vectors, measure reliance by asking...

Concept-based explanation of gene expression prediction from H&E images

Recent advances in pathology foundation models have enabled accurate prediction of spatial transcriptomics (ST) from routine H&E images. However, existing explainability methods for vision transformer (ViT)-based models are largely limited to local heatmaps and do not reveal how morphological concepts contribute to ST...

ICON Decomposition : Multivariate Concept-Level Explanations of Deep Representations for Model Auditing

Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations

Lie detection probes aim to predict from a language model's internal states whether its output is truthful or dishonest. However, role-play complicates what"truth"means for an LLM: language models can adopt a wide range of personas that take very different claims to be true, including personas whose beliefs clearly con...

Can Data Attribution Filter Out Subliminal Learning? Not Reliably

The results suggest that gradient-based attribution can identify data responsible for subliminal learning in some settings, but that some approximations are more reliable than others.

ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers

Residual-aware Layer-wise Relevance Propagation (ResLRP) is introduced, a simple extension of LRP whose propagation rules explicitly account for cancellations in residual branches, are exactly conservative, and provably bound relevance explosion.

Safety Signals to Verify NetOps Agents with Action-Level Granularity

This work shows that agent verifiers leveraging internal signals predict both harm and progress more reliably than a baseline using observable signals only, and aims to use these signals as safety feedback to an agent harness to abstain from risky actions and protect the target system.

ICON decomposition : Auditing deep neural networks with multivariate, variance-based concept-level explanations

CON decomposition is introduced, which quantifies how much of a layer's variance each concept explains given all other concepts and the outcome, and how much none of them explains, yielding layer-comparable, calibrated scores that suppress false positives.