Capacity and Redundancy Trade-offs in Multi-Task Learning
A Capacity--Redundancy (CR) identity is investigated that decomposes the sum of per-task predictive informations into joint predictive information that includes label redundancy defined via total correlation (TC), and a residual coupling term that quantifies interference left unresolved by the shared representation.