Hierarchical Structured Abstraction as an Inductive Bias for Out-Of-Distribution Generalisation

Abstract

Flat machine learning models fit training-distribution statistics and degrade when that distribution shifts at test time. We present a cross-domain research arc demonstrating that hierarchical structured abstraction, which decomposes a prediction task into fewer, topologically-grounded decisions over domain-specific primitives, acts as an effective inductive bias that mitigates out-of-distribution (OOD) degradation. Four domains are examined: (i) dialog relation extraction, where a two-layer MLP-LLM architecture exploiting closed-class word structure improves over flat LLM extraction by $\mathbf{1 3. 6 \%}$ in a small-scale pilot; (ii) conceptual combination, where graphaugmented generator-validator optimisation improves relational linkage by 6.2 points and attains the best overall combination quality over zero-shot prompting; (iii) code generation with coupled semantics, where a knowledge-graph intermediate layer with simulated-annealing selection outperforms Reflexion and OpenEvolve on 7 of 8 benchmark problems at 300-600 evaluations; and (iv) graph navigation, where a pointer network restricted to topological landmarks with exact BFS bridging achieves 95.3% vs. 91.0% OOD success on maze navigation and 85.1% vs. 78.0% on zero-shot road-network transfer, both at a $20 \times$ compute reduction. A cross-domain theoretical analysis shows that the errorcompounding bound from the navigation work (characterising when a hierarchical policy dominates a flat one as a function of decision-count ratio $T / L$) is consistent with the observed gains across all four settings, and probing experiments confirm the mechanism is representational, not merely inferential.