The Compositional Advantage: Why Structure Helps Reasoning

The three tasks where a block-diagonal FFN beat the dense baseline by the largest margins are all compositional: Boolean SAT, logical reasoning, and multi-hop QA. Here is why that pattern is not an accident.