Reflqt.
HOMERESEARCHPUBLICATIONSBLOGEXPLOREABOUTCONTACT →
§ art-018-bd4-downstream · April 24, 2026 · BREAKTHROUGH

Block-Diagonal Transformers Win 8 of 17 Downstream Tasks

Refleqt Labs · 7 min read

Across 17 downstream tasks at five seeds each, a block-diagonal FFN beat the dense baseline eight times, tied nine times, and lost zero -- with compositional tasks dominating the win column.

inside the ffnblock diagonaldownstream tasksstructured matrices
RELATED
APR 19Block-Diagonal Matrices: Why Sparsity Structure Matters
7 min→
APR 24When Your Results Are Real But Your Sample Size Is Not
6 min→
APR 24The Compositional Advantage: Why Structure Helps Reasoning
7 min→
Reflqt.

Every claim, plotted. A research lab in public.

Research
All entriesPublicationsBlog
Explore
All demosAlgebraKnowledge Layer
Lab
About · Open letterContactGitHub
© 2026 REFLQT. LABSv2026.04 · plotted