I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

This paper presents a principled two-stage approach for distilling knowledge from Transformer models into Mamba-based State Space Models, using linearized attention as an intermediate step and achieving near-teacher performance while maintaining the efficiency benefits of SSMs.