Imagine you're a bouncer at a nightclub with a guest list. Normally the list arrives before the crowd — easy. But what if each person in line has to whisper their own name to you, and the list only becomes readable after half the crowd has already passed through? That's Mamba's retrieval problem. The selection mechanism (the bouncer) has full authority to admit or reject tokens, but the relevance signal (the guest list) hasn't resolved by the time the critical gating decisions are made. The bouncer is powerful enough; the list just arrives late. The paper's committed claim: Mamba's in-context retrieval failure is NOT primarily about fixed-size state capacity. Even in query-first settings — where the query precedes the context and state capacity should be sufficient — Mamba substantially underperforms Transformers. The bottleneck is a timing problem across network depth, not a memory problem. This directly challenges the dominant explanation in the SSM community. The standard story says Mamba's recurrent state is a fixed-size bottleneck that loses information as context grows — a compression argument. Yarden, Ben-Artzy, and Schwartz show this explanation is incomplete by constructing the most favorable possible setup: query-first ordering, where the model knows what to look for before it sees any candidates. If state capacity were the binding constraint, query-first should close the gap. It doesn't. The mechanistic dissection is where the paper earns its weight. The authors intervene directly on Mamba's selection gates — the input-dependent parameters that control how much each token updates the recurrent state. They show these gates have sufficient expressive range: artificially setting gates to oracle values (fully open for relevant tokens, fully closed for distractors) swings accuracy between near-zero and near-perfect. The mechanism isn't broken — it's misinformed. Probing token representations across layers reveals that relevance information (whether a token matches the query) emerges gradually through depth, and the gates begin reflecting it only several layers after it first appears. In the layers where gating decisions matter most for retrieval, the relevance signal hasn't yet propagated into the representations the gates consume. This is a depth-alignment problem, not a width problem. Transformers sidestep it entirely because attention is computed globally at each layer — every token can directly compare itself to the query at every depth. Mamba's recurrence forces a sequential, layer-by-layer information propagation where the relevance signal must percolate through representations before the gates can use it. The architecture's inductive bias creates a temporal lag between signal availability and decision point. The implications are concrete for anyone building with SSMs. Hybrid architectures that insert attention layers at retrieval-critical depths could address this specific bottleneck without abandoning SSM efficiency everywhere. The paper also suggests that training procedures or architectural modifications that accelerate relevance-signal emergence in early layers could help. But the deeper lesson is that expressiveness is not the same as alignment — a mechanism can be theoretically capable of solving a task while systematically failing because its inputs aren't ready when it needs to act. This is a clean, well-scoped mechanistic study. It doesn't propose a fix; it diagnoses a failure mode with surgical precision. The field fight between attention and recurrence won't be settled by one paper, but this one narrows the argument: the SSM camp can no longer point only at state capacity as the retrieval bottleneck. The problem is architectural and temporal, and fixing it requires rethinking how information flows across depth, not just how much state fits in memory.
