Imagine you're proofreading a sentence and you swap one synonym for another — 'big' for 'large.' If the reader's comprehension changes, you know they weren't really reading for meaning; they were keying on the specific word. But if you swap 'big' for 'quiet,' and comprehension changes, that's a real semantic shift. This paper applies exactly that logic to audio-language models: swap a sound event for another instance of the same class (synonym), and if the model's answer flips, you've caught sensitivity to waveform artifacts rather than genuine auditory understanding. The committed claim: by comparing matched same-class and different-class replacements of a single sound event in a synthetic soundscape, you can isolate whether a model's answer change reflects recognition of the event's semantic class or mere sensitivity to the act of editing the waveform. This is a methodological contribution, not a new model — it's a diagnostic protocol. The construction is careful. Both replacement types use the same temporal interval, the same level estimation, the same donor-fitting procedure, and the same crossfade. The only variable is whether the replacement sound belongs to the same class (preserving the correct answer) or a different class (changing it). Across 600 synthetic soundscapes and three models from two architecture families, different-class replacements caused 49.7 to 60.9 percentage points more answer changes than same-class replacements. Same-class accuracy stayed within 5.2 pp of baseline. That's a large, clean separation. The selective-accuracy metric adds a second layer: among trials where both the original and same-class answers were correct, soundscape-weighted accuracy for the inserted different class ranged from 73.7% to 81.9%. This means the models aren't just flipping coins when the class changes — they're often correctly identifying the new class, suggesting genuine class-level recognition rather than arbitrary confusion. The integrity picture is solid for a methods paper. The protocol is reproducible: code and frozen evidence are publicly available. The 600 synthetic soundscapes provide a controlled testbed, though synthetic audio is inherently cleaner than real-world recordings. Three models across two architecture families gives some breadth, but the paper doesn't name specific model identities in the abstract, making it harder to assess whether the strongest current audio-LMs were tested. The limitation that matters: all soundscapes are synthetic. Real-world audio has overlapping events, variable recording conditions, and ambient noise that could narrow the same-class vs. different-class gap. The paper's clean separation may represent an upper bound on how well this protocol discriminates in the wild. The authors haven't run the natural-audio version — likely because constructing matched same-class and different-class replacements in real recordings is much harder to control. This is a tooling contribution, not a capability breakthrough. Its value scales with adoption: if the audio-AI community uses this protocol as a standard diagnostic, it becomes important infrastructure. If it stays in one lab, it's a well-executed methods paper and not much more. The 49.7–60.9 pp gap is the headline number — it's large enough that the distinction between class-driven and artifact-driven answer changes is unambiguous in the synthetic setting.
