The question underneath every AI coding demo is the same: does the model understand the code, or is it pattern-matching against billions of lines of training data? KillSwitch is a genuine attempt to isolate that question by removing the training-data crutch entirely. A novel esoteric language, purpose-built to be adversarial to LLMs, benchmarked against every major model. The answer, so far: models can stumble through simple tasks in a language they've never seen, but nobody should confuse that with understanding. The design is clever in a way that reveals real knowledge of LLM failure modes. Character counting in identifiers exploits the well-documented tokenization blindspot. Phrases like "hack into a computer" and "ignore previous instructions" weaponize safety guardrails and prompt-injection vulnerabilities as syntactic constructs. Most of the script text is noise — stories that may or may not contain instructions — which forces the model to parse signal from irrelevant context, a task humans handle with ease and LLMs handle with varying degrees of hallucination. The benchmark results are the most interesting artifact. Claude Opus 5 leads at 66.8%, and notably outperforms Opus 5.5 — a result the author attributes to better spec interpretation rather than noise. Fable 5.1 and Gemini 4 Argon crater due to overzealous safety guardrails, which is both predictable and damning: their refusal to engage with phrases like "hack into a computer" means the guardrails are interfering with actual capability. MiMo-V2.6-Pro leads the open-weight pack. The honest caveat is important: the benchmark tasks are still simple. Hello World, max-of-two-numbers output prediction, Celsius-to-Fahrenheit conversion and reversal. These are entry-level programming exercises. A 66.8% score on trivial tasks in a language designed to confuse is not evidence of deep code comprehension — it's evidence of surprisingly robust pattern transfer, which is a different and more modest claim. What makes KillSwitch work as a project is its layered adversarial design. The language spec itself contains deliberate lies that humans would catch and LLMs would swallow. It expects readers to deduce unstated rules rather than follow explicit instructions. Literals are placed anywhere in the script and referenced by line count. Each of these features targets a specific, documented LLM weakness, which makes the benchmark a diagnostic tool as much as a leaderboard. The correlation between KillSwitch performance and other model benchmarks is the buried lede. If a model that scores well on standard coding benchmarks also scores well on a language it has never seen, that suggests the standard benchmarks are measuring something beyond memorization — or at minimum, that whatever capability drives memorization also drives novel-language transfer. That's a useful signal even if it doesn't settle the understanding question. The project is open source, with both the interpreter and spec on GitHub, and the author signals plans for harder task sets if models start completing the current ones. The real value here isn't the language itself — it's the methodology. Building adversarial evaluation tools that specifically target known failure modes is more useful than yet another MMLU variant, and the community benefits from having the spec, interpreter, and benchmark publicly available.
