The Transformer architecture is seven years old and powers essentially everything in modern AI, yet most people who use LLMs daily could not explain what a Query matrix does or why attention scores get masked. Transformer Explainer exists to fix that specific gap: it is an interactive visual walkthrough of GPT-2 (small), the 124-million-parameter model that shares its DNA with GPT-4, Llama, and Gemini but is small enough to run in a browser and simple enough to diagram without lying. The approach is mechanistic, not metaphorical. You feed it a prompt — "Data visualization empowers users to" — and watch the text get tokenized, embedded into 768-dimensional vectors, positionally encoded, and then passed through 12 Transformer blocks. Each block's multi-head self-attention and MLP layers are visually decomposed: you can see Query, Key, and Value matrices being computed, the dot product producing attention scores, the mask zeroing out future tokens, and softmax converting raw scores into probability distributions. The MLP's 4× expansion from 768 to 3072 dimensions and compression back is shown explicitly. Nothing is handwaved. The QKV web-search analogy — Query as search text, Key as page titles, Value as page content — is the kind of intuition-builder that actually sticks. It maps the mechanism (similarity matching followed by content retrieval) rather than just the surface topic. The masking explanation is equally clear: the model cannot peek at future tokens during training, so the upper triangle of the attention matrix is set to negative infinity before softmax. This is the sort of detail that separates understanding from vibes. What distinguishes this from the dozens of "Transformers explained" blog posts is the interactive component. The tool is powered by a live GPT-2 (small) model, so you are not reading about abstract matrix multiplications — you are watching real numbers flow through real weights. The embedding layer view (Figure 1) shows the four-step pipeline from raw text to final embedding. The attention view (Figure 3) shows dot product, scaling, masking, softmax, and dropout as sequential operations on actual matrices. This is closer to 3Blue1Brown's pedagogical style than to a typical Medium post. The scope is deliberately limited and that is a strength. By choosing GPT-2 (small) — 12 blocks, 12 attention heads, 768-dimensional embeddings, 50,257-token vocabulary, ~39 million parameters in the embedding matrix alone — the tool avoids the impossible task of visualizing a 405-billion-parameter model while preserving every architectural principle that matters. The paper correctly notes that GPT-2 shares the same components as current state-of-the-art models. The differences (scale, training data, RLHF, mixture-of-experts) are important but orthogonal to the architectural fundamentals this tool teaches. The limitations are obvious and honest. This is not a research contribution — it is a pedagogical one. It does not cover training, fine-tuning, RLHF, or any post-2019 architectural innovations (rotary positional embeddings, grouped-query attention, sliding-window attention). It teaches the decoder-only Transformer and nothing else. For its intended audience — engineers, students, and curious practitioners who want to understand the machinery before engaging with the literature — that is exactly the right scope. The craft here is in the reduction. Compressing the full Transformer pipeline into an interactive visual flow without losing mathematical precision is genuinely hard. The equations are present (the QKV computation formula is shown explicitly), the dimensions are concrete (768, 3072, 50257), and the operations are sequential and traceable. This is what good technical education looks like: mechanism over mystique, precision over hand-waving.
