You know how autocomplete on your phone works? It guesses the next word based on what you've typed so far. Now imagine your phone had no language model at all — just a zip program and a copy of Shakespeare. Every time you needed the next word, it tried every possibility and picked whichever one the zip file shrank the most. That's GziPT. The core insight is the compression–prediction equivalence from information theory: any compressor implicitly assigns probabilities. The number of bits needed to encode a symbol is −log₂(p), so if gzip compresses a continuation cheaply, it's effectively saying "I expected that." The author exploits this by scoring candidate byte sequences via len(gzip(context + candidate)) — smaller compressed output means higher implicit probability. The corpus is loaded into gzip's 32 KiB sliding window as a kind of zero-parameter "training set," and anything that echoes the corpus compresses small. The generation problem is harder than scoring. gzip returns integer byte lengths, so single-byte candidates frequently tie — the signal drowns in quantization noise. The fix is beam search: maintain the top-k partial continuations, extend each by every byte that appears in the corpus, score all extensions by compressed length, prune back to k, and repeat for a horizon of bytes before committing. This lookahead smooths the quantization floor enough to produce recognizable structure. A practical detail matters: only the most recent tail of generated output stays in the scoring context. DEFLATE encodes nearby matches more cheaply than distant ones, so without this truncation the model degenerates into verbatim self-copying loops. The tail window acts as a crude recency bias, forcing the generator to draw from the corpus rather than its own output. The output — real, unedited — reads like Shakespeare through a blender. Character names land correctly, punctuation and line structure are plausible, but semantic coherence collapses within a clause. This is exactly what you'd expect from a model whose "memory" is a 32 KiB byte-match window with no abstraction hierarchy. It's a ceiling demonstration: here is what pure local pattern matching buys you, and here is where it stops. The project builds on the 2023 paper "Language Modeling is Compression," which established the theoretical equivalence but reported poor generation results. The author's contribution is showing that beam search — mentioned but not implemented in the paper — materially improves output quality. The entire implementation is one file of standard-library Python using zlib. No dependencies, no GPU, no training loop. This is not a practical language model. It's a pedagogical artifact that makes information theory tactile. The value is in what it clarifies: that prediction and compression are the same operation viewed from different angles, that neural language models are doing something gzip does badly but structurally similarly, and that the gap between "some structure" and "coherent text" is where learned abstractions earn their keep.
