You know how a seasoned bartender doesn't read a cocktail book every time someone orders — they glance at the ingredients on the counter and their hands move? That's what Jeff does with text classification. Instead of generating tokens one by one (the LLM default, which is like reading the recipe aloud before every pour), Jeff looks at the situation description and the options once, does a single forward pass, and returns calibrated probabilities. No text generation, no regex parsing of the output. One look, one answer, 22 milliseconds. The committed claim: full-weight fine-tuning of 0.8B and 2B parameter models on synthetic data from an open teacher model, trained on a single consumer-grade GPU in 2-3.5 hours, produces zero-shot classifiers that match or beat Jev (a commercial system running on undisclosed but much larger models) on classification and grounding benchmarks. The Jeff-Qwen3.5-2B hits 83.1% overall across five public benchmarks versus Jev's 83.0%. On Financial PhraseBank, all three Jeff variants score ~96% against Jev's 77%. The tradeoff is explicit and honest: on reasoning-heavy benchmarks (BBH, JudgeBench, JevBench hard), Jeff stays well below Jev, as the authors plainly state you'd expect at this parameter count. The architecture is straightforward: full-weight fine-tuning (not LoRA, not adapters) of pretrained decoder-only transformers, one epoch, batch size 256, cross-entropy loss over option-letter tokens. The key design choice inherited from AutoJev is a trained answer readout — the model doesn't generate free text but produces a probability distribution over option tokens in a single pass. A fitted temperature parameter handles calibration post-training. The synthetic training data was generated entirely by Qwen3.8-Flash-Next running on two DGX Sparks, with a leak filter to prevent benchmark contamination. No closed-model outputs in the training set; a closed model was used only for spot-checking synthetic data quality. The integrity picture is mixed in instructive ways. On the positive side: five public benchmarks totaling 4,599 questions, code released under MIT, training recipe fully documented, benchmark contamination explicitly filtered, and the authors are candid about where Jeff loses (reasoning tasks, game play inconsistencies). On the negative side: the Jev and AutoJev comparison numbers were measured on different samples of the same benchmarks — not identical test sets — and latency comparisons between Jeff (local GPU) and Jev (API calls including network) aren't apples-to-apples, which the authors acknowledge. The game evaluations (Doom, Frogger, Pac-Man) are fun demonstrations but 20-episode runs with a single seed aren't statistical proof of anything. The most revealing result isn't in the benchmark table — it's the games. Jeff-0.8B matched or beat a hand-coded rule bot on Doom and Frogger while the larger Jeff-2B actually played worse. The untrained Gemma 4 E2B scored highest on benchmarks among the untrained models but played games worst. This is the paper's most honest contribution: a concrete demonstration that benchmark accuracy doesn't predict real-world decision-making quality, and that bigger isn't always better for fast heuristic judgment. The fine-tuning transfer result deserves attention: a voice-navigation domain fine-tune on ~11,000 examples moved held-out accuracy from 31.7% to 95.8% in under 30 minutes on one GPU. This is the practical unlock — Jeff as a starting point for domain-specific classifiers rather than an end product. The 22-28ms latency on consumer hardware (RTX PRO 6000, M4 Max) puts this squarely in the real-time decision loop for production systems. The project is explicitly positioned as independent work building on Denis Yarats' open-source AutoJev recipe (MIT licensed), not affiliated with TypeSafe (makers of Jev). The entire pipeline — training data generation, fine-tuning, evaluation, serving — runs on local hardware with no cloud GPU dependency. This is the kind of reproducible, transparent ML work that advances the field by showing what small models can actually do when the task is scoped correctly.
