Terence Eden, a developer working on an NLnet-funded project called ActivityBot, ran a simple experiment: he paid volunteers €25 each to share their screen, follow his README instructions from scratch, and narrate their confusion aloud. Total cost: roughly €150 for six sessions. The yield was a list of failures no automated tool or self-review could have caught. The bug list is instructive precisely because it's mundane. A broken demo link. Users who read READMEs in the terminal, where formatting assumptions collapse. People who didn't know how to rename a hidden file. Quoted text that looked mandatory in some places and optional in others. An entire section the author found technically interesting that confused every single tester. These aren't edge cases — they're the default experience for anyone who isn't the author. The method borrows directly from usability testing as practiced at GOV.UK, where Eden previously did technical writing. The key discipline: think-aloud protocol. Volunteers narrate what they're doing, what they expect, and where they get stuck. The author watches the gap between intent and comprehension open in real time. After each session, Eden updated the README and re-tested with the next person — iterative improvement with a human feedback loop. Eden explicitly rejects the suggestion to use an LLM to simulate user confusion, and his reasoning is worth taking seriously. The signal isn't just the failure list — it's the frustration in someone's voice, the pause before they ask a question they think is stupid, the cat that walks across the keyboard mid-session. These are the cues that tell you which failures are annoying and which are blocking. An LLM can generate plausible confusion; it cannot generate real confusion with real stakes. The economics are striking. €150 bought six sessions of high-quality, context-specific feedback on a piece of documentation that will be read by every new user of the project. Compare that to the developer-hours lost when users hit the same broken link, the same hidden-file confusion, the same unexplained terminology — multiplied across every installation attempt. The ROI is asymmetric: small upfront cost, compounding downstream benefit. The deeper pattern here is about who bears the cost of bad documentation. In the status quo, every user independently discovers every documentation failure — a massively duplicated cost distributed across the least powerful participants in the ecosystem. Eden's method inverts this: the developer absorbs a small, one-time cost to eliminate repeated friction for all future users. This is textbook generative behavior — spending effort once to create value that compounds. The implicit argument is that documentation quality is a design problem, not a writing problem. You cannot edit your way to clarity if you cannot see your own assumptions. Fresh eyes aren't optional; they're structural. The fact that this is considered novel enough to blog about tells you how low the bar is in open-source documentation practice.
