
Read the paper
Do Embedding-Map Distances Guide LLM Ideation? Four Tests Against What Came Next
on GitHub
Imagine you have a map of every restaurant in your city, plotted by cuisine similarity. The popular ideation trick says: find the biggest empty patch on the map and open a restaurant there — that's where unmet demand lives. This paper runs four careful tests asking whether that trick actually works when the "map" is an embedding space and the "restaurants" are games, papers, or patents. The answer, across every test, is that the empty patches are empty for a reason — they're mostly noise, not opportunity. The committed claim: embedding-space distances do not reliably guide ideation toward what will actually appear next. Test 1 shows that random far-away directions in embedding space don't even correspond to coherent text — cosine similarity between far points and later real items barely reaches 0.10, and an LLM decoder shown neighbors with score-only feedback simply rewrites those neighbors rather than producing genuinely novel content. A neighbor-free search can produce coherent text (median cosine 0.57 to interior gaps) but only 0.07 to truly far points. The map's blank spots aren't latent gold mines; they're off the edge of meaningful language. Test 2 uses k-means cells on a past-only map to ask whether "registered holes" — empty or sparse cells — predict where future items will land. The hole score mostly tracks cell size and position, not genuine opportunity. Three density-based gap scores forecast which cells grow fastest at or below chance (AUC 0.30 to 0.66) and add nothing to momentum in Steam and arXiv data. Patents were inconclusive. The one interesting nugget: more isolated new Steam games get slightly better reviews, but that's a weak and isolated signal. Test 3 is the most subtle. When an LLM is asked to continue an arXiv paper, its proposals stay closer to the source than papers that actually cite the original — tested under three user prompts and against gpt-4o rewrites and OLMo completions from April–December 2023. Critique prompts lengthen the step beyond content-free revision, but revealing a real follow-up's title only turns the LLM's ideas by 0.032 along the follow-up direction. Reviewer, oracle, and distance feedback add fractions of that turn (0.13, 0.08, 0.34 respectively) with upper bounds at 0.46, 0.46, and 0.70, and those turns aren't separable from the revision step itself. Translation: LLMs generate plausible continuations, but distance-based steering barely nudges them toward what actually happened. Test 4 checks whether inspiration material at intermediate distances — the Goldilocks zone — produces the best ideation. No inverted-U was detected, and the anticipation margin tracked format more than content. The expected sweet spot between too-close and too-far simply doesn't materialize in the data. Methodologically, this paper is unusually careful for the ideation-with-AI literature. The original tests were pre-registered before running. Thirteen follow-up tests were requested during two AI-agent reviews and registered after main results but before execution. All registrations are dated by local git commits. Code and data pointers are public. The integrity story is strong relative to the subfield, though git-commit dating is weaker than a formal timestamping service. The practical takeaway is deflationary. If you're building an AI-assisted ideation tool that works by finding "gaps" in embedding space and decoding them into ideas, this paper's evidence says you're chasing noise. The empty regions of an embedding map are not a roadmap to future innovation — they're artifacts of the embedding's geometry. Real innovation appears to follow paths that density-gap metrics cannot distinguish from momentum. The gap between what embeddings encode and what constitutes a viable new idea remains wide and poorly understood.
The analogy
Imagine a map of every restaurant in your city plotted by cuisine similarity. The popular ideation trick says: find the biggest empty patch and open a restaurant there. This paper tests whether empty patches on the map are real opportunity or just noise from how the map was drawn. Across four tests, the blanks are mostly cartographic artifacts, not unmet demand.
Restaurant map
Blank patches on a cuisine-similarity city map as supposed signals of unmet dining demand
Embedding gaps
Empty regions in embedding space as supposed signals of unexplored innovation opportunities
Honest assessment
This is a meticulous, solo-author negative result that deflates a popular but lazy assumption in AI-assisted ideation: that embedding-space gaps are innovation opportunities. The integrity is strong — pre-registered, public code, three domains — but the architecture is off-the-shelf and the milestone is conceptual rather than quantitative. Practitioners building ideation tools should read this. Researchers will find it useful as a methodological template for how to rigorously test intuitive claims about latent space geometry. It's not flashy, but it's honest, and the field needs more papers like it.
What Is CLAIMS?
60
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
4/5
Ladder
3/5
Architecture
2/5
Integrity
4/5
Milestone
2/5
Successor
3/5
What is this paper actually committing to?
Genuine negative result deflating a popular but untested assumption. Not a first-of-kind discovery but a first rigorous falsification — the ideation-via-embedding-gaps recipe fails four distinct tests.
The field fight
Can embedding-space geometry guide AI-assisted ideation toward genuinely novel ideas?
Side A
Empty regions in embedding space signal unexplored opportunity; decode far points into novel ideas
Held by: Latent-space exploration tools, generative design, AI brainstorming startups
Side B
Embedding gaps are geometric artifacts, not innovation signals; real novelty follows paths that distance metrics can't predict
Held by: This paper, skeptics of AI creativity tools
Milestone tracker
Unlocks: A validated signal for AI-assisted ideation that beats simple momentum
Current
0.66
AUC
best gap score (this paper)
Next
0.75
AUC
meaningful prediction threshold
Conceptual barrier — not clear any distance-based method can get there
Verified Facts (9)
No later item reaches a raw cosine above about 0.10 with far random directions
After centring or whitening, 22 of 24 cells stay out of reach
Neighbour-free search writes coherent text at median cosine 0.57 and 0.47 to interior gaps
Three density-based gap scores forecast which cells grow fastest at or below chance (AUC 0.30 to 0.66)
LLM continuations of arXiv papers stay closer to the source than papers that cite it
Revealing a real follow-up's title turns the ideas by 0.032 along a follow-up-specific direction
Reviewer, oracle and distance feedback add 0.13, 0.08 and 0.34 of that turn (upper bounds 0.46, 0.46, 0.70)
Original tests were registered before running, follow-ups after the main results
48 pages, 5 figures
Practical Implications (4)
If you're building an AI ideation tool that finds 'gaps' in embedding space and generates ideas there, this evidence says your core signal is noise — reconsider the architecture
Momentum-based forecasting (what's growing fastest) is at least as good as gap-based methods for predicting where new work appears
LLM-generated continuations of research papers stay closer to the source than actual follow-on work — LLMs extend rather than innovate from a given paper
Pre-registration with public code is achievable even for solo-author ideation research; this paper sets a methodological bar the subfield should adopt
More latest news
Read all