Turbopuffer, the serverless vector database used by Cursor and Notion, is undertaking a fundamental storage architecture overhaul. The company's v3 redesign removes the approximate nearest neighbor (ANN) vector index from its role as the primary index — the key around which all documents, attributes, and full-text search postings are organized — and demotes it to "just another" secondary index. The change is motivated by three concrete engineering bottlenecks that the vector-primary design imposes. The original v1 architecture stored documents as an ID and a vector, clustered hierarchically using SPANN (later SPFresh) for incremental indexing. Everything was keyed by ClusterId and LocalId — the "ANN address." This worked well for cheap, reasonably fast vector search on object storage with tiered NVMe SSD/memory caches, and scaled to single indexes of 100B+ vectors serving 200ms p99 reads at 1,000+ QPS. But as turbopuffer added attribute filtering, BM25 full-text search, regex, fuzzy matching, sparse vectors, and aggregations (the informal v2 era), the ANN-primary layout became a straitjacket. Three specific costs have accumulated. Storage amplification: multi-vector document representations (nesting, late interaction) force full document duplication per vector. Write amplification: SPFresh rebalancing cascades to moving all document contents and inverted indexes, meaning a single vector update can move hundreds of attributes. Limited vectorization: modern query engines run tight loops over blocks of thousands of rows (DuckDB uses 2,048, ClickHouse up to ~65k, Lucene 256), but ANN clusters constrain block sizes to 100–200 documents, starving CPU pipelines for non-vector query plans. The vectorization constraint is quantified by turbopuffer's own experience. Their first full-text search implementation partitioned posting lists along ANN cluster boundaries, producing a median block of ~1.5 postings. Reworking postings into fixed 256-doc blocks made the FTS index 10x smaller and queries up to 20x faster. That improvement was possible because posting lists are stored separately and point at documents. Aggregations and scans read documents directly and remain stuck at cluster-sized blocks under the current layout. The v3 fix is conceptually simple — stop keying on the ANN address — but the migration is non-trivial. Turbopuffer reports that 100% of CI now passes on v3, with the team entering performance optimization. Benchmarks will be published incrementally as they work toward and beyond parity with the current production system. The company currently hosts 1T+ documents, handles 10M+ writes per second, and serves 25k+ queries per second. The strategic implication is clear: turbopuffer is repositioning from "vector database" to "search engine that also does vectors." The post explicitly frames the goal as moving many more SQL queries to turbopuffer. This mirrors a broader industry pattern where specialized vector databases are being absorbed into general-purpose analytical engines (or evolving into them), as the hype cycle around standalone embedding search matures into practical hybrid workloads. The risk is real. Turbopuffer's competitive moat was built on ANN performance tuned to object storage economics. Any regression in vector search latency during the transition could cost customers who came for exactly that capability. The bet is that the efficiency gains on non-vector queries — and the ability to serve SQL-shaped workloads — will expand the addressable market faster than any ANN regression shrinks it.
