RemoteANN: Cloud Native Vector Search
RemoteANN does nearest-neighbor search directly on object storage with up to 16.1x fewer requests and 5.9x higher throughput than baselines.
Abstract
The paper introduces RemoteANN, achieving up to $16.1 imes$ fewer object-storage requests and $5.9 imes$ higher throughput than baselines when searching near-neighbors in cloud-based vector data, demonstrating it is the speed and cost leader at approx. 97% recall. RemoteANN employs IVF with RaBitQ built for Cloud, leveraging locality-aware vector placements, analytical pruning, fine-grained Range-GET, request merging, and NVMe caching, to optimize index and query processing with cloud specifications. The system includes QGKMeans for fast index construction, ensuring operational feasibility at large scales by employing intelligent centroid assignment that matches state-of-the-art clustering speed while retaining recall performance in benchmarks. Problem setting and system thesis RemoteANN addresses approximate nearest-neighbor search when the vector payload is stored in cloud object storage and the query engine runs on a separate compute instance. This deployment model offers elasticity, durability, and storage-cost advantages, but it changes the dominant performance objective relative to in-memory or local-SSD ANN systems. Object storage imposes substantial time-to-first-byte latency and charges primarily by discrete retrieval requests. Consequently, an index that is acceptable on local storage can become inefficient when each cluster or graph hop induces a remote request. The paper’s central thesis is that cloud-native vector search requires joint optimization of the ANN index, physical data layout, query execution, and storage hierarchy. RemoteANN therefore selects an IVF structure rather than a graph index, because IVF can retrieve candidate clusters with a bounded number of remote round trips. It combines IVF with RaBitQ quantization, locality-aware vector placement, analytical pruning, fine-grained Range-GET operations, request merging, and local NVMe caching. Index construction is accelerated separately through QGKMeans. This is a systems paper rather than a proposal for a new ANN index family in isolation. Its contribution lies in aligning the index’s logical access pattern with the latency and pricing model of object storage. The evaluation reports up to 5.9×5.9\times higher throughput and up to 16.1×16.1\times fewer object-storage requests than representative baselines at approximately 97% recall (2610.25172). Architecture and storage layout RemoteANN uses a three-level hierarchy: DRAM on the compute node, local SSD, and remote object storage. …
Explained
What happened
RemoteANN (NTU Singapore and ByteDance; arXiv 2610.25172) is a vector search engine built to run directly on cloud object storage like Amazon S3, with only small index metadata on the compute nodes. At 97% recall, versus representative baselines:
- Up to 5.9x higher query throughput.
- Up to 16.1x fewer object-storage requests, which cuts the per-request API bill.
Only the abstract was available, so the numbers are the headline ones.
Why it works
Storing vectors in S3 separates storage from compute: cheap, elastic, durable. But object storage has high latency per request (tens of milliseconds) and charges per request. Classic vector indexes like HNSW assume cheap random access in memory or on SSD; on S3 every hop in the graph becomes a slow, billed GET.
RemoteANN designs for that cost model:
- Index: an IVF (inverted file) index groups vectors into clusters, and RaBitQ quantization keeps a compact compressed copy for fast first-pass scoring. Vectors are reordered inside and between clusters so that what a query needs sits close together in the file.
- Queries:
- Prune clusters using the compact metadata, so most of the data is never touched.
- Fetch only needed byte ranges with Range GET requests.
- Merge nearby requests into one bigger GET.
- Cache hot data on local NVMe SSD.
- QGKMeans, a faster clustering method, reduces index build time while keeping cluster quality.
The bigger pattern
Design for the storage's cost model. On object storage the cost is the number of requests and the latency per request, not bytes or CPU. The winning moves are classic: locality (put related data together), batching (merge requests), and tiered caching. The same shift already happened for analytics (Parquet on S3 with range reads), logs and Kafka tiered storage. Vector search is next.
What it means in practice
- For large embedding collections (video frames, catalogue items), S3-backed vector search can be far cheaper than keeping everything in RAM, if the engine is built for it. Ask vendors how they handle request count and locality.
- The same tricks help any S3-heavy service you build: lay files out for sequential range reads, coalesce requests, and cache hot ranges locally.
- When estimating cost, model requests per query, not just storage GB.