newsInfoQ·24 Oct 2026·Renato Losio

Cloudflare Reclaims 100 TB of RAM by Optimizing Consistent Hashing

Cloudflare cut over 100 TB of RAM from its load balancer by optimizing its Rust consistent hashing implementation.

consistent-hashing load-balancing memory-optimization

Depth 5/5Score 6.75

Extract

Cloudflare recently documented how it reduced the memory footprint of its internal load-balancing service by more than 100 TB by optimizing the Rust implementation of consistent hashing in pingora-ketama. The changes compacted hash-ring entries and used statistical analysis of hash distribution and collision rates to reduce the number of hashes per backend by about 90%.
By Renato Losio

Explained

What happened

Cloudflare cut more than 100 TB of RAM across its fleet by shrinking the hash rings in pingora-ketama, its open-source Rust library for consistent hashing. The library powers PBR (Pingora Backend Router), a load balancer that sends each URL to one cache server per data center, so each asset is cached once. Some PBR processes had reached 6 GB, mostly ring data, because dozens of rings coexist (one per feature and compliance combination).

Two changes:

Smaller entries. Each ring point was a u32 hash plus a u32 server index, 8 bytes. Now it is a 4-byte hash plus a u16 index packed into [u8; 6], 6 bytes (25% smaller). They avoided #[repr(packed)] because unaligned references are undefined-behaviour risks in Rust. The u16 caps a ring at ~65k servers.
Far fewer points. Every server had 160 virtual points (a default inherited from NGINX), multiplied by weight. A server with weight 625 got 100,000 points. They cut points per server by about 90%.

Why it works

In consistent hashing, each server gets many virtual nodes on a ring so load spreads evenly. More points means less skew, but with sharply diminishing returns: the spread falls with the square root of the point count. Cloudflare's numbers showed going from 10,000 to 100,000 points per server improved balance by only ~0.7%.

Worse, too many points hurt. With 32-bit hashes and millions of points, birthday-paradox collisions silently overwrite each other's slots. In a 2,048-server simulation, error actually went up when points rose from 10k to 100k. So the 90% cut cost nothing measurable.

The bigger pattern

Inherited defaults are tuned for someone else's scale. 160 points was reasonable for a few nginx upstreams, not for weighted rings with thousands of servers replicated dozens of times per process. Also a nice example of struct layout and alignment mattering at fleet scale: 2 bytes per entry, times billions of entries.

Safe rollout: the old (v1) and new (v2) rings ran side by side in memory, picked per request by hash, ramped by traffic percentage and data center, watching cache hit ratio, origin traffic and RSS. Since changing the ring remaps keys, they had to watch for cache misses during migration.

What it means in practice

If you shard Redis or Kafka consumers with consistent hashing, check your virtual node count. A few dozen to ~100 per node is usually enough. Measure the coefficient of variation instead of guessing.
Any change to the ring remaps keys, which means a burst of cache misses. Roll out ring changes gradually, like Cloudflare did.
At large scale, memory layout (padding, index width) is a real cost lever. Look at the biggest in-memory structures first.