Storage and memory expenses have surged dramatically in the past year, forcing Cloudflare to optimize how it deploys resources across its globally distributed infrastructure. The company operates multiple large-scale storage systems, with its CDN at the center, all dependent on squeezing maximum efficiency from available capacity to serve its customer base.
To address this challenge, Cloudflare developed a prototype system that increases effective cache capacity by encoding eligible content with Zstandard compression inside Pingora, its custom proxy. The approach exchanges a small rise in CPU usage for substantial reductions in storage footprint and bandwidth consumed between data centers.
Built during an internship as part of the 1.1.1.1 Intern Program, the Cache Transcoding system compresses responses using Zstandard, or zstd, at the moment they enter the cache and before being written to disk. The compressed version persists as the asset moves through Tiered Cache between facilities, with decompression occurring only when the response reaches the client.
Early tests showed that eligible assets shrank to roughly one-third their original on-disk size. While the origin-facing proxy incurs a modest CPU penalty, the payoff proves substantial: Cloudflare gains petabytes of usable cache capacity and cuts inter-data center traffic. The compression work happens once when an asset is cached, but the storage and bandwidth gains accumulate every time that asset is retrieved.
Understanding Zstandard
Zstandard, or zstd, is a lossless compression algorithm created by Yann Collet at Facebook and released as open source in 2016. Lossless compression ensures that after decompression, every byte matches the original exactly, allowing assets to be stored in a different form without altering their content.
Zstd balances compression effectiveness with processing speed—a critical design goal for Cloudflare's use case. In earlier browser compression tests, it compressed data 42% faster than Brotli while achieving nearly identical file sizes, and produced files 11.3% smaller than gzip at comparable speeds. This equilibrium matters because Cache Transcoding would process enormous volumes of traffic, requiring both encoding and decoding to remain performant.
The prototype uses zstd level 3, which delivers most compression gains without turning cache fills into a CPU bottleneck. Traditionally, Cloudflare stores assets using whatever content encoding the origin provides. If an origin sends uncompressed data, it stays uncompressed on disk and during inter-data center transfers. Cache Transcoding introduces compression at the cache layer itself.
Selective compression: not all content benefits
Transcoding does not mean compressing everything indiscriminately. Images, video, and fonts arrive already compressed. In Cloudflare's traffic sample, this media category represented 21.4% of requests but consumed 63.3% of bytes—compressing it again would waste CPU cycles for negligible gain.
Compressible text tells a different story. HTML, JSON, CSS, and JavaScript accounted for 67.3% of requests and 22.3% of bytes. Within this text category, roughly 71% arrived uncompressed with no Content-Encoding header, and these assets compress effectively.
In the controlled test corpus, eligible assets compressed by approximately 2.8 times. Encoding costs 4.31 nanoseconds per byte (roughly 232 MB/s) and occurs once per cache fill. Decoding costs 1.56 nanoseconds per byte (roughly 641 MB/s) and happens on every serve. While encoding carries higher per-byte expense, assets are served far more frequently than they are cached, making the math favorable.
By changing how assets are represented in storage, existing hardware can hold more customer content. Fewer bytes on disk mean each server retains more objects, increasing cache density and reducing the chance that useful content gets evicted due to unnecessary space consumption. The smaller representation also benefits Tiered Cache by reducing data transferred between Cloudflare data centers, making backbone usage more efficient.
Compression cost paid once, benefits multiplied
Compression always carries a cost in CPU cycles for both encoding and decoding. The critical question is whether byte savings justify the processing expense. At zstd level 3, Cloudflare's model kept extra CPU overhead to a few percent under the traffic and reuse patterns tested.
The team initially explored limiting transcoding to popular content, reasoning that hot assets are reused more frequently. Testing showed this approach did not help. Since decoding happens every time an asset is served, restricting the feature to only the hottest content reduced storage savings without cutting CPU usage proportionally.
A simpler policy proved superior. Transcoding all eligible compressible text at or above 4 kibibytes (KiB) captured nearly all measured storage benefits while staying within the CPU budget.
How Cache Transcoding operates
On a cache miss, the Pingora-based proxy encodes the response body using zstd before writing it to disk. Cache metadata records that the stored representation is compressed and preserves the original content length. Before the response leaves the proxy, the body is decoded back to its original form.
On a cache hit, the stored zstd object is read from disk and decoded. With Tiered Cache, the compressed representation moves from the upper tier to the lower tier in compressed form. Decoding happens only at the client-facing hop.
When the upper tier must fetch identity bytes from the origin on a full cache miss, those bytes are encoded once, stored as zstd, and transferred to the lower tier in compressed form. The lower tier also stores the zstd representation, then decodes it for the request path.
If the lower tier misses but the upper tier has the object, the origin is not contacted. The compressed object moves directly between cache tiers, remaining compressed on the wire and on disk, then decoded once at the lower tier.
If the lower tier already holds the object, no network transfer or encoding is required. The lower tier reads the zstd bytes from disk, decodes them, and passes the original asset onward.
A storage encoding marker prevents an object from being encoded more than once. A cache layer receiving an object from another tier can see it is already stored using zstd and preserve it in that form.
Why certain text gets transcoded
The fastest compression operation is the one that never happens. Cache Transcoding therefore applies a series of eligibility checks to avoid content unlikely to benefit.
The prototype only transcodes a 200 OK response when Content-Encoding is unset, the Content-Type is compressible text, and the response has a known Content-Length of at least 4 KiB. Slice subrequests, responses using active upstream compression, range requests, precompressed responses, unknown length bodies, and binary content remain unchanged.
The 4 KiB threshold eliminated a large number of tiny requests while excluding only about 1% of otherwise eligible bytes. Lowering it would add per-object overhead without saving much additional storage.
Both the threshold and zstd level are parameters rather than fixed limits. The team started with zstd level 3 and a 4 KiB minimum to measure the architecture conservatively. With the initial CPU budget understood, testing can determine whether higher compression levels improve the ratio enough to justify their additional cost.
Testing at scale
The prototype was exercised against a controlled test zone with each request correlated across request logs, Prometheus metrics, and Jaeger traces. The correctness campaign covered cache misses, cache hits, single-hop fills, Tiered Cache fills, and more. The team varied cache keys to make each request follow a specific path and used traces to confirm where encoding and decoding occurred.
One performance campaign sent more than a million requests across 10 cache servers. Half ran with Tiered Cache disabled and the other half with it enabled, allowing measurement of local cache behavior separately from transfers between cache tiers.
The two test assets were approximately 195 KiB and 272 KiB, and both compressed by roughly 2.8 times. This was deliberately a compressible test corpus, providing clear signal for validating the architecture but not representing every text object on the Internet. A broader corpus is needed before treating the measured compression ratio as a fleet-wide constant.
Compress once, benefit repeatedly
The experiment demonstrated significant efficiencies still available across Cloudflare's caching service that can benefit all customers. Cache Transcoding shows the trade-off is favorable under the tested conditions. The architecture preserved content integrity and remained within the CPU budget.
Next steps include evaluating higher zstd levels, testing a broader range of content types and object sizes, and tuning parameters from the eligibility criteria. Future work can also examine range requests, pre-compressed origin responses, and passing the compressed object directly to downstream components that already support it without decoding.