Key Takeaways
- A load test against a warm cache measures the cache. A load test against a cold or just-purged cache measures your origin. You need both numbers, and you need to know which one you are looking at.
- The risky moment is the purge, not the steady state. When a popular entry disappears, every request for it can reach the origin at once unless the cache collapses them into one.
- Test three states on purpose: cold, warm, and mid-invalidation. Report the cache hit ratio next to every latency figure.
What does cache invalidation have to do with load testing?
Cache invalidation removes or marks stale a stored response so the next request fetches a fresh copy from the origin. In a load test it decides how much of your traffic is answered by a cache and how much reaches the application and database. The same test can show 40 ms or 900 ms depending on that split, so the cache state is part of the test setup, not a detail.
This post is about the testing side. For how much of a CDN-fronted site’s traffic never reaches your servers in the first place, read edge computing and load testing, which this post does not repeat.
How HTTP caches decide what is fresh
HTTP caching is defined in RFC 9111. A cache may reuse a stored response without contacting the origin while it is fresh, and freshness comes mainly from Cache-Control: max-age or Expires. Once the response is stale, the cache validates it with a conditional request (ETag with If-None-Match, or Last-Modified), and a 304 Not Modified lets it reuse the stored body (rfc-editor.org/rfc/rfc9111, read October 2026).
The directives that matter for test design, as MDN defines them (developer.mozilla.org, Cache-Control, October 2026):
| Directive | What it does | Why a tester cares |
|---|---|---|
max-age=N | Response is fresh for N seconds after it was generated | Sets how often each entry goes back to the origin |
s-maxage=N | Same, for shared caches only (CDN, proxy); overrides max-age there | Browser and CDN lifetimes can differ |
no-cache | May be stored, but must be validated before every reuse | Every request costs a validation round trip |
no-store | Must not be stored by any cache | Nothing is cached, the origin takes every request |
stale-while-revalidate=N | A stale response may be reused while a fresh one is fetched | Hides origin latency from users right after expiry |
stale-if-error=N | A stale response may be reused when the origin returns 500, 502, 503 or 504 | Origin failures can stay invisible in your results |
Two consequences for a load test. First, a script that sends Cache-Control: no-cache on every request measures the validation path, not what a browser does. Second, with stale-if-error in place, an origin that is failing under load may still return 200s from the cache, so check the cache status header as well as the status code.
Cold, warm and mid-purge: three states to test
Cold cache. Nothing is stored yet. Every first request for each URL goes to the origin. This is what a new deployment, a new CDN location or a restarted Redis looks like. A cold run is the worst case your origin must survive.
Warm cache. Entries are stored and fresh. Most requests never reach the origin, so throughput and latency look excellent. This number is real, but it only describes the sustained state.
Mid-invalidation. Entries are purged or expire while load is running. This is the state most tests never exercise, and it is where most cache-related incidents start.
A simple illustration, with invented numbers: a catalogue page takes 1,000 requests per second, and the CDN answers 95 percent from cache. The origin sees 50 requests per second. If a purge empties the entry and nothing coalesces the misses, the origin can see up to 1,000 requests per second for the same page until the first response is stored, twenty times its normal load. Your origin was sized for 50.
Cache stampedes: what a purge does under load
A stampede (also called a thundering herd) happens when many requests miss on the same key at the same moment and all go to the origin. RFC 9111 describes the mitigation: a cache can collapse requests, merging several incoming requests for one resource into a single forwarded request after a miss, which reduces origin load. It also notes the cost, since requests that cannot be served from the collapsed response must be forwarded separately and may be slower.
What you test is whether that protection exists and works in your stack:
- Does your CDN collapse concurrent misses, and is it enabled for your content types?
- Do you use
stale-while-revalidate, so that users keep getting the old copy while one request refreshes it? - Does an application-level cache (for example Redis) have a lock, or a jittered TTL, so that thousands of keys written together do not all expire together?
A cheap way to expose the problem: warm the cache, purge a hot path while a constant load runs, and watch origin requests per second and p95 latency for the next minute. Flat means the protection works. A spike shaped like a cliff means it does not.
Invalidation methods and what each does to your test
| Method | Behaviour | Test implication |
|---|---|---|
| Wait for TTL expiry | Entries expire on their own schedule | Entries written together expire together unless TTLs are jittered |
| Purge a single URL | Next request refetches that URL | Good for targeted stampede tests |
| Purge by wildcard or everything | Large part of the cache disappears | Behaves like a cold start under live traffic |
| Versioned file names | New name means a new cache key; the old one is no longer requested | No purge, so no stampede on that file; AWS recommends it for frequently updated files |
| Redis eviction | Memory limit reached, keys removed by policy | Hit ratio drops without any deploy or purge |
Vendor behaviour to know before you schedule a purge test:
- CloudFront: after an invalidation, the next viewer request goes back to the origin. The first 1,000 invalidation paths per month are free, a wildcard path counts as one, and AWS recommends versioned file names over invalidation for files you update often (docs.aws.amazon.com, CloudFront invalidation pages, October 2026).
- Cloudflare: purge by single URL, prefix, hostname, cache tag or everything, with per-plan rate limits. After a purge, check that the
CF-Cache-Statusheader is no longerHIT(developers.cloudflare.com/cache/how-to/purge-cache, October 2026). Cloudflare’s page gives no propagation time, so measure it yourself. - Fastly: purge by URL, by surrogate key, or everything, and soft purges exist (their documentation index lists them; the detail pages were not read for this article).
- Redis: with
maxmemoryset, a policy such asallkeys-lruevicts keys when memory runs out, andnoevictionreturns errors for writes instead. Redis documentskeyspace_hits / (keyspace_hits + keyspace_misses)as the cache hit percentage (redis.io/docs/latest/develop/reference/eviction, October 2026).
Per RFC 9111, a cache only invalidates the stored responses it can see, so an invalidation does not guarantee consistency across every cache between a user and your origin: a browser cache or a corporate proxy may still hold the old copy.
How to design the test
- Measure and record the cache state. Log the CDN’s cache status header (
X-Cache,CF-Cache-Status,Age) for a sample of requests and compute the hit ratio per run. - Run cold, then warm. Start the first run right after a purge or deployment, then repeat the identical script. The gap between the two runs shows how much your capacity depends on the cache.
- Realistic URL mix. One URL repeated by every virtual user is a 100 percent hit rate after the first request. Use a data file of real paths, weighted the way production traffic is. See common performance testing mistakes.
- Add the purge as an event. Trigger the purge from the test (an API call to your CDN, or a scheduled step in your pipeline) at a known second so you can line it up with the latency graph.
- Read percentiles, not averages. A purge hurts a small share of requests badly, which an average hides. See why percentiles beat averages.
- Look behind the cache. Watch the origin’s CPU, database connections and queue depth. The connection pool is often the first thing to run out, as covered in optimizing database connection pools.
- Expect bursts. A stampede after a purge is a short spike, so combine this with spike testing.
Check the CDN’s testing rules first. Load testing a CDN you do not own usually needs the provider’s approval; the edge post linked above covers this.
Running it with LoadFocus
LoadFocus load testing runs HTTP tests from its cloud regions, which matters here because each region reaches a different edge cache, so one region’s warm cache says nothing about another’s. A scheduled run right after a deploy gives you the cold-cache number every time; a second run a few minutes later gives the warm one. LoadFocus does not purge your cache for you: trigger the purge through your CDN’s own API or console.
Frequently Asked Questions
What is cache invalidation in performance testing?
It is the removal or expiry of cached entries during or before a test. It matters because it decides whether requests are answered by the cache or by your origin, so it determines what your latency and throughput numbers actually describe.
Should I clear the cache before a load test?
Run both. A cold-cache run shows the worst case your origin must handle, and a warm run shows the sustained case. Report the two separately with the cache hit ratio for each, instead of one blended average.
What is a cache stampede?
A stampede is many simultaneous cache misses for the same key, all sent to the origin together. It typically follows a purge or the expiry of a very popular entry. Request collapsing at the cache and stale-while-revalidate both reduce it.
How do I measure cache hit ratio during a test?
Read the cache status response header your CDN adds, or the cache’s own counters. For Redis, INFO stats reports keyspace_hits and keyspace_misses, and hit ratio is hits divided by hits plus misses.
Is invalidation or versioned file names better?
For files that change often, AWS recommends versioned names for CloudFront because they avoid invalidation cost and old-copy problems in browser and proxy caches. Purges stay necessary for content that must keep its URL, such as an HTML page.
Sources
- RFC 9111, HTTP Caching (read October 2026)
- MDN, Cache-Control (October 2026)
- AWS, Invalidate files to remove content and Pay for file invalidation (October 2026)
- Cloudflare, Purge cache (October 2026)
- Redis, Key eviction (October 2026)
Published with PostNext
