To cut API response times, load test the endpoint users complain about, split its time by dependency, shorten the critical path (the longest chain of waits), and retest the same scenario after each change. Parallelizing independent calls and caching slow, rarely changing data are the two changes that most often move the number.
This post was first published as a case study. The company and the figures in that version could not be verified, so it has been rewritten as a how-to. The numbers below are arithmetic examples, not results from a real customer.
Key takeaways
- Measure where one request spends its time before changing code. The slowest dependency is often not the one the team suspects.
- Response time of calls made one after another is their sum. For independent calls run together it is the slowest one.
- Caching only helps if it shortens the critical path, and the p95 only moves when the hit rate is high enough.
- Retest with the same load after every change, and judge by p95 and p99, not the average.
How do you find what makes an API slow?
Start from one endpoint with a measured problem, not from the whole API. Pick the one with the worst p95 response time or the most user complaints, and run a load test against it at a realistic request rate. Then look at where one request spends its time: your own code, the database, and each outbound call to another service.
Two sources give you that split. Distributed tracing or timing logs inside the service show time per dependency. A cloud load test shows how the whole response time behaves as load rises. LoadFocus load testing runs tests from 25 cloud regions and reports response time with 50th, 90th and 99th percentiles per run, and thresholds on p95 and p99 response time, error rate and throughput let you fail a run automatically (loadfocus.com/load-testing, read 2026-10-23). The split of a single request into DNS, connect, TLS and server wait is covered in how to reduce API latency with distributed load testing, so this post starts where that one ends: the server wait is large, and you need to know why.
Why is the database often not the problem?
Teams usually suspect the database first, because queries are the part they know how to tune. It is a fair first guess and sometimes right, but it is only a guess until the timing is split by dependency. An endpoint that calls other services can spend most of its time waiting for them while the database answers in a few milliseconds.
A worked example, with invented numbers. An endpoint builds a dashboard response from three upstream services and its own logic:
| Step | Time |
|---|---|
| Call A (user settings) | 120 ms |
| Call B (analytics summary) | 240 ms |
| Call C (notifications) | 180 ms |
| The service’s own work (including its database query) | 60 ms |
Run one after another, the total is 120 + 240 + 180 + 60 = 600 ms. The upstream calls account for 540 of those 600 milliseconds, which is 90 percent. Tuning the 60 ms of own work, even to zero, saves at most 10 percent. That is the whole case for measuring first.
How much does parallelizing independent calls save?
Calls that do not depend on each other’s results can run at the same time. The wait is then the slowest of them, not the sum. In the example, A, B and C only need the user id, so they can start together:
| Version | Calculation | Response time |
|---|---|---|
| Sequential | 120 + 240 + 180 + 60 | 600 ms |
| Parallel | max(120, 240, 180) + 60 | 300 ms |
That halves the response time with no new infrastructure. Two limits apply. Calls that need an earlier call’s output cannot be moved, and calls with side effects (writes, payments) need care before they run together. Also watch the upstream service’s rate limit: three calls at once per request is three times the burst. The post on common causes of API rate limiting explains how that shows up.
What should you cache, and when does it help?
After parallelizing, the critical path is call B at 240 ms. Caching call A (120 ms) would change nothing, because B still takes 240 ms. The useful target is the slowest call on the path: B, if the analytics summary changes rarely enough to store for a while.
Say B is served from a cache with 90 percent of requests hitting it and a near-zero cost on a hit. Then the critical path on a hit becomes max(120, 0, 180) = 180 ms, plus 60 ms of own work, so 240 ms. On a miss it is the same 300 ms as before.
| Cache hit rate | Mean response time | p95 response time |
|---|---|---|
| No cache | 300 ms | 300 ms |
| 90% | 0.9 x 240 + 0.1 x 300 = 246 ms | 300 ms |
| 96% | 0.96 x 240 + 0.04 x 300 = 242 ms | 240 ms |
The mean barely moves, and at a 90 percent hit rate the p95 does not move at all, because 10 percent of requests are still misses and the 95th percentile falls inside them. Only when misses drop below 5 percent does the p95 follow. This is why you check percentiles after a caching change, and why a load test with varied request data matters: a test that repeats one request flatters the cache.
Decide what can be stale, and for how long. Settings that change on a profile edit can be cleared on that edit; a summary that is recomputed hourly can expire hourly. When a popular entry expires and many requests rebuild it at once, they all hit the slow call together (a cache stampede). Letting one request refresh the entry while the others wait or use the old value avoids that.
What else shortens the critical path?
- Set timeouts on every outbound call. One upstream stuck for 30 seconds holds a worker for 30 seconds, and it drags your p99 with it.
- Return less. If the client shows two fields, do not send forty. Smaller bodies cut serialization and download time.
- Move work out of the request. Anything the user does not need before the response (sending an email, writing an audit record) can go to a queue.
- Reuse connections to upstream services and the database instead of opening one per request.
For the wider list of causes and how to recognize each, see 10 common API performance issues and how to detect them.
How do you confirm the change worked?
Use the same scenario, the same request mix and the same load before and after, and compare p50, p95, p99 and error rate. If the change helped, the tail moved, not just the average. Change one thing per run; if parallelizing and caching ship in one release, you cannot say which one paid off. Then guard the result: put thresholds on p95 and p99 in the pipeline so a slower release fails the build (the LoadFocus GitHub Action and verdict API support this, loadfocus.com/load-testing, read 2026-10-23), and use API monitoring to watch production between tests.
Frequently Asked Questions
How do I reduce API response time without a rewrite?
Measure which dependency dominates a slow endpoint, then run independent calls in parallel, cache slow data that changes rarely, add timeouts, and return smaller responses. These are small code changes compared with a redesign, and each can be checked with a repeat load test.
What is a good API response time?
It depends on the endpoint and what calls it. There is no single target, so set one per endpoint from what the user flow can tolerate, and track p95 and p99 against it. For definitions of the metrics, see what response time is in performance testing.
Is parallelizing API calls always safe?
No. It is safe for independent reads. Calls that depend on each other’s output, or that write data, need to stay ordered, and the upstream service must accept the extra concurrent requests.
Why did my average improve but my p95 did not?
Because a minority of slow requests still dominate the tail. With a cache, for example, the p95 only improves once fewer than 5 percent of requests miss it. Look at the percentiles, not the mean.
Should I load test before or after optimizing?
Both. A baseline test shows where the time goes, and a test with the same scenario after each change shows what the change was worth.
Sources
- LoadFocus load testing and API monitoring pages (loadfocus.com/load-testing, loadfocus.com/api-monitoring, read 2026-10-23).
- All timings in the worked examples are invented for illustration and recomputed in this post.
