Key Takeaways
- Measure percentiles, not just averages. P95 and P99 response times show the slow requests a mean hides. See why percentiles beat averages.
- Split every slow request into its parts (DNS, connect, TLS, time to first byte, download) before touching code. The part that grew tells you which of the ten issues below you have.
- Separate 5xx from 429. A 500 is the server failing; a 429 is the server refusing on purpose. They need different fixes.
- Most fixes are boring and well known: an index, a smaller payload, a cache header, a timeout, a background job. Detection is the hard part, so test after every release and monitor in production.
How Do You Troubleshoot Slow API Responses?
Start with the symptom, not the code. Check three numbers per endpoint: the P95 response time, the error rate split into 4xx and 5xx, and requests per second. If latency grows only as traffic grows, you have a capacity or contention problem (issues 5, 6 and 9). If it is slow at any load, look at the query, the payload or a dependency (issues 1, 4, 5 and 7). If errors appear before latency does, look at issues 2, 3 and 7.
Two tools cover most of this. A load test shows how the API behaves as traffic rises, and continuous monitoring shows how it behaves in production. LoadFocus API monitoring checks status code, response body and response time on a schedule from 25 cloud regions and splits each run into wait, DNS, TCP connect, first byte and download. A cloud load test then finds the traffic level where things start to break. Neither replaces server-side tracing and profiling, which show what happens inside your own code.
The 10 Issues at a Glance
| # | Issue | Typical symptom | First thing to check |
|---|---|---|---|
| 1 | High latency | Slow P95 on one endpoint | Where the time goes (first byte vs download) |
| 2 | HTTP 500 errors | 5xx spike, often after a deploy or under load | Logs and stack traces for that time window |
| 3 | HTTP 429 responses | 4xx spike with healthy servers | Rate limit policy vs real client traffic |
| 4 | Large payloads | Download time dominates | Response size per endpoint |
| 5 | Database bottlenecks | Latency steps up with load | Slow query log, connection pool usage |
| 6 | Blocking code | Latency and CPU grow together | CPU profile or flame graph |
| 7 | External dependencies | Slow only on calls that use a third party | Time spent in outbound calls |
| 8 | Caching mistakes | Every request hits the backend | Cache hit ratio and Cache-Control headers |
| 9 | Resource limits | Slowdown plus CPU, memory or network at a ceiling | Host and container metrics |
| 10 | Version regressions | Slower right after a release | The same test on the old and new build |
1. High Latency and Slow Response Times
Latency is the symptom users feel first, and it can come from almost anywhere in the request path. Measure it per endpoint, track P95 and P99 next to the mean, and break each request into phases. A long wait before the first byte points at the server or its dependencies. A long download points at payload size or the network. A long connect or TLS phase points at DNS, connection setup or a distant region. The article on what response time is in performance testing defines each part.
Synthetic checks from several regions tell you whether the slowness is global or only affects users far from your servers.
A hypothetical example: a product listing endpoint answers in 4.9 seconds. The query filters a large table on a column with no index, so the database scans every row. Adding an index on that column and selecting only the columns the client uses typically brings such a query down to a fraction of that time, but the size of the gain depends on your data, so measure before and after with the same test.
2. HTTP 500 Internal Server Errors
A 500 means the server hit a condition it did not handle: an uncaught exception, a missing dependency, or an exhausted resource such as memory or database connections. RFC 9110 defines it as a generic server error, so the status code alone never tells you the cause.
To find the cause, line up the time of the error spike with three things: the last deployment, resource metrics (CPU, memory, open connections) and the application log for that window. Errors that appear only at a certain request rate in a load test usually point at a resource limit; errors that follow a deploy point at the change. If memory climbs steadily until the 500s start, read how to identify and fix memory leaks with load testing.
3. HTTP 429 Too Many Requests
A 429 is not a malfunction. RFC 6585 defines it as the server telling a client it has sent too many requests in a given time, and allows a Retry-After header saying how long to wait (RFC 9110 defines the header’s format). The question is whether the limit is right.
Track 429s per client and per endpoint next to throughput. A burst from one client suggests a retry loop or a batch job. A broad rise across clients suggests the limit is below real demand. Clients should back off exponentially with some random jitter so they do not retry in lockstep, and servers should send Retry-After. For the causes in more depth, see 10 common causes of API rate limiting.
4. Large Payloads and Inefficient Data Transfer
An endpoint that returns every field of every record makes each request slower and costs bandwidth on both sides. Log response sizes per endpoint and look at the outliers. If the download phase of a request dominates its total time, size is the likely cause.
| Technique | What it does | Trade-off |
|---|---|---|
| Pagination | Caps the number of records per response | Clients need extra requests for the rest |
| Field selection | Returns only the requested fields | Needs an API design change |
| Compression (gzip, Brotli) | Shrinks text responses on the wire | CPU cost, no gain on already-compressed data such as images |
| Binary formats (Protocol Buffers) | Smaller than JSON for the same data | Needs client and server support, harder to read |
A hypothetical example: a list endpoint returning thousands of products with images, specs and reviews in one response, where the client displays only a name and a price. Paginating and returning two fields cuts the response by orders of magnitude, and the effect is easy to confirm by comparing response size and download time before and after.
5. Database Bottlenecks
Slow queries, missing indexes and a saturated connection pool are the usual suspects. Turn on the slow query log in your database and review the worst offenders. Use the database’s own query plan tool (for example EXPLAIN in PostgreSQL and MySQL) to see whether a query scans a whole table.
The connection pool is the quiet one. When every connection is in use, new requests queue and latency jumps without any single query being slow. In a load test this shows up as a step change in response time at a certain number of concurrent users, while database CPU may look fine. Compare pool usage and queue time with the latency curve. Fixes range from an index or a rewritten query to read replicas, caching, or a changed schema, and the larger ones need planning.
6. Blocking Backend Code and Heavy Computation
Work done inside the request handler, such as image processing, large report generation or a heavy aggregation, holds a worker for its whole duration. In a runtime with few workers or one event loop (Node.js, for example), a single CPU-heavy request delays everything queued behind it.
Look for latency and CPU rising together under load, then confirm with a profiler or flame graph, which shows the functions that consume the time. The usual fix is to accept the request, put the work on a queue, return a job id, and let a background worker do it. That changes the API contract, so clients must poll or receive a callback.
7. External API Dependencies
Every call to a payment provider, a maps service or an AI API puts someone else’s latency and limits inside your response time. Distributed tracing, for instance with OpenTelemetry, shows how much of each request is spent waiting on outbound calls. Without tracing, log the duration and status of every outbound call.
Protect yourself with explicit timeouts on every outbound call, a capped number of retries with backoff, a circuit breaker that stops calling a failing service for a while, and a fallback (cached or reduced data) where the product allows it. Watch 5xx and 429 rates from each provider separately, because they tell you whether the provider is failing or limiting you.
8. Caching Misconfigurations
When caching is absent or ineffective, every request reaches the database. Check the cache hit ratio at the CDN, the reverse proxy and the application cache, and read the response headers: Cache-Control, ETag and Age show what clients and proxies are allowed to reuse. A no-store on a response that could be cached, or a very short lifetime, is a common mistake.
The opposite risk is stale data. Decide per endpoint how stale is acceptable, set lifetimes to match, and plan how entries are invalidated when the underlying data changes. A load test that repeats the same request will flatter a cache; use realistic request variety so you measure the hit ratio real traffic would get.
9. Infrastructure Resource Constraints
CPU, memory, network bandwidth and disk I/O all have ceilings, and an API that behaves well at normal load can degrade quickly when one is reached. High CPU slows every request. Memory exhaustion causes swapping, restarts or out-of-memory kills. Network saturation causes timeouts.
Monitor those four resources on every host or container behind the API and alert before they reach the limit. Scaling up or out buys time, but if the cause is a leak, a missing index or blocking code, more hardware only moves the failure point and raises the bill. Run a load test, find the user level where a resource saturates, and then decide whether to tune or to scale. The article on how throughput and response time relate explains why latency climbs sharply near the limit.
10. Versioning and Integration Regressions
A new API version, a library upgrade or a change in middleware can make an endpoint slower without changing its behavior, so functional tests stay green. The reliable defence is a repeatable performance test with a stored baseline: run the same scenario against the old and the new build and compare P95, error rate and throughput. Put it in your CI pipeline so a regression fails the build before release, and keep monitoring after release, because production traffic differs from any test.
If you support several API versions or client versions at once, test the busiest paths in each. Full coverage of every combination is rarely affordable.
Which Issue Should You Fix First?
Rank by user impact, then by how cheap the fix is. Quick wins with high impact (an index, compression, a cache header, a missing timeout) go first. Architectural changes such as moving work to a queue or redesigning a schema need a measured case, so put numbers behind them: current P95, the target, and the load at which the problem appears.
| Issue | Typical detection effort | Typical fix effort |
|---|---|---|
| High latency | Low, monitoring shows it | Depends on cause |
| 500 errors | Low, error rate and logs | Varies widely |
| 429 responses | Medium, needs traffic breakdown | Low to medium |
| Large payloads | Low, response sizes | Low |
| Database bottlenecks | Medium to high, needs profiling | Medium to high |
| External dependencies | Medium, needs tracing | Medium |
These ratings are general rules of thumb, not measurements.
Frequently Asked Questions
What are the first signs of an API performance problem?
Rising P95 or P99 response time and a growing share of 5xx or 429 responses, usually before the average moves. Alert on percentiles and error rate, and compare against a baseline from a calm period.
Which metrics matter most for API performance?
Response time at P95 and P99, error rate split by status class, throughput in requests per second, and resource use (CPU, memory, connections) on the servers. Time to first byte helps separate server time from transfer time.
How do I tell whether latency comes from the server or the network?
Break the request into phases. A long wait for the first byte with a fast download means the server or its dependencies are slow. A long connect or TLS phase, or slowness only from distant regions, points at the network path. Running the same check from several regions makes the difference visible.
What is the difference between a 500 and a 429?
A 500 means the server failed to handle the request. A 429 means the server refused it because the client exceeded a rate limit. Fix a 500 in the application or its resources; fix a 429 by tuning the limit or by making clients back off.
How often should I run API performance tests?
Before each release that touches the API, after infrastructure changes, and on a schedule for critical endpoints. Continuous monitoring covers the time between tests.
Sources
- RFC 9110, HTTP Semantics: 500 Internal Server Error and Retry-After (rfc-editor.org, October 2026)
- RFC 6585, Additional HTTP Status Codes: 429 Too Many Requests (rfc-editor.org, October 2026)
Published with PostNext
