API Performance Metrics: Latency, Throughput, Error Rate

What are API performance metrics?

API performance metrics are measurements of how fast an API responds, how much traffic it handles and how often it fails, taken from real traffic or from a test. The core four are latency, throughput, error rate and saturation. Teams use them to set targets, find bottlenecks and decide when to scale.

The four golden signals

Google's SRE book names four signals that cover most of what you need to watch on a service (see Monitoring Distributed Systems). The example values below are illustrative.

Latency: use percentiles, not the average

The average hides the slow tail. The SRE book gives the classic case: a service with an average latency of 100 ms at 1,000 requests per second can easily have 1% of requests taking 5 seconds. Report percentiles instead, and read more in why percentiles beat averages.

Measure the latency of successful and failed requests separately. A fast error can make your numbers look better than the user experience really is, and a slow error is worse than a fast one. For the basics, see latency.

Throughput

Throughput is the amount of work done per unit of time. For an API that is usually requests per second (RPS). Related numbers are concurrent users (virtual users in a test) and bandwidth. Throughput on its own says little: watch it together with latency and errors. The point where latency starts climbing while throughput stops growing is your practical capacity, see how throughput and response time are related.

Error rate

The share of requests that fail. Split it by cause, because each one points somewhere different:

• 5xx responses: server faults, your problem.
• 4xx responses: mostly client mistakes (401, 404, 429), but a sudden jump can mean a broken release or an expired credential.
• Timeouts and connection errors: the request never completed.
• Wrong content with a 200: the SRE book counts these as errors too. Assert on the body, not only the status code.

Saturation

Saturation shows how close the system is to its limit. Many systems slow down well before 100% utilization, so watch CPU, memory, disk I/O, network, database connection pool use and queue depth, and set a utilization target. A rising p99 over a short window is often the first sign.

Other useful timings

• Time to first byte: server think time before the payload starts, see time to first byte (TTFB).
• DNS, TCP connect and TLS time: network overhead before your code even runs.
• Download time: payload size and bandwidth.
• Per-endpoint and per-region breakdowns: an overall number can hide one slow endpoint or one slow region.

SLI, SLO, SLA and error budget

The 43 minutes come from simple arithmetic: 0.1% of a 30-day month (43,200 minutes). More on contracts in SLA management.

A worked example

Say you load test a hypothetical /orders endpoint at 50, 100 and 200 virtual users. At 50 and 100 users p95 stays around 180 ms and errors are near zero. At 200 users p95 jumps to 900 ms and 2% of requests time out, while throughput barely moved from the 100-user run. The knee is around 100 users, so that is your safe capacity, and the database connection pool (a saturation metric) is the first thing to inspect.

How to measure API performance

• Load testing: generate controlled traffic before a release and compare runs. See load testing.
• Synthetic API monitoring: scheduled checks against production that record response time, status and body. See API monitoring.
• APM and real user data: server-side instrumentation and field data show what actual traffic experiences. See application performance monitoring.

No single source sees everything: tests give a reproducible baseline, monitoring catches regressions in production, and APM explains why.

Common mistakes

• Reporting averages only. The slow tail is what users complain about.
• Measuring only in staging. Production data, caches and traffic mix differ.
• No targets. Metrics nobody compares to an SLO get ignored.
• Alerting on every 5xx. Alert on the error rate over a window instead.
• One aggregate for every endpoint. Tag by endpoint and version.
• Testing once. Re-test after major changes.

Measure API performance with LoadFocus

LoadFocus load testing runs JMeter, k6 and no-code tests from 25 cloud regions and reports response time, latency, throughput, errors and the 50th, 90th and 99th percentiles, with pass or fail thresholds on p95 or p99, error rate and throughput. API monitoring checks your endpoints on a schedule (every 5 minutes on the free plan) and can fail a check on status, body, headers or response time. For the wider picture of testing endpoints, see API testing.

FAQ: API performance metrics

What is a good API latency?

It depends on the use case and on what users expect. Many teams aim for a p95 under a few hundred milliseconds for interactive endpoints, and tighter for internal or real-time calls. Set the target from your own users and measure p95 and p99, not the average.

How do I find the maximum throughput of an API?

Run a load test that raises traffic step by step and watch latency and errors. The point where latency climbs or errors appear while throughput stops growing is the practical maximum.

What error rate is acceptable?

There is no universal number. It comes from your SLO: a payment or login endpoint needs a much lower rate than a non-critical one. Define it per endpoint and measure it over a window such as 30 days.

Should I track p95 or p99?

Both. p95 describes the quality for most users and p99 shows the experience of the unlucky ones. If your traffic is large, even 1% is a lot of people.

What is an error budget?

The amount of failure your SLO still allows. At 99.9% availability over 30 days it is about 43 minutes. When the budget is nearly spent, teams usually slow down risky releases and fix reliability first.