Key Takeaways
- “Mobile performance testing” covers three different jobs: how the app runs on a device (start-up, rendering, memory, battery), how it behaves on a bad network, and how the backend and web pages behave under mobile traffic. The cloud-versus-local answer is different for each.
- Use local devices and the platform profilers for debugging on-device performance. Use a hosted device cloud when you need coverage you cannot afford to own. Use cloud load generators for the backend, because that load has to come from somewhere other than your laptop.
- Emulators and simulators are good for trends, not for final numbers. Apple’s own engineers say to test on real hardware before shipping.
- Regulation rarely forces local-only testing. It forces decisions about what data the test uses.
What does mobile performance testing actually measure?
A mobile app is slow for one of three reasons, and each is tested in a different place.
| What you are measuring | Where it runs | Best tested |
|---|---|---|
| On-device speed: start-up, frame rendering, memory, battery | On the phone itself | Locally with the platform profilers, then on a spread of real devices |
| Behaviour on a poor network: slow, high-latency, dropped connections | Device plus a throttled connection | Locally with network throttling, or on devices in a device cloud |
| Backend and API speed under many users | Your servers, driven by load generators | Cloud load testing from several regions |
| Mobile web page speed (Core Web Vitals, Lighthouse) | A browser emulating or running on a phone | Scheduled synthetic checks plus real-user data |
Most arguments about “cloud or local” come from mixing these rows. A device cloud tells you nothing about how your API copes with 5,000 simultaneous sign-ins, and a load test tells you nothing about frame drops on a three-year-old phone.
When is local testing the better choice?
Local testing means your own devices, your own emulators and simulators, and your own network. It wins for debugging, because you can attach a debugger and a profiler and rerun a change in seconds.
Android Studio’s Profiler offers a system trace for CPU and rendering problems, a power usage view, heap dumps, allocation tracking and live CPU, memory, network and energy views (developer.android.com, read 2026-10-23). Xcode’s Instruments covers the same ground on Apple platforms. Neither needs a hosted service.
Local network throttling is also free. The Android Emulator takes -netspeed and -netdelay flags. The documented speed profiles include gsm, gprs, edge, umts, hsdpa, lte and evdo, and lte is listed as 58,000 kbps up and 173,000 kbps down. There is no named 5G profile in that list, but you can pass your own up:down numbers. On iPhone and iPad, Apple’s Network Link Conditioner can be switched on from Xcode’s Devices and Simulators window or in Developer Settings on the device. Both are approximations of a network, not recordings of a carrier.
Local also wins for builds you cannot send out of the building and for hardware that no cloud lists, such as a prototype or a vendor-specific device.
When does a cloud device farm earn its place?
A device cloud gives you physical phones you do not have to buy, charge, update and store. It pays off when your users run on many models and OS versions and you need to run the same test on a lot of them in parallel, for example on every release candidate.
Firebase Test Lab is a documented example: it runs Android and iOS tests on real, production devices in a Google data center (firebase.google.com/docs/test-lab, read 2026-10-23). The same page says Test Lab is deprecated and will shut down on 30 September 2027, with a migration path to the Developer Device Platform on Google Cloud. That is a good reminder to check a service’s roadmap before building a pipeline on it. We name no other device cloud and print no prices here, because plans change and we did not read any vendor’s page for this post.
A device cloud does not remove the need for judgement. You still choose which devices count. Pick them from your own analytics: the three or four models that carry most of your sessions, the oldest OS you support, and the weakest hardware.
Are emulators good enough for performance numbers?
For trends, yes. For absolute numbers, no. In Apple’s WWDC19 session on Instruments, the presenter explains that the Simulator runs on your Mac, with the Mac’s CPU, memory, file system behaviour and thermal limits, which differ from those of a smaller embedded device. He says Instruments is “great for looking at trends”, and tells developers to test on real hardware before shipping (developer.apple.com/videos/play/wwdc2019/411, read 2026-10-23).
The practical rule: use emulators in the inner loop to see whether a change made something faster or slower, and use real devices, local or hosted, for the numbers you report or gate a release on. Keep one low-end device in the lab, because that is where start-up and jank problems show first.
Where does the backend test belong?
In the cloud, and from outside your office network. A load test run from the same network as the system under test removes the internet between users and server, and a single machine runs out of CPU and bandwidth well before a real launch crowd arrives. Our post on network latency and cloud load test accuracy covers why the origin of the load matters.
LoadFocus’s load testing runs JMeter and k6 tests from 25 cloud regions, with pass/fail thresholds on P95 and P99 response time, error rate and throughput, a GitHub Action, a Verdict API and a Jenkins plugin for CI (loadfocus.com/load-testing, read 2026-10-23). That is backend and API load. The page does not describe real-device testing, so LoadFocus is not a device farm and does not measure frame rate or battery on a phone.
For the web side, page speed monitoring runs scheduled Lighthouse checks for desktop and mobile with bandwidth and CPU throttling, and tracks Core Web Vitals with budgets (loadfocus.com/page-speed-monitoring, read 2026-10-23). The Core Web Vitals targets are LCP at 2.5 seconds or less, INP at 200 milliseconds or less and CLS at 0.1 or less, measured at the 75th percentile of page loads split by mobile and desktop (web.dev/articles/vitals, read 2026-10-23). Lab runs like these help you find and fix a cause. The score Google uses comes from field data, as the synthetic monitoring vs real user monitoring post explains.
Does compliance force you to test locally?
Less often than people say. HHS guidance on HIPAA and cloud computing says a covered entity or business associate may use a cloud service provider for electronic protected health information if it has a business associate agreement with that provider (hhs.gov/hipaa/for-professionals/special-topics/cloud-computing, October 2016 guidance; the page returned an access error to our fetch tool, so we confirmed the wording through summaries of it, and you should read the page yourself). GDPR regulates how personal data is processed and transferred. It does not ban hosted services.
The simpler question for a mobile test is whether it needs real personal data at all. Most performance tests do not: use synthetic accounts and generated records, and the compliance argument mostly goes away. Keep local or self-hosted testing for the cases that remain, such as an unreleased build under contract restrictions, a system reachable only on a private network, or a contract that names where data may sit. Our cloud load testing vs on-premise for startups post goes through the same compliance reasoning in more detail, and cloud testing security best practices covers what to ask a vendor.
How should you choose? A decision checklist
| If this is true | Lean toward |
|---|---|
| You are chasing a crash or a jank problem on one device | Local, with the platform profiler |
| Your users span many models and OS versions and you ship often | A hosted device cloud, chosen from your analytics |
| You need a defensible number for a release gate | Real devices, local or hosted, not an emulator |
| You want to know how the API holds up at launch traffic | Cloud load generators in several regions |
| You must reproduce a bad cellular connection | Local throttling first, a device cloud for variety |
| The build cannot leave your network | Local or self-hosted |
| You have no hardware budget and a small team | Emulators for the loop, one low-end real device, cloud load tests for the backend |
Most teams end up with all three: local devices for debugging, a hosted farm for coverage, and cloud load tests for the server. Costs follow the same split. Counting your own cost means device purchase and replacement, engineer time to maintain the lab, and idle time, set against a hosted plan you read off the vendor’s page today. We print no figures because neither side’s numbers were checked for this post.
Frequently Asked Questions
What is mobile performance testing?
It is testing how fast and stable a mobile app or mobile website is for users: start-up time, frame rendering, memory and battery use on the device, behaviour on slow networks, and how the backend responds under many users. Each part needs different tools.
Is cloud testing better than local testing for mobile apps?
Neither wins everywhere. Local testing is better for debugging and for builds that cannot leave your network. A hosted device cloud is better for coverage across many devices. Backend load tests belong on cloud load generators.
Are emulators accurate enough for performance testing?
They are good for spotting whether a change made things faster or slower. Apple’s engineers advise testing on real hardware before shipping, because the Simulator inherits the Mac’s CPU, memory and thermal behaviour.
Can I simulate 3G or 4G on my own machine?
Yes. The Android Emulator has -netspeed and -netdelay options with named profiles such as edge, umts and lte, and Apple’s Network Link Conditioner throttles a device or a Mac. These are approximations of real networks.
Does HIPAA or GDPR require on-premise testing?
No. HHS says covered entities can use a cloud provider for ePHI under a business associate agreement, and GDPR governs how personal data is handled, not where a test tool runs. Using synthetic data in tests avoids most of the question.
Does LoadFocus test on real mobile devices?
No. LoadFocus load tests the backend and APIs from 25 cloud regions, and page speed monitoring checks pages in a mobile or desktop profile with throttling. It does not provide a real-device farm.
How do I add mobile performance checks to CI?
Run the backend load test in your pipeline with a pass/fail threshold, and run on-device tests on emulators or a device cloud for each release candidate. Our continuous performance testing post covers the pipeline side.
Sources
- Android Studio: profile your app performance (read 2026-10-23)
- Android Emulator command-line options: -netspeed and -netdelay (read 2026-10-23)
- Apple WWDC19: Getting Started with Instruments (read 2026-10-23)
- Apple WWDC19: Designing for Adverse Network and Temperature Conditions (surfaced in a search; not read in full)
- Firebase Test Lab (read 2026-10-23)
- web.dev: Web Vitals (read 2026-10-23)
- HHS: HIPAA and cloud computing
Published with PostNext
