Performance Testing iOS PhoneGap Apps

PhoneGap apps tend to have a bad reputation when it comes to user experience, and for the most part this is due to poor performance. Nowadays, I often hear about teams starting work on a mobile app, and there is inevitably a debate about whether to go native or hybrid. The argument on the native side is usually something like this:

We cannot compromise on user experience.

Demanding the best user experience is a good stance to take. However, there are plenty of successful PhoneGap apps out there in the wild right now. You just don’t know that they are PhoneGap apps because once an app reaches a certain level of performance, customers are happy and it doesn’t matter how the app was developed.

When you set out to build a PhoneGap app, your goal should be to make it feel like a native application instead of just a web page running in a UIWebView. You can do this by learning and implementing the techniques on this site, and emphasizing performance testing early and often.

In this post I will cover my testing process for iOS PhoneGap apps, with a focus on performance. The process will be similar for other platforms such as Android and the latest Blackberry phones which both use WebKit-based browser engines, but each of these platforms deserves a separate post where testing is concerned.

Test on a Physical Device Early

In my opinion, testing is one of the biggest wins for PhoneGap apps, and this is especially true on iOS. Before you can test a native app on a physical device you have to pay Apple. With a PhoneGap app, you can easily test on a real device before forking over any cash to Apple. Just host your source files on a web server somewhere, and point mobile Safari at the server. On iOS devices you can simulate full-screen by pinning the page to your home screen.

Naturally, when you test in this manner you will not have access to some of the device functionality exposed through the PhoneGap API. However, by using a variant of feature detection and stubbing you can have fairly high confidence that the parts of your application that rely on device functionality will work correctly when you finally deploy your app to a physical device.

PhoneGap App Testing Progression

Here is the testing process I follow for iOS PhoneGap apps:

Desktop Chrome or Safari

I test as much functionality as possible in a desktop browser like Chrome or Safari. This includes running automated test suites. I generally prefer the developer tools in Chrome, but testing in Safari has the advantage of being closer to the environment your app will run in when it is on the device. Safari also recently shipped with remote development tools that can help immensely when you need to debug issues that only happen on device (and this will happen to you eventually).

iOS Simulator Safari

Test in the iOS simulator’s web browser when you need an environment closer to what you will get when you ship your app. This is most useful for flushing out iOS Safari-specific display issues and seeing how your app handles simple touch input. Note that you cannot draw any conclusions about your app’s performance from this step, the simulator will be considerably faster than a physical device in most cases.

iOS Device Safari

This is where you should start focusing very heavily on performance. Pay special attention to animations and interactive elements. Mobile Safari does have Nitro enabled, so you’ll need to keep in mind JavaScript execution will be faster than what you will see when you deploy your app to a device. This is also a good time to test more complicated multi-touch gestures.

PhoneGap app on iOS Simulator

This is the last step you can perform without enrolling in the Apple developer program. You’ll have access to most device functionality like contacts and filesystem, but there is some device functionality (like push notifications) that you can only test on a physical device. Again, you cannot rely on the iOS simulator for performance testing.

PhoneGap app on iOS device

Now that you’ve shelled out the $100 to enroll in the developer program, you need to get ready to deploy your app to the App Store! Focus on testing all device functionality you cannot test in the simulator, and pay very close attention to performance. Ideally, you’ll see performance very similar to what you saw in step 3, just slightly slower due to the lack of Nitro in UIWebView.

The progression from browser to simulated mobile browser to physical mobile browser should happen rapidly with each feature you complete. Testing the fully packaged PhoneGap app on either the simulator or physical device can happen less frequently depending on your situation. The more your app relies on device functionality, the more time you’ll want to spend testing a fully packaged app.

Test PhoneGap Apps on Old Hardware

Modern mobile devices have considerably less horsepower than modern laptops and desktops, and while Moore’s Law dictates that this will improve over time, for the moment you need to build your app with these constraints in mind. Not only do you need to test on physical hardware, you need to test on old hardware. This is where you will experience the most extreme performance challenges. However, if you can make your PhoneGap app fast on old hardware (and you can if you keep reading this blog!) you will be in great shape on the more recent hardware.

A good rule of thumb is to test on the oldest possible device for the lowest iOS major version you intend to support. So if your app requires iOS 6 or higher, that means you need to test on an iPhone 3GS or an iPad 2. If your app supports iOS 4 or higher, you need to test on an iPhone 3G or a first generation iPad, provided you haven’t upgraded the iPad to iOS 5 already! Unfortunately, I don’t have my first generation iPad anymore, but I tested with it religiously at my last job and I found it extremely helpful in keeping me ahead of looming performance problems.

To demonstrate the hardware differences, I put together a simple benchmark that renders an animation as fast as the device will allow, either with or without hardware acceleration. The table below shows how many frames per second different devices achieved on the benchmark.

Device	Version	Acceleration On	Acceleration Off
Chrome	26.0.1410.65	195	192
iPhone Simulator	6.1	228	228
iPad 2	6.1.3	220	55
iPhone 4	6.1.3	150	55
iPod Touch 2	4.2.1	45	31

Clearly, on physical iOS devices hardware acceleration makes a huge difference, and the difference is particularly noticeable on more recent devices. However, on desktop browsers and simulated iOS devices, hardware acceleration makes little or no difference. I’ll cover some techniques for squeezing as much performance as possible out of hardware acceleration in a whole series of future articles. But for now, hopefully this helps to illustrate why testing on old, physical hardware early on will help you out in the long run!