You’re beginning to understand just how different mobile is from web. User expectations, engagement, and experiences are different, and best practices for architecting and scaling apps are also different. It’s now time to think about how to measure app health from a mobile perspective. Monitoring for app crashes isn’t sufficient. You need to think about monitoring and measuring app stability and performance to understand the health of your mobile app.

App users don’t have patience for poor stability or performance. You need to be able to monitor and measure exactly what they are experiencing. There’s a simple formula to keep in mind: Stability + Performance = Overall App Health.

In this post, we’ll cover some key stability and performance metrics you should measure and monitor to maintain a healthy app. In addition, we’ll explain why prioritizing issues comes down to understanding their impact on your users and your business.

Stability is about more than crashes

Stability encompasses anything that contributes to app failures, whether it’s caused by the user, the app, or the OS. If your app crashes regularly, you won’t have any users. But stability goes far beyond crashes – that’s table stakes for app health. If you have other stability issues, you’ll also experience high churn rates. The majority of users will abandon an app after only three app failures.

It’s important to monitor and measure stability metrics including:

  • Percentage of cold start sessions: This can indicate the presence of other underlying issues. Your app might be using too much memory or CPU or hanging the main thread. If your app freezes, users may force-quit it and trigger a subsequent cold start.
  • Percentage of sessions with low memory warnings: This can indicate memory leaks that ultimately affect performance.
  • Percentage of sessions in low power mode: You may have introduced a feature that is draining the battery and producing a poor user experience.
  • App not responding (ANR): ANRs lead to stutters and freezes, which can feel the same as a crash to your users. Monitor ANR exits and ANRs that recover, because users will quickly abandon apps that offer slow or frustrating experiences.

Some mobile tools only measure an app’s crash-free rate. But this is just one of many components that should be measured when considering the health score of your mobile app. A healthy app also enables your users to access and engage with your features quickly. The speed with which users can traverse your app and complete key flows like startup and purchase is known as app performance.

Measure performance to gain a competitive edge

Apps increasingly differentiate on performance. Just like with poor stability, users have extremely low tolerance levels for poor app performance. This is why it’s important to get a baseline of robust performance metrics, so you’ll know immediately when degraded performance is impacting your user experience.

It’s particularly important to test and measure these metrics under actual user conditions! Consider where your users are and what devices or operating systems they are using so you can replicate these environments as closely as possible.

Some of the most important performance metrics you should measure include:

  • Startup time: This directly impacts sales conversions and user engagement. Amazon has calculated that a 100ms reduction in startup time results in a 1% increase in company sales. Have you measured your startup time recently? Just as importantly, do you know the startup time of your biggest competitors?
  • Login success/failures: Get a baseline of this KPI, so you will know if it spikes. It may mean your app is logging users out.
  • Timing for first party networking calls: If you are thinking mobile-first, you should have device-side monitoring for all network calls to understand the user experience. For example, your server-side tools won’t pick up timeouts, connection errors, or frozen screens stemming from too many concurrent network calls.
  • Success and failure rates for key user flows: The better you understand your user flows, the easier it will be to measure key experiences that impact engagement and revenue.

You can lose users even if your app isn’t failing in the traditional sense – crashes or errors. If you only monitor stability metrics, you won’t be spotting churn triggered by slowdowns in your app. By also monitoring performance metrics, you can easily spot regressions that may be costing you users and revenue.

Understand the user impact of stability and performance

Now that you understand the most important stability and performance metrics you should be measuring, it’s also important to understand how to prioritize those when your mobile monitoring tools indicate a problem. Your resources are not limitless. But how do you determine which ones deserve immediate attention?

The answer boils down to the impact an issue has on your users and your business, and how it ultimately affect your KPIs. Because KPIs can differ across verticals, the type of app you’re developing, and the maturity of your app, there isn’t a universal answer to the question of which ones to prioritize when your monitoring tools surface an issue.

When you consider business impact, consider the maturity of your app. Mature apps tend to closely track stability across releases to prevent regressions, while young apps may sacrifice some stability to increase feature velocity and gain more users rapidly. Retention becomes more important once your user growth slows and you can’t expect to acquire new users as easily.

The impact to your users depends on the type of app you’re building, and what the expected user experience is. E-commerce impact measurements are different than social media or gaming impacts. For example, checkout time is critical in e-commerce, user engagement or photo uploading time are more relevant to social media, and in-app purchase conversion metrics may be the most applicable to gaming.

Figuring out the most relevant impact metrics for your app will direct your attention to the issues most likely to affect your bottom line.

How Embrace can help

Embrace is a data driven toolset to help mobile engineers build better experiences. Reproduce any issue with full access to unsampled user sessions, which contain all the user actions and technical events of any user journey.

That means you can look up any user and recreate exactly what they experienced in your app, even if it spans multiple foreground sessions.

In one platform, you can identify, prioritize, and resolve all user and business-impacting issues, such as slow network calls, background crashes, broken web views, or failing user flows. In addition, Embrace provides a unified app health metric combining stability and performance. Alongside your metrics and KPIs, this enables you to easily track feature and release health to spot regressions immediately.

To learn more about measuring and monitoring app health, check out our recent webinar about the Four Pillars of Building a Great Mobile App, or read our more detailed eBook.

If you’d like to learn more about Embrace, you can check out our website or request a demo.