In order to build the best experiences, engineers need the best data available.

Mobile makes this challenge all the more complex, with variables that include different device types, different operating systems, and different connectivities, to name just a few.

With a growing number of tools that all offer varying levels of visibility into your mobile app health, performance, and stability, it can be difficult to know exactly which metrics you should be tracking.

In this post, we’ll outline the most important performance metrics you need to track, and how they help you get to the root cause of issues more quickly, so you can build better mobile experiences.

Startup Time

Mobile users typically check their apps on the go and are accustomed to instantaneous results. Therefore, they won’t sit around and wait for the app to load. For example, if the Uber app takes too long to load, the user will probably switch to the Lyft app instead. In addition, that user that had a poor experience in the Uber app and a decent experience in the Lyft app is much more likely to become a loyal Lyft user.

Now, not only have you lost revenue from that user’s session, but your cost per acquisition and churn increased, not to mention the LTV that particular customer could have brought the company.

Therefore, ensuring your app’s startup time meets user expectations is essential to its success.

However, if the mobile team only has access to the average startup time, they probably miss key changes and respond reactively rather than proactively.

For example, let’s say a company is launching its app into a new market, and the overall feedback is negative. The average startup time shows that the app is a few milliseconds longer than it was before the launch, but it doesn’t indicate that it became dramatically slower during the launch. So there must be another problem, right?

Unfortunately, because the percentage of users in the new market is only a fraction of the overall user base, it makes sense that even an abysmal startup time in the new market would only minimally impact the total user base’s startup time.

Instead, the team needs to be able to segment the data to better understand how startup time impacts the business.

For example, how are high-value users suffering from slow startups? Are users in a new market experiencing worse performance? Are certain devices experiencing slower startups?

This data puts your company back in control of the situation and omits the guesswork in time-sensitive scenarios that impact your revenue.

To learn more, check out our eBook on how to improve mobile app startup time, here.

Crash Rate

Crashes are a surefire way to anger customers, and for a good reason! It’s essentially the equivalent of a customer walking into a brick-and-mortar store where the staff kicks them out the door halfway through their shopping.

This is a major issue for the company’s brand for two reasons:

  1. It damages the reputation of your brand as customers feel their time is disrespected.
  2. It hurts your revenue as customers can’t complete their immediate transaction, and you lose the lifetime value of that customer if they decide to switch to a competitor.

Here are just a few examples from various industries where a crash directly translates to lost revenue:

  • E-commerce Apps: If an e-commerce app crashes during checkout, the customer won’t be able to make their purchase and likely won’t be back.
  • POS Systems: If a POS system crashes during a live event, none of those live customers will be able to make a purchase or enter the venue.
  • Smart Device Apps: If a smart device such as a toothbrush crashes during the setup process, it’s very likely the customer will return the product.

However, while tracking the average crash rate is a good start, it’s still not enough to understand how crashes impact your business.

For example, if the current crash rate is just 0.5%, you might not see the need to drill deeper. However, what if all of the crashes that occur happen at the checkout screen? That tiny percentage could be cheating the business out of significant revenue.

So, in addition to looking at the major metric numbers, it’s also important to have data that shows patterns in the crash rate. Specifically, how are various high-value areas in your app performing? Which devices tend to experience the most crashes? How is the app performing for high-value users? Are there any regions that experience particularly poor crash rates? And if so, should the app be fixed or removed from those regions?

By segmenting crash details, your team can better prioritize fixing issues.

ANR Rate

Application Not Responding (ANR) errors are typically described as freezes or glitches.

Essentially, if the main thread is blocked, the applications cannot run effectively. Therefore, the user cannot proceed, which can have a major impact on your business.

For example, a retailer we worked with had an issue with ANRs. This issue was causing startup times to increase by almost 60%, translating to an estimated revenue loss of $6.5 million per year. With the right data, the engineering team was able to quickly address the issue and reclaim that lost revenue.

Beyond revenue, ANRs can also negatively impact an app’s rankings in the Google Play Store and make it less visible to new customers.

To effectively track ANRs, you can look at the stack trace and see how users responded to the issue.

From there, the mobile team can prioritize what to fix first based on how many people leave at various thresholds, where high-value users are most affected, and which device types and screens are most affected by ANRs.

To learn more, check out this post on investigating Android crashes like ANRs.

Region Metrics

Tracking region metrics is also very important as users in different regions have different devices and different connectivities.

This can have a major impact on the business for several reasons.

First, regions differ in their importance to a business’s bottom line.

For example, perhaps only a portion of your users are in Singapore, but they might make up a large portion of your total annual revenue. Therefore, checking region metrics on a granular level will spotlight opportunities to improve the app, particularly for high-value users.

Tracking segmented region metrics is also essential when launching into a new region.

For example, suppose you expand into Australia, yet that geographic area only makes up 5% of all users on launch. In that case, it won’t impact median/average metrics enough to allow the team to effectively track performance.

It’s also important to take into account cultural differences, and with a tool that provides region-specific metrics, the team can test specific aspects of the app just for that region.

For example, an e-commerce checkout screen that converts well in the United States may not convert as well in Dubai.

In addition, granular region metrics make it easy to roll out new features slowly. For example, the team can roll out a new feature into a smaller region and see how it performs. If it performs well, roll it out to more and more regions with higher value users.

This data can guide your team in answering critical questions like:

  • Do we need to build a new app for this region?
  • How is this new launch performing compared to launches in other regions?
  • Which regions are the most problematic? And should we stop serving them altogether?
  • ​​How are various critical areas of the app performing in one region versus another?

Session Duration

Another key metric to keep an eye on is session duration, as it signals how long users are using your app. If the average session duration changes dramatically in a week, it’s a pretty good hint that there is an issue with the app.

For example, if you have a gaming app and notice the average session duration dropped from 15 minutes to 5 minutes, there’s a good chance that users had a poor experience.

With this clue, you can ask questions like:

  • Did we ship a bad release?
  • Are users less engaged with the app?
  • Is there a corresponding change in another metric that can help explain the change in session duration?

Tracking session duration also allows the team to investigate patterns among the affected individual sessions and discover the root cause of issues. By analyzing session duration in parallel with other metrics, mobile engineers can draw a clearer picture of which issues are most intrusive to users.

For example, an e-commerce store might have an OOM issue on the main scrolling feed of products that correlates strongly with a shorter session duration, whereas slow network calls on another screen have little to no correlation with the session duration.

Therefore, tracking session duration is a great way to prioritize which issues should be fixed first as it directly reveals reduced user engagement.

Churn/Retention Rate

It costs a lot more to acquire a new customer than it does to keep a current customer happy, and a leading cause of churn in mobile app users is a poor user experience.

Downloading a new app only takes a few seconds, so if an app’s experience interrupts the user in a way that takes more than a few seconds of their time, don’t expect them to stick around.

Therefore, track not only the overall churn and retention rate, but also the churn and retention rates of segments of the user base (by devices, connectivities, regions, etc.).

This will illuminate various opportunities to improve and prioritize retention. For example, you may find that while the churn rate is very high in one particular region, that region has few high-value users. Therefore, you may decide to invest engineering resources somewhere else where they will translate to larger business outcomes.

Tracking churn and retention is also a great way to understand how users respond to new releases and various experiments. If there is a correlation between high churn and a new feature update, that’s a strong indicator that the team needs to roll back that feature update.

Detailed data will allow the team to track the churn of various segments (such as specific regions or devices) to which the update is rolled out.

Without segmented data, it will be difficult to see the impact that various feature updates have on small testing groups. Therefore, only once the feature is rolled out to a large group of users (and presumably caused many monthly active users to leave) will it become apparent that the feature rollout was premature.

User Termination Rate

While some user terminations are nothing more than a user decluttering their phone, many terminations occur because the app has frozen and the user has no choice but to swipe up and terminate the session.

Of course, a user that is forced to terminate their session will likely be disgruntled and choose a competitor’s app instead.

To prevent this, monitoring the average user termination rate is an excellent indicator of potential issues that could lead to churn, including:

  • Failed ad loads are causing freezes
  • Slow or broken user flows
  • Failing purchases
  • Server outages causing login errors
  • Excessive media loading that causes slowdowns

In addition to providing an overview of the average user termination rates, mobile teams need to know the source of every poor user experience. Therefore, one of the key features we built into Embrace is the ability to see which screens cause frustrated users to abandon your app.

Therefore, rather than wasting valuable time and losing sales while engineers guess where the terminations are occurring, the mobile team is immediately directed to the issue so that they can fix the problem as efficiently as possible.

Timing of Key User Actions

There are likely a few user actions within your app that absolutely must work 100% of the time. For example, if an office offers keyless entry, that feature must work every time. Otherwise, people may not be able to enter the office without calling additional support.

Therefore, select a few key user actions and add them to the list of performance metrics the mobile team is tracking.

In many cases, customer reviews don’t say where they experienced an issue in the app, so tracking specific user actions is a great way to catch issues that otherwise aren’t immediately apparent.

This also helps you answer questions like:

  • How many users experience issues in these critical areas of the app?
  • Is there a direct correlation between issues in specific areas of the app and churn?
  • How long do users try before giving up and abandoning the app?

For example, one of our customers noticed that about 1% of all purchase attempts resulted in purchase failures. However, both of the associated network calls were resolving successfully, so there were no obvious errors to inspect.

Therefore, they began tracking the exact moment a customer would make a purchase and found that the two network calls were happening out of order in 1% of purchase attempts, which resulted in a failed purchase. Though customers had been complaining about this issue, the mobile team couldn’t pinpoint the root cause without knowing the timing, outcome, and order of all events within the affected sessions. High-fidelity user experience data was vital in helping them reclaim 1% of their total sales. For a company doing $10 million in annual sales, that’s $100,000 that would otherwise be lost each year!

Set Your Mobile Team Up For Success

If you care about mobile experiences then you need the data that enables your team to see all of the metrics mentioned above.

Embrace helps you build better mobile experiences by giving you just that data, making your engineers more efficient and less bogged down by tedious work.

Learn more and explore Embrace today.