Mobile app development is uniquely challenging as there are many variables involved, from different device types to different connectivities. These variables not only make it difficult to maintain high performance, but they also lead to the question of what metrics your team should be tracking.

Accessing the data mobile teams need to track key performance metrics efficiently is a major issue. In many cases, teams have a tool that provides generic metrics, though it doesn’t provide enough data to tell the story behind why that metric is what it is. This makes it difficult (or even impossible) for mobile engineers to resolve the issue.

Therefore, this post will show you the mobile app metrics the team should track and how to use data to quickly identify the culprit and resolve the issue.

Using these detailed performance metrics, your brand will be able to proactively fix issues before they impact customers.

Startup Time

Mobile users typically check their apps on the go and are accustomed to instantaneous results. Therefore, they won’t sit around and wait for the app to load. For example, if the Uber app takes too long to load, the user will probably switch to the Lyft app instead. In addition, that user that had a poor experience in the Uber app and a decent experience in the Lyft app is much more likely to become a loyal Lyft user.

Now, not only have you lost revenue from that user’s session, but your cost per acquisition and churn increased, not to mention the LTV that particular customer could have brought the company.

Therefore, ensuring your app’s startup time meets user expectations is essential to its success.

However, if the mobile team only has access to the average startup time, they probably miss key changes and respond reactively rather than proactively.

For example, let’s say a company is launching its app into a new market, and the overall feedback is negative. The average startup time shows that the app is a few milliseconds longer than it was before the launch, but it doesn’t indicate that it became dramatically slower during the launch. So there must be another problem, right?

Unfortunately, because the percentage of users in the new market is only a fraction of the overall user base, it makes sense that even an abysmal startup time in the new market would only minimally impact the total user base’s startup time.

Instead, the team needs to be able to segment the data to better understand how startup time impacts the business.

For example, how are high-value users suffering from slow startups? Are users in a new market experiencing worse performance? Are certain devices experiencing slower startups?

This data puts your company back in control of the situation and omits the guesswork in time-sensitive scenarios that impact your revenue.

To learn more, check out our eBook on how to improve mobile app startup time.

Crash Rate

Crashes are a surefire way to anger customers, and for a good reason! It’s essentially the equivalent of a customer walking into a brick-and-mortar store where the staff kicks them out the door halfway through their shopping.

This is a major issue for the company’s brand for two reasons:

  1. It damages the reputation of your brand as customers feel their time is disrespected.
  2. It hurts your revenue as customers can’t complete their immediate transaction, and you lose the lifetime value of that customer if they decide to switch to a competitor.

Here are just a few examples from various industries where a crash directly translates to lost revenue:

  • E-commerce App: If an e-commerce app crashes during checkout, the customer won’t be able to make their purchase and likely won’t be back.
  • A POS System: If a POS system crashes during a live event, none of those live customers will be able to make a purchase or enter the venue.
  • Smart Device App: If a smart device such as a toothbrush crashes during the setup process, it’s very likely the customer will return the product.

However, while tracking the average crash rate is a good start, it’s still not enough to understand how crashes impact your business.  

For example, if the current crash rate is just 0.5%, you might not see the need to drill deeper. However, what if all of the crashes that occur happen at the checkout screen? That tiny percentage could be cheating the business out of significant revenue.

So, in addition to looking at the major metric numbers, it’s also important to have data that shows patterns in the crash rate. Specifically, how are various high-value areas in your app performing? Which devices tend to experience the most crashes? How is the app performing for high-value users? Are there any regions that experience particularly poor crash rates? And if so, should the app be fixed or removed from those regions?

By segmenting crash details, your team can better prioritize fixing issues.

To learn more, check out our on-demand webinar about how to solve those impossible crashes.

OOM Rate

Most companies are obsessed with keeping their crash rate low, but they are puzzled by the large volume of complaints from users about “crashes.”

Unfortunately, most of these “crashes” aren’t really crashes at all. Instead, they are often OOM (out of memory) problems, though they look and feel like crashes to the end-user. In fact, OOMs are frequently an order of magnitude more common than crashes.

For example, a company’s 99.9% crash-free rate doesn’t tell the full story. If their OOM-free rate is 98%, they have 20 times the number of “crashes” they think they do. Therefore, tracking OOMs should be a top priority, specifically for social media, e-commerce, and other apps that contain a lot of media that will take up memory resources.

So how can OOMs be resolved?

First, look at the data to see which screens are disproportionately impacted by OOMs. Then, you can prioritize the screens that have the highest combination of OOMs and business value. For example, fixing a purchase screen with a moderate OOM issue will be a priority over fixing a product detail screen that receives less traffic yet has a higher OOM percentage.

The data can also show which users are most impacted (buyers vs. browsers) as well as the culprit of the problem (the app is loading too many elements, the users have old devices, etc.).

To learn more, check out our on-demand webinar about how to debug memory leaks.

ANR Rate

Just as customers often describe OOMs as crashes, ANRs (application not responding) are typically described as freezes or glitches.

Essentially, if the main thread is blocked, the applications cannot run effectively. Therefore, the user cannot proceed, which can have a major impact on your business.

For example, a retailer we worked with had an issue with ANRs. This issue was causing startup times to increase by almost 60%, translating to an estimated revenue loss of $6.5 million per year. With the right data, the engineering team was able to quickly address the issue and reclaim that lost revenue.  

Beyond revenue, ANRs can also negatively impact an app’s rankings in the Google Play Store and make it less visible to new customers.

To effectively track ANRs, you can look at the stack trace and see how users responded to the issue.

From there, the mobile team can prioritize what to fix first based on how many people leave at various thresholds, where high-value users are most affected, and which device types and screens are most affected by ANRs.

To learn more, check out this post about a better way to solve ANRs.

Region Metrics

Tracking region metrics is also very important as users in different regions have different devices and different connectivities.

This can have a major impact on the business for several reasons.

First, regions differ in their importance to a business’s bottom line.

For example, perhaps only a portion of your users are in Singapore, but they might make up a large portion of your total annual revenue. Therefore, checking region metrics on a granular level will spotlight opportunities to improve the app, particularly for high-value users.

Tracking segmented region metrics is also essential when launching into a new region.

For example, suppose you expand into Australia, yet that geographic area only makes up 5% of all users on launch. In that case, it won’t impact median/average metrics enough to allow the team to effectively track performance.

It’s also important to take into account cultural differences, and with a tool that provides region-specific metrics, the team can test specific aspects of the app just for that region.

For example, an e-commerce checkout screen that converts well in the United States may not convert as well in Dubai.

In addition, granular region metrics make it easy to roll out new features slowly. For example, the team can roll out a new feature into a smaller region and see how it performs. If it performs well, roll it out to more and more regions with higher value users.

This data can guide your team in answering critical questions like:

  • Do we need to build a new app for this region?
  • How is this new launch performing compared to launches in other regions?
  • Which regions are the most problematic? And should we stop serving them altogether?
  • ​​How are various critical areas of the app performing in one region versus another?

Session Duration

Another key metric to keep an eye on is session duration, as it signals how long users are using your app. If the average session duration changes dramatically in a week, it’s a pretty good hint that there is an issue with the app.

For example, if you have a gaming app and notice the average session duration dropped from 15 minutes to 5 minutes, there’s a good chance that users had a poor experience.

With this clue, you can ask questions like:

  • Did we ship a bad release?
  • Are users less engaged with the app?
  • Is there a corresponding change in another metric that can help explain the change in session duration?

Tracking session duration also allows the team to investigate patterns among the affected individual sessions and discover the root cause of issues. By analyzing session duration in parallel with other metrics, mobile engineers can draw a clearer picture of which issues are most intrusive to users.

For example, an e-commerce store might have an OOM issue on the main scrolling feed of products that correlates strongly with a shorter session duration, whereas slow network calls on another screen have little to no correlation with the session duration.  

Therefore, tracking session duration is a great way to prioritize which issues should be fixed first as it directly reveals reduced user engagement.

Churn/Retention Rate

It costs a lot more to acquire a new customer than it does to keep a current customer happy, and a leading cause of churn in mobile app users is a poor user experience.

Downloading a new app only takes a few seconds, so if an app’s experience interrupts the user in a way that takes more than a few seconds of their time, don’t expect them to stick around.

Therefore, track not only the overall churn and retention rate, but also the churn and retention rates of segments of the user base (by devices, connectivities, regions, etc.).

This will illuminate various opportunities to improve and prioritize retention. For example, you may find that while the churn rate is very high in one particular region, that region has few high-value users. Therefore, you may decide to invest engineering resources somewhere else where they will translate to larger business outcomes.  

Tracking churn and retention is also a great way to understand how users respond to new releases and various experiments. If there is a correlation between high churn and a new feature update, that’s a strong indicator that the team needs to roll back that feature update.

Detailed data will allow the team to track the churn of various segments (such as specific regions or devices) to which the update is rolled out.

Without segmented data, it will be difficult to see the impact that various feature updates have on small testing groups. Therefore, only once the feature is rolled out to a large group of users (and presumably caused many monthly active users to leave) will it become apparent that the feature rollout was premature.

User Termination Rate

While some user terminations are nothing more than a user decluttering their phone, many terminations occur because the app has frozen and the user has no choice but to swipe up and terminate the session.

Of course, a user that is forced to terminate their session will likely be disgruntled and choose a competitor’s app instead.

To prevent this, monitoring the average user termination rate is an excellent indicator of potential issues that could lead to churn, including:

In addition to providing an overview of the average user termination rates, mobile teams need to know the source of every poor user experience. Therefore, one of the key features we built into Embrace is the ability to see which screens cause frustrated users to abandon your app.

Therefore, rather than wasting valuable time and losing sales while engineers guess where the terminations are occurring, the mobile team is immediately directed to the issue so that they can fix the problem as efficiently as possible.

Timing of Key User Actions

There are likely a few user actions within your app that absolutely must work 100% of the time. For example, if an office offers keyless entry, that feature must work every time. Otherwise, people may not be able to enter the office without calling additional support.

Therefore, select a few key user actions and add them to the list of performance metrics the mobile team is tracking.

In many cases, customer reviews don’t say where they experienced an issue in the app, so tracking specific user actions is a great way to catch issues that otherwise aren’t immediately apparent.

This also helps you answer questions like:

  • How many users experience issues in these critical areas of the app?
  • Is there a direct correlation between issues in specific areas of the app and churn?
  • How long do users try before giving up and abandoning the app?

For example, one of our customers noticed that about 1% of all purchase attempts resulted in purchase failures. However, both of the associated network calls were resolving successfully, so there were no obvious errors to inspect.

Therefore, they began tracking the exact moment a customer would make a purchase and found that the two network calls were happening out of order in 1% of purchase attempts, which resulted in a failed purchase. Though customers had been complaining about this issue, the mobile team couldn’t pinpoint the root cause without knowing the timing, outcome, and order of all events within the affected sessions. High-fidelity user experience data was vital in helping them reclaim 1% of their total sales. For a company doing $10 million in annual sales, that’s $100,000 that would otherwise be lost each year!

Set Your Mobile Team Up For Success

If the mobile app is a critical aspect of your business, you need data that enables the team to see all of the metrics mentioned above.

That said, it’s also important to give your mobile team appropriate tooling that will allow them to identify and fix issues as they arise. For example, your current tooling might provide an average OOM rate, but if there is no data on which screens are most affected, your team will be left guessing in the dark.

Fortunately, Embrace is an ideal solution as our extensive data collection of every session of every user makes it easy to identify, prioritize, and solve the issues that matter to your business.  

This makes your engineers more efficient as they spend less time experimenting with different possible solutions and allows them to take a more proactive approach to dramatically minimize mobile app performance issues.

If you’d like to see what Embrace can do for your company, request a demo today!