What is it?

This is a feature for mobile teams to view and compare metrics in a way that scales according to their needs as the company and teams grow. The feature enables you to look at the metrics you care about for the groups you’re interested in with granular specificity.

Need to generate your own comparison reports but don’t have the necessary infrastructure already built? Embrace built this just for you!

Why were people asking for this?

This is a feature designed to scale with the needs of your app and team as they mature. In the early stages of mobile development an engineer will simply monitor crashes with each release. But as the app becomes more complex and features are added, the company grows and new teams care about different metrics as they enter new markets. This constant influx of additional features to the codebase becomes a compounding problem for each team to easily filter for the metrics they are responsible for and determine how each feature is affecting the app as a whole.

Growth is good! And as the company and the app grows, so too should its toolset. Solving issues becomes not just an exercise in what happened, but also:

  • When did it start happening?
  • Is it tied to a feature?
  • Who is responsible for the bad code?

All of these and more can be easily answered with our new Metric Comparisons feature.

How can it be used?

This is especially helpful when something is wrong but you are having difficulty pinpointing what exactly is wrong because all of the data is entwined together. Some companies are actively hampering their own feature development by using the “best” practice of isolated roll-outs of new features and code to ensure that when something breaks they can narrow it down by the release!

This is not AGILE at all! A mobile company’s workflow should serve the company’s needs, not the toolset’s needs. Embrace’s new Metric Comparisons feature returns control of data back to the teams, whether it’s Engineering that needs to identify and solve individual issues that have cropped up, Product’s needs to monitor the health of individual features, or Managers that want to monitor the app’s health across all features and versions that exist in the wild. The metric comparison feature exists to eliminate the guesswork involved with all of these coalesced interests!

How does it work?

The Metric Comparisons feature allows mobile teams to have full control over how they view and compare their metrics. We’ll walk through the steps to create a comparison now.

1. Segments — When Product, Engineering, or Management wants to know something the first thing we need to do is narrow it down. Segments are the categories you can compare metrics across. For example, you can compare stability metrics across countries if you are entering a new market. Or you can compare performance metrics across features to spot anomalies leading to poor adoption. Here are the built-in segments you can use out of the box:

  • App version
  • Build
  • Country
  • Environment
  • Model
  • OS major version
  • OS version
  • Persona
  • Session property key

For instance, if you’re grouping by session property, you can easily track new features, experiments, etc. by key and value.

Let’s go through an example to create a Metric Comparison. Say you have an app that’s based in America and are expanding into India. You want to compare the performance and stability of your app in these two regions.

Your segments would be USA and India. If you want to compare across several versions of the app as well, you can select the versions you care about.

Example:
Versions:
– 2.2, 2.3
Segment:
– Country: USA, India
– Filter – Region: USW, East India

You can further refine segments by adding filters. In this case, we want to focus not at the entire country level, but at the regional level within the countries.

2. Filter — (Optional) Sometimes, segmentation isn’t enough to refine the dataset down to what you are currently looking for. In that case, select global filters for all segments to focus the metric data on user and device demographics you want to compare.
Perhaps you only care about how older Android versions are doing in our USA and India example. You could add a global filter here to further refine your dataset.

Example:
Versions:
– 2.2, 2.3
Segment:
– Country: USA, India
– Filter – Region: USW, East India
Global Filters:
– Android Versions 6, 7, 8

3. Metrics — (Required) This is where you set the metrics you want to track. Some options include the following:

  • Crash counts
  • Percentage of sessions with crashes
  • Percentage of userbase with crashes
  • Counts of Application Not Responding (ANRs) or percentage
  • Average duration of network request
  • Log counts
  • Failed Moments
  • And more!

You can add filters here to refine the metrics.

Example:
Versions:
– 2.2, 2.3
Segment:
– Country: USA, India
– Filter – Region: USW, East India
Global Filters: Android Versions 6, 7, 8
Metric:
– Moment Count
– Filter – Moment Name = “startup”

This would allow you to compare the startup times. You can continue to add filters as well, such as if you were only interested in comparing startup times that take longer than a specific threshold.

4. Graphs — (Optional) You can add supplemental graphs to visualize trends. Sometimes a table isn’t enough, and you need a visualization. You can display the same data in the table as a chart, or create a complementary chart to provide accompanying context.

In this example, our metrics comparison would look something like this:

Example:
Versions:
– 2.2, 2.3
Segment:
– OS Version: 9, 10, 11
Metrics:
– Crash Count
– Percent of Network Requests
– Filter – Status Code: 400-599
– Percent of Network Requests
– Filter – Connection Errors

The chart in this example shows the total network request counts by OS version. That way, an engineer can judge whether they care about a higher number of failures in OS’s with lower session counts.

How Embrace Helps Mobile Teams

Embrace is an observability and developer analytics platform built for mobile teams. For mobile teams creating new features or expanding into new markets, you need fine-grained control over the performance and stability monitoring in your app. We are a one-stop shop for your mobile app’s needs, including error debugging and monitoring performance and feature releases.
Want to see how Embrace can help your team ship better apps? Request a customized demo and see a path to improvement immediately!