Experiments and Feature Flags

Experiments and feature flags

info

Experiments and feature flags are available in the Embrace Apple SDK 6.22.0 and later. If you are on an earlier version, update the SDK before you integrate.

Most apps run A/B tests and roll features out behind flags, but the results usually live in a different tool than your crash, performance, and session data. Embrace closes that gap: when you tell the SDK which experiments and feature flags a user is exposed to, every session, log, crash, and trace produced from that point on carries that context.

That lets you answer questions like:

• Did the new checkout variant increase the crash rate?
• Is the app slower to start for users in the treatment group?
• Which flag was enabled on the sessions where this error spikes?

How it works

You declare enrollment in experiments and feature flags by calling the Embrace APIs every time your app starts up, as early as possible, so as much of the telemetry from that app instance will be annotated correctly. The Embrace SDK is not an experimentation platform and does not decide which variant a user gets, so it is up to you to do the bookkeeping and update the SDK if experiment and feature flag state changes. In other words, you keep using whatever system you already have for experimentation and pass the resulting assignment to Embrace so your telemetry can be segmented by it.

A few properties are worth knowing before you integrate:

• Experiment state is associated with the app instance. Once declared, an experiment or feature flag is attached to everything the SDK reports for as long as the app runs, irrespective of session boundaries.
• State is not persisted across launches. Cold-starting the app clears the declared state, so you have to declare it again on every start.
• Experiments and feature flags are tracked separately. Each is identified by its kind together with its ID, so an experiment and a feature flag can share the ID (e.g. dark-mode) and stay independent of each other.
• The first call wins. The first call for a given ID fixes its variant and start time. Later calls using that same ID change nothing, even after it has been untracked.
• Untracked experiments and feature flags are kept. Untracking sets an end time rather than removing it, so a variant the user was exposed to earlier in the app instance stays visible in the data.

info

These APIs do nothing until the SDK has been set up. Calls made before EmbraceIO.setup(options:) are ignored rather than queued, so declare enrollment after setup.

Tracking experiments

Use trackExperiment(id:variant:startedAt:) for a single experiment:

import EmbraceIO

EmbraceIO.shared.trackExperiment(
    id: "checkout-redesign",
    variant: "treatment"
)

Both variant and startedAt are optional. Omitting startedAt uses the moment of the call, which is what you want when you declare enrollment as soon as you resolve it:

// No variant — you only care that the user is in the experiment
EmbraceIO.shared.trackExperiment(id: "new-onboarding")

// Enrollment started earlier, and you are catching Embrace up
EmbraceIO.shared.trackExperiment(
    id: "checkout-redesign",
    variant: "treatment",
    startedAt: assignment.receivedAt
)

When you resolve several assignments at once — the usual case, right after your experimentation SDK finishes fetching — declare them in a single call with trackExperiments(_:):

import EmbraceIO
import EmbraceSemantics

let experiments = assignments.map { assignment in
    TrackedExperiment(
        id: assignment.key,
        variant: assignment.variant
    )
}

EmbraceIO.shared.trackExperiments(experiments)

Tracking feature flags

Feature flags work exactly like experiments and follow the same rules. They are kept as a separate kind so the dashboard can present them as a distinct experience:

import EmbraceIO
import EmbraceSemantics

// A single flag
EmbraceIO.shared.trackFeatureFlag(id: "new-video-player", variant: "enabled")

// Several at once
EmbraceIO.shared.trackFeatureFlags([
    TrackedFeatureFlag(id: "new-video-player", variant: "enabled"),
    TrackedFeatureFlag(id: "offline-mode", variant: "disabled"),
    TrackedFeatureFlag(id: "beta-search")
])

Untracking an experiment or feature flag

When an experiment or feature flag no longer applies for an app instance, untrack it so telemetry from that point forward is annotated correctly:

// A single experiment or feature flag
EmbraceIO.shared.untrackExperiment(id: "checkout-redesign")
EmbraceIO.shared.untrackFeatureFlag(id: "new-video-player")

// Several at once, with an explicit end time
EmbraceIO.shared.untrackExperiments(ids: ["checkout-redesign", "new-onboarding"])
EmbraceIO.shared.untrackFeatureFlags(ids: ["offline-mode"], endedAt: userLoggedOutAt)

Untracking is scoped to the kind you call it on: untracking the experiment dark-mode leaves a feature flag with the same ID untouched. Only the first call for a given ID takes effect, and IDs that were never tracked are ignored.

You do not need to untrack anything at app exit. The declared state lives only for the current process.

A typical integration

The common shape is to declare enrollment as soon as your experimentation SDK hands you assignments, after Embrace has been set up:

import EmbraceIO
import EmbraceSemantics

func applicationDidFinishLaunching() {
    try? EmbraceIO.setup(options: .init(appId: "myApp")).start()

    experimentationClient.fetchAssignments { assignments in
        EmbraceIO.shared.trackExperiments(
            assignments.experiments.map {
                TrackedExperiment(id: $0.key, variant: $0.variant)
            }
        )

        EmbraceIO.shared.trackFeatureFlags(
            assignments.flags.map {
                TrackedFeatureFlag(id: $0.key, variant: $0.value)
            }
        )
    }
}

If a user's assignments change mid-run — after a login, or a remote config refresh — declare the new ids as they appear and untrack the ones that no longer apply. Re-declaring an id you already tracked does nothing, so you can safely call trackExperiments(_:) with your full assignment set after every refresh.

Limits and validation

The SDK bounds what it accepts so that a misconfigured experimentation setup cannot grow your payloads without limit. The defaults are:

Limit	Default
Experiments/flags per process	500
Identifier length	128
Variant length	128

The per-process limit covers experiments and feature flags together, both those still active and ones you have untracked. Embrace can adjust these limits remotely, within fixed ceilings.

Validation is per entry, so one bad entry never invalidates the rest of a batch:

• Surrounding whitespace is trimmed from IDs and variants.
• An ID that is empty once trimmed is dropped.
• An entry whose ID or variant is over the allowed length is dropped. Values are never truncated, since a truncated ID would refer to a different experiment.
• nil, an empty string, and a whitespace-only string all mean the same thing for variant: no variant.
• Duplicate IDs within a single call resolve to the first one.

Invalid and over-the-limit entries are dropped silently without throwing, so it's worth checking your IDs and variants against these limits while you integrate.

Where is experiment data recorded in telemetry?

Tracked experiments and feature flags are attached as an emb.experiments attribute on the session part span and on every log the SDK emits, which is what makes them available for filtering and comparison in the Embrace dashboard. Because the value is also mirrored to storage, a crash report recovered on the next launch still carries what was declared when the crash happened.

If you export your telemetry to another backend, the same emb.experiments attribute travels with the exported spans and logs.