Add A/B Testing to Your App in Minutes Using Kii

What is A/B testing?

In a nutshell, A/B testing allows you to deploy multiple scenarios to your app’s users in order to determine the optimal combinations around app experience, features, usability etc. This is a great way to test your hypotheses about what your users prefer, and helps insure that you are optimizing for user adoption, retention, monetization and other KPIs that matter for your business. In a ‘Hello World’ example, think of it as testing whether a button is clicked more when it is red or green. You won’t know unless you test it on users!

Which problem does this solve?

Other than optimizing your app, A/B testing can also improve your development cycle if you’re using the right tools. Kii’s latest toolset for A/B testing is a great example.

The traditional way to A/B test looks something like this:
A/B old cycle

You must deploy your app with version A, wait for it to roll out to a significant number of users, gather analytics, redeploy with version B, wait for it to roll out, gather analytics, then you can finally make your decision on whether version A or B yields the best results. This method takes lots of time on both the development side and simply waiting for the app store and your users to update.

With Kii’s A/B testing, you can deploy both versions simultaneously, gather analytics and propagate the optimized version to all of your users without ever redeploying. This saves lots of time in the development cycle and gives you more control over what your users are seeing!

What are some example use cases?

This will vary with every app, but to get your creative ideas flowing:

  • Optimize the position/color/text of UI elements
    • Is my conversion button better as blue or green?
    • What is the best text for my conversion button?
  • Pricing Models
    • A/B test whether users will pay an extra dollar for an In-App-Purchase
    • Larger one-time purchase or smaller subscription?
  • Entire flows/experiences
    • In my tutorial, is it better to show one big page or several smaller pages?
    • Should the news feed or the messages view be shown as the main tab?

How can I integrate it in my app?

Kii has just launched an easy-to-use toolset for A/B testing, which can be added to your app in a matter of minutes. For a more in-depth, cross-platform guide, check out our documentation.

Configuring

It’s easy to set up your experiments in Kii’s developer portal. Once you have your free Kii account, check out the A/B testing tab on the left-hand side of your app’s page. You can easily configure an experiment as shown:

ab configuration

In this case, we’re going to determine if a button is clicked more when the color is red vs. green.

The Code

In order to execute the experiments within your app, it will need to download some configuration from the cloud and upload any experiment results that the user generates. Fortunately, Kii’s tools allow us to do that very easily. All you need to do is wire up the logic of your experiment!

Retrieving an Experiment

To start, you must retrieve the experiment’s configuration from the backend

// Get experiment synchronously
NSError* error = nil;
KiiExperiment *exp = [KiiExperiment getExperimentSynchronous:@"EXPERIMENT_ID"
                                                   withError:&error];
if(error != nil){
  // handle error
}

Note: this is a very basic iOS snippet. For asynchronous and other platforms, check out our documentation.

Apply a variation

The SDK will automatically segment the users, so you simply need to ask which variation should be applied and apply it to your UI or flow as needed:

NSError *error;
KiiVariation *variation = [experiment appliedVariationWithError:&error];
if (error != nil) {
  // If our SDK hasn't been able to download a variation yet,
  // simply use a default
  variation = [experiment variationByName:@"A"];
}

// Get the details for the applied variation
NSString *buttonColor = variation.variableDictionary[@"buttonColor"];
....
// Apply the color to the button
....

Sending Events

Finally, we need to track conversions – so that means sending events when the button is viewed and when the button is clicked. Be sure to use the variables you defined in the configuration as shown:

// The button is displayed and "eventViewed" event triggered.
NSDictionary *viewEvent = [variation eventDictionaryForConversionWithName:@"eventViewed"];
[KiiAnalytics trackEvent:experiment.name
              withExtras:viewEvent];

// The button is clicked and "eventClicked" event triggered.
NSDictionary *clickEvent = [variation eventDictionaryForConversionWithName:@"eventClicked"];
[KiiAnalytics trackEvent:experiment.name
              withExtras:clickEvent];

Viewing Results

The Kii developer portal makes it easy to track your experiment’s progress, and even provides statistical confidence to determine whether or not an experiment has enough data to be considered complete. Check out how it looks here:

abresults

Deploying

Let’s say your experiment has achieved 100% statistical confidence and you are ready to officially deploy variation B to all your users. Don’t worry about changing code and updating your app – simply hit ‘Finish’ in your experiment’s page. The optimized version will automatically be rolled out to all your users without having to wait for a version update!

Wrapping up

A/B testing is a must for developers who are trying to optimize their apps and compete in the app store. Rather than spending your time building infrastructure to handle this, a tool like Kii’s A/B testing is a great way to add these professional features in your app within minutes. You focus on building a great app, and we’ll help take care of the rest!

To get started, head over to developer.kii.com and sign up for a free account – no credit card required. If you’d like to check out our in-depth guides, head over to documentation.kii.com.

If you’d like to see more about A/B testing and how to integrate it into your app, check out our video below

Happy coding 🙂

- Share -

Chris Beauchamp

More posts from

  • Kii IoT platform enables customers to create smart, connected solutions and services
  • Long track record of supporting customers like Docomo, Toshiba, Kyocera and many others
  • Strong ecosystem partners across the globe, consisting of carriers, device/sensor manufacturers, system integrators, network/infrastructure providers, chip vendors and solution developers
  • HQ in Tokyo, US HQ in Silicon Valley & offices in China, Taiwan, Hong Kong and Europe (UK, Spain, Germany)
  • www.kii.com