What Is A/B Testing?
A/B testing feature lets you safely roll out features and optimize the user experience through controlled experiments. Using Remote Config feature under the hood, A/B testing let's you test different app behaviors or appearances across variants, then use the results to make smarter product decisions.
A/B testing helps you:
- Optimize user experience by understanding what your users prefer.
- Validate new features before rolling them out broadly.
- Reduce risk by testing changes on a smaller audience before applying them to everyone.
Server Side Setup
An A/B testing experiment defines:
- Which users will be eligible/targeted (e.g., random split, specific segments, cohorts).
- How many user variants will be used.
- The goals that will be tracked to measure success.
You can follow this guide for detailed steps on creating an experiment.
A/B Testing works with the help of Remote Config feature. While creating an A/B Testing experiment you will be asked to create a parameter and its variants (its competing values.)
Let's say now you have created a test with two variants (one is selected as control group) for the parameter button_color; blue (control) and green (variant A.) You can now move on to SDK integration to use those values and make your users participate in the test.
SDK Integration
Make sure the SDK and SDK version you will use supports the A/B testing and Remote Config features. To configure the SDKs in the simplest manner, you can follow the quick access guide below to start up your A/B testing journey in the SDKs.
The role of the SDK integration is to:
- fetch the Remote Config parameters (like button_color: blue)
- enroll the user to the experiment
Eligible users will get a variant of the parameter you have set in the experiment when Remote Config values are fetched by the SDK. Which variant the user will get (blue or green) is determined by the server alone. However if the user should participate (enroll) in the experiment is your decision.
Enrollment determines whether a user should be participating an experiment with the variant they received. You can enroll directly when the values are fetched or you can enroll any other time you like.
If you don't enroll an eligible user to an A/B test, that user will not participate in the experiment but will still receive the default parameter value.
SDK only fetches these values for you. Your app has the responsibility to use it for the desired effect.