This article will help you:
- Create new variants, which will be compared to the control (or baseline)
- Add additional code to your variants, to create more dynamic user experiences
The next step in designing your experiment is to create at least one variant. Amplitude Experiment will compare your variants with the control, which is usually your product’s current user experience. (This way, Amplitude Experiment measures the performance of the variant against a known quantity, the performance of your app as it is today.)
Variants are added on the Configure tab.
Start by specifying your experiment settings. Choose the deployment (or deployments) for your experiment from the Deployment drop-down menu in the Settings panel. For more information about working with deployments, see this article on configuring Amplitude Experiment.
Next, you have the option to add your experiment to an exclusion group. Amplitude Experiment will treat it as mutually exclusive to other experiments in that group. Users who see one experiment in an exclusion group will not see any of the group’s other experiments. To add your experiment to an exclusion group, follow the steps outlined in our Help Center article on mutually exclusive experiments in Amplitude Experiment.
Next, click Advanced Settings to set up the following options:
- Sticky bucketing. In some cases, you may want to change your allocation after starting the experiment. This could lead to users being allocated to different variants from the one they've already seen. Enabling sticky bucketing will ensure users continue to receive the same variant once they've been allocated to one. You can do this by switching the Sticky Bucketing toggle to Enabled. Bear in mind that if you want to roll out the winning variant of an experiment to all users, you will need to disable sticky bucketing so Amplitude can re-allocate all your users to that variant. See this article in the Amplitude Developer Center to learn more.
- Evaluation mode. Use this dropdown to select remote or local evaluation of your experiment. For more information, see this article in the Amplitude Developer Center.
- Bucketing salt. Changing this can result in users switching between variants in your experiment. For that reason, we recommend you do not change this unless you know what you're doing. For more information, see our Help Center article on how randomization works in Amplitude Experiment.
Add a variant to your experiment
To add a variant to your experiment, scroll down to the Variants section and click + Create Variant. The Create Variant modal will appear.
Type in a name and a description for your variant in the appropriate fields. Amplitude Experiment will automatically generate the variant value from the name you enter. The variant value is a string that you’ll use as a flag in your codebase. When you’re done, click Create Variant.
There is no limit to the number of variants you can add to an experiment, but adding too many can make it harder for your experiment to reach statistical significance. Try to keep your experiments limited to a handful of variants, at most.
By default, Amplitude Experiment will use the first variant you add as the control. You can change this by dragging a different variant to the top of the Variants section. Whichever variant is labeled A when your experiment launches will be the control.
Add a payload to your variant
You can also add a payload to your variants. A payload is a JSON object that can dynamically change a variant’s experience without requiring you to write more code.
For example, imagine you’re testing a new splash screen on a marketing webpage. You might get early results that suggest different content might improve the performance of the splash. Instead of going into your codebase and making changes to the variant there, you can just include those changes in a payload, and Amplitude Experiment will implement them automatically.
To add a payload to a variant, click next to Payload in the variant’s panel. The Edit Payload modal appears. Paste or type your code into the window and click Apply to set the payload for this variant.
Be sure to save your changes regularly.
<%= heading %>