The experiment design phase: Add your variants

The final step in designing your experiment is to create at least one variant. Experiment will compare your variants with the control, which is usually your product’s current user experience. (This way, Experiment measures the performance of the variant against a known quantity, the performance of your app as it is today.)

Add a variant to your experiment

To add a variant to your experiment, click + Create Variant. The Create Variant modal will appear.



Type in a name and a description for your variant in the appropriate fields. Experiment will automatically generate the variant value from the name you enter. The variant value is a string that you’ll use as a flag in your codebase. When you’re done, click Create Variant.

There is no limit to the number of variants you can add to an experiment, but adding too many can make it harder for your experiment to reach statistical significance. Try to keep your experiments limited to a handful of variants, at most.

Add a payload to your variant

You can also add a payload to your variants. A payload is simply extra data that can dynamically change a variant’s experience without requiring you to write more code.

For example, imagine you’re testing a new splash screen on a marketing webpage. You might get early results that suggest different content might improve the performance of the splash. Instead of going into your codebase and making changes to the variant there, you can just include those changes in a payload, and Experiment will implement them automatically.

To add a payload to a variant, click  next to Payload in the variant’s panel. The Edit Payload modal appears. Paste or type your code into the window and click Apply to set the payload for this variant.

Be sure to save your changes regularly.

Now that you’ve finished designing your experiment, the next step is to roll it out to your users.