QA your experiment

Once you’ve designed an experiment, you’ll need to QA it, both before and after rolling it out to your users. Amplitude Experiment gives you two easy ways to do that.

QA before rollout

Before any users see your experiment, you’ll want to make sure that the variants you’ve developed look and function exactly the way you intended.

Because Experiment allows you to allocate specific variants by user or device ID, you can quickly and easily ensure that your test devices are served the relevant variants when they enter your experiment. Simply scroll to the Individual Users section and enter the device ID of a testing device, or the user ID you have reserved for QA and internal use.

Just open your product and trigger the exposure event for the experiment. You should see the variant you specified above.

QA after rollout

After rollout, you’ll be able to track how many of your users have been exposed to each variant on a daily basis. Experiment breaks this out for you in the Assignments panel, in both chart and tabular form.

This is a useful way to QA the assignment process. If you notice that one variant is enrolling significantly more or significantly fewer users than you expected, it could indicate an issue you should investigate.

If you do spot some outliers or anomalies that concern you, click Root Cause Analysis to conduct a deep dive into the potential causes. To learn more about how the Root Cause Analysis feature works, see this article in the Help Center.