Roll out your experiment

Rolling out an experiment is a straightforward process: Specify the SDKs you’ll be using, generate some code samples for use in your codebase, create the different user experiences the experiment will include, and then specify the user segments and cohorts that will be included in the experiment. That’s all there is to it.

NOTE: This article continues directly from the Help Center article on designing your experiment. If you haven’t read that and followed the process it describes, do so before continuing here.

To roll out your experiment to your users, follow these steps.

1. In the Setup panel, choose the implementation language you’ll be using for this experiment—Android, iOS, JavaScript Client, JavaScript Server, or React—from the Language drop-down menu. Experiment will generate code snippets for your codebase, which you can copy and modify as needed.
   
   For more details on working with Experiment SDKs, see our SDK documentation.
   
   
2. Next, choose the deployment for your experiment from the Deployment drop-down menu.
   
   For more information about working with deployments in Experiment, see this article on configuring Amplitude Experiment.
   
   
3. In the Allocation panel, you can define user segments that will see your experiment, specify the percentage of users who will be exposed to your experiment, and set the relative distribution weights of each variant.
   
   
   
   Defining a user segment is useful if you’d like to limit experiment exposure to users in a specific geographical location, or those who belong to certain demographic groups, or those who meet certain usage thresholds in your product (i.e., power users).
   
   To define a user segment, scroll down to the Rule Based User Segments section and click into Segment 1. Then follow the same steps you’d use to build a user segment in Amplitude Analytics:
   

• • • • • • To target your experiment based on cohorts, select Cohort and choose a cohort you've previously created in Amplitude Analytics. Cohort membership is evaluated and updated once an hour. 
          • To target based on user properties from Amplitude Analytics, select the user property you're interested in and enter the appropriate values. User properties come from the end user profile maintained by Amplitude Analytics, and are updated based on your analytics instrumentation. (If you need to evaluate user properties in real-time, add the value of the user property to the context you pass to the SDK or REST API.)
          • To target based on custom user properties (i.e., properties that don't exist in Amplitude), type the user property name into the dropdown and enter values to match on for the property. (Rules containing custom user properties can only evaluate against values explicitly defined in the context passed to the SDK or REST API, since Amplitude does not know about these properties otherwise.)

Anyone not assigned a variant as part of a rule-based user segment—and who was not targeted individually—will be assigned a variant based on the configuration in the All Non-Targeted Users section (discussed in Step 4 below).

All of Amplitude’s user properties and cohorts are available to use in defining user segments. Note that the value for any user property included in a rule-based user segment will be the most recent value received by Amplitude. There is no limit on the number of user segments you can include here. Any user who belongs to more than one segment included in an experiment will be assigned to the first one they match. 

NOTE: Caching expiration differs with the properties you use to define user segments. All user properties will always include the latest values your product has sent, defaulting to the most recent value in the event that Amplitude has not received your product’s data. Cohorts will be synced every hour.

4. Next, set the rollout percentage for this experiment. This is the percentage of the users included in the experiment’s user segments who will participate in the experiment. Choose one of the preset values, or manually enter your desired value.
   
   After that, you’ll need to tell Experiment how to distribute your variants. This will determine how many users who’ve been exposed to the experiment will see each variant. The weights are relative values: for example, if you give variant A a weight of 1 and variant B a weight of 4, then four times as many users will see variant B than variant A. (Bucketing is based on the Amplitude ID that Amplitude assigns, according to the device_id and user_id fields of the user context.)
   
   You can then set separate rules for everyone not covered by the segment you created in step 3. If, for example, you only want the experiment rolled out to the specific cohorts you targeted earlier, scroll down to the All Non-Targeted Users section and set the rollout percentage to zero.
   
   Note that Experiment evaluates for users included in rule-based user segments before those not covered by a user segment. However, it evaluates for any individual user or device IDs prior to both.
   
   You can see the total effective distribution of variants across all users in the Summary section.
   
   
5. Next, you have the option to add your experiment to an exclusion group. Experiment will treat it as mutually exclusive to other experiments in that group. Users who see one experiment in an exclusion group will not see any of the group’s other experiments.
   
   To add your experiment to an exclusion group, follow the steps outlined in our Help Center article on mutually exclusive experiments in Amplitude Experiment.
   

NOTE: In some cases, you may want to change your allocation after starting the experiment. This could lead to users being allocated to different variants from the one they've already seen. Enabling sticky bucketing will ensure users continue to receive the same variant once they've been allocated to one. You can do this by switching the Sticky Bucketing toggle to Enabled.

Bear in mind that if you want to roll out the winning variant of an experiment to all users, you will need to disable sticky bucketing so Amplitude can re-allocate all your users to that variant.

4. In the upper-right corner, switch the Inactive toggle to on. Your experiment is now active and visible to your users.

Now that you’ve rolled out your experiment, the next step is to learn from it.