The experiment design phase: Create a new experiment

  • Updated

This article will help you:

  • Create and initialize a new experiment
  • Add context to your experiment description, so other stakeholders will understand it

The decisions you make in the design phase set the stage for your experiment’s success. By putting more thought into your experiment’s purpose and goals before you start, you’ll be far more likely to glean useful, actionable insights from it.

To create a new experiment, first ensure you’ve created a deployment and installed the SDK. Then follow these steps:

  1. Open Amplitude Experiment and click + New. In the Create New… fly-out panel, select Experiment.
  2. In the Create Experiment modal, choose the project that will house this experiment from the Projects drop-down menu. 
  3. Enter a name and a description for your experiment in the appropriate fields.

    NOTE: Amplitude Experiment uses flags to include experiments within your product. It will automatically generate the flag key for your experiment from the name you choose; this key will act as an identifier for the flag used in your codebase.

  4. Specify the evaluation mode for your experiment, either Remote (i.e., it will be evaluated on Amplitude servers) or Local. Then specify the bucketing unit you want to use for this experiment.

    TIP: In general, the appropriate bucketing unit will be the user. However, in some B2B use cases, you might want to use company ID or city as the bucketing unit. For example, bucketing by company ID ensures that all users in a particular company will have the same user experience. Be sure the Stable Unit Treatment Value Assumption holds for whichever unit you choose.

  5. When you’re done, click Create. Amplitude Experiment will open a blank template for your experiment.

At this point, Amplitude Experiment will take you to your experiment’s Plan tab. The Details panel should already be filled out with the information you provided when creating your experiment, as should the bucketing unit in the Experiment Goals panel. Double-check these to make sure they’re correct, then move on to the next phase: defining your experiment’s goals.