Dig deeper into experimentation data with Experiment Results

Previously, Amplitude customers had to use Amplitude Experiment’s feature flagging system if they wanted to take advantage of experimentation features built into the platform. With Experiment Results, Amplitude Analytics customers who have invested in a non-Amplitude feature flagging platform, whether third party or homegrown, can now take advantage of Amplitude’s planning, tracking, and analysis tools for Experiment—while still using the A/B tracking data generated by their own feature flagging platform.

Before you begin

Prior to using Experiment Results, you’ll need to ensure you’ve instrumented the metric events that are relevant to your experiment. Without them, you’ll be unable to create the success metrics and goals that Experiment Results needs to compare each variant in its analysis.

You will also want to make sure you’ve instrumented the necessary exposure events, which represent the delivery of a variant to a user participating in the experiment. See the Amplitude developer documentation on exposure tracking for more detailed information.

Analyze an A/B test using Experiment Results

To create an A/B test and see the results, follow these steps:

1. From the left-hand sidebar, click + New and select Experiment Results from the menu.
2. In the Events module, click Define Metric to begin setting up your primary metric.
3. In the Define Metric fly-out panel, use the drop-down menu to specify the metric type. There are five options:

• • • Unique conversions
    • Event totals
    • Sum of property value
    • Average of property value
    • Funnel Conversion

The first four are available for individual event metric analyses, while funnel conversion allows you to define a multi-step journey that must be completed in order for conversion to be counted.

4. Next, specify the event that will be used for this metric. You can also filter the event using a + where clause. When you’re finished, click Done.
   
   
   
   
5. Optionally, click Add secondary metric to add a second, subordinate metric to the analysis. You can add multiple secondary metrics as necessary.
6. Click Select event… to define your experiment’s exposure event. The exposure event is the event users must trigger to become part of the experiment.
7. In the Segmentation module, add your variants. All experiments require at least one variant, which is known as the control. You can use a previously-saved segment by clicking Saved Segments and selecting the one you want from the list. You can also create a new variant by clicking Select property… under Control.
   
   
   
   Choose the properties and values that will define your variant and click Apply.
   
8. Click Add experiment variant to add more variants as necessary to reflect the experiment setup in your feature flagging system. 

Amplitude will calculate your statistical results on the fly and display them in the stats table, located in the Metrics Module. 

Interpret your results

While the specifics may vary depending on the metric types you’re using, you’ll see three charts depicting your results:

• Absolute values by variant
• Absolute values with confidence interval over time
• Cumulative exposure