This article covers frequently-asked questions about Amplitude's Experiment Results chart.
Questions
- Why is my graph displaying an error state?
- Why is reaching significance taking longer than it should?
Answers
Why is my graph displaying an error state?
A common mistake is to attempt to generate a chart using only one variant.
The Experiment Results chart needs something to compare your control to in order to generate results. If you neglect to include both the control and at least one variant, your chart will not display anything.
Why is reaching significance taking longer than it should?
When using a T-test, be sure to plug in the number of samples per variant before launching the experiment. You can get this information from the duration estimator. With a T-test, you will have to wait until your experiment reaches the specified sample size before Experiment Results will run the p value and confidence interval computations.
With sequential testing, even with a large MDE, it can take some time to reach statistical significance if your experiment’s lift is small. A T-test will generally require fewer samples to detect the same lift.
<%= heading %>