Insights: Spot anomalies in your metrics quickly with alerts

Amplitude's alerts feature is built using Prophet, an advanced data mining and machine learning technique that automatically detects any anomalies in your product data, and instantly brings these hidden trends to your attention. It does this by first identifying expected values, and the confidence intervals around them, and then analyzing the overall trend of the data and combining it with the weekly trend of the data. 

Before you begin

Alerts are only available to Growth and Enterprise customers who have purchased the Insights add-on. If you'd like to upgrade your Amplitude subscription, contact your CSM.

Secondly, there are a few details about alerts you should be aware of:

• You can set alerts for multiple events and user segments at the same time.
• If you use a group-by on a property, you alert will track metrics against the top 1000 segments only.
• Currently, custom alerts are available  in Event Segmentation and Funnel Analysis charts, and only on those set to a daily or hourly frequency.
• A Funnel Analysis must measure conversion over time in order to support alerts.
• In an Event Segmentation chart, if you need to track weekly or monthly KPIs, use a 7-day or 30-day rolling window. Also, alerts are not supported for the Frequency and Formula tabs, or for the bar chart visualization. Some custom formulas are supported, as long as they produce a chart with an X-axis time series.
• Because funnel analyses are calculating conversion of the funnel based on the conversion window, alerts cannot detect anomalies until that window has elapsed. Using a conversion window longer than a day will delay monitor alerts.
• Only chart owners can set alerts. If a chart you would like to receive alerts for already exists but was created by someone else, make a copy and save it before setting up an alert. Additionally, any changes you make to a chart will automatically apply to the alert tracking it.

Automatic monitors are setup on every event you log in Amplitude and "track all of your events for anomalies and unexpected trends."  For these, the user has no control over setup. The event will monitored for anomalies once it has a volume of 100 or more events per day in at least 15 of the last 30 days. These anomalies are detected when a value falls outside of the 99% confidence interval of historical data.

Set an alert

There are three types of alerts in Amplitude: automatic, custom, and smart.

An automatic alert is set up for every event you instrument in Amplitude. It's designed to track all events for anomalies and unexpected trends. This happens automatically; there is noting for you to do in setting up an automatic alert.

Amplitude will monitor an events for anomalies once it achieves a volume of 100 or more events per day in at least 15 of the last 30 days. Amplitude considers an anomaly to have occurred when an event's value falls outside of the 99% confidence interval of historical data. We use 120 training days for automatic monitors.

To subscribe to automatic alerts and receive emails when an anomaly is detected, navigate to Settings > Projects, find the project you want to monitor, and open the Automatic Monitors tab. Switch the Not Subscribed toggle to Subscribed.

To set a custom or smart alert for a chart you own, follow these steps:

1. Navigate to the chart you want to set an alert for. Note that the chart must be saved before you can set an alert for it.
2. Click the bell icon to open the quick setup tool for alerts and decide whether you want a smart alert or a custom alert:

• • • • A smart alert will look for unexpected changes outside of a 99% confidence interval. 
      • A custom alert allows you to be more specific about the conditions under which you'll receive a custom alert: whether it's above or below a specific value, or whether it differs from a previous value by a specified amount. You can also set a custom alert based on confidence interval.

3. To set a smart alert, click Advanced Settings only if you want the alert to be sent to other team members, in addition to yourself. Add those emails in the appropriate field of the Set Alert modal, and click Set Alert. If you do not wish to add other emails, just click Save instead.
4. To set a custom alert, click the Custom radio button to open the Set Alert modal. Specify your alert conditions, add the emails of everyone who should receive this alert, and click Set Alert.

For custom or smart alert, the training days are 120 for daily interval charts and 14 for hourly interval charts.

Confidence intervals and statistical significance in custom alerts

When setting up a custom alert, you can choose to receive alerts when significance thresholds of 95%, 98%, or 99% are breached. These confidence intervals are determined by taking your historical data and identifying where 95%, 98%, and 99% of all data points fall.

The higher the required significance, the less "noisy" your alerts will be. In charts, the blue band represents the range of the confidence interval. A 95% confidence interval will have a narrower band than a 99% confidence interval, because the 99% confidence interval captures more historical data points.

View and manage alerts

You can see a list of recently-triggered alerts for a project by clicking Notifications in the left-hand sidebar, then opening the Alerts tab. You can see a list of all existing alerts for a project by clicking  > Projects, selecting the project you're interested in, and opening either the Custom Monitors (for custom alerts) or the Automatic Monitors (for smart alerts) tab. Click on an alert to edit or manage it.

Alert emails

When an alert is triggered, Amplitude will send an email to everyone who is designated to receive them. This email will be sent by 8:00am in the project's timezone for daily metrics, or the hour after an anomaly in an hourly metric is detected.

Click on a chart in the email to be taken directly to that chart in Amplitude. You'll see a side-panel that reiterates the issue Amplitude alerted you about, so you won't lose important context in the moment.