Alert fatigue is when you're so frequently alerted to something you begin to ignore it and is often caused by being alerted too frequently about non-urgent things.
Alert fatigue is why we don't do fire drills every week, as the more often we're alerted to non-urgent situations, the more likely we are to ignore a future alert when there is an urgent situation. Ignoring or silencing monitoring exception notifications can be dangerous, just like ignoring a fire alarm or taking the batteries out of a smoke detector.
If you ignore or turn off alerts, you're likely suffering from alert fatigue, putting your application at risk of severe problems. The good news is once you identify alert fatigue, you can remedy it.
In this article, we'll show you ways to configure AppSignal only to alert you when necessary.
To better manage your notifications, you first need to consider what you want to be notified about and when you want to be notified about it. In this instance, look at your application, consider which components are critical and non-critical, and decide what notification strategy you want. You also need to consider the frequency of any critical elements of your application to decide how often you are notified to avoid being over-notified when an exception occurs on a minutely basis.
Benign errors present no threat to your application. There may be something that fails frequently but has no negative impact on your application, for example, an API that third parties frequently experiment with. In that case, you may never want to be notified or only be notified the first time an exception occurs.
There may also be elements of your application that are critical, such as code that charges customers for using your application, in this case, you may want to be notified every time an exception occurs.
You may also have code that executes frequently, for example, a minutely job. If logic executed by the job fails, to avoid your inbox being filled with dozens of emails, you likely don't want to be notified on every occurrence and may prefer to be notified on the first occurrence per deployment.
In AppSignal when an error is detected in your application, an incident is created, and you can configure how and when you are alerted about an incident. We have the following notification frequencies:
You will be notified every time an incident occurs, with a five minute cooling period.
For example: if an incident is triggered 20 times a minute, for 20 minutes, you'll receive a notification for the duration of the incident on minutes 0, 5, 10, 15, and 20, rather than every time the occurrence occurs.
You will be notified the first time an incident occurs within a deployment. You will need to have deploy markers set up to use this frequency.
For example: If you deploy new code to production, and an error occurs 5 times a minute, you'll only be notified of the first occurrence.
This is for re-occurring issues. If a previously closed issue occurs again, you'll only be notified the first time it happens.
For example, a previously closed error re-occurs once every 5 minutes. You'll only be notified of the first occurrence.
You will never be notified of a specific incident.
A notification is sent every nth time per hour or day, depending on your preferences.
For example, an error is triggered 85 times per day. You've configured AppSignal to notify you on every 20th occurrence of the error. In this case you'd be notified 4 times.
Notification frequency settings are just one of the many developer-driven features that make AppSignal the APM of choice for thousands of devs! AppSignal integrates with many popular third-party platforms like Discord and Slack to notify you when and where it matters most.