You've got your data in, searched it, and transformed and viewed it. You know how Splunk Enterprise makes it easy to recognize, at a glance, when your data is telling you that something is wrong and action is needed. With alerts, there's no glance required: Splunk Enterprise can tell you the moment something happens--an action is logged, a value is reached, a threshold is passed, and so on--and even trigger actions based on what's happening.
If you're just interested in creating custom alert actions, which are developer-crafted, customizable actions that are initiated when an alert is triggered, you might want to go directly to the "New adventures require new tools: alerting" chapter in the Journey.
An alert is a notification mechanism to let you know when an event of interest has occurred. You can configure Splunk Enterprise to trigger an alert whenever a search returns a result that matches a threshold or trend setting that you specify.
When getting started with alerts, consider the following:
Consider also the different types of alerts available in Splunk Enterprise:
If you want more information about the different alert types, along with example scenarios for each, see "Alert types and scenarios" in the Alerting Manual.
You've probably already come up with a few possible uses for alerting in your Splunk Enterprise deployment. Here are a few more to get you thinking even more about how to make alerting work for you:
More alert action scenarios are available in topic "Alert examples" in the Alerting Manual.
You always start the alert creation process in the same way: create a search and then save it as an alert.
Enter a search query. In the upper-right corner, click Save As, and then Alert. The Save As Alert window appears.
In the Settings section, you enter a title for the alert, an optional description, specify whether the alert is shared with others or private, and what type it is.
Select the Throttle checkbox to suppress alert triggering for a period of time that you define. This prevents you from being overwhelmed with triggered alert actions, should your alert be triggered more frequently than you had expected.
The final step in creating an alert is specifying a trigger action, or alert action. We go into detail about alert actions in the next section.
Alert actions are initiated when an alert is triggered. You specify an alert action in the final step of creating an alert.
You have several options when it comes to alert actions:
To manage or configure the installed alert actions, go to Settings > Alert Actions. From here, you can click Browse more to search for and browse custom alert actions that others have created and posted to Splunkbase.
The next section describes custom alert actions in more detail and tells you where to go to learn how to create them.
The custom alert action framework lets developers add new functionality and integration for alerts. Custom alert actions, like alerts, can be access control list (ACL)-managed, packaged, and distributed within apps, but they are fully modular, and can be reused throughout Splunk Enterprise.
A custom alert action is packaged within an app, and is made up of several files laid out in a set directory structure. A generic directory structure for a typical custom alert action appears within detailed instructions for creating a new custom alert action in "Custom alert actions overview" in the Developing Views and Apps for Splunk Web Manual. We also list the structure of our reference app's custom alert action--along with an account of our real-life experience creating a custom alert action from scratch--in the "New adventures require new tools: alerting" chapter in the Journey.
The process of creating a custom alert action is straightforward but multifaceted. To ease the process, we've created a custom alert action template and included it with the PAS reference app download package. In the spikes folder, open the alertaction_app_template folder and you'll see all of the necessary files, already placed in the correct file structure. From there, follow the four basic steps for building a custom alert action into an app:
Depending on how you engineer your custom alert action, you may need your users to set it up first. For example, our reference app's custom alert action requires users to set up their JIRA server and other configuration settings before they can assign the alert action to an alert. Be sure to add a setup step to your documentation.