Develop your ITSI module

This section describes how to develop your own Splunk IT Service Intelligence (ITSI) module, with the following assumptions:

  • You are at least an intermediate-level user of ITSI. You understand the basic concepts of ITSI, and are familiar with the main components and understand how ITSI works with them.
  • You have researched and planned your ITSI module.
  • You have data, either real or sample, to use for developing your module.
  • If you need to get your data into Splunk using an add-on, the add-on is ready to use. If you need to create an add-on, see the Splunk Add-on Builder User Guide.

A module is a Splunk add-on under the covers, and as such it interacts with ITSI using configuration files, modular inputs, and data models. ITSI integrates the information from these files across the ITSI app as part of ITSI workflows.

[ITSI Module Workflow]

Here's the basic workflow to developing a module for ITSI:

  1. Set up your environment
  2. Request a developer license
  3. Create the module components in ITSI
  4. Create a module container
  5. Add the components to your module
  6. Package the module
  7. Document, publish, and certify
  8. Configuration summary
 

Set up your environment

Set up a development environment that is isolated from your production environment. For example, set up a standalone instance of Splunk Enterprise with Splunk IT Service Intelligence on a laptop. For installation details, see Install Splunk ITSI in the Installation and Configuration Manual.

 

Request a developer license

If you are not a current customer of Splunk IT Service Intelligence (ITSI) and are interested in developing modules for ITSI, you should request a Splunk Enterprise Developer License to get started. The ITSI Development License is included in the Splunk Enterprise Developer License and the same rules and restrictions apply.

To gain access to Splunk ITSI for your development efforts, send an email to itsi-devlicense@splunk.com with the following information:

  • First name
  • Last name
  • Splunk.com username (the same account you used to request your developer license)
  • Company
  • Contact phone number
  • A brief summary of your project and goals
  • Would you like to speak to anyone on the product team about your project?

Thank you for your interest in developing Splunk ITSI modules, we look forward to seeing what you can do!

 

Create the module components in ITSI

To develop an ITSI module, you will create the following components using ITSI:

  • Entities
  • Entity discovery searches
  • Base searches
  • Services
  • KPIs
  • Drilldowns
 

Create the entities

During your research and planning exercise, you should have identified the entities for your ITSI module.

In Splunk ITSI, create the entities for your module. For instructions, see Define Entities in the Installation and Configuration Manual.

When creating an entity from a CSV or search, we recommend doing the following:

  • In the Search step, assign a role to the entity (for example, add a command to the search such as "eval itsi_role=<rolename>"), which adds a column (and a corresponding metadata field) to the search results. Feel free to create your own roles for identifying entities.
  • In the Select Columns step, use the Combine existing matching entities into a single record option for the Import Type.
  • In the Select Columns step, select the type of column that matches the data from the search as follows:
    • The Entity Title Column is the unique title used to create, look up, append, and replace entities.
    • The Entity Alias Column provides additional values for identifying an alias (such as the hostname, IP address, or MAC address) and makes the column entry a searchable alias (field).
    • The Entity Informational Column provides additional information about the entity (such as the datacenter, product, version, region, or role) and makes the column entry a tag that provides user-facing validation.
    • The Entity Description Column describes the entity.
  • At the end of the wizard, click Set up Recurring Import, which creates a modular input that runs on a regular interval (for example, every four hours) to discover new entities in your data and automatically add them. You can fill out additional details in the configuration page that follows.
    Tip  You will need the name of this modular input when you create your module package, so make a note of the name you used.
 

Explore your data and create searches

Using the Splunk Search Processing Language (SPL) in Splunk Web, use search to explore your data and become familiar with it:

  • Using the metrics you identified your research and planning exercise, figure out where they appear in your data and how often they appear.
  • Determine whether these metrics appear individually or are produced by combining other types of data or through calculations.
  • Create searches for the KPIs you identified during your research and planning exercise.

The next step is create base searches to retrieve the metrics for your KPIs. A base search is like a common denominator for multiple searches that you can use as a starting point for retrieving different metrics. Using base searches reduces the search load and helps improve search performance. For details, see Create KPI base searches in the Installation and Configuration Manual.

Data models are good for structuring your data and combining data from different technologies, but we recommend that you use KPI base searches because data model searches tend to be slower and more resource intensive. For details, see Data models in the Installation and Configuration Manual.

Data models and modular inputs must be part of the ITSI app context to work with entity discovery searches. We suggest the following practices:

  • When using an existing data model, create and use a clone of the date model, and select IT Service Intelligence for the clone's app context.
  • Create entity discovery searches by importing them in bulk, then select the Set up Recurring Import option to save searches as modular inputs. For details, see Import from search in the Installation and Configuration Manual.
 

Create a service and the KPIs

Create a service for your module, and then create the KPIs for the base searches you defined. For instructions, see Configure ITSI Services in the Installation and Configuration Manual.

We recommend using threshold time policies. For details, see KPI threshold time policies in the Installation and Configuration Manual.

When preparing a service for use in a module, a good practice is to clone a live service and put the new service in a disabled state. Then, you can modify entity filter rules and other pieces of the service without affecting a live one.

 

Create drilldowns

Drilldowns are created per entity. A drilldown might be a URI that points to a web page or another Splunk app, or a search that is opened in the Search & Reporting app.

At this time, to include drilldowns in your module, you must manually create a deep_dive_drilldowns.conf configuration file and add a stanza for each drilldown to define the rules and properties. For specifications, see the deep_dive_drilldowns.conf.spec file in $SPLUNK_HOME/etc/apps/itsi/README/.

You'll need to copy this configuration file to your module directory just before you package it.

 

Create a module container

Use the ITSI Module Builder to create a module. Start by creating a module container:

  1. In Splunk Web, open the IT Service Intelligence app.
  2. Click Configure > Modules.
  3. Click Create Module.
  4. Enter a Title and a Description for your module.
    An App ID is created automatically using the title, in the format DA-ITSI-title.
  5. [ITSI Module Container]

  6. Click Create.
  7. The ITSI Module Builder displays your module container.

  8. Edit the properties of the module as needed.
  9. Upload a readme file (or add one later).
  10. Upload app icons (or add them later).
    For more about app icons, see Add icons to your app in the Web Framework documentation.
  11. [ITSI Module Container Properties]

  12. Click Save Metadata to save your changes.

The ITSI Module Builder creates a default package structure for the module under $SPLUNK_HOME/etc/apps/App_ID with the following files:

Directory Contents
DA-ITSI-App_ID readme file
    /local app.conf
inputs.conf
itsi_kpi_base_search.conf
itsi_kpi_template.conf
itsi_service_template.conf
    /metadata default.meta
    /static appIcon.png
appIcon_2x.png

You can also create a module container using the command-line tool, create_itsi_module.py, located in $SPLUNK_HOME/etc/apps/SA-ITOA/bin/itsi_module_cli.

 

Add the components to your module

Once you have created the different components for your module—ITSI services, data models, and entity discovery searches—and verified that the data they produce looks correct, you are ready to add content to the module container. Any related items such as KPIs and base searches will be retrieved and added to your module along with the components.

To add components to your module:

  1. Open the module container.
    • In the IT Service Intelligence app, click Configure > Modules.
    • Click the module from the ITSI Modules list.
  2. Click Add Content.
  3. Click the type of component to add:
    • Services
    • Data Models
    • Entity Discovery Searches
  4. From the list, click the checkbox next to the components to add, and then click Add to Module.

    [ITSI Module Container, adding services]

  5. Repeat steps 2-3 as needed to add all your components to your module.
  6. The Module Builder templatizes your content.

  7. When you have finished, close the Add Content to Module box.
    The module container now contains tabs listing the components you added to your module.
  8. [ITSI Module Container Properties]

You can also add content to your module container using the command-line tool, generate_itsi_module_contents.py, located in $SPLUNK_HOME/etc/apps/SA-ITOA/bin/itsi_module_cli.  

Package the module

Before you package your module, do the following:

  • Add the drilldown configuration file (deep_dive_drilldowns.conf) if you created one to the /local directory of the module.
  • For any files outside of a /bin directory, change all file permissions to 644, which gives read-write permission to the file owner but read-only access to the group and other users.
  • Remove any hidden files from your package. For example, the .DS_Store file is a common artifact that lingers in many packages.

To package the module into an .SPL package file:

  1. Open the module container.
    • In the IT Service Intelligence app, click Configure > Modules.
    • Click the module from the ITSI Modules list.
  2. Click Export Module.

The ITSI Module Builder validates and generates a compressed package of the following module objects, and merges files from the /local directory to /default:

  • Service templates
  • KPIs
  • Base searches
  • KPI groups
  • Data models
  • Entity discovery searches
  • Modular inputs
  • An auto-generated app configuration file, app.conf
  • Artifacts in your module root directory

Once you have created the SPL package, you can test it by installing it and then viewing it in ITSI to verify that the UI shows no errors in the module.

You can also package your module using the command-line tool, package_itsi_module.py, located in $SPLUNK_HOME/etc/apps/SA-ITOA/bin/itsi_module_cli.  

Document, publish, and certify

Now that you've packaged your module, here are other steps for you to take:

  • Document your module. Provide documentation to the users of your module that describes any requirements, dependencies, and configuration details that your module requires. You should also describe the KPIs, thresholds, and data models used by your module, along with any other necessary information.
  • Publish your module to Splunkbase. Share your ITSI module with the community. For details, see the Working with Splunkbase manual.
  • Get your module certified. Get your module (which is actually an add-on) certified by the Splunk App Certification service to demonstrate that your module has met a specific set of requirements above and beyond the normal Splunkbase standards. For details, see the App Certification documentation.
 

Configuration summary

The following table summarizes the relationship between ITSI terms and how they map to configuration files.

Domain TermITSI Terms ITSI Integration Points Configuration File
Metric KPI: a single metric, tracked per entity.
KPI template: a collection of KPIs.
KPI base search: a single search that is used to populate multiple metrics, which is important for performance.
Service creation
Service definition
itsi_kpi_template.conf
itsi_kpi_base_search.conf
Entity Entity: a single asset, with attributes.
Entity rules: used to capture a collection of entities.
Entity listing
Automated entity creation
Service definition
inputs.conf
Service configuration Service: a set of KPIs and entities that models an IT infrastructure.
Service template: a predefined set of KPI templates and entity rules that define a service.
Service creation
Service definition
itsi_service_template.conf