Use the AppInspect CLI tool

Once you've installed the Splunk AppInspect dependencies and the Splunk AppInspect CLI tool itself, you can use it to inspect your app package.

This topic demonstrates how to use AppInspect. It contains the following sections:

AppInspect concepts

To understand how AppInspect works, you should first understand these basic concepts.

Checks

AppInspect inspects your app by running checks, or individual validations of structural or functional properties within a Splunk app. Each check encapsulates specific validation logic, focuses on a single responsibility, and is independent from other checks.

Groups

Related sets of checks are organized into a file called a group. All groups are stored in the .../splunk_appinspect/checks/ directory. For example, the file .../splunk_appinspect/checks/check_json_files.py performs checks that are relevant to JSON files, including one called check_validate_json_data_is_well_formed, whose sole responsibility is to make sure that JSON is well-formed.

Tags

AppInspect uses tags to provide a mechanism of selection and filtering for validation. Each check is tagged with a keyword that specifies what the check is relevant to. For example, the tag "splunk_appinspect" indicates that the check is relevant to app certification.

For more information about the built-in tags in Splunk AppInspect, see Tag reference.

Modes

Splunk AppInspect can be run with different modes. A mode determines the run-time output of Splunk AppInspect. There are two available modes:

  • Test mode is the default mode, and is ideal for testing during development. It runs only tests that don't require manual intervention.
  • Appcert mode is meant to be run before submitting your app to Splunk for certification. It runs all tests.

Results

A result is the output of a check that the check returned. Each AppInspect check result will include one of the following result states:

Result state Meaning Description Resolution
E Error A failure has occurred within AppInspect. N/A; this code does not indicate a problem in the app package.
F Failure The Splunk app being tested has violated some aspect of the Splunk App Certification criteria. The output returned with this result indicates how to fix the issue encountered.
M Manual check Functionality might exist that has not been automated. The output returned with this result indicates what should be checked to make sure that the Splunk app being tested adheres to the Splunk App Certification criteria.
N/A Not applicable The feature being tested does not exist in the Splunk app being tested and can be ignored. The output returned with this result explains why the check is not applicable.
P Pass (Success) The feature being tested exists, and the functionality adheres to the App Certification criteria. N/A
Skipped Skipped The feature being checked in the Splunk app is missing a required resource needed to perform the check. The output returned with this result explains what should be included to enable checking of this feature.

Note: The warn method on the reporter object will always report success, but it will generate additional messages that you might need to address.

Quickstart

If you've installed the required dependencies and AppInspect, and you've got your Splunk app tarball ready to test, you're ready to run any of the following:

Use AppInspect for automated unit testing

splunk-appinspect inspect path/to/splunk/splunk_app.tgz
// This is equivalent to
splunk-appinspect inspect path/to/splunk/splunk_app.tgz --mode test

Review a Splunk app before requesting certification

splunk-appinspect inspect path/to/splunk/splunk_app.tgz --mode precert --included-tags splunk_appinspect

Review an app for compliance with Cloud requirements

splunk-appinspect inspect path/to/splunk/splunk_app.tgz --mode precert --included-tags cloud
Note: For more information about the requirements for Splunk apps intended for Splunk Cloud, see Splunk Cloud app requirements and best practices.

Commands

Splunk AppInspect supports different commands to help separate concerns for usage. The available commands are:

Command Description
list Generates information about Splunk AppInspect.
inspect Validates a Splunk app.

Filtering on a tag

When using a command, you can filter the output by using the following command options:

Command option Description
--included-tags <tag_to_select> Selects checks that are marked with the specified tag.
--excluded-tags <tag_to_deselect> Deselects checks that are marked with the specified tag.

Filtering on multiple tags

You can append additional instances of the --included-tags and --excluded-tags command options to add more filtering.

Filtering precedence

The --included-tags option will always take precedence over the --excluded-tags option.

// This will show only groups and checks with the appapproval tag
splunk-appinspect list groups checks --included-tags appapproval --excluded-tags appapproval

Using the Splunk AppInspect list command

With the list command, AppInspect can list some or all of the checks that AppInspect runs, as well as its groups, tags, and version information.

Use any combination of groups, checks, or tags to list the groups, checks, and tags, respectively. Use version to see the version of AppInspect currently running.

list command examples

List groups, checks, and tags

// This is the command to list items
splunk-appinspect list groups checks tags
  
// This is the output of the command truncated
 
================================================================================
Standard Certification Checks
================================================================================
App.conf standards (check_app_configuration_file)
    - Name: check_app_icon
        - Description:  Check that the app has an icon, in PNG format, located at
 static/appIcon.png (36x36px) and static/appIcon_2x.png (72x72px).
        - Version: (1.0.0-*)
        - Tags: splunk_appinspect, appapproval
 
...
 
Configuration file standards (check_configuration_files)
    - Name: check_config_file_parsing
        - Description:  Check that all config files parse cleanly- no trailing whitespace after
 continuations, no duplicated stanzas or options.
        - Version: (1.1.12-*)
        - Tags: splunk_appinspect
...
JSON file standards (check_json_files)
    - Name: check_validate_json_data_is_well_formed
        - Description:  Check that all JSON files are well formed.
        - Version: (1.1.0-*)
        - Tags: splunk_appinspect, cloud
...
 
================================================================================
All Tags
================================================================================
advanced_xml                3
alert_actions_conf          7
appapproval                 19
...

List only groups and checks

splunk-appinspect list groups checks
  
================================================================================
Standard Certification Checks
================================================================================
App.conf standards (check_app_configuration_file)
    - Name: check_app_icon
        - Description:  Check that the app has an icon, in PNG format, located at
 static/appIcon.png (36x36px) and static/appIcon_2x.png (72x72px).
        - Version: (1.0.0-*)
        - Tags: splunk_appinspect, appapproval
 
...
 
Configuration file standards (check_configuration_files)
    - Name: check_config_file_parsing
        - Description:  Check that all config files parse cleanly- no trailing whitespace after
 continuations, no duplicated stanzas or options.
        - Version: (1.1.12-*)
        - Tags: splunk_appinspect
...
JSON file standards (check_json_files)
    - Name: check_validate_json_data_is_well_formed
        - Description:  Check that all JSON files are well formed.
        - Version: (1.1.0-*)
        - Tags: splunk_appinspect, cloud
...

List only tags

// This is the command to list items
splunk-appinspect list tags
  
// This is the output of the command
================================================================================
All Tags
================================================================================
advanced_xml                3
alert_actions_conf          7
appapproval                 19
cloud                       51
custom_search_commands      8
custom_search_commands_v2   5
custom_visualizations       3
custom_workflow_actions     2
deprecated_feature          15
developer_guidance          1
django_bindings             1
manual                      56
markdown                    1
modular_inputs              1
modularinput                6
offensive                   1
restmap_config              2
savedsearches               3
security                    9
splunk_6_3                  8
splunk_6_4                  7
splunk_appinspect           138
web_conf                    3

List AppInspect version number

In Splunk AppInspect v1.3.0 and later, use the version modifier to list the version of AppInspect that is currently installed.

// This is the command to list the version number
splunk-appinspect list version
  
// This is the output of the command
Splunk AppInspect Version 1.3.1.78

List filtering examples

Filter with included and excluded tags

In this example, AppInspect includes checks tagged as "manual" and excludes checks tagged as "cloud." If there is a check that is tagged with both "manual" and "cloud," that check will be displayed.

// Returns only checks with the manual tag and excludes checks tagged with `cloud`
// if a check contains the tags `manual` and `cloud` it will be returned
splunk-appinspect list --included_tags manual --excluded_tags cloud

Filter with multiple included tags

You can also use multiple tags to determine inclusion or exclusion from the listing. For example, the following example specifies that we want to see checks tagged with "manual," as well as checks tagged with "splunk-appinspect." To do this, it uses the --included_tags flag twice.

splunk-appinspect list --included_tags manual --included_tags splunk-appinspect

Filter with multiple excluded tags

Similarly, the following example specifies that we want to exclude any checks tagged with "manual," as well as any checks tagged with "splunk-appinspect." To do this, it uses the --excluded_tags flag twice:

splunk-appinspect list --excluded_tags manual --excluded_tags splunk-appinspect

Using the Splunk AppInspect inspect command

When you use AppInspect to inspect your Splunk app, it's always done in a specific mode. The modes available for inspection are "precert" and "test." For more information about modes, see Modes, elsewhere in this topic.

Splunk AppInspect's inspect command:

  • Runs in test mode by default.
  • Writes the failures at the end of an inspection.
  • Writes a result summary at the end of an inspection.
  • Sends all output to stdout unless specified otherwise:
    • Use the --output-file option followed by a filename to write the results to that file. If you don't also specify the --data-format option, AppInspect will write the results to the file in human-readable format.
    • Use the --data-format option followed by a format to specify the data format to use when writing results to a file—for example, JSON or XML.

Test mode

Test mode is meant to facilitate command line integration much like a linter, or a testing tool for continuous integration, and builds.

By default, test mode:

  • Only returns successes—represented by a '.'—and failures—represented by an 'F'.
  • Will not run checks marked with the "manual" tag, by default. To override this behavior, use the --included_tags manual option, or run in precert mode by using --mode precert.
Note: The following two commands are functionally equivalent. The output of each is different, but AppInspect runs all of the same checks.

splunk-appinspect inspect --mode precert <app>

and

splunk-appinspect inspect --included-tags manual <app>

Test mode results example output

Following is example output of test mode results:

Validating: <APP_NAME> Version: <APP_VERSION>
....F........................................................................
...............F.....
App.conf standards
  Check that default/app.conf setting is_configured = False.
    FAILURE: The app.conf [install] stanza has the `is_configured` property set to true. This is indicates a setup was already performed.
Deprecated features from Splunk 6.4.
  Check Simple XML files for <single> panels with deprecated options
 'additionalClass', 'afterLabel', 'beforeLabel', 'classField', 'linkFields',
 'linkSearch', 'linkView'
    FAILURE: 3 <single> panel(s) contain the option(s): linkView File: default/data/ui/views/example.xml
 
<APP_NAME> Report Summary:
       skipped:  0
       success: 66
  manual_check:  4
       failure:  2
       warning:  0
         error:  0
not_applicable: 30
-------------------
         Total: 102

Precert mode

Precert mode is meant to aid those developers looking to certify their Splunk apps. Use precert mode before submitting your Splunk app to Splunkbase for certification. When you run AppInspect in precert mode, you're seeing the same checks that Splunk performs as one of the first steps in the certification process. If any tests fail in this mode during the certification process, a Splunk app will not move onto the next part of the certification process.

By default, precert mode:

  • Returns all results.
  • Runs all checks with all tags unless otherwise specified.

Precert mode results example output

Here is example output of precert mode results.

Validating: <APP_NAME> Version: <APP_VERSION>
[  P  ] - check_app_icon - Check that the app has an icon, in PNG format, located at
            static/appIcon.png (36x36px) and static/appIcon_2x.png (72x72px).
...
[  F  ] - check_that_setup_has_not_been_performed - Check that default/app.conf setting is_configured = False.
...
[ N/A ] - check_that_local_passwords_conf_does_not_exist - Check that local/passwords.conf does not exist. Password files are
            not transferable between instances.
...
[  M  ] - check_hard_coded_paths - Check for hard-coded filepaths in scripts relative to author's
            local developer environment, or absolute paths.
...
 
App.conf standards
  Check that default/app.conf setting is_configured = False.
    FAILURE: The app.conf [install] stanza has the `is_configured` property set to true. This is indicates a setup was already performed.
Deprecated features from Splunk 6.4.
  Check Simple XML files for <single> panels with deprecated options
 'additionalClass', 'afterLabel', 'beforeLabel', 'classField', 'linkFields',
 'linkSearch', 'linkView'
    FAILURE: 3 <single> panel(s) contain the option(s): linkView File: default/data/ui/views/example.xml
Operating system standards
  Check for hard-coded filepaths in scripts relative to author's local
 developer environment, or absolute paths.
    MANUAL_CHECK: Found possible hard-coded path '/example_path]' File: metadata/default.meta:9 Line: 9
Appropriate use of sensitive functionality
  Check that file access outside of the app's home directory,
 $SPLUNK_HOME/var/log, $SPLUNK_HOME/var/run, and system temporary directories
 is explained in the app's documentation.
    MANUAL_CHECK: File access will be inspected during code review.
XML file standards
  Check to ensure any XML files do not embed JavaScript via inline
 calls or external references.
    MANUAL_CHECK: Embedded javascript has been detected. Total line(s) of code found: 1. File: default/data/ui/views/example.xml.
Saved search standards
  Check that saved searches are enabled.
    MANUAL_CHECK: The search [Generate Example every 15 mins] in savedsearches.conf is disabled.
    MANUAL_CHECK: The search [Generate Example every 30 mins] in savedsearches.conf is disabled.
    MANUAL_CHECK: The search [Generate Example every 5 mins] in savedsearches.conf is disabled.
 
<APP_NAME> Report Summary:
       skipped:  0
       success: 66
  manual_check:  4
       failure:  2
       warning:  0
         error:  0
not_applicable: 30
-------------------
         Total: 102

inspect command examples

Inspect an app using test mode

splunk-appinspect inspect path/to/splunk/app
// This is equivalent to
splunk-appinspect inspect path/to/splunk/app --mode test

Inspect an app using test mode using filters

splunk-appinspect inspect path/to/splunk/app --mode test --included-tags manual --included-tags appapproval --excluded-tags cloud

Inspect an app using test mode with logging

splunk-appinspect inspect path/to/splunk/app --mode test --log-level DEBUG --log-file log_output.log

Inspect an app using test mode and output as JSON to a file

splunk-appinspect inspect path/to/splunk/app --mode test --log-level DEBUG --log-file log_output.log

Inspect an app using test mode and output as JUnitXML to a file

splunk-appinspect inspect path/to/splunk/app --mode test --data-format junitxml --output-file results.xml

Inspect an app using precert mode

splunk-appinspect inspect path/to/splunk/app --mode precert

Inspect an app using precert mode using filters

splunk-appinspect inspect path/to/splunk/app --mode precert --included-tags manual --included-tags appapproval --excluded-tags cloud

Inspect an app using precert mode with logging

splunk-appinspect inspect path/to/splunk/app --mode precert --log-level DEBUG --log-file log_output.log

Inspect an app using precert mode and output as JSON to a file

splunk-appinspect inspect path/to/splunk/app --mode precert --data-format json --output-file results.json

Inspect an app using precert mode and output as JUnitXML to a file

splunk-appinspect inspect path/to/splunk/app --mode precert --data-format junitxml --output-file results.xml

Logging

The Splunk AppInspect inspect command has logging built into it. To use AppInspect logging, add one or both of the following two flags to the inspect command:

Flag Description
--log-level The logging level to use. This can be any one of the following, in order of severity from lowest to highest:
  • NOTSET
  • DEBUG
  • INFO
  • WARNING
  • ERROR
  • CRITICAL
If this flag is not specified, only ERROR and CRITICAL log levels are logged by default.
--log-file The file to which the logging information is written. If this flag is not specified, the logging will be written to the command line by default.

Because AppInspect uses Python's default logging library, if you are familiar with native Python logging, you can extend its capabilities. For more information, see the Python logging module documentation.

Getting help

General help

// To view general help
splunk-appinspect --help

list command help

// To view list help
splunk-appinspect list --help

inspect command help

// To view inspect help
splunk-appinspect inspect --help

Additional help resources

For additional help, see Get help with AppInspect.