Vet apps and add-ons for Splunk Cloud

Cloud vetting is a process that determines whether an app or add-on can be used in Splunk Cloud.

Apps and add-ons must be evaluated to ensure the security of the Splunk Cloud environment, as well as the security of the data stored in that environment. In addition, Splunk Enterprise and Splunk Cloud have several differences. While most apps and add-ons on Splunkbase are suitable for an on-premises Splunk Enterprise environment, they have not all been evaluated for transmitting and storing data in a cloud environment. Some apps might not be authorized for Splunk Cloud because they are intended for on-premises Splunk Enterprise systems or for installation on Splunk forwarders.

To install an app or add-on from Splunbase in your Splunk Cloud environment, you must request cloud vetting to be performed on the app or add-on by creating a support ticket with Splunk Support and Services. If the app or add-on passes the cloud vetting, it is installed on your Splunk Cloud instance.

For more information on the cloud vetting process for developers, jump to a section:

The cloud vetting process

Cloud vetting is an automated process, but sometimes requires a manual review:

  • Splunk Inc. runs the Splunk AppInspect API to perform automated cloud vetting.
  • When necessary, a Splunk employee performs a manual cloud vetting process to further evaluate the app or add-on.

Prepare your app or add-on for cloud vetting

Your app or add-on must meet a set of criteria to pass cloud vetting. Follow these steps to prepare your app or add-on for cloud vetting:

  1. Review the Development requirements for Splunk Cloud in this topic.
  2. Verify that you have fulfilled all of the Splunk Cloud requirements by running the Splunk AppInspect API with the cloud tag.
  3. For example, use the AppInspect CLI as follows:

    splunk-appinspect inspect app_path/app_filename.tgz 
    --mode precert --included-tags cloud

    Or, use the AppInspect REST API as follows:

    curl -X POST \
    	-H "Authorization: bearer <token>" \
    	-H "Cache-Control: no-cache" \
    	-F "app_package=@\"app_path/app_filename.tgz\"" \
    	-F "included_tags=cloud" \
    	--url ""

    For more about the AppInspect tool, see Overview of Splunk AppInspect.

  4. Review the inspect command results:
    • One or more failures indicate that your app or add-on failed cloud vetting and is not approved for installation on Splunk Cloud. Identify and fix the failures, and then run the command again.
    • One or more manual checks indicate that the app or add-on requires manual checking as part of the cloud vetting process. When you submit your app, a Splunk employee must manually perform cloud vetting. The manual cloud vetting process takes longer. To increase the chances of passing cloud vetting, follow the Development recommendations for Splunk Cloud in this topic.
    • Apps or add-ons that return zero failures or manual checks are likely to be quickly approved for installation on Splunk Cloud.

Splunk Cloud criteria and tips

Splunk uses a set of criteria to vet an app or add-on for Splunk Cloud. You can view a full list of these criteria on the AppInspect Check Criteria page, where all Cloud-vetting criteria are tagged "cloud" (indicated by "x" in the cloud field). These criteria are subject to change as new security threats are discovered and the Splunk platform is updated.

Splunk Cloud app developers and Splunk Cloud apps users assume responsibility for ensuring proper usage of any third-party services that they choose to use in connection with Splunk Cloud, including compliance with any relevant terms and licenses. As a reminder and pursuant to Splunk Cloud Terms of Service, Splunk is not liable for any problems that might arise from sending data to those third-party services, including, without limitation, any disclosure, modification or deletion of data resulting from access to such third-party services, and does not provide any support for those services. For more information, see Splunk Cloud Terms of Service FAQs.

Development requirements for Splunk Cloud

If you are developing an app or add-on for installation in Splunk Cloud, do the following:

  • Write all scripts for 64-bit Linux. All scripts must be able to run on Linux because the Splunk Cloud service runs on Linux-based servers.
  • Ensure that all network communication is encrypted and secured with the SSL protocol. Any configurable options that affect network communication must be secure as well.
  • Ensure that all credentials that are used and stored by the app, such as API keys, account passwords, and Base64-encoded private keys, are encrypted using the storage/passwords REST endpoint. For more information, see Access endpoint descriptions in the REST API Reference Manual and Setup page example with user credentials.
  • Provide source code for review, with the packaged app or by including a link to a public open-source repository.
  • Package the app using the Packaging Toolkit CLI according to these guidelines for Splunkbase apps:
    • Remove all hidden files.
    • Remove all executable binary files, unless source code is provided.
    • Remove .pyo and .pyc files.
    • Make sure all the files have the right permissions.
    • Package the app as a .tgz, .tar.gz, or .spl file.

    For information about the Packaging Toolkit CLI, see Packaging Toolkit CLI.

    For a complete list of packaging standards including correct file permissions and file names, see Splunk app packaging standards and Splunk Packaging Toolkit (SLIM) validation.

  • Document all dependencies, installation procedures, and post-installation validation procedures, including a list of dependencies with their version numbers. To test your app effectively, the Splunk platform needs to know which SDKs, apps, or add-ons are required to run your app.
  • Test your app's performance. Apps that cause significant performance degradation might be rejected.
  • Enter any credentials required for the app to function in a setup or configuration page. For more information, see Create a setup page in the Splunk Add-on Builder User Guide and Splunk Meet the Experts Advanced Visualizations on Github.

Avoid these practices when you develop apps and add-ons for Splunk Cloud:

  • Do not require privilege elevation with sudo, groupadd, useradd, su, or other similar utilities.
  • Do not use the reverse shell technique. For more information, see ReverseShell on the Ubuntu Wiki site.
  • Do not allow your app or add-on to manipulate files outside of the app directory, except in the following circumstances:
    • When writing to the Splunk software log directory, $SPLUNK_HOME/var/log.
    • When creating modular input checkpoints. For more information, see Data checkpoints in the Developing Views and Apps for Splunk Web manual.
  • Do not allow your app or add-on to manipulate processes outside of the control app, including the Splunk software instance or processes created by other apps.
  • Do not allow your app or add-on to manipulate the operating system.
  • Do not allow your app or add-on to allow file management through the user interface.
  • Do not allow your app or add-on to use any of the reserved ports 443 (inbound only), 8080, 8089, 8443, 9887, or 9997.
  • Do not allow your app or add-on to restart the Splunk Cloud server.
  • Do not allow your app or add-on to monitor the Splunk Cloud infrastructure.
  • Do not allow your app or add-on to send user data from the Splunk Cloud server to a third party without the user's explicit consent.
  • Do not provide automatic update features for scripts, executables, or libraries.

Common failures and preventive actions

The following sections list common types of failures you might run into when vetting apps and add-ons for Splunk Cloud.

Unsecured network communication

Appinspect uses check_for_unencrypted_network_communications to check that all network communications are encrypted using SSL/TLS. Vetting fails when data communications between an app and Splunk Cloud are not secured. For more information, see Splunk AppInspect check criteria.

The following table lists common causes for unsecured network communication vetting failures:

Cause for failure Preventive action
The http scheme configured from the Alert Actions, Data Inputs or App configuration pages is not validated, which allows the app to send and receive unencrypted data on the network. Always validate user input and accept only https communications.
The app uses insecure protocols such as UDP, FTP, and SMTP. Use an encrypted SSL/TLS channel for data communications. For example:
smtp = smtplib.SMTP(smtpHost,smtpPort) 
smtp = smtplib.SMTP_SSL(smtpHost,sslPort)
ftp = ftplib.FTP_TLS()

Secret disclsure

AppInspect uses check_for_secret_disclosure to check whether passwords and secrets are protected or are subject to disclosure.

Vetting fails if an app contains credentials as plain text in Splunk Cloud. A secret credential is anything that allows a hacker to impersonate a user and access the user's private information. Secret credentials contain passwords for any account, access token, or key for any platforms such as a Splunk session key and other identification information.

If a field is not a secret credential, do not give it a name that resembles one, such as client_token, access_key, or password.

The following table lists common causes for secret disclosure vetting failures:

Cause for failure Preventive action
Some secret credentials are exposed in plain text in the source code, README file, lookup table, .conf files, or other files. For example, the password in the README file is not encrypted.
username = admin
password = 23ds_ewwq!
Remove any secret credentials from the source code.
Credentials configured on the UI are stored as plain text, usually in the $APP_HOME/local/xx.conf file.
For example, setting encrypted to false will cause Cloud vetting to fail.
Use the storage/passwords endpoint to encrypt secret credentials. Also, make sure you do not have unencrypted secret credentials temporarily stored in a plain text in the file system. Refer to Setup page example with user credentials and passwords.conf for details.
Credentials are displayed in console or stored as plain text in log files, as is implemented the following code:
send_url = url + "?api_key" + api_key
Remove credentials or replace them with dummy credentials in logs or console.
The user can set proxy as follows on the UI and save the proxy setting as plain text in a local .conf file.
Provide separate fields such as host, port, username, password - on the UI for the user to configure proxy. When the user enters the proxy password, mask the field and store the password in the storage/password endpoint to encrypt it.

System environment issues

The following table lists failures caused by system environment-related issues:

Cause for failure Preventive action
The app contains code that modifies OS environment variables. For example:

Remove any code that modifies OS environment variables.
The app imports libraries from other non-Splunk apps. For example:
(["etc", "apps", "<OTHER_APP_NAME>",
Package the required library inside the app to subject it to Cloud vetting.
The app contains hard-coded paths that do not work in Splunk Cloud. Use the $SPLUNK_HOME environment variable whenever possible.


Some operations outside of an app's boundary cause the app to fail the vetting process. The following table lists failures caused by the app's boundary:

Cause for failure Preventive action
The app manipulates files outside of the app's boundary. Make sure all file manipulations are limited inside the app's boundary, which includes the following paths:
The app reads sensitive files external to the app. For example, reading system process information from the /etc/shadow path, collecting data from the Splunk Cloud infrastructure, or retrieving search results containing keyword "security risks" using system commands such as top (Linux) and btool (Splunk). Remove any function that retrieves sensitive information from outside of the app.
The app contains binary files that cannot be vetted. Provide source code of the binaries for Cloud Vetting, either by packaging it in the app or including github links in the file.

Shell injection vulnerabilities

If an app is potentially subject to shell injection, it will fail Cloud vetting. The following table lists failures caused by shell injection vulnerabilities:

Cause for failure Preventive action
The app behaves as a reverse shell, which is not permitted in Splunk Cloud.
For example, the user configures the Path field on the UI. The value gets passed to a subprocess parameter, which can result in reverse shell injection.
Remove or validate all the functions and paths that allow command line access to prevent reverse shell injection.

Optimize your apps or add-ons for Splunk Cloud using the following best practices

You can use the following best practices when you develop apps or add-ons for Splunk Cloud:

  • Create setup pages that allow users to configure the app or add-on. Splunk Cloud users cannot access the shell or server file system, and cannot manipulate Splunk configuration files directly. For more information, see Create a setup page for a Splunk app.
  • Clearly comment your code to accelerate the review process.
  • Provide sample data with your app or add-on using the Splunk Event Generator utility (Eventgen), along with an eventgen.conf file. Sample data helps demonstrate how your app or add-on functions during the code review. For more information, see the Eventgen app on Splunkbase.
  • Specify when an app or add-on requires multi-threading. Apps or add-ons that require more than one thread per search might be forced to run on their own search head.
  • Ensure that your app or add-on exits properly, including freeing memory, terminating processes, and closing files.
  • Provide version and build numbers in the app.conf file. For more, see app.conf in the Admin Manual.
  • Do not use "#!" characters to specify the Python interpreter in scripts. Splunk Cloud uses a customized Python interpreter to invoke all scripts.
  • Do not use the Python command unset LD_LIBRARY_PATH because it might prevent scripts from properly mapping Splunk software libraries.