Overview of the Splunk SDKs

So, you want to roll your own? Go for it. The Splunk SDKs are written on top of the Splunk REST APIs. The intent is to give you broad coverage of the REST API in a language-specific fashion to ease your access to the Splunk engine.

Currently, Splunk has SDKs for these languages:

What you can do

Here are a few things that our customers are doing. We want to hear what you are doing or want to do. Ping us at Splunk Dev Info and tell us all about it.

  • Integrate with third-party reporting tools and portals
  • Log directly to Splunk
  • Integrate Splunk search results into your application
  • Extract data for archiving
  • Build a UI on the web stack of your choice
  • ...and so much more

REST API coverage by Splunk SDKs

The most basic way to programmatically access Splunk's resources is by using the REpresentational State Transfer (REST) model to make HTTP requests. Splunk provides a REST API that lets you interact with a Splunk instance and do most everything that you can using Splunk Web—including authentication, creating and running searches, managing search jobs, creating and managing indexes and inputs, and configuring Splunk. All of these things can be done with HTTP GET, POST, and DELETE operations using Splunk's REST API.

However, we want to make it even easier for you to develop Splunk applications using common programming languages, so we've created software development kits (SDKs) to help out. We've got Splunk SDKs for Python, Java, JavaScript, PHP, Ruby, and C#. But first, here's some background on how the SDKs relate to the REST API.

 

A little about the Splunk REST API

Each of Splunk's resources (apps, users, searches, jobs, indexes, inputs, and others) has a corresponding REST endpoint that indicates the resource's category (for example, the operation for streaming search results is GET search/jobs/export). To use the REST API to interact with Splunk's resources, you send a request to the management port of a Splunk server (which is port 8089 by default). The request requires admin access and is over HTTPS, using the URI of the REST endpoint. You can use any web browser, command-line tool, REST client, scripting language, or programming language that supports making HTTP calls. Curl and Wget are common tools. By default, responses are returned in Atom Syndication Format (an Atom Feed) with entries containing information about the Splunk resource.

The URI for the request includes the location of the Splunk server splunkd, the user/app context, and a REST endpoint that corresponds to the resource category. You also need to provide login credentials for Splunk and any additional parameters for the request.

The Atom Feed response contains the following containers:

  • <feed> is a top-level element containing meta data that pertains to the entire response, such as the time of the request or the total number of results.
  • <entry> corresponds to each individual result, containing meta data that pertains to one result and its content.
  • <content> includes the key-value pairs that make up each result.

So what does that look like in practice? Here's an example of an HTTP POST request that creates a new index with a given name ("myindexname"):

  • The username:password credentials are admin:pass.
  • The location of splunkd is https://localhost:8089/servicesNS.
  • The user/app context is admin/search.
  • The endpoint is data/indexes.

You put it all together at the command line with curl and you get this: 

curl -k -u admin:pass https://localhost:8089/servicesNS/admin/search/data/indexes \
     -d name=myindexname

To see what the Atom Feed response looks like, see the example under the POST data/indexes endpoint.

This is just a taste of the Splunk REST API. The endpoints are fully documented in the REST API Reference, along with information about how to use them.

 

What the Splunk SDKs do for you

Although you can use the REST API directly, you can also use the Splunk SDKs to interact with Splunk. Essentially, these SDKs are wrappers around the REST API that do a lot of the work for you, such as:

  • Handling HTTP access. The SDKs provide HTTP access and handle the certificates for HTTPS.
  • Authenticating. When you log in using a username and password, Splunk returns a session key. The SDKs automatically remember and append this session key to subsequent requests.
  • Managing namespaces. A namespace is the user/app context for accessing a resource, which is specified by a Splunk username, a Splunk app (such as the default Search app), and a sharing mode. The SDKs send requests based on the namespace that was used for logging in, or you can specify a namespace to access a specific resource. For example, you can list all apps, or only the apps that a specific user has access to.
  • Simplifying access to REST endpoints. The SDKs provide access to the REST API in the native style of different programming languages. For example, here's how to set a Splunk app's description using the REST API:
      • curl -k -u admin:changeme https://localhost:8089/services/apps/local/myApp \
    -d description="My Killer App"

      Here's how you'd do this with a Java setter method:

      app.setDescription("My Killer App");

      To show you how the other languages are used, here's a Python example that submits an event to an index:

      index = service.indexes["my_index"]
index.submit("some event", source="www", sourcetype="web_event")

      For comparison, here's the same example using curl to access the REST API:

      curl -k -u admin:changeme "https://localhost:8089/services/receivers/simple?source=www&sourcetype=web_event \
    -d index="my_index" \
    -d "some event"
  • Building the correct URL for an endpoint. The SDKs build out the complete REST URLs in the correct format, with the namespace and any additional parameters you specify.
  • Displaying simplified output for searches. The REST API returns search results (events) in XML, JSON, or CSV—but in a raw format. The SDKs provide results readers (helper classes for Python and Java, a code example for JavaScript) that parse these events and return them in a simplified structure with clear key-value pairs.
 

Do you still need to know the REST API?

While the Splunk SDKs provide a layer over the Splunk REST API, you still should become familiar with the REST endpoints and understand how to navigate them. The abstraction layer can be thin depending on which SDK language you are using, and not every feature has a corresponding class or method in each SDK.

In general, you interact with the REST API by getting and setting the parameters that are available for each endpoint. Similarly, the SDKs use key-value pairs to set parameters when there isn't a specific SDK API to do the job. That's when being familiar with the REST API helps—in these cases when you want to get or set parameters that are not defined by the SDK, you need to know which parameters are allowed for that resource, being careful to specify the case-sensitive name, and provide a value in the correct format.