Overview of the Splunk SDK for Ruby (Deprecated)

Deprecation notice

The Splunk SDK for Ruby is deprecated.

What deprecation means:

  • On May 1, 2017, the resources relating to the Splunk SDK for Ruby will be removed from dev.splunk.com and will only be available in the GitHub repository.
  • Apps that use the Splunk SDK for Ruby will continue to work as they do now.
  • Apps that use the Splunk SDK for Ruby will continue to be eligible for Splunk App Certification.
  • Splunk will no longer provide any feature enhancements to the Splunk SDK for Ruby.

Recommendation on new app development and app migration:

  • Because Splunk is no longer investing in the Splunk SDK for Ruby, we recommend that any new app development be done using other approaches:

  • For existing apps that use the Splunk SDK for Ruby, while not necessary, we request that developers begin the migration process away from the Splunk SDK for Ruby. We encourage developers to provide feedback to Splunk at devinfo@splunk.com if there are any issues with migration.

Notice of removal:

  • The Splunk SDK for Ruby will continue to be available on GitHub, should other developers want to clone or fork the project.

Welcome to the Splunk® Software Development Kit (SDK) for Ruby!

This SDK is open source and uses the Apache v2.0 license. If you want to make a code contribution, go to the Open Source page for more information.

This overview tells you more about:

What you can do with the Splunk SDK for Ruby

This SDK contains library code and examples designed to enable developers to build applications using Splunk. With the Splunk SDK for Ruby you can write Ruby applications to programmatically interact with the Splunk engine. The SDK is built on top of the REST API, providing a wrapper over the REST API endpoints. So with fewer lines of code, you can write applications that:

  • Search your data, run saved searches, and work with search jobs
  • Manage Splunk configurations and objects
  • Integrate search results into your applications
  • Log directly to Splunk
  • Present a custom UI

About the Splunk SDK for Ruby API layers

The Splunk library included in this SDK consists of two layers of API that can be used to interact with splunkd - the binding layer and the client layer. First, however, a word about XML...

A word about XML

Ruby ships with the REXML library by default, but for most real world work, you may want to use Nokogiri, which can be orders of magnitude faster. The Splunk SDK for Ruby supports both. By default it will try to use Nokogiri, but will fall back to REXML if Nokogiri is not available. The value of the library in use is kept in the global variable $splunk_xml_library (which will be either :nokogiri or :rexml).

You can force your program to use a particular library by calling require_xml_library(library) (where, again, library is either :nokogiri or :rexml). This method is in lib/splunk_sdk_ruby/xml_shim.rb, but will be included when you include the whole SDK.

If you force your program to use a particular library, the SDK will no longer try to fall back to REXML, but will issue a LoadError.

The binding layer

This is the lowest layer of the Splunk SDK for Ruby. It is a thin wrapper around low-level HTTP capabilities, including:

  • authentication and namespace URL management
  • accessible low-level HTTP interface for use by developers who want to be close to the wire
  • Atom feed parser

Here is a simple example of using the binding layer. This example makes a REST call to Splunk returning an Atom feed of all users defined in the system:

require 'splunk-sdk-ruby'

c = Splunk::Context.new(:username => "admin",
                        :password => 'password')
c.login()

# Will print an Atom feed in XML:
puts c.request(:resource => ["authentication", "users"]).body

You can read the Atom feed into a convenient Ruby object with the AtomFeed class. It has two getter methods: metadata, which returns a hash of all the Atom headers; and entries, which returns an array of hashes describing each Atom entry.

require 'splunk-sdk-ruby'

c = Splunk::Context.new(:username => "admin",
                        :password => 'password')
c.login()

response = c.request(:resource => ["authentication", "users"])
users = Splunk::AtomFeed.new(response.body)
puts users.metadata["updated"]
puts users.entries[0]

The client layer

The client layer builds on the binding layer to provide a friendlier interface to Splunk that abstracts away many of the lower level details of the binding layer. It currently abstracts the following (with more to come):

  • Authentication
  • Apps
  • Capabilities
  • Server Info
  • Loggers
  • Settings
  • Indexes
  • Roles
  • Users
  • Jobs
  • Saved Searches
  • Searching (one-shot, asynchronous, real-time, and so on)
  • Restarting
  • Configuration
  • Messages
  • Collections and Entities

Here is example code to print the names of all the users in the system:

service = Splunk::connect(:username => 'admin', :password => 'password')
service.users.each do |user|
  puts user.name
end

For more examples, see the examples/ directory in the Splunk SDK for Ruby.

The Splunk SDK for Ruby components

The Splunk developer platform consists of three primary components: Splunkd, the engine; Splunk Web, the app framework that sits on top of the engine; and the Splunk SDKs that interface with the REST API and extension points.

The Splunk SDK for Ruby lets you target Splunkd by making calls against the engine's REST API and accessing the various Splunkd extension points such as custom search commands, lookup functions, scripted inputs, and custom REST handlers.

Read more about:

The Service class

The Service class is the primary entry point for the client library. Construct an instance of the Service class and provide the login credentials that are required to connect to an available Splunk server. There are different ways to construct the instance and authenticate.

First, add the following to the top of your source file to include the Splunk SDK for Ruby:

require 'splunk-sdk-ruby'

All the code in the SDK is in the Splunk module. Once you have included the SDK, create a connection to your Splunk instance with the following (changing host, port, username, and password to your values):

service = Splunk::Service.new(:host => "localhost",
                              :port => 8089,
                              :username => "admin",
                              :password => "changeme").login()

Once the Service instance is created and authenticated, you can use it to navigate, enumerate, and operate on a wide variety of Splunk resources.

Entities and collections

The Splunk REST API has over 160 endpoints (resources) that provide access to almost every feature of Splunk. The Splunk SDK for Ruby exposes many of these endpoints as entities and collections of entities. The base abstractions are as follows:

  • Entity: An abstraction over a Splunk entity (such as a single app, saved search, job, or index), providing operations such as update, remove, read properties, and refresh.
  • Collection: An abstraction over a Splunk collection (such as all apps, all saved searches, all jobs, or all indexes), providing operations such as creating new entities and fetching specific entities.

Each collection type can be accessed on the Service object. For example, the following example shows how to retrieve a collection of all the apps installed on Splunk:

require 'splunk-sdk-ruby'
service = Splunk::connect(...) # Fill in your host and authentication details
service.apps.each do |app|
    puts app.name
end

Namespaces

To account for permissions to view apps, system files, and other entity resources by users throughout a Splunk installation, Splunk provides access to entity resources based on a namespace. This is similar to the app/user context that is used by the Splunk REST API when accessing resources using endpoints.

The namespace is defined by:

  • An owner, which is the Splunk username, such as "admin". A value of "nobody" means no specific user. The "-" wildcard means all users.
  • An app, which is the app context for this resource (such as "search"). The "-" wildcard means all apps.
  • A sharing mode, which indicates how the resource is shared. The sharing mode can be:
    Sharing mode
    Description
    "user"The resource is private to a specific user, as specified by owner.
    "app"The resource is shared through an app, as specified by app. The owner is "nobody", meaning no specific user.
    "global"The resource is globally shared to all apps. The owner is "nobody", meaning no specific user.
    "system"The resource is a system resource (owner is "nobody", app is "system").

In general, when you specify a namespace you can specify any combination of owner, app, and sharing the SDK library will reconcile the values, overriding them as appropriate. If a namespace is not explicitly specified, the current user is used for owner and the default app is used for app.

Here are some example combinations of owner, app, sharing:

  • List all of the saved searches for a specific user named Kramer: kramer, -, user
  • Create an index to be used within the Search app: nobody, search, app

Splunk's namespaces give access paths to objects. Each application, user, search job, saved search, or other entity in Splunk has a namespace, and when you access an entity via the REST API, you include a namespace in your query. What entities are visible to your query depends on the namespace you use for the query.

Some namespaces can contain wildcards or default values filled in by Splunk. We call such namespaces wildcard, since they cannot be the namespace of an entity—only a query. Namespaces that can be the namespace of an entity are called exact.

We distinguish six kinds of namespace, each of which is represented by a separate class:

  • DefaultNamespace, used for queries where you want to use whatever would be default for the user that is logged into Splunk, and is the namespace of applications (which themselves determine namespaces, and so have to have a special one).
  • GlobalNamespace, which makes an entity visible anywhere in Splunk.
  • SystemNamespace, which is used for entities like users and roles that are part of Splunk. Entities in the system namespace are visible anywhere in Splunk.
  • AppNamespace, one per application installed in the Splunk instance.
  • AppReferenceNamespace, which is the namespace that applications themselves live in. It differs from DefaultNamespace only in that it is a exact namespace.
  • The user namespaces, which are defined by a user and an application.

In the user and application namespaces, you can use "-" as a wildcard in place of an actual user or application name.

These are all represented in the Ruby SDK by correspondingly named classes: DefaultNamespace, GlobalNamespace, SystemNamespace, AppNamespace, and UserNamespace. Each of these have an empty mixing Namespace, so an instance of any of them will respond to is_a?(Namespace) with true.

Some of these classes are singletons, some aren't, and to avoid confusion or having to remember which is which, you should create namespaces with the namespace function.

What namespace the eai:acl fields in an entity map to is determined by what the path to that entity should be. In the end, a namespace is a way to calculate the initial path to access an entity. For example, applications all have sharing="app" and app="" in their eai:acl fields, but their path uses the services/ prefix, so that particular combination, despite what it appears to be, is actually an AppReferenceNamespace.

Searches

One of the primary features of Splunk is running searches and retrieving search results. There are multiple types of search:

  • Normal: A normal search runs asynchronously and allows you to monitor its progress. This type of search is ideal for most cases, especially for searches that return a large number of results.

    The Splunk::Jobs#create method creates a normal search job.

  • Blocking: A blocking search runs synchronously and does not return a search job until the search has finished. This type of search can be used when you don't need to monitor progress, and does not work with real-time searches.

    The Splunk::Jobs#create_oneshot and Splunk::Jobs#create_export methods both create blocking searches.

  • Oneshot: A oneshot search is a blocking search that is scheduled to run immediately. Instead of returning a search job, this mode returns the results of the search once completed.

    The Splunk::Jobs#create_oneshot method creates a oneshot search.

  • Saved search: A saved search is simply a search query that was saved to be used again and can be set up to run on a regular schedule. The results from the search are not saved.

    The Splunk::SavedSearch class represents a saved search.