How to run searches and display results using the Splunk SDK for JavaScript

Searches run in different modes, determining when and how you can retrieve results:

  • Normal: A normal search runs asynchronously. It returns a search job immediately. Poll the job to determine its status. You can retrieve the results when the search has finished. You can also preview the results if "preview" is enabled. Normal mode works with real-time searches.
  • Blocking: A blocking search runs synchronously. It does not return a search job until the search has finished, so there is no need to poll for status. Blocking mode doesn't work with real-time searches.
  • Oneshot: A oneshot search is a blocking search that is scheduled to run immediately. Instead of returning a search job, this mode returns the results of the search once completed. Because this is a blocking search, the results are not available until the search has finished.
  • Export: An export search is another type of search operation that runs immediately, does not create a job for the search, and starts streaming results immediately.
    At this time, the Splunk® SDK for JavaScript does not support this type of search.

For those searches that produce search jobs (normal and blocking), the search results are saved for a period of time on the server and can be retrieved on request. For those searches that stream the results (oneshot and export), the search results are not retained on the server. If the stream is interrupted for any reason, the results are not recoverable without running the search again.

The job APIs

The classes for working with jobs are:

Access these classes through an instance of the splunkjs.Service class. Retrieve a collection, and from there you can access individual items in the collection and create new ones.

Code examples

This section provides examples of how to use the job APIs, assuming you first connect to a Splunk instance:

The following parameters are available for search jobs:

To list search jobs for the current user

This example gets the collection of jobs available to the current user and lists the search ID for each job:

// Retrieve the collection of jobs
var myJobs = service.jobs();

myJobs.fetch(function(err, jobs) {
  
  // Determine how many jobs are in the collection
  var jobsList = myJobs.list() || [];
  console.log("There are " + jobsList.length + " jobs available to the current user");

  // Loop through the collection and display each job's SID
  for(var i = 0; i < jobsList.length; i++) {
    console.log((i + 1) + ": " + jobsList[i].sid);
  };
});

To create a blocking search and display properties and results of the job

Running a blocking search creates a search job and runs the search synchronously in "blocking" mode. The job is returned after the search has finished and all the results are in.

When you create a search job, you need to set the parameters of the job as a dictionary of key-value pairs. For a list of all the possible parameters, see Search job parameters.

This example runs a blocking search, waits for the job to finish, displays some statistics for the completed job, and then displays the results:

// Search everything and return the first 100 results
var searchQuery = "search * &#124; head 100";

// Set the search parameters
var searchParams = {
  exec_mode: "blocking",
  earliest_time: "2012-06-20T16:27:43.000-07:00"
};

// A blocking search returns the job's SID when the search is done
console.log("Wait for the search to finish...");

// Run a blocking search and get back a job
service.search(
  searchQuery,
  searchParams,
  function(err, job) {
    console.log("...done!\n");

    // Get the job from the server to display more info
    job.fetch(function(err){
      // Display properties of the job
      console.log("Search job properties\n---------------------");
      console.log("Search job ID:         " + job.sid);
      console.log("The number of events:  " + job.properties().eventCount); 
      console.log("The number of results: " + job.properties().resultCount);
      console.log("Search duration:       " + job.properties().runDuration + " seconds");
      console.log("This job expires in:   " + job.properties().ttl + " seconds");

      // Get the results and display them
      job.results({}, function(err, results) {
        var fields = results.fields;
        var rows = results.rows;
        for(var i = 0; i < rows.length; i++) {
          var values = rows[i];
          console.log("Row " + i + ": ");
          for(var j = 0; j < values.length; j++) {
            var field = fields[j];
            var value = values[j];
            console.log("  " + field + ": " + value);
          }
        }
      })

    });
    
  }
);

To paginate through a large set of results

The maximum number of results you can retrieve at a time from your search results is determined by the maxresultrows field, which is specified in a Splunk configuration file. We don't recommend changing the default value of 50,000. If your job has more results than this limit, just retrieve your results in sets (0-49999, then 50000-99999, and so on), using the "count" and "offset" parameters to define how many results to retrieve at a time. Set "count" (the number of results in a set) to maxresultrows (or a smaller value), and increment "offset" by this same value to page through each set.

The following example shows how to retrieve search results in sets, using a count of 10 for this example:

// Search everything and return the first 100 results
var searchQuery = "search * | head 100";

// Set the search parameters
var searchParams = {
  exec_mode: "blocking",
  earliest_time: "2012-06-20T16:27:43.000-07:00"
};

// Create a blocking search, which returns the job's SID when the search is done
console.log("Wait for the search to finish...");

// Create the search, wait for it to finish, and get back a job
service.search(
  searchQuery,
  searchParams,
  function(err, job) {
    console.log("...done!\n");

    // Get the job from the server to display more info
    job.fetch(function(err){
      // Display properties of the job
      console.log("Search job properties\n---------------------");
      console.log("Search job ID:         " + job.sid);
      console.log("The number of results: " + job.properties().resultCount);
      console.log("Search duration:       " + job.properties().runDuration + " seconds");
      console.log("This job expires in:   " + job.properties().ttl + " seconds");

      // Page through results by looping through sets of 10 at a time
      var resultCount = job.properties().resultCount; // Number of results this job returned
      var myOffset = 0;         // Start at result 0
      var myCount = 10;         // Get sets of 10 results at a time

      // Run an asynchronous while loop using the Async.whilst helper function to
      // loop through each set of results 
      splunkjs.Async.whilst(

        // Condition--loop while there are still results to display
        function() {
          return (myOffset < resultCount);
        },

        // Body--display each set of results
        function(done) {
          // Get one set of results
          job.results({count: myCount,offset:myOffset}, function(err, results) {

            // Display results
            var fields = results.fields;
            var rows = results.rows;
            for(var i = 0; i < rows.length; i++) {
              var values = rows[i];
              console.log("*** Row " + i + ": ");
              for(var j = 0; j < values.length; j++) {
                var field = fields[j];
                var value = values[j];
                console.log(" " + field + ": " + value);
              }
            }
            
            // Increase the offset to get the next set of results
            // once we are done processing the current set.
            myOffset = myOffset + myCount;
            done();
          })
        },

        // Done
        function(err){
          if (err) console.log("Error: " + err);
        }
      );
    });
  }
);

To create a normal search, poll for completion, and display results

Running a normal search creates a search job and immediately returns the search ID, so you need to poll the job to find out when the search has finished.

When you create a search job, set the parameters of the job as a dictionary of key-value pairs. For a list of all the possible parameters, see Search job parameters.

This example runs a normal search, waits for the job to finish, and then displays the results along with some final statistics:

// Search everything and return the first 100 results
var searchQuery = "search * | head 100";

// Set the search parameters
var searchParams = {
  exec_mode: "normal",
  earliest_time: "2012-06-20T16:27:43.000-07:00"
};

// Run a normal search that immediately returns the job's SID
service.search(
  searchQuery,
  searchParams,
  function(err, job) {

    // Display the job's search ID
    console.log("Job SID: ", job.sid);

    // Poll the status of the search job
    job.track({period: 200}, {
      done: function(job) {
        console.log("Done!");

        // Print out the statics
        console.log("Job statistics:");
        console.log("  Event count:  " + job.properties().eventCount); 
        console.log("  Result count: " + job.properties().resultCount);
        console.log("  Disk usage:   " + job.properties().diskUsage + " bytes");
        console.log("  Priority:     " + job.properties().priority);

        // Get the results and print them
        job.results({}, function(err, results, job) {
          var fields = results.fields;
          var rows = results.rows;
          for(var i = 0; i < rows.length; i++) {
            var values = rows[i];
            console.log("Row " + i + ": ");
            for(var j = 0; j < values.length; j++) {
              var field = fields[j];
              var value = values[j];
              console.log("  " + field + ": " + value);
            }
          }
        });
        
      },
      failed: function(job) {
        console.log("Job failed")
      },
      error: function(err) {
        done(err);
      }
    });

  }
);

To create a basic oneshot search and display results

Unlike other searches, the oneshot search does not create a search job, so you can't access it using the splunkjs.Service.Jobs class. Instead, use the splunkjs.Service.oneshotSearch method. To set properties for the search (for example, to specify a time range to search), you'll need to create a dictionary of key-value pairs. Some common parameters are:

  • output_mode: Specifies the output format of the results (XML, JSON, JSON_COLS, JSON_ROWS, CSV, ATOM, or RAW).
  • earliest_time: Specifies the earliest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string.
  • latest_time: Specifies the latest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string.
  • rf: Specifies one or more fields to add to the search.

For a full list of possible properties, see the list of Search job parameters, although most of these parameters don't apply to a oneshot search.

This example runs a oneshot search within a specfied time range and displays the results.

Note: If you don't see any search results, that means there aren't any in the specified time range. Just modify the date and time as needed for your data set.

// Search everything and return the first 10 results
var searchQuery = "search * | head 10";

// Set the search parameters--specify a time range
var searchParams = {
  earliest_time: "2011-06-19T12:00:00.000-07:00",
  latest_time: "2012-12-02T12:00:00.000-07:00"
};

// Run a oneshot search that returns the job's results
service.oneshotSearch(
  searchQuery,
  searchParams,
  function(err, results) {
    // Display the results
    var fields = results.fields;
    var rows = results.rows;
    
    for(var i = 0; i < rows.length; i++) {
      var values = rows[i];
      console.log("Row " + i + ": ");
      
      for(var j = 0; j < values.length; j++) {
        var field = fields[j];
        var value = values[j];
        console.log("  " + field + ": " + value);
      }
    }
  }
);

Collection parameters

By default, all entries are returned when you retrieve a collection. But by using the parameters below, you can also specify the number of entities to return and how to sort them. These parameters are available whenever you retrieve a collection.

ParameterDescription
countA number that indicates the maximum number of entities to return.
offsetA number that specifies the index of the first entity to return.
searchA string that specifies a search expression to filter the response with, matching field values against the search expression. For example, "search=foo" matches any object that has "foo" as a substring in a field, and "search=field_name%3Dfield_value" restricts the match to a single field.
sort_dirAn enum value that specifies how to sort entities. Valid values are "asc" (ascending order) and "desc" (descending order).
sort_keyA string that specifies the field to sort by.
sort_modeAn enum value that specifies how to sort entities. Valid values are "auto", "alpha" (alphabetically), "alpha_case" (alphabetically, case sensitive), or "num" (numerically).

Search job parameters

Properties to set

The parameters you can use for search jobs correspond to the parameters for the search/jobs endpoint in the REST API.

This list summarizes the properties you can set for a search job. For examples of setting these properties, see To create a blocking search and display properties and results of the job and To create a normal search, poll for completion, and display results.

ParameterDescription
searchRequired. A string that contains the search query.
auto_cancelThe number of seconds of inactivity after which to automatically cancel a job. 0 means never auto-cancel.
auto_finalize_ecThe number of events to process after which to auto-finalize the search. 0 means no limit.
auto_pauseThe number of seconds of inactivity after which to automatically pause a job. 0 means never auto-pause.
earliest_timeA time string that specifies the earliest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. For a real-time search, specify "rt".
enable_lookupsA Boolean that indicates whether to apply lookups to events.
exec_modeAn enum value that indicates the search mode ("blocking", "oneshot", or "normal").
force_bundle_replicationA Boolean that indicates whether this search should cause (and wait depending on the value of "sync_bundle_replication") bundle synchronization with all search peers.
idA string that contains a search ID. If unspecified, a random ID is generated.
index_earliestA string that specifies the time for the earliest (inclusive) time bounds for the search, based on the index time bounds. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string.
index_latestA string that specifies the time for the latest (inclusive) time bounds for the search, based on the index time bounds. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string.
latest_timeA time string that specifies the latest time in the time range to search. The time string can be a UTC time (with fractional seconds), a relative time specifier (to now), or a formatted time string. For a real-time search, specify "rt".
max_countThe number of events that can be accessible in any given status bucket.
max_timeThe number of seconds to run this search before finalizing. Specify 0 to never finalize.
namespaceA string that contains the application namespace in which to restrict searches.
nowA time string that sets the absolute time used for any relative time specifier in the search.
reduce_freqThe number of seconds (frequency) to run the MapReduce reduce phase on accumulated map values.
reload_macrosA Boolean that indicates whether to reload macro definitions from the macros.conf configuration file.
remote_server_listA string that contains a comma-separated list of (possibly wildcarded) servers from which to pull raw events. This same server list is used in subsearches.
rfA string that adds one or more required fields to the search.
rt_blockingA Boolean that indicates whether the indexer blocks if the queue for this search is full. For real-time searches.
rt_indexfilterA Boolean that indicates whether the indexer pre-filters events. For real-time searches.
rt_maxblocksecsThe number of seconds indicating the maximum time to block. 0 means no limit. For real-time searches with "rt_blocking" set to "true".
rt_queue_sizeThe number indicating the queue size (in events) that the indexer should use for this search. For real-time searches.
search_listenerA string that registers a search state listener with the search. Use the format: search_state;results_condition;http_method;uri;
search_modeAn enum value that indicates the search mode ("normal" or "realtime"). If set to "realtime", searches live data. A real-time search is also specified by setting "earliest_time" and "latest_time" parameters to "rt", even if the search_mode is normal or is not set.
spawn_processA Boolean that indicates whether to run the search in a separate spawned process. Searches against indexes must run in a separate process.
status_bucketsThe maximum number of status buckets to generate. 0 means to not generate timeline information.
sync_bundle_replicationA Boolean that indicates whether this search should wait for bundle replication to complete.
time_formatA string that specifies the format to use to convert a formatted time string from {start,end}_time into UTC seconds.
timeoutThe number of seconds to keep this search after processing has stopped.

Properties to retrieve

This list summarizes the properties that are available for an existing search job:

PropertyDescription
cursorTimeThe earliest time from which no events are later scanned.
delegateFor saved searches, specifies jobs that were started by the user.
diskUsageThe total amount of disk space used, in bytes.
dispatchStateThe state of the search. Can be any of QUEUED, PARSING, RUNNING, PAUSED, FINALIZING, FAILED, DONE.
doneProgressA number between 0 and 1.0 that indicates the approximate progress of the search.
dropCountFor real-time searches, the number of possible events that were dropped due to the "rt_queue_size".
eai:aclThe access control list for this job.
eventAvailableCountThe number of events that are available for export.
eventCountThe number of events returned by the search.
eventFieldCountThe number of fields found in the search results.
eventIsStreamingA Boolean that indicates whether the events of this search are being streamed.
eventIsTruncatedA Boolean that indicates whether events of the search have not been stored.
eventSearchSubset of the entire search before any transforming commands.
eventSortingA Boolean that indicates whether the events of this search are sorted, and in which order ("asc" for ascending, "desc" for descending, and "none" for not sorted).
isDoneA Boolean that indicates whether the search has finished.
isFailedA Boolean that indicates whether there was a fatal error executing the search (for example, if the search string syntax was invalid).
isFinalizedA Boolean that indicates whether the search was finalized (stopped before completion).
isPausedA Boolean that indicates whether the search has been paused.
isPreviewEnabledA Boolean that indicates whether previews are enabled.
isRealTimeSearchA Boolean that indicates whether the search is a real time search.
isRemoteTimelineA Boolean that indicates whether the remote timeline feature is enabled.
isSavedA Boolean that indicates whether the search is saved indefinitely.
isSavedSearchA Boolean that indicates whether this is a saved search run using the scheduler.
isZombieA Boolean that indicates whether the process running the search is dead, but with the search not finished.
keywordsAll positive keywords used by this search. A positive keyword is a keyword that is not in a NOT clause.
labelA custom name created for this search.
messagesErrors and debug messages.
numPreviewsNumber of previews that have been generated so far for this search job.
performanceA representation of the execution costs.
priorityAn integer between 0-10 that indicates the search's priority.
remoteSearchThe search string that is sent to every search peer.
reportSearchIf reporting commands are used, the reporting search.
requestGET arguments that the search sends to splunkd.
resultCountThe total number of results returned by the search, after any transforming commands have been applied (such as stats or top).
resultIsStreamingA Boolean that indicates whether the final results of the search are available using streaming (for example, no transforming operations).
resultPreviewCountThe number of result rows in the latest preview results.
runDurationA number specifying the time, in seconds, that the search took to complete.
scanCountThe number of events that are scanned or read off disk.
searchEarliestTimeThe earliest time for a search, as specified in the search command rather than the "earliestTime" parameter. It does not snap to the indexed data time bounds for all-time searches (as "earliestTime" and "latestTime" do).
searchLatestTimeThe latest time for a search, as specified in the search command rather than the "latestTime" parameter. It does not snap to the indexed data time bounds for all-time searches (as "earliestTime" and "latestTime" do).
searchProvidersA list of all the search peers that were contacted.
sidThe search ID number.
ttlThe time to live, or time before the search job expires after it has finished.