How to use manual batching with Splunk logging for JavaScript

This topic demonstrates how to manually batch events before sending them to HTTP Event Collector (EC) on Splunk Enterprise or Splunk Cloud. The manual_batching.js example included in the examples directory of the Splunk logging for JavaScript package represents a sample implementation. It has been pasted below.

Note: The examples are not installed when using the npm installation method. To obtain copies of the examples, download the Splunk logging for JavaScript package.

Example walkthrough

This example includes the basic logic to batch events before sending them to HTTP Event Collector on Splunk Enterprise or Splunk Cloud. Batching is enabled by disabling the autoFlush property, which causes events to be sent every time the send function is called.

First, we declare SplunkLogger and config variables, and add an error handler (Logger.error), as described in the basic.js example.

Then, we define the event payload in the payload variable. At minimum, we need some sort of message to send. The other keys, metadata and severity, are optional. In this case, we've added two key-value pairs, but the contents of the message key can be anything at all. The contents of metadata will be assigned to this event when Splunk Enterprise or Splunk Cloud indexes the event. If any of these values (source, sourcetype, and so on) differ from the default values on the server, the values specified here will override the default values. Of course, your JavaScript app will determine what goes into the actual payload contents.

Next, we "send" the payload to the event queue by calling Logger.send. In this case, since autoFlush has been disabled, we're effectively queuing the payload.

After queuing the first event, we define another payload variable (payload2) and load it with some more sample data. We then queue this payload as well, by calling Logger.send.

Finally, we send the batched events by manually calling Logger.flush. We then log the response from Splunk Enterprise or Splunk Cloud.

manual_batching.js

/*
 * Copyright 2015 Splunk, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License"): you may
 * not use this file except in compliance with the License. You may obtain
 * a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations
 * under the License.
 */

/**
 * This example shows how to batch events with the
 * SplunkLogger by manually calling flush.
 *
 * By setting maxbatchCount=0, events will be queued
 * until flush() is called.
 */

// Change to require("splunk-logging").Logger;
var SplunkLogger = require("../index").Logger;

/**
 * Only the token property is required.
 * 
 * Here, maxBatchCount is set to 0.
 */
var config = {
    token: "your-token-here",
    url: "https://localhost:8088",
    maxBatchCount: 0 // Manually flush events
};

// Create a new logger
var Logger = new SplunkLogger(config);

Logger.error = function(err, context) {
    // Handle errors here
    console.log("error", err, "context", context);
};

// Define the payload to send to Splunk's Event Collector
var payload = {
    // Message can be anything, doesn't have to be an object
    message: {
        temperature: "70F",
        chickenCount: 500
    },
    // Metadata is optional
    metadata: {
        source: "chicken coop",
        sourcetype: "httpevent",
        index: "main",
        host: "farm.local"
    },
    // Severity is also optional
    severity: "info"
};

console.log("Queuing payload", payload);
// Don't need a callback here
Logger.send(payload);

var payload2 = {
    message: {
        temperature: "75F",
        chickenCount: 600,
        note: "New chickens have arrived"
    },
    metadata: payload.metadata
};

console.log("Queuing second payload", payload2);
// Don't need a callback here
Logger.send(payload2);

/**
 * Call flush manually.
 * This will send both payloads in a single
 * HTTP request.
 *
 * The same callback can work for send() and flush().
 */
Logger.flush(function(err, resp, body) {
    // If successful, body will be { text: 'Success', code: 0 }
    console.log("Response from Splunk", body);
});