How to use a custom event format with HTTP Event Collector stream for Bunyan

This topic demonstrates how to use a custom event format for SplunkLogger. The custom_format.js example included in the examples directory of the HTTP Event Collector (HEC) stream for Bunyan package provides a sample implementation. It has also been pasted below.

This section of the documentation contains information about the custom_format.js example included in the examples directory of the Splunk logging for JavaScript package.

Note: The examples are not installed when using the npm installation method. To obtain copies of the examples, download the Splunk HTTP Event Collector stream for Bunyan package.

Example walkthrough

This example includes the logic to send data to HTTP Event Collector in a custom event format that is defined by overriding the stream's default eventFormatter() function.

First, we add require statements for Bunyan and the HEC stream for Bunyan.

Then, we declare a config variable to store the configuration information for the Splunk Enterprise instance or Splunk Cloud server. Only the token property is required, but in this example, we've set the token and url properties.

Next, we create a Bunyan stream (splunkStream), plus an error handler.

At this point, we override the default eventFormatter() function using setEventFormatter(). The eventFormatter() function takes message and severity inputs and returns any type, though a string or object are recommended. In the example, we're building up a string of key-value pairs if the message is an object; otherwise the message value is a value for the message key. We prefix the event string with the severity in square brackets.

Then, we create a logger (Logger) using the bunyan.createLogger() function, including the streams array as one of its inputs. Inside the streams array, we include splunkStream.

Next, we define the event payload in the payload variable. We've added fields for the event data itself (temperature and chickenCount in this case). Then we added several special keys to specify metadata that is to be assigned to the event data when HTTP Event Collector receives it. If any of these values (source, sourcetype, and so on) differ from the default values on the server, the values specified here will override the default values. Of course, your JavaScript app will determine what goes into the actual payload contents.

Because the default value of the maxBatchCount variable is 1, calling the send() function will immediately send the payload.

Finally, use the Logger's info function to send the payload along with a status message.

custom_format.js

/*
 * Copyright 2015 Splunk, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License"): you may
 * not use this file except in compliance with the License. You may obtain
 * a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations
 * under the License.
 */

/**
 * This example shows how to use a custom event format
 * for the Splunk Bunyan logger.
 */

// Change to require("splunk-bunyan-logger");
var splunkBunyan = require("../index");
var bunyan = require("bunyan");

/**
 * Only the token property is required.
 */
var config = {
    token: "your-token-here",
    url: "https://localhost:8088"
};
var splunkStream = splunkBunyan.createStream(config);

splunkStream.on("error", function(err, context) {
    // Handle errors here
    console.log("Error", err, "Context", context);
});

/**
 * Override the default eventFormatter() function,
 * which takes a message and severity, returning
 * any type; string or object are recommended.
 *
 * The message parameter can be any type. It will
 * be whatever was passed to Logger.send().
 * Severity will always be a string.
 *
 * In this example, we're building up a string
 * of key=value pairs if message is an object,
 * otherwise the message value is as value for
 * the message key.
 *
 * This string is prefixed with the event
 * severity in square brackets.
 */
splunkStream.setEventFormatter(function(message, severity) {
    var event = "[" + severity + "]";

    if (typeof message === "object") {
        for (var key in message) {
            event += key + "=" + message[key] + " ";
        }
    }
    else {
        event += "message=" + message;
    }
    
    return event;
});

// Setup Bunyan, adding splunkStream to the array of streams
var Logger = bunyan.createLogger({
    name: "my logger",
    streams: [
        splunkStream
    ]
});

// Define the payload to send to HTTP Event Collector
var payload = {
    // Our important fields
    temperature: "70F",
    chickenCount: 500,

    // Special keys to specify metadata for HTTP Event Collector
    source: "chicken coop",
    sourcetype: "httpevent",
    index: "main",
    host: "farm.local"
};

/**
 * Since maxBatchCount is set to 1 by default,
 * calling send will immediately send the payload.
 * 
 * The underlying HTTP POST request is made to
 *
 *     https://localhost:8088/services/collector/event/1.0
 *
 * with the following body (the pid will be different)
 *
 *     {
 *         "source": "chicken coop",
 *         "sourcetype": "httpevent",
 *         "index": "main",
 *         "host": "farm.local",
 *         "event": "[info]name=my logger pid=35265 temperature=70F chickenCount=500 msg=Chicken coup looks stable. v=0"
 *     }
 *
 */
console.log("Sending payload", payload);
Logger.info(payload, "Chicken coup looks stable.");