Enable logging to HTTP Event Collector in your Java project

This topic describes how to integrate the Splunk logging for Java library into your Java project to enable logging to Splunk HTTP Event Collector. To learn how to add logging to TCP inputs, see Enable logging to TCP inputs in your Java project.

You can log individual events or batches of events to HTTP Event Collector. Splunk logging for Java enables you to set timestamp, sourcetype, source, index, and host for all events or per event.

Follow these steps to enable logging to HTTP Event Collector:

  1. Add the Splunk logging for Java library to your project
  2. Add the logging library you have chosen to use (Logback, Log4j 2, or java.util.logging) to your project.
  3. Make sure HTTP Event Collector is enabled on your Splunk Enterprise Instance and that you have a valid token. For more information, see “About Event Collector Tokens” in Use HTTP Event Collector in the Getting Data In Manual.
  4. Configure your logger to use the HTTP Event Collector.
  5. Use your logger to log events.

Add the Splunk logging for Java library to your project

Follow the instructions on the Install page according to how you prefer to add Splunk logging for Java library to your project:

Add the logging framework you have chosen to use. The three major frameworks are Logback, Log4j 2, and java.util.logging (which is included with the JDK). Follow the instructions provided here to add your preferred logging framework to your project.

Logback

If you are using Maven, add the following to the dependencies section of your pom.xml file to include the necessary libraries:

<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-classic</artifactId>
  <version>1.1.3</version>
</dependency>
<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-core</artifactId>
  <version>1.1.3</version>
</dependency>

Otherwise, download and add the following .JAR files to your classpath, as described on the Logback setup page:

  • logback-core-1.1.3.jar
  • logback-classic-1.1.3.jar

Log4j 2

If you are using Maven, add the following to the dependencies section of your pom.xml file to include the necessary library:

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>2.3</version>
</dependency>

Otherwise, download and add the following .JAR file to your classpath, as described on the Log4j download and setup page:

  • log4j-core-2.3.jar

java.util.logging

The java.util.logging library ships with the JDK, so just add the following import statement:

import java.util.logging.*; 

SLF4J

To use SLF4J, download and add the following .JAR files to your classpath, as described in the SLF4J user manual:

  • slf4j-api-1.7.12.jar
  • slf4j-simple-1.7.12.jar

Then include the import statement and (at least) the Java code shown here:

import org.slf4j.*;

...

    Logger logger = LoggerFactory.getLogger("splunk.logger");
    logger.info("This is a test");

Enable HTTP Event Collector

HTTP Event Collector is turned off by default in Splunk Enterprise and Splunk Cloud. Before you can send events to it, you need to enable it. Talk to your Splunk administrator about enabling HTTP Event Collector, or follow the instructions in Use HTTP Event Collector.

Once HTTP Event Collector is enabled, you need some additional information before you can log events to it. At the very least, you need:

  • The hostname and port of the Splunk Enterprise or Splunk Cloud server with HTTP Event Collector enabled. The default port number for HTTP Event Collector is 8088, but your Splunk administrator may have changed it.
  • A valid, enabled token. An HTTP Event Collector token is a string that identifies a client (your Java app) to HTTP Event Collector. Your Splunk administrator (or a designated token administrator) can generate and give you a valid token. For more information, see "About Event Collector tokens" in the Getting Data In Manual.

In addition, you can specify the following optional attributes:

  • The Splunk index you want to index the events.
  • The Splunk source value you want to assign to the events.
  • The Splunk sourcetype value you want to assign to the events.
  • Event batching parameters, including batch interval, maximum number of events in a batch, and maximum size of events in a batch.

Configure your logging system

In this section you will see how to configure to use each of the 3 loggers and then how to log to them from within an application using slf4j. You can also use each logger directly, which is not shown.

Note: Example configuration files are available in the Splunk logging for Java download in splunk-library-javalogging/src/test/resources.

The log4j2.xml and logback.xml files should be placed somewhere in the classpath of your program. Specify jdklogging.properties to your program by passing the following to the Java executable:

-Djava.util.logging.config.file=/path/to/jdklogging.properties

Logback

To start logging to HTTP Event Collector using Logback, first add the framework as described previously, in "Add the Splunk logging for Java library to your project."

Then, create the logging configuration XML file. You might want to make a copy of the logback_template.xml file from the Splunk logging for Java repo and then edit it as necessary. The template is at the following path: /src/test/resources/logback_template.xml. A bare-minimum version of the file is included below. It includes just the url and token elements, plus the optional disableCertificateValidation element. This example does not include the optional elements source, sourcetype, batch_interval, batch_size_bytes, and batch_size_count. Be sure to customize the XML configuration file to correspond to your setup.

Note: When testing this functionality, set the batch_size_count property to "1". Setting this property to "1" means that once one event is queued, it's considered a batch. This is useful when you're testing this functionality by logging an event or two, but is not recommended for production use. When you put this solution into production, start with a setting of "10" and adjust as necessary. Leaving out this property sets it to its default, which is "10".
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <Appender name="%user_logger_name%" class="com.splunk.logging.HttpEventCollectorLogbackAppender">
        <url>%scheme%://%host%:%port%</url>
        <token>%user_httpeventcollector_token%</token>
        <disableCertificateValidation>true</disableCertificateValidation>
        <layout class="ch.qos.logback.classic.PatternLayout">
            <pattern>%msg</pattern>
        </layout>
    </Appender>
    <logger name ="%user_logger_name%" level="debug">
        <appender-ref ref="%user_logger_name%" />
    </logger>
</configuration>

Next, add the following to the imports section of your code to import the appropriate libraries: 

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.splunk.logging.*;

In your code, create a logger by using the getLogger method of the LoggerFactory class: 

Logger logger = LoggerFactory.getLogger(loggerName);

Finally, add code to log data at the appropriate levels. For example, the following logging statements from the Splunk logging for Java test code log messages at the info level:

logger.info("This is a test event for Logback test");
logger.error("This is a test error for Logback test");

Log4j 2

To start logging to HTTP Event Collector using Log4j 2, first add the framework as described previously, in "Add the Splunk logging for Java library to your project."

Then, import the XML configuration file. You might want to make a copy of the log4j2_template.xml file from the Splunk logging for Java repo and then edit it as necessary. The template is at the following path: /src/test/resources/log4j2_template.xml. A bare-minimum version of the file is included below. It includes just the name, url, and token attribute, plus the optional disableCertificateValidation attribute. This example does not include the optional elements source, sourcetype, batch_interval, batch_size_bytes, and batch_size_count. Be sure to customize the XML configuration file to correspond to your setup.

Note: When testing this functionality, set the batch_size_count property to "1". Setting this property to "1" means that once one event is queued, it's considered a batch. This is useful when you're testing this functionality by logging an event or two, but is not recommended for production use. When you put this solution into production, start with a setting of "10" and adjust as necessary. Leaving out this property sets it to its default, which is "10".
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" name="example" packages="">
    <Appenders>
    <Http
        name="http"
        url="%scheme%://%host%:%port%"
        token="%user_httpeventcollector_token%"
        disableCertificateValidation="true"
    >
        <PatternLayout pattern="%m"/>
    </Http>
</Appenders>
<!-- Define a logger named 'testSplunkLog4j' which writes to the socket appender we defined above. -->
    <Loggers>
        <Root level="INFO"/>
        <Logger name="%user_logger_name%" level="info">
        <AppenderRef ref="http"/>
        </Logger>
    </Loggers>
</Configuration>

Next, add the following to the imports section of your code to import the appropriate libraries:

import org.apache.logging.log4j.*;
import com.splunk.logging.*;

In your code, create a logger by using the getLogger method of the LoggerContext class: 

Logger logger = org.apache.logging.log4j.core.LoggerContext.getLogger(loggerName);

Finally, add code to log data at the appropriate levels. For example, the following logging statements from the Splunk logging for Java test code log messages at the info level:

logger.info("This is a test event for Logback test");
logger.error("This is a test error for Logback test");

Though HTTP Event Collector accepts only JSON-formatted event data packets, the event data payload can be in any format you want, as long as it is surrounded by curly brackets.

java.util.logging 

To start logging to HTTP Event Collector using java.util.logging, first specify jdklogging.properties to your program by passing the following to the Java executable:

-Djava.util.logging.config.file=/path/to/jdklogging.properties

Then, customize the jdklogging.properties file. You might want to make a copy of the logging_template.properties file from the Splunk logging for Java repo and then edit it as necessary. template is at the following path: /src/test/resources/logging_template.properties. A bare-minimum version of the file is included below. It includes just the url, level, and token properties, plus the disableCertificateValidation property. This example does not include the optional elements source, sourcetype, batch_interval, batch_size_bytes, batch_size_count, and index. Be sure to customize the XML configuration file to correspond to your setup.

Note: When testing this functionality, set the batch_size_count property to "1". Setting this property to "1" means that once one event is queued, it's considered a batch. This is useful when you're testing this functionality by logging an event or two, but is not recommended for production use. When you put this solution into production, start with a setting of "10" and adjust as necessary. Leaving out this property sets it to its default, which is "10".
# Implicitly create a logger called 'splunkLogger', set its level to INFO, and
# make it log using the SocketHandler.
%user_logger_name%.level = INFO
%user_logger_name%.handlers = com.splunk.logging.HttpEventCollectorLoggingHandler

# Configure the com.splunk.logging.HttpEventCollectorLoggingHandler
com.splunk.logging.HttpEventCollectorLoggingHandler.url = %scheme%://%host%:%port%
com.splunk.logging.HttpEventCollectorLoggingHandler.level = INFO
com.splunk.logging.HttpEventCollectorLoggingHandler.token = %user_httpeventcollector_token%
com.splunk.logging.HttpEventCollectorLoggingHandler.disableCertificateValidation=true

Next, add the following to the imports section of your code to import the appropriate libraries: 

import java.util.logging.*;
import com.splunk.logging.*; 

In your code, create a logger by using the getLogger method of the Logger class: 

Logger logger = Logger.getLogger(loggerName); 

Finally, add code to log data at the appropriate levels. For example, the following logging statements from the Splunk logging for Java test code log messages at the info level:

logger.info("This is a test event for Logback test");
logger.error("This is a test error for Logback test");

Though HTTP Event Collector accepts only JSON-formatted event data packets, the event data payload can be in any format you want, as long as it is surrounded by curly brackets.

If you log a sequence of events to HTTP Event Collector that contains invalid events, it will process until it hits the first invalid event, and then return an error.