Create a Lambda function in Java

This topic describes one way to create an AWS Lambda function that sends data to HTTP Event Collector (HEC) using Splunk logging for Java. The guidance provided here assumes that you are already familiar with the programming model documentation in the AWS Lambda Developer Guide:

You can write the function using an IDE such as Eclipse, though an IDE is not required. Integrate Splunk logging for Java and the logging and other libraries you need, and then package everything in a ZIP file and upload it to the AWS Lambda console.

This documentation guides you through our simple Java Lambda example function in detail. It contains the following sections:

Prerequisites

This walkthrough uses the Eclipse Java IDE, the AWS Toolkit for Eclipse, and Apache Maven to create a Lambda function. The M2Eclipse Maven plug-in for Eclipse is built into the latest version of Eclipse. These tools are not required, but are recommended to save you some time.

The one AWS Lambda prerequisite that must be in place is that Lambda must be logging its own diagnostics to Amazon CloudWatch, and not forwarding them elsewhere.

  1. Install Eclipse. Amazon recommends the Eclipse IDE for Java EE Developers.

  2. Install the AWS Toolkit for Eclipse.

Next, you'll create a new AWS Lambda Java project.

Create a new project

  1. In Eclipse, on the File menu, point to New, and then click Project. In the New Project window, expand AWS, and then click AWS Lambda Java Project. Then, click Next.

    New Project wizard

  2. In the Create a new AWS Lambda Java project setup wizard, fill in the following values, and then click Finish:

    • First, name the project and the package. We've named both loggertest.
    • Next to Input Type, choose Custom from the pop-up menu, and then replace Object with String. We do this because AWS Lambda considers HEC events as generic strings.
    • Next to Output Type, replace Object with String.

    Create a new AWS Lambda project window

    After you click Finish, the .java file opens in Eclipse. A README file also appears, if you left the option checked. It's a good idea to review the README for any breaking changes that may differ from the instructions given here.

  3. Convert the project to a Maven project. In the Package Explorer, right-click the project name, point to Configure, and then choose Convert to Maven Project.

    Convert to Maven Project window
  4. In the Create New POM window, change any values you want to, and then click Finish.

    Create new POM window

The project converts to a Maven project, and the pom.xml file opens.

Add dependencies and a repository

Continue by configuring the project as follows:

  1. Add the following repository to the project's pom.xml file:

    • Splunk releases (id: splunk, name: splunk-releases, url: http://splunk.artifactoryonline.com/splunk/ext-releases-local)

    You can do this using whatever method you prefer, such as using the plug-in UI or editing the pom.xml file manually, but you should end up with something like the following repositories element in pom.xml:

    <repositories>
        <repository>
            <id>splunk</id>
            <name>splunk-releases</name>
            <url>http://splunk.artifactoryonline.com/splunk/ext-releases-local</url>
        </repository>
    </repositories>
    

    Once you do this and save your changes, Eclipse brings in the necessary repository.

  2. Add the following dependencies to the project's pom.xml file:

    • JUnit (groupId: junit, artifactId: junit)
    • Apache Log4J 2 (groupId: org.apache.logging.log4j, artifactId: log4j-core)
    • Splunk library for Java (groupId: com.splunk.logging, artifactId: splunk-library-javalogging)

    You can do this using whatever method you prefer, such as using the plug-in UI or editing the pom.xml file manually, but you should end up with something like the following dependencies element in pom.xml:

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.splunk.logging</groupId>
            <artifactId>splunk-library-javalogging</artifactId>
            <version>1.5.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.1</version>
        </dependency>	
    </dependencies>
    

    Once you do this and save your changes, Eclipse brings in the necessary dependencies.

Add Log4j 2 support

Our example uses Apache Log4j 2 for logging to HEC on Splunk Cloud. Be aware that this is a separate mechanism from the AWS Lambda custom appender for Log4j 1.2.

  1. Add a log4j2 folder to the project. In the Project Explorer, right-click the name of the project, point to New, and then click Folder. Next to Folder name, type log4j2.

  2. Create a new file named log4j2.xml inside the log4j2 folder using the method described in the previous step for a new file. The log4j2.xml file contains configuration information for Log4J 2. Inside the log4j2.xml file, you add a <Configuration> element, and inside that, both an <Appenders> element and a <Loggers> element. Inside <Appenders>, you'll add an Http appender that corresponds to your HEC input. To the <Loggers> element, you'll also add a logger to the Http appender.

    Use the following example XML as a template, replacing <splunk_host:port> with your Splunk Cloud URL and the port you've assigned to HEC (the default port is 8088) and <token> with the HEC token you want to use.

    <Configuration status="info" name="example" packages="">
        <!-- Define an appender that writes to a TCP socket. We use Log4J's SocketAppender, 
            which is documented at https://logging.apache.org/log4j/2.x/manual/appenders.html#SocketAppender. 
            You can find the documentation on how to open TCP inputs on Splunk at http://docs.splunk.com/Documentation/Splunk/latest/Data/Monitornetworkports. 
            Note that TCP inputs are *not* the same as Splunk's management port. -->
        <Appenders>
            <Http name="http" url="<splunk_host:port>"
                token="<token>" disableCertificateValidation="true" batch_size_count="1" >
                <PatternLayout pattern="%m" />	
            </Http>
        </Appenders>
        <!-- Define a logger named 'splunk.logger' that writes to the socket appender 
            we defined above. -->
        <Loggers>
            <Root level="INFO">
            </Root>
            <Logger name="splunk.logger" level="info">
                <AppenderRef ref="http" />
            </Logger>
        </Loggers>
    </Configuration>
    Note: We've set the batch_size_count property to "1" here, but that should only be used for testing. Setting this property to "1" means that once one event is queued, it's considered a batch, which isn't really a batch at all. When you put this solution into production, start with a setting of "10" and adjust as necessary. Leaving out this property sets it to its default, which is "10".
  3. Now, add the folder you just added to the build path. Open the project properties window by pressing Command-I (or CTRL-I) or by right-clicking the project name and then clicking Properties.

    Properties window

    Choose the Java Build Path properties pane. Click the Source button in the pane, and then click Add Folder. Select the checkbox next to the log4j2 folder you just created, and then click OK.

  4. To LambdaFunctionHandler.java, add the following import statements for Log4j 2. These are all required since our example depends on a specific version of Log4j:

    import org.apache.logging.log4j.*;
    import org.apache.logging.log4j.core.Appender;
    import org.apache.logging.log4j.core.LoggerContext;
    import org.apache.logging.log4j.core.appender.AbstractAppender;
    import org.apache.logging.log4j.core.config.Configuration;

Add code

Now it's time to add code. Open LambdaFunctionHandler.java, and then do the following:

  1. Just inside the LambdaFunctionHandler class, add the following LoggerContext final variable:
        final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
    
  2. Clear the contents of the handleRequest() method, and paste in the following:
        Logger logger = LogManager.getLogger("splunk.logger");
            
        logger.info(input);
    
        try {
            Thread.sleep(100);                 
        } catch(InterruptedException ex) {
            Thread.currentThread().interrupt();
        }
           
        return input;

    The code adds a sleep() method to delay exiting the Lambda function. This is necessary to ensure that the events are sent to Splunk Cloud, as Splunk logging for Java sends data in an asynchronous manner. Depending on the size of the payload or number of events, and whether your Splunk Cloud instance is in the same region, you may want to adjust this number.

Build and run the project

To test out your code, you need to build and upload it to AWS Lambda. To learn how to do this, follow the instructions in the AWS Toolkit for Eclipse User Guide, in Using AWS Lambda with the AWS Toolkit for Eclipse. Specifically, start at the third section of the tutorial, Allow Lambda to assume an IAM role. The first two sections are covered previously in this topic.

As you follow the instructions in the tutorial, change any tutorial-specific settings to correspond to the example you've been building here.

You will:

  • Allow Lambda to assume an IAM role.
  • Create an Amazon S3 bucket for your Lambda code (if you decide not to use an existing bucket).
  • Upload the code.
  • Invoke the Lambda function.

When you invoke the Lambda function, first check the Eclipse console to see the function output, and to catch any error messages. If the function ran successfully, log onto Splunk Cloud and search for data using the token or input name you gave your HEC input.