How to create a modular input using the Splunk Plug-in for Eclipse

The Splunk® Plug-in for Eclipse provides a new project type to create a modular input in Java. The created project consists of a Splunk Enterprise app containing a single modular input—with all the necessary shims and configuration files—and an Apache Ant build file to compile the source code into a runnable JAR file for Splunk.

This topic contains the following sections:

Create a new Splunk Enterprise modular input in Java project

To create a new Splunk Enterprise modular input in Java project:

  1. In Eclipse, on the File menu, point to New, and then click Project.

  2. In the New Project window, expand the Splunk category and click Splunk modular input in Java project. Then click Next.

    Screen shot of the Java New Project wizard
  3. In the Splunk modular input in Java project wizard, enter a name for your modular input. The name should contain only letters, numbers, the '.' (period) character, and the '_' (underscore) character. Then, enter the project's settings:

    1. Enter the author's name in the field next to Author. If you intend to publish the modular input to Splunkbase, you should enter your splunk.com user name here.

    2. Enter the version number in the field next to Version.

    3. If you want your new modular input to appear in your Splunk Enterprise launcher, check the box next to Show modular input in Splunk launcher, and then enter a label for the modular input next to Label to display and a description for it next to Description.

    4. Keep the box next to Generate a working example implementation checked. This will generate a fully functional modular input. (You can read more about this example modular input in How to create modular inputs in the Splunk SDK for Java.) You can use this modular input as a starting point. If you want to create your own modular input from scratch, uncheck the box next to Generate a working example implementation.

      Screen shot of the Splunk modular input in Java project wizard

  4. Click Finish.

The wizard creates a new project that includes a Main.java program in the src directory, and opens Main.java.

Screen shot of the Eclipse window with a new Java modular input project

The modular input project contains several more directories than a usual Java project. Here is a quick overview of what they are:

DirectoryPurpose
srcAll the source code to your program, including Main.java in the default package.
darwin_x86_64
linux_x86
linux_x86_64
windows_x86
windows_x86_64
Directories containing the shims Splunk Enterprise needs to run Java on different platforms.
default
README
Directories containing configuration files for Splunk.
jarsThe target directory which the modular input will be built into.
build.xmlAn Ant build file that compiled the modular input.

The project is wired into Eclipse to provide the support you expect from an IDE, but building the final jars for use in Splunk is done by the Ant build file. You should only have to edit files in the src, lib, and README directories.

Add an argument to the modular input

To add an argument to the modular input, you must put it two places:

  1. Add the line argument = <value> to README/inputs.conf.spec, where argument is the name of the argument to be added.

  2. Add an Argument object to the Scheme object returned by Main.getScheme in your implementation.

For instance, to add an argument named count to the modular input, you would add the following line to the inputs.conf.spec file (in the README directory):

count = <value>

The contents of the inputs.conf.spec file would then appear as follows:

[random_numbers://default]
*Autogenerated by Splunk Plug-in for Eclipse.
min = <value>
max = <value>
count = <value>

Next, in the Main.java file (in the src directory) insert your new argument before the return statement of the getScheme method. Shown here is the last portion of the contents of the getScheme method. Note the new countArgument object declaration:

...

Argument maxArgument = new Argument("max");
maxArgument.setDataType(Argument.DataType.NUMBER);
maxArgument.setDescription("Maximum value to be produced by this input.");
maxArgument.setRequiredOnCreate(true);
scheme.addArgument(maxArgument);

Argument countArgument = new Argument("count");
countArgument.setDataType(Argument.DataType.NUMBER);
countArgument.setDescription("Number of events to generate.");
countArgument.setRequiredOnCreate(true);
scheme.addArgument(countArgument);

return scheme; // ...and don't forget to return the scheme.   

Feel free to edit the new code block with a valid description and any other properties you want the object to have.

Of course, now that you've added a new argument, you'll need to use it in your modular input. For instance, if we want to change the current sample to generate events based on a count provided by the user, we can use the count argument in streamEvents to limit the number of random numbers generated. To do this, we'll need to make a few more additions to the code:

  • Declare a count variable and initialize the thread with that variable. In the streamEvents method, add the following line of code immediately after the max variable declaration:

    int count = ((SingleValueParameter)inputs.getInputs().get(inputName).get("count")).
    

    Also, replace the t thread declaration line with the following:

    Thread t = new Thread(new Generator(ew, inputName, min, max, count));
    

    The streamEvents method now appears as follows:

        @Override
        public void streamEvents(InputDefinition inputs, EventWriter ew) throws
                MalformedDataException, XMLStreamException, IOException {
            for (String inputName : inputs.getInputs().keySet()) {
                // We get the parameters for each input and start a new thread for each one. All the real work
                // happens in the Generator class below.
                double min = ((SingleValueParameter)inputs.getInputs().get(inputName).get("min")).getDouble();
                double max = ((SingleValueParameter)inputs.getInputs().get(inputName).get("max")).getDouble();
                int count = ((SingleValueParameter)inputs.getInputs().get(inputName).get("count")).getInt();
                Thread t = new Thread(new Generator(ew, inputName, min, max, count));
                t.run();
            }
        }
    
  • Update the Generator class declaration Next, update the Generator class declaration. First, declare count as a private integer, right after the min and max declarations:

    private int count;
    

    Now, update the constructor to include count when it's initialized. First change the following line:

    public Generator(EventWriter ew, String inputName, double min, double max) {
    
    It should now read:
    public Generator(EventWriter ew, String inputName, double min, double max, int count) {
    

    And add a line to initialize count to within the constructor:

    this.count = count;
    

    Finally, change the while loop within the run method to a for. This will mean first deleting the following line:

    while (true) {
    

    And pasting the following line in its place:

    for(int i=1; i <= count; i++) {
    

    Your Generator class now looks like the following:

        class Generator implements Runnable {
            private double min, max;
            private int count;
            
            EventWriter ew;
            String inputName;
    
            public Generator(EventWriter ew, String inputName, double min, double max, int count) {
                super();
                this.min = min;
                this.max = max;
                this.count = count;
                this.ew = ew;
                this.inputName = inputName;
            }
    
            public void run() {
                // First we log an INFO message that this thread has started. This will show up in splunkd.log and in
                // Splunk Enterprise's _internal index.
    
                // EventWriter provides both log and synchronizedLog (one a synchronized version of the other). In
                // this case, synchronizing at the level of each log message and event is exactly what we want. In
                // more complicated cases, you may want to use the unsynchronized version and do your own
                // synchronization.
                ew.synchronizedLog(EventWriter.INFO, "Random number generator " + inputName +
                        " started, generating numbers between " +
                        Double.toString(min) + " and " + Double.toString(max));
    
                final Random randomGenerator = new Random();
    
                for(int i=1; i <= count; i++) {
                    // Write a new event. The minimum that you must set on an event is the stanza it is supposed to
                    // go to (which you can skip if your modular input is not single instance, and the data of the
                    // event.
                    Event event = new Event();
                    event.setStanza(inputName);
                    event.setData("number=" + (randomGenerator.nextDouble() * (max - min) + min));
    
                    try {
                        ew.writeEvent(event);
                    } catch (MalformedDataException e) {
                        ew.synchronizedLog(EventWriter.ERROR, "MalformedDataException in writing event to input" +
                                inputName + ": " + e.toString());
                    }
    
                    try {
                        Thread.sleep(500);
                    } catch (InterruptedException e) {
                        return;
                    }
                }
            }
        }
    
  • You've now updated your modular input sample to generate events based on a count provided by the user.

    Use the modular input

    To use the modular input in Splunk Enterprise, first compile the source code into a runnable JAR (which is done by build.xml), and then package it into an .spl file. The Ant build file created by the project wizard handles both of these tasks.

    In Eclipse, right-click on build.xml in the Package Explorer, point to Run As, and then click Ant Build (take care not to choose the Ant Build option that is followed by ellipses ('...')).

    Eclipse packages the app into an .spl file in a new dist directory. If the dist directory doesn't appear right away, right-click on the random_numbers package (the top node) in the Package Explorer and click Refresh.

    To install the .spl file, in Splunk Home on the Apps menu, click Manage Apps. Click the Install app from file button. Click Choose File, find the .spl file to upload, and then click Upload. You will be prompted to restart Splunk Enterprise.

    Add libraries to the modular input project

    Many modular inputs depend on another JAR file for their functionality. To add a JAR to your modular input:

    1. Place the JAR file in the lib directory of the project.

    2. In Eclipse, right-click Referenced Libraries in the Package Explorer, point to Build Path, and then click Configure Build Path. The Properties window appears.

      Screen shot of the project properties window
    3. Click Add JARs, and in the JAR Selection window, navigate to the lib directory and the JAR file you added. Select the JAR file and click OK.

      Screen shot of the JAR selection window

    Eclipse now knows to use the JAR for autocomplete and internal compilation. The JAR file will be packaged into your final JAR by the Ant build file.