How to debug modular inputs in Splunk SDK for C# v2.x

This topic describes how to debug modular inputs using the Splunk SDK for C# 2.x, plus how to effectively use the diagnostic information generated by the SDK.

This topic contains the following sections:

Modular input debugging

A common developer request has been for a mechanism to debug modular inputs created with the Splunk SDK for C#. With the Splunk SDK for C# version 2.1.0 and later and Microsoft Visual Studio, now you can, using remote debugging. The basic process is:

  1. Enable: Enable remote debugging.
  2. Deploy: Compile and deploy the updated binary to a Splunk Enterprise instance, and then enable the new input in Splunk Enterprise.
  3. Attach: Add a break point to the code, and then remotely attach to a running modular input process.
  4. Debug: Step through and over, use the watch window, and so on.

Note: Though this should also work with Xamarin Studio, it has not been tested, and is not supported at this time.

Enable

To enable debugging, use the new overload of the Run<T> method, which accepts a debugger attach point and an optional timeout. The attach point (a member of the DebuggerAttachPoints enumeration) specifies where you want to debug, which can be either during the optional validation stage, or during the event streaming stage. The timeout is the length of time the input will wait for a debugger to attach before it continues processing; the default value is 30 seconds.

Here is an example of how to enable debugging for event streaming:

public static int Main(string[] args)
{
    return Run<Program>(args, DebuggerAttachPoints.StreamEvents);
}

Deploy

Next, compile and deploy the updated binary to your Splunk Enterprise instance. Then, enable the input in Splunk Enterprise. Be sure to disable any previous versions of the input before enabling the latest version.

Attach

Add a break point to the code:

  • If you specified the StreamEvents debugger attach point, add a break point within the StreamEventsAsync method.
  • If you specified the ValidateArguments debugger attach point, add a break point within the Validate method.

Notes: You can only attach for the StreamEventsAsync and Validate methods.

The Validate method has about 30 seconds before Splunk Enterprise kills the process, so work fast if you want to debug validation.

Now, in Visual Studio, open the Attach to Process window and attach to the running modular input process.

Note: If you can't see the modular input process (which is identified by the name of the modular input, followed by .exe), select the Show processes from all users checkbox.

Debug

As soon as you attach to the process, Visual Studio breaks on the break point you specified. At this point, you can step through and over, use the watch window, and so on.

Modular input diagnostics

You can use the Splunk SDK for C# to capture diagnostic information about failures in modular inputs.

In the Splunk SDK for C# version 2.1.0 and later, the following data is captured:

  • All trapped and untrapped exceptions, including the stack trace. This applies to Validate, Scheme, and StreamEventsAsync.
  • Wherever possible, the input instance name is captured. For instance, if you have several GitHub commits inputs, you'll know exactly which one is failing.
  • For Scheme, the instance name is not captured, as it is not instance based. In this case the type of input will still be captured.
  • For Validation, the XML message that Splunk Enterprise sends to you containing the values that are being validated is captured. The XML validation messages are captured as Info messages. Each one is separated by pipes ("|"), and then concatenated.

You can also easily generate your own diagnostic messages in your code using a strongly-typed overload to the EventWriter.LogAsync method.

For instance, following is an example of a custom diagnostic message from within the GitHub commits code. Note the info message that is logged using the new method:

if (!shaKeys.Contains(githubCommit.Sha))
{
    await StreamCommit(githubCommit, eventWriter, owner, repository);
    await fileWriter.WriteLineAsync(githubCommit.Sha); // Write to the checkpoint file
    shaKeys.Add(githubCommit.Sha);
    await eventWriter.LogAsync(Severity.Info, repository + " indexed a Github commit with sha: " + githubCommit.Sha);
}