The Windows release of Splunk Preview debuts with WMI. So, what is WMI for all you splunkheads out there? It’s an OS interface which allows “instrumented components to provide information and notification”. WMI gives you the ability to query system instrumentation data such as system performance, event logs, end countless other events that occur on the system. It also has the capability of doing this agent-less from remote machines. The most exciting feature is the ability to do collection of Windows event logs from other machines on your network simultaneously. A Splunk install is not required on every single node that generates this data, and you don’t need to do anything special to facilitate this. Assuming you’ve set up proper authentication between the machines, of course. Setting up proper WMI security is a hot topic on its own.

From the standpoint of configuration and what WMI is capable of doing, in the context of Splunk, WMI can be used in two ways: to pull event logs and to query instrumentation data. Assuming that you have enough credentials to poll event logs agentlessly, you can simply specify host name and the log file you are interested in. This is an example of retrieving “Application” event logs from a remote machine named “remotehost”:

[WMI:RemoteApplication]
namespace = \\remotehost\root\cimv2
interval = 10
event_log_file = Application
disabled = 0

The other aspect of WMI warrants more explanation. To get data from WMI providers, you query them using WQL (WMI query language), which is a subset of SQL. Simply specify a query, and all fields returned by the provider will be automatically collated as an event. (Some queries return multiple results, and hence generate multiple events.) An example query will be select FreeMegabytes from Win32_PerfFormattedData_PerfDisk_LogicalDisk, which will poll free disk space from all logical disk partitions on the

Hey everyone, just wanted to let you know that a preview release of Splunk just left the docks.

http://www.splunk.com/index.php/preview

I want to introduce to you one the latest features for Windows Splunk - the monitoring of Windows registry in real time for activity/events, and the indexing and searching these events with Splunk.

While working on this we had a few challenges:

First, there aren’t any published win32 APIs that does this in user mode. The best that you can do with win32 API is to poll the registry for certain registry key/hives, and you’ll be notified when if the key or subkey of the hive has been changed. Even when you get a notification for a change, you will not be told which key exactly has changed, you’ll have to figure that out yourself .

Second, scalability. You can’t possibly poll all of the registry in user mode for changes. There are simply too many keys to query.

The solution is to write a device driver that hooks to the kernel and intercepts all registry events. The driver bubbles up the events to the user mode for filtering and tagging, and finally pipe them to Splunk for indexing. Obviously, this driver needs to be very stable and reliable, needs to scale to the point where if you want to monitor all of the events in the registry, and it should be able to handle the load.

With this preview release we launched the first version of the splunk-regmon tool. The tool writes events to standard output, and using Splunk’s ExecProcessor(popen). Splunk is able to get these events and send them through the indexing pipeline. A basic filtering is in place, hard coded for now to only monitor registry events related to changes - i.e. Create, Delete, Set, etc. Create type events

Splunk is on track to become a billion-dollar company and you, the intrepid sysadmin/developer, are going to help us get there. Now, this is not a statement that I’m making as an analyst who “covers” the enterprise software market, and compiles a list of “top software companies to watch”. I’m writing this as Splunk’s Platform Architect, a techie whose goals are to ensure that what comes out of our development group is compelling and exciting to those that are actually working with the product.

It is this developer-centric ethos that sets us apart from so many of the other enterprise software firms and has already paid dividends on community goodwill. Instead of making prospective buyers jump through registration hoops just to view a guided webcast tour, Splunk provides fully functional software downloads to try out on your own data, inside your own network, free from webinar smoke and mirrors.

We don’t just want you to try out the software, we want you to try doing things that aren’t covered in our brochureware, things that sound ludicrous at first but are doable. In fact, in a perverse way, we hope that you do break our product because it reveals new limitations for us to solve, ultimately leading to a product that lets you do your job the way you want, yet easier and faster.

This is where the Splunk Platform comes into play. We want to increase the ubiquity of Splunk by, 1) exposing major components of Splunk as individual services, and, 2) allowing external developers to build on top of Splunk and leverage our award-winning IT search infrastructure. Starting with version 3.2 (you can download the preview version today), there is a new REST API that provides unprecedented access and consistency to every aspect of the Splunk Server. We are

Hey all,

It’s taken longer than we would have liked but our 3rd preview build has been posted.
Get’um here

A bunch of work has gone into windows stability, tons of bugs were fixed, and a bunch of customer requests have been implemented ( we will let you know out of band ). We expect that this release should be more stable, slightly faster, and less buggy.

Left to do, we still have a bunch of IE work, performance improvements, and cleaning up of some features like interactive field extraction and event type discovery.

Its still not production ready so don’t even think of trying it out for real - and there is no guarantee that migration will work from a preview to GA ( we will migrate from 3.1.x to GA but not preview ). Also, don’t run splunk as root - its just not good to do until we run through all our testing.

As always, please send us feedback at splunkpreview@splunk.com or hit us up on IRC (irc.efnet.org #splunk).
The last round of info from Preview #2 was awesome please keep it up!

e.

With the new release of Splunk Preview out, I’ve run into a problem keeping the different versions straight on my laptop. I have the free version, the Preview, the official release, and a version of current running - often times simultaneously. It’s getting messy.

What you really want to do is refer to them with different subdomain names, where something like http://splunkpreview.mydomain.com/ would bring up Splunk without having to remember the port number.

If you are running Apache, (like I am on Leopard) you get a reverse proxy server for free. With just a few lines of configuration, you can alias subdomains (or domains for that matter) to your heart’s content.

You also get the ability of putting content behind some basic authentication provided via Apache’s HTTP auth methods. This comes in handy if you’d like to link to your Splunk install from a publicly facing page, but don’t want people to know what type of content is behind the authentication. It also works for limiting access to a particular IP address group or domain.

I’ve put together a screencast covering how to do this from OS X’s version of Apache. Click on the thumbnail below to play the screencast.

Happy new year (bit early) all dev.splunk.com readers….
We have just posted our second 3.2 preview release. (build number 30455)

Its packed with holiday goodness, albeit very raw.

First you will notice we have posted a windows build. Its been in the cooker since last Feb and thanks to Mitch, Ledio, Igor and a bit of Amrit we now have a single code base that rocks on linux, mac, solaris, freebsd, aix, AND windows. This was not an easy feat as evidenced by our gift of a pony (soft and electronic) to Mitch for his effort. Its still very raw (the build not the pony), and has a tendency to crash because of a memory fragentation and limited vm space. Which will be fixed by GA… MarkB. will post more on the build so stay tuned for details. Its a big deal for us so be patient and we sure could use feedback on how to make it the best it can be.

Also in this release you will see the UI starts to get some of the async search results. Over the next few releases we will be moving to fully async search in the UI. It will take a few turns but this preview has some of the first cut.

There are a bunch of other improvements; scheduled searches got a bit of a cleanup in the UI and the backend has been improved as well. Performance, bugs, and other tweaks are also spread throughout. I’ll get others to post specifics.

In the mean time, as always its a huge help to us in dev if you can kick the tires before we freeze for GA. Please send feedback to splunkpreview@splunk.com, post comments to this blog, or drop by and tell us in person.

Again, thanks for the help

Last week I made a video about how to setup new roles in Splunk 3.2 preview release. The video will demonstrate creating a new type of power user, with the same capability of a standard power user, and the addition of the ability to manage and create new users. You will also see how to create new roles by configuring authorize.conf.

(Update): While watching the video again and realized I sent a mixed message about where to edit configuration in splunk. I made it clear that you want to edit in the local bundle directory, and if you look at the terminal that is where I was editing my configuration, however, I later said “default over-rides local, so always edit in default”, this is WRONG. Always make your personalized configuration changes in the local directory, if the configuration file doesn’t exist there, create one or copy it from default and edit that one.

Take a look at the video and let me know if you have any questions about this stuff.

Quicktime Video (625×352)

For those interested in monitoring real-time data being consumed by Splunk we’ve introduced a new feature called Live Tail to the latest preview release. Additionally, we’ve added a nifty new REST endpoint /v3/splunk/tail for your custom application needs.

More information can be found in these videos:

• A quick walkthrough of the new preview release feature Live Tail, its UI, and some sample code - See Video
• An overview of the architecture used to integrate real-time data from Splunk Live Tail in a web browser. Challenges and workarounds when using JavaScript/Flash hybrids - See Video

Happy Streams!

Preview is out the door, woohoo! So up here in support I’m busy with the existing versions so I hadn’t checked out many of the new features. I wanted to mess with real data I care about, so I figured I’d copy my existing index and drop it into my splunkpreview directory. I host a handful of domains at home (on Leopard Server) and I’m using Splunk to watch various things I want to know, like who’s commenting on my blog and how many dictionary attacks I’ve had today. I thought it would be nifty to look at the same data in both 3.1.3 (my current production version) and preview.

The first time I tried it, I thought I’d be clever and set it all up before first startup with my whole index, users, saved searches and basically everything. Because, well, I clone this stuff all the time between 3.1.x versions when I’m setting up repro environments for customer issues. Wrong! Not sure what I forgot, but for my efforts I got a nice big segfault. Well, nothing a little rm won’t fix.

Take two. This time I installed preview plain and made sure it was up and running. I made myself a new admin user, deleted the default admin/changeme one so it was set up (mostly) like my original. Then I shut down both instances. I did clean all on the new one (very important to confirm you are in the right window!) and then copied over the contents of $SPLUNK_HOME/var/lib/splunk/defaultdb and fishbucket. defaultdb because that’s my main index and fishbucket so (in theory) when I start indexing it will pick up where it left off and not forget what happened before. I also moved inputs.conf, props.conf, transforms.conf and savedsearches.conf. Copying over savedsearches only worked because I knew my

Hi Kids,

So we have added in flexible roles into the preview release. Well, what does that mean.
We will now allow folks to create their own roles. The previous ones of Admin, Power
and User will be included as defaults.

There is currently no GUI available for editing roles but you can directly edit the
config file $SPLUNK_HOME/etc/bundles/default/authorize.conf.

To add in these roles we did an audit of our system and broke down various actions
into capabilities. These capabilities can be grouped together to create any role.
Please bear with us here, this is just a first cut and we may not have chopped up
things in a way that makes sense to you. This is the beauty of preview, you got a suggestion
about capabilities you’d like to see added or removed then comment or mail us.
The more feedback we get at this stage the faster this feature will improve.

A role in the splunk system contains the following things.
1. A list of capabilities that role can perform.
2. A list of roles that are contained within this role ( their capabilities will be imported into our role)
3. A list of search filters that should be applied when searching as this role.

Below demonstrates how to define a role called kwyjibo that can edit users information and
make changes to the authentication system. It imports in the capabilities of the roles User and Power.

[role_kwyjibo]
edit_user = enabled
change_authentication = enabled
bounce_authentication = enabled
importRoles = Power;User
srchFilter =

If you have any questions, comments please let me know.

Rory