Using splunk in Fedora9 x86_64

For those who use Linux as their primary desktop, using splunk can be a chore. Splunk dashboards are built on Flash9. So, you will likely need the following commands (as root, or sudo) to get Flash working.

  • rpm -ivh http://fpdownload.macromedia.com/get/flashplayer/current/flash-plugin-9.0.124.0-release.i386.rpm
  • yum install nspluginwrapper.{i386,x86_64} pulseaudio-lib.i386
  • yum install flash-plugin
  • yum erase rhythmbox.*
  • mozilla-plugin-config -i -g -v
  • mozilla-plugin-config nspluginwrapper -i /usr/lib/mozilla/plugins/libflashplayer.so

(Optionally, if you haven’t imported the Adobe GPG key, you will have to run the following command)

  • #rpm –import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

Splunk2LCD : Display your Alerts on an LCD

This morning I got a nice little LCD from Crystalfontz that allows me to connect to it via the open source project lcdproc. After a bit of compiling and installing, LCDproc (which runs natively on linux, darwin (osx) and most other unix distros) connects to any serial, parallel or USB LCD device. In this case, the Crystalfontz LCD is 4 line by 20 character display.

Splunk2LCD

Once configured and connected, you start the server and accept connections.

I then grabbed the IO-LCDproc perl module and modified it to display to the LCDproc server. You can get the IO-LCDproc through CPAN.

Here is the code that would go in your $SPLUNK_HOME/bin/scripts directory

[source:python]
#!/use/bin/perl -w
use IO::LCDproc;
use IO::Socket;
use strict;

&usage if (! $ARGV[0]);

my $client = IO::LCDproc::Client->new(host => “localhost”, name => “MYNAME”, port => “13666″);

my $screen = IO::LCDproc::Screen->new(name => “screen”);

my $title = IO::LCDproc::Widget->new( name => “date”, type => “title”);

my $first = IO::LCDproc::Widget->new(
name => “first”, align => “center”, type => “string”, xPos => 1, yPos => 2,
data => “test”);
my $second = IO::LCDproc::Widget->new(
name => “second”, align => “center”, type => “string”, xPos => 1, yPos => 3
);
my $third = IO::LCDproc::Widget->new(
name => “third”, align => “center”, type => “string”, xPos => 1, yPos => 4
);

Splunking your iPhone

Had a little fun last night. Enabled syslogd on the iPhone and sent the logs to a splunk instance via UDP/514

Process is hacking your iPhone and install ssh. Enable syslogd by the following method. (Thanks to core on #iphone)

syslog
20:00 so to get syslog running you need /etc/syslogd.conf from your mac
20:01 then break the syslog in /System/Library/LaunchDaemons/apple.com.syslogd by putting in bad values
20:01 then restart the phone and run 20:01 /usr/sbin/syslogd -bsd_out 1 &

Then edit /etc/syslog.conf and append *.* @loghost

Restart syslogd and you’re set.

Then just set splunk up to listen on 514/UDP and you have iPhone logs.

Interesting bit found? launchd, the service that starts up the daemons on the iPhone just keeps respawning services. The iPhone lacks a standard service control mechanism such as the sysv-compatible init process.

Rant on Syslogd

Syslogd really should either be modified or ditched for syslog-ng. As anyone who looks at logs knows, its crucial to have full, standard time stamps. This should include, HH:MM:SS:MS YYYY-MM-DD.Rfc3164 states :

5.1 Dates and Time

It has been found that some network administrators like to archive their syslog messages over long periods of time. It has been seen that some original syslog messages contain a more explicit time stamp in which a 2 character or 4 character year field immediately follows the space terminating the TIMESTAMP. This is not consistent with the original intent of the order and format of the fields. If implementers wish to contain a more specific date and time stamp within the transmitted message, it should be within the CONTENT field. Implementers may wish to utilize the ISO 8601 [7] date and time formats if they want to include more explicit date and time information. Additional methods to address this desire for long-term archiving have been proposed and some have been successfully implemented. One such method is that the network administrators may choose to modify the messages stored on their collectors. They may run a simple script to add the year, and any other information, to each stored record. Alternatively, the script may replace the stored time with a format more appropriate for the needs of the network administrators. Another alternative has been to insert a record into the file that contains the current year. By association then, all other records near that informative record should have been received in that same year. Neither of these however, addresses the issue of associating a correct timezone with each record.

IMHO, this is backwards. We shouldn’t require developers to put the year in the content field or have people post process logs to include the year.. Syslog should properly write out the year.

selinux and splunk

If you’ve enabled selinux for whatever reason, you need to either disable it or configure it to allow splunk to run.

To configure selinux to allow splunk to run, you need to run the chcon command on the splunk lib directory. Here is what you type :

chcon -c -v -R -u system_u -r object_r -t lib_t $SPLUNK_HOME/lib 2>&1 > /dev/null
You can also disable the check when splunk starts by adding this line to the $SPLUNK_HOME/bin/setSplunkEnv script

export SPLUNK_IGNORE_SELINUX=1

Telling Splunk to not phone home for update info.

(2.1.1 only)

We’ve had a few people ask for this. Its going to be in the documentation eventually, but until then here is how you do it.

Edit $SPLUNK_HOME/etc/myinstall/search.xml

Change :

<updateCheckerBaseURL>http://quickdraw.splunk.com/js/</updateCheckerBaseURL> <updateCheckerBaseURL>0</updateCheckerBaseURL>

(2.1)

$SPLUNK_HOME/share/splunk/search/static/js/update_checker_pro.js.

At the top of the file, and within that same setup function, comment out these two lines:
createUpdateCheckerScriptlet();
setTimeout(’possiblyFallBackToCannotConnectMessage()’, 5000);

Allowing users to log in with HTTP GET in 2.1x

I’ve had to field a few of these requests so here goes.

Assuming you understand that by doing this, you send users and passwords in clear text and the risks involved.

There is a way to do this through http GET, but it requires modifying a bit of python.

Edit line 395 of XMLResourse.py located in $SPLUNK_HOME/lib/python2.4/site-packages/splunk/search/XMLResource.py

def render_GET(self, request) :
# backdoor so scripts can auto-login just with a GET request instead of having to craft a proper HTTP POST. Doesnt help said script keep track of the cookie, which is the hard part.
#if (”usr” in request.args) and (”pwd” in request.args) :
# return self.render_POST(request)
logger.debug(”LoginResource.render_GET”)
sessNS = request.getSession().sessionNamespaces

Uncomment out the if and return lines and restart splunk.

To log in, you would enter this URL

http://your.host/login?usr=username&pwd=password