Logging examples

This topic provides examples of what to do and what not to do when creating events for Splunk software to index.

Use clear key-value pairs

In this example of what not to do, the term "error" is too vague and the keys must be assigned to the values because they are not provided:

BAD:

Log.debug("error %d 454 - %s", userId, transId)

In this improved version, the event is easier to parse because the key-value pairs are clearly provided. Searching on "orderstatus=error" will retrieve exactly the events you want. Also, you can query Splunk for reports that use orderstatus, such as requesting its distribution (for example, completed=78%, aborted=21%, error=1%), which is something you couldn't do if you only had the keyword "error" in your log event.

GOOD:

Log.debug("orderstatus=error, errorcode=454, user=%d, transactionid=%s", userId, transId)

Break up multi-value information

Parsing this multi-value event is difficult and so is adding data for each value of app:

BAD:

<TS> phonenumber=333-444-4444, app=angrybirds,facebook

This improved version breaks multi-value information into separate events, so the key-value pairs are more clear:

GOOD:

<TS> phonenumber=333-444-4444, app=angrybirds, installdate=xx/xx/xx
<TS> phonenumber=333-444-4444, app=facebook, installdate=yy/yy/yy

Use headings as keys

You can use headings as keys, as shown in the following example. Splunk software can interpret the column headers as keys and each line as values (although this does break the rule about avoiding multi-line events):

<TS>
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
Root 41 21.9 1.7 3233968 143624 ?? Rs 7Jul11 48:09.67 /System/Library/foo
Rdas 790 4.5 0.4 4924432 32324 ?? S 8Jul11 9:00.57 /System/Library/baz

Use multiple unique IDs to connect events

If you can't use one unique identifier, use a transitive connection from one event to another. For example, here are three separate events:

Event A: transid=abcdef
Event B: transid=abcdef, otherid=qrstuv
Event C: otherid=qrstuv

You can associate Event A with Events B and C, because of the connection between the two IDs in Event B.