This topic provides examples of what to do and what not to do when creating events for Splunk software to index.
In this example of what not to do, the term "error" is too vague and the keys must be assigned to the values because they are not provided:
Log.debug("error %d 454 - %s", userId, transId)
In this improved version, the event is easier to parse because the key-value pairs are clearly provided. Searching on "orderstatus=error" will retrieve exactly the events you want. Also, you can query Splunk for reports that use orderstatus, such as requesting its distribution (for example, completed=78%, aborted=21%, error=1%), which is something you couldn't do if you only had the keyword "error" in your log event.
Log.debug("orderstatus=error, errorcode=454, user=%d, transactionid=%s", userId, transId)
Parsing this multi-value event is difficult and so is adding data for each value of app:
<TS> phonenumber=333-444-4444, app=angrybirds,facebook
This improved version breaks multi-value information into separate events, so the key-value pairs are more clear:
<TS> phonenumber=333-444-4444, app=angrybirds, installdate=xx/xx/xx <TS> phonenumber=333-444-4444, app=facebook, installdate=yy/yy/yy
You can use headings as keys, as shown in the following example. Splunk software can interpret the column headers as keys and each line as values (although this does break the rule about avoiding multi-line events):
<TS> USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND Root 41 21.9 1.7 3233968 143624 ?? Rs 7Jul11 48:09.67 /System/Library/foo Rdas 790 4.5 0.4 4924432 32324 ?? S 8Jul11 9:00.57 /System/Library/baz
If you can't use one unique identifier, use a transitive connection from one event to another. For example, here are three separate events:
Event A: transid=abcdef Event B: transid=abcdef, otherid=qrstuv Event C: otherid=qrstuv
You can associate Event A with Events B and C, because of the connection between the two IDs in Event B.