You can use Splunk Enterprise, Splunk Cloud, Splunk Light, or Splunk Cloud Light (collectively, "Splunk software") immediately in your everyday development activities to help improve the quality of your applications and reduce the time it takes to resolve issues. With Splunk software you can quickly and easily search your log files. You can also make some slight code changes to improve the readability of your logs across the entire organization.
Logs can be a hassle to deal with: they don't have a real structure, you might not know what's in them if you didn't create them yourself, and the sheer number of files and types of logs can be overwhelming. On the other hand, logs are really useful: they contain a gold mine of analytic information, they can help you find problems, and they can tell you about your IT infrastructure, the behavior of your users, and identify potential attackers.
Chances are, you're already using logs for debugging purposes. But you can do more with your log files. Keep reading to learn how to get the most from your logs and how to set them up right. The results are worth it.
There are two types of logging:
Typically, IT organizations develop systems that focus on specific silos of technologies, functions, people, or departments. Individually, these silos provide a limited view. But they all generate data—data that is non-standard, unstructured, high-volume, and growing every millisecond of every day. Whatever you're logging, every environment has some set of IT data to leverage:
Often, companies must write their own applications from scratch to perform statistical analysis on the data contained in these events. Similarly, companies build dashboards by hand to display that analysis for historical comparisons, future projections, performance tuning, billing, security, and so on.
Splunk software doesn't care about the format or schema of your data—queries and searches can be ad-hoc, and your data can come from any textual source. With Splunk software, you can use the logs generated from your application to achieve the same powerful analytics without building special-purpose software. Logging semantic data in addition to debugging data can significantly enrich virtually all applications.
The following stub code shows a purchase transaction:
void submitPurchase(purchaseId) { log.info("action=submitPurchaseStart, purchaseId=%d", purchaseId) // These calls throw an exception on error: submitToCreditCard(...) generateInvoice(...) generateFullfillmentOrder(...) log.info("action=submitPurchaseCompleted, purchaseId=%d", purchaseId) }
This pseudo-code logs the purchase ID for each action. With the two events generated from the above pseudo-code, you could learn a lot about your purchase transactions. You could:
Each of these items is a simple search in Splunk software that you can save, schedule, and share by e-mail. And there is no need to architect a complex solution—the more data you log, the more you get out of it. It's really that simple. Use Splunk software instead of complex and inflexible architectures like RDBMS/SQL.
It's important to set up your logging in the right way to get the maximum benefit with your application. Start by thinking about how other people will view your logs. Events might get separated from the logs. Certain events might get copied and passed along to a variety of people inside and outside of an IT organization. Without using robust logging techniques, you can lose meaningful data, making it harder to derive insight from the logs. The logging best practices that follow can help make your debugging logs more relevant and powerful for you and others.
Now, let's get into the specifics with Logging Best Practices.