We're planning a journey, and like any real journey we need to know who's coming with us, have an idea of where we want to get to, and what some of the choices along our way will be. We need to make some plans before we start. This journey is not a journey to a place, but the journey we took to learn how to build Splunk® apps in the real world.
This chapter introduces the companies that will collaborate on this journey, the people from those companies who will come along with us, and most importantly the Splunk apps that we intend to build as the goal of our journey. This chapter outlines some of the key features and functionality that we plan to include in the finished apps and introduces some of the issues we are considering right at the start of our journey. The chapters that follow enable you to follow along as we continue on this journey to build our Splunk apps.
This guide describes the journey undertaken by the project team and is intended to give you an insight into how to set about developing Splunk apps, to recommend some good and proven practices, and to provide a useful reference when you come to build your own Splunk apps. However, like on any real journey, we make mistakes, have arguments, and change our minds along the way; so in addition to showing you how best to do things, we highlight the pitfalls and issues that we encounter, and the solutions we find. We do not intend this guide to cover everything related to developing Splunk apps, it focuses on the specific issues we faced in building our two sample apps. However, our two sample apps are representative of the typical apps Splunk customers do build.
The primary objective of our journey is to deliver applied engineering guidance along with Splunk apps that act as reference implementations. We hope this guidance will help to lower the entry barrier, reduce the time to market, and decrease the risk for you, our Splunk customers and partners. The following five principles characterize our guidance philosophy:
Each chapter of this guide covers a specific theme such as UI design or data onboarding and each follows the basic chronology of the journey as we develop our apps. The chapters also include commentary from various experts, descriptions of some of the team's working practices where they shed light on what happened, and discussions about why we made particular decisions. We hope this guide will mature through a dialog with you, our audience of Splunk app developers.
Three different companies collaborate on this journey: Splunk, Conducive, and Auth0. Conducive already uses Splunk Enterprise and has previously developed Splunk apps; Auth0 is new to Splunk Enterprise. Both Conducive and Auth0 have ideas about how they can use Splunk apps to add value to their own products and services and want to partner with Splunk to pursue those plans further. This means that the journey includes two separate development projects for two separate Splunk apps: one for Auth0 to enable insights into how users are interacting with the Auth0 service, and one for Conducive to build a Splunk app to assist in monitoring secure document repositories (for more information, see Our apps below). We built both of these Splunk apps using Splunk simple XML. The next chapter, Platform and tools: a kitbag for our Journey, discusses why we chose this specific technology in more detail.
Conducive is a system integrator with one of the focus areas being standards compliance. Their user audience includes business managers, compliance officers, auditors and investigators.
Some of the specific goals that the Conducive team has for their involvement in the project are to learn how to:
Auth0 is an Identity-as-a-Service infrastructure provider. Their user audience is primarily developers.
Some of the specific goals Auth0 has for their involvement in this journey are to enhance their monitoring and reporting functionality with a Splunk app.
Their learning objectives include:
A panel of experts from a number of roles follow the journey, and the following chapters include their comments on the development effort. The roles these experts perform are:
|A developer, expert in using Splunk and developing Splunk apps. He wants to see a well written Splunk app that follows proven practices.|
|An architect who wants to see a well-architected solution.|
|A UX designer who wants to see an engaging, intuitive, easy-to-use UI for the app.|
|A tester who wants to ensure the highest possible quality for the reference implementations.|
|A system administrator who wants to know how to deploy and monitor the apps, and understand how to configure them.|
|A representative of the business who knows how the apps will be used and their expected benefit to the business.|
|A release manager who wants to ensure that the apps are packaged in a way that makes them easy to deploy and to update in the future.|
|A security expert who wants to ensure that the apps comply with any security requirements in the target environment.|
|A performance tester who wants to ensure that the apps performance is acceptable with production loads.|
During the journey, the team develops two apps, one with Conducive and one with Auth0. This book focuses primarily on the Conducive app but discusses the Auth0 app where it provides a contrast or highlights different issues. The following two sections in this chapter provide overviews of the apps as we envision them at the start of the project.
If you're planning to list your app on the Splunkbase website, you should follow the naming guidelines at dev.splunk.com/goto/namingguide.
The PAS Splunk reference app is intended to enable an organization to monitor a highly secure document repository such as one hosted on Box.com. Conducive wants the organization to use the app to see who has looked at, updated, modified, deleted, or downloaded documents in the repository. The app should enable the organization to be both reactive and proactive in its approach so that in addition to being able to see what happened, the app notifies the organization of potential security issues when it detects unusual behavior associated with either a user or a document.
Potential users of the app include anyone with responsibility for protecting sensitive data. We expect the end users of the app to be non-technical business users such as:
These end user roles are typically non-technical, and they should be able to use the app without technical help: an intuitive UI is important.
In addition to the end users, there is a technical role for configuring the app with rules for triggering alerts specific to the organization and its requirements. Some organizations may configure these rules once when they deploy the app, others may have the requirement to be able to update these rules in response to changes in the business environment or to emerging threats. This may require some basic technical knowledge of Splunk Enterprise. The app should include a set of basic default rules to get started.
The app should include the following five key elements:
These were our goals at the start of the journey. The finished apps do not incorporate everything we envisaged, primarily because we ran out of time and resources needed to implement everything in our backlog.
At the start of the project, Auth0 is using Splunk Enterprise to monitor some basic events related to users of the Auth0 services. For example, whenever a new user signs up with Auth0, the Auth0 service sends event data to a Splunk REST endpoint. This event data includes information such as the event type, the target app, the client IP address, and the userid. Auth0 then display this information in a simple dashboard.
Auth0 would like to build a Splunk app that will replace the current solution and have the following features:
Auth0 are also considering making a Splunk app available to their customers to enable them to track business metrics such as:
You check the details of your Splunk Enterprise license if you are planning to provide your customers with reports and data from your Splunk Enterprise installation.
The app should display this information in way that's attractive to customers.
The following chapters describe what happened on our journey, with each chapter focusing on a high level theme. Each chapter covers the issues the team encountered in tackling the various stories and use cases, how they resolved those issues, the decisions they made, and the lessons they learned. You'll also learn more about the specifics of the Splunk apps we have introduced in this chapter. Some chapters will also include sections that describe how the team works or arrives at particular decisions.
The following flowchart illustrates the typical workflow for developing a Splunk app. Not all projects will follow these steps exactly, but they provide a rough outline of the key stages for most Splunk app projects:
To start, look at a sample of the real business data and begin to understand what information is available.
Then you must work out how to get that data into Splunk. Options include, using file monitors, open ports, scripted inputs, or modular inputs. The chapter "Working with data: where it comes from and how we manage it" discusses these options for the PAS and Auth0 apps.
During the development process use simulated data so you are not reliant on a third-party system while you are building the app. Typically, this helps you develop and test faster. The chapter "Platform and tools: a kitbag for our journey" discusses this.
Use Splunk Enterprise features such as transforms to manipulate the incoming data to make it easier to search. The chapter "Working with data: where it comes from and how we manage it" discusses these options.
Create a skeleton app using the Splunk Enterprise web UI or the command-line tools. You can then add dashboards and visualizations to this app. This is discussed in the chapter "Platform and tools: a kitbag for our journey."
Run the basic searches and add them as panels to the dashboards in your skeleton app. This is discussed in the chapter "Platform and tools: a kitbag for our journey."
Prepare your app for deployment by adding features such as user roles and permissions. The chapter "Packaging and deployment: reaching our destination" discusses these options.
Our team has a variety of technical skills relevant to the project that include:
Some of the team also have experience of building Splunk apps using technologies such as Splunk SPL and the simple XML development model for apps.
Note that code samples and screenshots shown in the following chapters do not necessarily match the code and UI in the completed apps. The samples show the code and UI at that particular point in the journey, and in some cases they may be modified at a later stage in our journey. In particular, some screenshots and code sample refer to the warum app; this was our original internal code name for the app that was changed near the end of the project to pas.
The majority of the screenshots in this guide were taken on a Windows machine, in some cases where there is a significant difference we show a *nix screenshot as well.