Visualizing data

Now that you've gotten your data in and searched the data to extract the information most relevant to you, you want to view the results. Moreover, you want to represent and organize the results in a way that visually gives you the most insight about your data.

"What is to be sought in designs for the display of information is the clear portrayal of complexity. Not the complication of the simple; rather the task of the designer is to give visual access to the subtle and the difficult - that is, the revelation of the complex."
    ~ Epilogue to The Visual Display of Quantitative Information by Edward R. Tufte

How can you leverage Splunk Enterprise to help you get insight into your data set? Even with the most carefully optimized search, you'll likely need more than tabulated search results. In this chapter, we'll explore some of the visualization options available to you as an app developer. These range from basic but powerful built-in tools for organizing and rendering your data to custom-built or third-party programs for displaying your data in the most useful way. The method you choose depends mainly on domain-specific knowledge, user interface (UI) requirements, and data representation complexity, all of which are likely the main reasons you're building a custom app.

Depending on your needs and the complexity of your app, you will need at least a working knowledge of the front-end technologies: HTML, JavaScript, and CSS. Minimally, you'll also need to know how to work with Simple XML. If you're doing more advanced programming, you should be familiar with the SplunkJS Stack and the Splunk SDK for JavaScript. For full app customization flexibility, you can use open source libraries and APIs you might already be familiar with, like jQuery, backbone.js, and D3.

Also, be aware that we make a distinction between UI and visualization throughout this chapter. A visualization is a representation of your data in either graphic or text form, whereas the UI provides the visual container in which your visualization is rendered, and might include interactive functionality if it's needed. For example, the following is a screen shot from the Splunk Dashboard Examples app. This app includes many dashboards like this one, with a visualization (in this case, a bubble chart) at the top of the dashboard and, further down in the UI, the code that makes it work.

Example visualization

Choose a UI technology

The following table shows the technologies available to you as an app developer. In order of complexity, the list runs from tweaking dashboards by using an interactive editor or editing the underlying Simple XML, to full programmatic development using HTML, CSS, and JavaScript. We highlight the pros and cons of each approach.

UI technology

Mechanism

Recommendation with description

Simple XML-based dashboards

Simple XML
XML file

Start here.

Lets you build simple dashboards. Splunk Enterprise includes interactive tools for creating and modifying dashboards, and an XML editor that you can use to edit dashboard definition markup.

This approach is ideal for simple apps that require only basic customization and that do not require complex navigation between different dashboards.

Simple XML-based dashboard extensions

Simple XML with JavaScript or CSS extensions
XML file

When you reach the limits of Simple XML, add JavaScript and CSS into your arsenal. 

Extends the capabilities of Simple XML by adding more layout options, visualizations, and custom behaviors.
This is useful for adding additional capabilities to a Simple XML dashboard, but it adds to the complexity of the app because your app now includes custom JavaScript and CSS resources. JavaScript libraries can include the SplunkJS stack, third-party libraries, or your own custom JavaScript code.

HTML dashboards

Simple XML converted to HTML with custom JavaScript
HTML panel in XML file

Not recommended. 

Converts a Simple XML dashboard to HTML, and then adds JavaScript. This gives you full control of the rendering of the page.

You should use caution with this method due to serious maintainability concerns: The generated dashboards end up being specific to the Splunk Enterprise version on which the dashboards have been generated, and therefore will likely not be future-proof.

Also, this is a one-way conversion. That is, you can't go back to Simple XML once you've converted to HTML.

You'll most likely start with Simple XML because it's the easiest way to begin, particularly for prototyping an interface. It has enough functionality and power to let you design your interface before you need to move to more advanced techniques. In fact, the UI and visualization requirements of many applications are often met using Simple XML alone. An example of an application that uses only Simple XML is the Auth0 app described in the Journey. The PAS reference app from the Journey, however, required user interaction and data representation that made it necessary for its developers to extend the Simple XML implementation with custom JavaScript and CSS.

Start with Simple XML by creating dashboards using the dashboard editor. Simple XML is the mechanism upon which dashboards are implemented. You can then modify the Simple XML for your dashboard using the Splunk Enterprise XML Source Editor. See "Build dashboards using Simple XML" in the "Dashboards and Visualizations Manual" to learn how to work with Simple XML. Refer to the Simple XML Reference in the same manual for a description of the Simple XML syntax.

You may want to take advantage of the built-in token passing and management mechanism. Searches in Simple XML dashboards use tokens as a variable to dynamically specify fields, field values, and search terms. Tokens are useful for capturing input values from forms, implementing dynamic drilldown operations, and specifying the conditional display of panels in a dashboard.

DEVIn a search, token name syntax uses $...$ delimiters.
 
 

If your application has more complex UI and visualization requirements, add JavaScript and CSS extensions to the dashboards you've created. JavaScript extensions are built using the SplunkJS stack (splunkjs/mvc/). The SplunkJS stack gives your app access to views and to the search manager. You can also use third-party JavaScript for even more custom functionality. For more information about using the SplunkJS stack, including reference documentation, see the Splunk Web Framework Component Reference.

Converting Simple XML to HTML to work on a dashboard directly in HTML, CSS, and JavaScript may offer you easier and more powerful customization. However, this approach is not recommended, because when you convert a Splunk dashboard to HTML, you can no longer edit it interactively within Splunk Web, nor can you generate PDFs or schedule delivery using Splunk Enterprise mechanisms. Use the conversion approach only when customization is needed that can't be achieved with either of the other mechanisms, but remember that your implementation might need to be updated with each major Splunk Enterprise release and might not be backward compatible. That is, the implementation will be specific to the Splunk Enterprise version for which it was implemented.

DEVYou can't reverse the development steps to generate Simple XML from your custom HTML. Because of these maintenance issues, if this much customization is needed, it might be better to build your app outside of Splunk Enterprise and use Splunk simply as a data store.

BUSIt is possible to export your dashboards into PDF. Find more information in "Generate dashboard PDFs" in the "Dashboards and Visualizations Manual."
 

A dashboard typically consists of panels, with each panel providing a different view of your data. You can choose a UI technology on a per-panel basis, so a dashboard can have a mix of panels created using only Simple XML and panels created using Simple XML with JavaScript or CSS extensions. We advise that you begin with the simplest implementation--using Simple XML--and progress to more advanced implementations only as needed.

Explore built-in visualizations

Splunk Enterprise comes with a number of built-in UI controls and visualizations defined in Simple XML. The following table lists some of the most common visualization elements and input types. All elements listed here are contained within form or panel elements within a dashboard. Simple XML also provides built-in support for enabling interaction between visualizations, such as specifying search parameters and drill-down dependencies. For an exhaustive list and detailed descriptions of all available elements, see the Simple XML Reference.

GENERAL ELEMENTS

label

Header text for a dashboard or form.

description

Text below a dashboard, form, or panel.

INPUT TYPES

checkbox

Checkbox input with option to populate labels and values with search and saved search results.

dropdown

Dropdown menu input with option to populate labels and values with search and saved search results.

multiselect

Multiple choice input with option to populate labels and values with search and saved search results.

radio

Radio button input with option to populate labels and values with search and saved search results.

text

Text input.

time

Time picker input.

PANEL ELEMENTS

chart

Search data displayed in a chart.

event

Search data displayed as individual events.

html

Inline HTML text display.

list

Data list display.

map

Geographic coordinates mapped as interactive markers on a world map. Choropleth maps are supported in Splunk Enterprise 6.3 and later.

single

Single value search result display with support for at-a-glance, single-value indicators with historical context and trend indicators.

table

Tabulated search results.

title

Specifies text for the title of a panel or for visualization elements.

Sparkline options

Sparkline display formatting options.

fields

Comma-separated list of fields display.


DEVFor a comprehensive collection of examples of each built-in visualization option, plus dozens of example dashboards, download and install the Splunk Dashboard Examples app. We've included an illustration of the most popular built-in visualizations here.

Example visualizations

Add custom visualizations to a dashboard

The built-in Splunk Enterprise data visualization capabilities might not fully meet the needs of your particular application. You might decide that your own visualization libraries, third-party libraries, or open source libraries would provide more pertinent ways to represent your data. Fortunately, Splunk Enterprise provides the ability to seamlessly integrate external JavaScript libraries with your client-side code. For example, some common third-party libraries that you might already find in use within apps on Splunkbase include D3.js and RaphaĆ«l.  Illustrated here is an example dashboard with a custom Sankey diagram visualization, from the Splunk Dashboard Examples app. This visualization uses a custom Sankey plug-in that utilizes jQuery and D3.js.

Another example visualization

The easiest way to integrate external JavaScript libraries is to call the library from your dashboard's Simple XML markup. While the implementation details can potentially be challenging depending on the complexity of your application and of the external library, you'll need to complete a few basic steps to use the library from your dashboard:

  1. As a starting point, it's helpful to use a visualization from an existing Splunk Enterprise dashboard example as a template and modify a copy of the example so that it calls the external library instead of the built-in visualization. Copy the template to $SPLUNK_HOME/etc/appserver/static/<myApp> and make the changes needed to call the external code.
  2. Next, you'll need to override the SimpleSplunkView.updateView method and implement your new visualization. Modify the updateView method to load the new libraries, use the visualization data received from the formatData method, write data to the target element, and make any desired CSS changes.
  3. Create or modify the target <div> or <svg> elements to hold the data.
  4. In the formatData method, reformat the search results to the format expected by your external visualization library. Copy the custom visualization code into the formatData method and make the changes needed to format search results into the visualization format. This data gets passed to the updateView method.
  5. Modify your CSS to make any desired styling changes to your visualization.

Make custom visualizations reusable

Suppose you want to use a custom visualization to give you an interesting view of your data that is not available with out-of-the-box Splunk Enterprise charts. The obvious solution is to create a new component that does exactly what you need for your application. The problem with this approach is the tendency to hard code definitions and functionalities to meet your immediate needs, which means that your nifty new component most likely can't be used by another application or even multiple times within the same app or dashboard. By designing and implementing the component for a single application, you lose the potential benefits you and your organization might have obtained from employing a few reuse principles.

The following reuse principles promote not only flexibility as a means for achieving reuse but also ease-of-use so the effort you put into building the component is more likely to be leveraged by other app developers.

Parameterization

Parameterize your component so the value of properties that are likely to differ among applications can be set at runtime, or as late as possible. Use late binding. Some examples are names of elements on a page like titles, and cardinality like number of rows and columns.

A prime candidate for parameterization is the name of <div> elements. Custom Splunk Enterprise components typically insert data into an empty <div> element in a panel. By parameterizing the name of the <div> element, you can use the name expected by the application. Parameterization also permits you to populate multiple <div> elements in the same panel. This also applies to CSS selectors like #viz-node_id, or to JQuery node names, depending on how the application is implemented.

Simplicity

Use good API design practices to define a simple interface for your component. This is admittedly subjective, but in general, the simpler the interface, the easier it is to reuse the component.

Dependencies

Each component should declare its own dependencies, such as underscore and any other components on which the custom component depends.

Location

Put your custom components in a dedicated location so they are easy to find and reference. One recommendation is to put your components in appserver/static/components/<componentPackageName> folders. Use the Node.js require method, and specify the correct component pathname.

Documentation

Reuse is greatly facilitated by having adequate documentation. Document usage, parameters, return values, and error conditions for the component constructor and all methods.

Comment your code to document the method interfaces. This keeps the documentation close to the code it describes, and is more likely to keep the documentation current with code changes.

We started this chapter with a quote from Edward R. Tufte's classic book on displaying data, and we think the following quote is a fitting summary for how you should think about designing your own visualizations.

"Excellence ... consists of complex ideas communicated with clarity, precision, and efficiency ...

  • show the data
  • induce the viewer to think about the substance rather than about methodology ...
  • avoid distorting what the data have to say
  • present many numbers in a small space
  • make large data sets coherent
  • encourage the eye to compare different pieces of data
  • reveal the data at several levels of detail ...
  • serve a reasonably clear purpose: description, exploration, tabulation, or decoration
  • be closely integrated with the statistical and verbal descriptions of a data set."
    ~ The Visual Display of Quantitative Information by Edward R. Tufte