The overwhelming consensus during the design phase was that the MVC design pattern was preferable for working with Splunk configuration objects.

This decision was based on experience gleaned from a previous app setup prototype that had used a Splunk module to provide app management. This proved to be difficult for several reasons:

• Module controller methods do not have access to decorators such as @route, making multi-step workflows and even success/failure notifications overly complex.
• By design, module controllers can only handle GET requests, whereas our setup controller must handle GET and POST
• Module controllers and JavaScript are designed to facilitate asynchronous interactions with a Splunk context, search, and job rather than to configure configuration objects using HTML forms

Thus, rather than try to fit a square peg in a round hole, a better approach was to create a custom controller and templates in order to facilitate supportable and reusable code.

The file that contains the WISetup controller class definition is located at:

    $SPLUNK_HOME/etc/apps/webintelligence/appserver/controllers/WISetup.py

The following section drills down into the Web Intelligence setup workflow controller code. The comments below do not appear in the actual controller file, but are provided for illustrative purposes.

Imports

NB: Standard library imports should be listed first, followed by third-party imports, followed by local imports. Following this standard makes imports easier for others to understand.

# we will need to create a logger below
import logging

# cherrypy is imported for cherrypy.session
import cherrypy

# import the generic controllers
import splunk.appserver.mrsparkle.controllers as controllers

# import the @expose_page() decorator to expose controller methods
from splunk.appserver.mrsparkle.lib.decorators import expose_page

# import the EventType model
from splunk.models.event_type import EventType

Globals

# set up a logger object - Splunk is for logs, after all
# the name of the logger dictates how logging is handled
# in the case below, logging will be normal verbosity to web_service.log
logger = logging.getLogger('splunk.appserver.controllers.WISetup')

# these are the five eventtype stanza names that must be 
# defined for the webintelligence app to function properly
required_keys = ['web-traffic', 'clientip-internal', 'internal-domain',
                 'brand-name', 'exclude-pageview']

Instantiation

# the controller inherits the methods of the BaseController
# the controller root will be at: /custom/webintelligence/WISetup/
# verify what methods are available via the /paths endpoint
class WISetup(controllers.BaseController):
    '''Web Intelligence Setup Controller'''

Methods

Since the requirements dictate that setup workflow should only expose read and update actions, only two public methods are required.

show

The show() method is exposed via the @expose_page() decorator at /custom/webintelligence/WISetup/show.

The show() method is responsible for building our form_content, which is a keyword dictionary of the unique eventtypes that the template setup.html requires as template_args.

The form_content will be passed to the Mako template setup.html as the template is rendered via render_template().

    # @expose_page() ensures only authenticated GET requests will reach this endpoint
    @expose_page(must_login=True, methods=['GET']) 
    
    # even though show does not use keyword arguments, accepting them by 
    # default tends to be a better practice than risking an exception
    def show(self, **kwargs):

        # empty dictionary
        form_content  = {} 
  
        # the cherrypy.session provides us with the requesting user name
        user = cherrypy.session['user']['name'] 

        # iterate through each key
        for key in required_keys:

            # use try/except in case one of the default eventtypes is absent
            try:

                # assign an EventType member to the form_content dictionary
                form_content[key] = EventType.get('/servicesNS/%s/%s/saved/eventtypes/%s' %\
                                                  (user, 'webintelligence', key)) 
 
            # a blank search will suffice if one of the default eventtypes is absent
            except:
                form_content[key] = {'search': ''} 

        # call render_template() using the template setup.html with template_args 'form_content'
        return self.render_template('/webintelligence:/templates/setup.html', 
                                    dict(form_content=form_content))

save

The save() method is used to handle updating the app's configured eventtypes when a POST is submitted from setup.html, as rendered by the show() method.

    # @expose_page() ensures only authenticated POST requests will reach this endpoint
    @expose_page(must_login=True, trim_spaces=True, methods=['POST']) 

    # **params provides the method with params, a keyword dictionary representing 
    # the parameters included in the POST request to this controller and method
    def save(self, **params):

        form_content = {}
        user = cherrypy.session['user']['name'] 

        # two loops are used, in case loading the models fails now or passive_save() fails later
        for k, v in params.iteritems():
           
            # use try/except, as non-period-delimited keys can be ignored by this handler
            # we know this because we used the following naming standard in the template
            #         <input type="text" name="${eventtype.name}" ... >
            try:
                key = k.split('.')[1]
            except IndexError:
                continue 
  
            if key and key in required_keys:
                try:
                    # load the given eventtype
                    form_content[key] = EventType.get('/servicesNS/%s/%s/saved/eventtypes/%s' % \
                                                      (user, 'webintelligence', key)) 
                   
                    # if two form elements have the same name, the controller receives a 
                    # list of unique values from cherrypy, which the internal helper method
                    # _parseListParam() is expecting (rather than a string)
                    if isinstance(v, list):
                        search = self._parseListParam(key, v)
                    else:
                        search = self._parseListParam(key, [v])

                    # if we successfully parsed out the user-supplied value for
                    # the eventtype, we need to store it in the model, but we don't
                    # want to update the entity until all params have been parsed
                    if search:
                        form_content[key].search = search

                # an exception would be unusual here, so capture any that occur
                except Exception, ex:
                    logger.debug(ex)
                    logger.error('Failed to load model for eventtype %s' % key)
 
                    # render the failure.html template, passing the specific eventtype
                    # name that raised the exception as well as the 500 response code
                    return self.render_template('webintelligence:/templates/failure.html',
                                                dict(name=key), 500) 

        # try to save, and on error, give the form back with pre-populated values
        for key in form_content.keys():
 
            # passive_save() is preferred to save() because it will not raise
            if not form_content[key].passive_save():

                # if passive_save() returned False, we return the setup template
                # with the form_content as well as the name of the specific key
                # that failed to save.  The template will interrogate the form_content
                # to discover any error messages that were placed in the offending
                # model's self.errors member in order to render the errors 
                return self.render_template('/webintelligence:/templates/setup.html', 
                                            dict(name=key, form_content=form_content))

        logger.info('Save successful')

        # if we got here, we deserve a beer, so render success.html
        return self.render_template('/webintelligence:/templates/success.html') 

In the event of a failure to save one of the eventtypes, the same template (setup.html) is rendered with an additional template_arg that specifies the name of the eventtype which could not be saved. The template is designed to render any error messages stored in form_content[key].errors:

NB: CherryPy controllers are instantiated at startup rather than on a per-request basis. Thus, any change to the controller code will require a restart of Splunk so that the controller instance can be reloaded. For this same reason, custom controllers should be as resilient as possible in handling exceptions upon instantiation.

/paths

If you need to make sure that your controller has started properly, check the /paths endpoint. If you do not see your controller advertised there after a restart of Splunk, then it has failed to load. To troubleshoot a custom controller, use web_service.log and web_access.log.

web_service.log

    $SPLUNK_HOME/var/log/splunk/web_service.log

Example of normal web_service.log custom controller entry:

2011-07-05 15:04:02,545 INFO	[4e138a51a31d4c7490] custom:210 - Registering custom app endpoint: webintelligence/WISetup

Example of web_service.log custom controller error entry:

2011-07-05 14:52:23,814 ERROR	[4e138797401a901490] custom:199 - cannot load specified module WISetup in app webintelligence

web_access.log

    $SPLUNK_HOME/var/log/splunk/web_access.log

Example of web_access.log custom controller GET request and 200 response:

10.3.1.69 - - [05/Jul/2011:20:49:15] "GET /en-US/custom/webintelligence/WISetup/show HTTP/1.1" 200 2292 "https://localhost:8000/en-US/app/webintelligence/setup" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/5.0.1" - 4e41ffbb481afc8450

Example of web_access.log custom controller POST request and 200 response:

10.3.1.69 - - [05/Jul/2011:20:49:15] "POST /en-US/custom/webintelligence/WISetup/save HTTP/1.1" 200 2401 "https://localhost:8000/en-US/custom/webintelligence/WISetup/show" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/5.0.1" - 4e41ffc4051b07fa90

Example of web_access.log custom controller GET request and 500 response:

10.1.5.148 - - [05/Jul/2011:16:42:16] "GET en-US/custom/webintelligence/WISetup/save HTTP/1.1" 500 2584 "https://localhost:8000/en-US/app/webintelligence/setup" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" - 4e14f2d7fd490db50

web.conf

In order for the custom controller to be instantiated and routed when Splunk Web is started, the following entry must exist in webintelligence/default/web.conf:

    [endpoint:WISetup]

NB: A restart of Splunk is required after initial installation of the Splunk for Web Intelligence App in order to bring up the setup controller.

Chrome

One of the requirements for the setup workflow is to retain the Splunk for Web Intelligence App's chrome:

The chrome is generated by the Splunk view system using Splunk modules AppBar and NavBar.

As such, the controller template needed to be rendered inside a module. IFrameInclude was the obvious choice. From IFrameInclude.conf:

[module]
# The JavaScript name of the module
className = Splunk.Module.IFrameInclude

# The module class to subclass from
superClass = Splunk.Module

description = This simple module uses an inline frame (iframe) to show content from 
              another URL.

[param:src]
required = True
label = This is the URL to a remote resource to be displayed in the module. Supports 
        remote URI's (ie., http://foobar.com/hello), local app static files (ie., 
        hello.html found in $SPLUNK_HOME/etc/apps/$APPNAME/appserver/static) and 
        relative locations (ie., /manager).

The setup view was configured using the following View XML:

<view template="search.html">
  <label>Setup</label>
  <module name="AccountBar" layoutPanel="appHeader"/>
  <module name="AppBar" layoutPanel="navigationHeader"/>
  <module name="Message" layoutPanel="messaging">
    <param name="filter">*</param>
    <param name="clearOnJobDispatch">False</param>
    <param name="maxSize">1</param>
  </module>
  <module name="Message" layoutPanel="messaging">
    <param name="filter">splunk.search.job</param>
    <param name="clearOnJobDispatch">True</param>
    <param name="maxSize">1</param>
  </module>
  <module name="TitleBar" layoutPanel="viewHeader">
    <param name="showActionsMenu">False</param>
  </module>
  <module name="IFrameInclude" layoutPanel="viewHeader">
    <param name="src">/custom/webintelligence/WISetup/show</param>
  </module>
</view>

This XML renders the following HTML - notice the IFrameInclude parent to the iframe with src="/en-US/custom/webintelligence/WISetup/show".

The app's setup workflow uses a custom controller, but the controller needed to be embedded in a normal Splunk view in order to preserve the app chrome.

The setup template rendered by the controller includes custom JavaScript that enables admins to add and remove HTML form elements dynamically. In some cases, adding and removing elements resulted in parts of the rendered template overflowing out of view.

To deal with these issues in the embedded iframe, the application.js file was used to bind events occurring within the iframe to a resize handler for IFrameInclude.

The following JavaScript was created in webintelligence/appserver/static/application.js:

    1    switch (Splunk.util.getCurrentView()) {
    2        case "setup":
    3        if (Splunk.Module.IFrameInclude) {
    4            Splunk.Module.IFrameInclude = $.klass(Splunk.Module.IFrameInclude, {
    5                onLoad: function(event) {
    6                    this.logger.info("IFrameInclude onLoad event fired.");
    7     
    8                    this.resize();
    9                    this.iframe.contents().find("body").click(this.resize.bind(this));
   10                },
   11       
   12                resize: function() {
   13	                this.logger.info("IFrameInclude resize fired.");
   14	    	    
   15		        var height = this.getHeight();
   16		        if(height<1){
   17 		            this.iframe[0].style.height = "auto";
   18 		            this.iframe[0].scrolling = "auto";
   19		        }else{
   20		            this.iframe[0].style.height = height + this.IFRAME_HEIGHT_FIX + 20 + "px";
   21		            this.iframe[0].scrolling = "yes";
   22		        }
   23           
   24                }
   26    
   27            });
   28        }
   29    }

In the code above, we ensure that changes we introduce only affect the 'setup' view by using getCurrentView() and the JavaScript 'switch' statement.

Next, we check each module's namespace, and if it is in fact an IFrameInclude, we use JQuery.klass() to create a new, unique instantiation of the IFrameInclude class.

We then override the onLoad() method of the IFrameInclude, essentially moving the meat of the original function to a new resize() method and using the onLoad() handler to bind click events within the iframe body to our new resize() method.