The journey continues: updating our equipment & dealing with OAuth

Is the journey ever really over? That's what we asked ourselves after successfully completing our first adventure and prioritizing what to achieve next. There was plenty more functionality we could add to the first version of the Splunk Reference app. 

It's time for the next leg of our journey: We're exploring the latest release of Splunk Enterprise and updating the tools we have available for building apps. Those of you who were with us during the first leg know that we didn't get to everything we originally planned. As with many software projects, the commissioning of a new release (labeled v1.5 in the code repo) allowed us to once again consider some of the use cases we had to leave behind.

Like our previous journey we'll focus on how we can use the enhanced features of the platform to extend the functionality of our sample app. Along the way we'll explain the paths we explored, the decisions we made, and the options we rejected. Much of the team remains the same, although you may detect a slightly different tone going forward. Our previous narrator wasn't able to join us this time, but we're sure you'll find our new one just as engaging. We also had an additional developer join the team; because the PAS app was architected to be modular in nature, he was able to quickly hit the ground running.

What's new?

In this chapter we'll explore integrating Google Drive more closely with the PAS app. Later chapters will discuss how to use alerting to trigger various actions and HTTP Event Collector to send high volumes of data directly to Splunk Enterprise.

Dealing with OAuth

For the first step on our new journey, we decided to revisit the Google Drive add-on that serves as a custom data provider. We weren't entirely satisfied with the current user workflow that required the user to have shell access in order to execute a Python shell script during the OAuth 2.0 refresh token pairing and access management setup process. Users may not be comfortable or familiar with the shell or executing Python scripts, and shell access is something currently not available to Splunk Cloud users.

DEVThe OAuth protocol provides client apps with secure delegated access to server resources on behalf of a resource owner. In our case, the Google Drive REST API is the resource and our add-on is a client. For more information about OAuth 2.0, see this simplified intro. For more information about authenticating with the Google Drive REST API, see "Authorizing Your App with Google Drive."

BUSIt is common for consumer facing ISVs to use OAuth to protect the APIs they offer their customers and partners. It is also gaining traction among enterprises.
 

Original workflow

The original user interaction workflow is summarized here. This corresponds to a "three-legged OAuth" scenario—where the add-on calls Google APIs on behalf of the user, requiring the user's consent. For more information about using OAuth 2.0 to access Google APIs, see the Google Identity Platform docs for OAuth 2.0. "Using OAuth 2.0 to Access Google APIs."
Legs 1 and 2 (Creating the Google Drive Client ID and Client secret):

  1. The user creates a new Google project within the Google Developers Console
  2. Within the new project, the user enables Google Drive API access.
  3. The user creates a new Client ID for native applications, and then jots down the Client ID and Client secret for later use.  

Leg 3 (Getting the authentication token):

  1. The user installs the Google Drive add-on from the PAS package. 
  2. The user goes to the command line and executes the configure_oauth.py script in the Google Drive add-on's bin directory. 
  3. When prompted by the script, the user enters the saved Client ID and Client secret values. 
  4. The user is then presented with a URL that the user copies into their computer's Clipboard. The script presents a prompt asking for a validation code, which is an alphanumeric string.
  5. In a local browser window (but keeping the terminal window open), the user pastes the URL from the command-line output into the address bar, and then presses Enter/Return to go to the URL.
  6. A dialog box appears, requesting access to the user's Google Drive data, which the user grants. 
  7. An alphanumeric validation code string appears and the user copies and pastes it into the waiting prompt in the Python shell script. 

The script then exchanges the validation code for OAuth 2.0 refresh tokens. The user can use the refresh tokens to access the necessary data indefinitely, or until the user revokes access

Possible alternatives

To avoid our existing "three-legged OAuth" scenario, we came up with two possible alternatives. The first was ultimately rejected, and involved the "two-legged OAuth" solution, or authorization using a service account. The second involved integrating the OAuth 2.0 setup into the PAS app Setup dashboard, and removing the need for users to implement the Python setup script manually.

Method 1: Auth using a service account

A service account is an account that belongs to an app rather than to an individual user. The service account must be modified to impersonate a user, and would need domain-wide access to do so. This is a valid way of doing things with OAuth 2.0, because otherwise the add-on would have to impersonate a user by storing and using the user's actual account and credentials.

SEC: Using service accounts undermines OAuth 2.0 security and is not an acceptable solution for publicly-released code.

Using a service account can impact performance if the account owns numerous shared items in the same domain. In fact, Google strongly recommends against using service accounts as the common owner for domain-wide files. Also, only users that have a paid Google Apps domain account can use this approach. Ultimately, this is what killed this possibility.

Method 2: Integrate OAuth 2.0 setup into PAS app Setup dashboard

The second possible solution that was ultimately accepted and implemented, involved integrating the OAuth 2.0 setup with the PAS app's Setup dashboard. That is, we modified the PAS reference app's Setup dashboard to include Google Drive module logic. Then, the app would simply ask the user for a Client ID, and then open a new browser window from which the user could obtain a validation code to copy and paste back into the app's Setup dashboard. This approach follows a similar flow to what was already in place, but removes the need for the user to touch the command line. Plus, it allowed us to continue accurately describing the add-on as an OAuth 2.0 "native application" client.

The new workflow is summarized here:

  1. The user creates a new Google project within the Google Developers Console.
     New Project screen
  2. Within the new project, the user enables both Google Drive API access and the Google Admin SDK by going to APIs & auth > APIs, and then clicking Enable API for each one. 
    Image of the Google Developers Console, Drive API section
  3. The user creates a new Client ID for native applications by going to APIs & auth > Credentials: First, the user creates an OAuth consent screen (if one doesn't already exist) by clicking OAuth consent screen on this page and following the instructions. Then, the user clicks Credentials and chooses OAuth 2.0 client ID from the Add credentials menu.
    Credentials screen
    On the Create client ID page, the user chooses an application type of Other, fills out the requested URL information, and then clicks Create.
    Create Client ID screen
    The console then displays the Client ID and Client secret. The Client ID and Client secret are always available from the Credentials page of the developers console, but the user should jot them down or copy and paste them in a safe place for quick access later in this process.
    Client ID and client secret screen
  4. On the PAS Setup dashboard, the app prompts the user to enter a Client ID and Client secret.
    Google Drive add-on setup
  5. The user then clicks the Get Code! button to get a validation code.
  6. A new browser window appears, and asks the user to allow or deny access to the user's Google Drive data.
    Google prompt
    DEVFor this to work, a redirect URI is baked into the endpoint that's called when the user attempts to authenticate the app on Google's site. This prevents open redirect attacks and is typically enforced by OAuth2 providers. This is outlined at developers.google.com/identity/protocols/OAuth2InstalledApp and the value is set at "urn:ietf:wg:oauth:2.0:oob".
  7. When the user accepts, an alphanumeric validation code string appears along with instructions to copy it back into the PAS app.
    Auth code display 
  8. The user copies and pastes the validation code into the field under Enter Authorization Code: on the PAS Setup dashboard, and then clicks Save Code! to save the configuration.
    Add-on setup paste code box

As before, the script then exchanges the validation code for OAuth 2.0 refresh tokens, which enable the user to access the data. 
Code accepted message
To implement the changes, we made modifications to the following files within the PAS app and Google Drive add-on:

  • pas_ref_app/appserver/static/setup.js This JavaScript file is responsible for the PAS app's setup logic. We added new UI control logic to determine when to show or hide elements during the user interaction flow. We also added new UI control logic to let users set up their Google Drive credentials right from the Setup dashboard. The dashboard now also shows whether the validation code has been generated or not.

The relevant portion of the setup.js file is shown here. This code validates the user entries for the Client ID and the Client secret, and then opens an auth window if they are both present:

setup.js Part 1

// Google Drive OAuth2 Checks
    $("#getAuth").click(function() {
        var clientId = $("#clientId").val();
        var clientSecret = $("#clientSecret").val();
        if(clientId.length == 0) {
            sendUxLog("User didn't enter a Client ID");
            $("#clentIdError").removeClass('hide');
        } else {
            // hiding error prompt since input value is present
            $("#clentIdError").addClass('hide');
        }
        if(clientSecret.length == 0) {
            sendUxLog("User didn't enter a Client Secret");
            $("#clentSecretError").removeClass('hide');
        } else {
            // hiding error prompt since input value is present
            $("#clentSecretError").addClass('hide');
        }
        if(clientId.length > 0 && clientSecret.length > 0) {
            sendUxLog("Opening Google Authentication window for user to obtain Auth Code.");
            window.open(GOOGLE_SIGN_IN_BASE_URL + clientId, "popupWindow", "width=600,height=600,scrollbars=yes");
            $("#codeEntry").removeClass('hide');
            $("#clentIdError").addClass('hide');
            $("#clentSecretError").addClass('hide');
        }
    });
  • pas_ref_app/default/data/ui/views/setup.xml This XML file is where the new static UI elements were added, including user input elements that take advantage of existing CSS styling.
  • googledrive_addon/bin/configure_oauth.py This is the Python script that users previously had to invoke through the shell. To this file we added more robust logging, including logging to a dedicated log file at $SPLUNK_HOME/var/splunk/log.
  • googledrive_addon/bin/googledrive.py This is the modular input script—the script that actually does the work of streaming Google Drive activity events back to Splunk Enterprise. To this Python script we also added more robust logging to the dedicated log file at $SPLUNK_HOME/var/splunk/log.

One final challenge: exchanging the validation code for the refresh token

The validation code that the user enters must be stored so that it is accessible to the existing script, so how do we get the string exchanged for a refresh token from the client side? We came up with two options: obtain the refresh token through JavaScript calls on the client side or create a new custom REST endpoint entry to call once the user enters the obtained string.

Method 1: JavaScript calls

The first possibility was quickly scuttled: Obtain the OAuth 2.0 refresh token through JavaScript calls on the client side. This was determined to be inadvisable from a best practices standpoint, as it would have required sensitive information to be stored directly on the client. Though this method might be acceptable for a quick fix in a sandboxed environment, it's not a good idea to implement it in code that is released publicly.

Method 2: New custom REST endpoint

The second possibility was the one we ultimately implemented: In the restmap.conf file, we created a new custom REST endpoint entry that the add-on calls once the user enters the validation code. That then executes the configure_oauth.py script that does the server-side refresh token exchange.
The relevant portion of the Google Drive add-on's restmap.conf file (which is located in googledrive_addon/default), is shown here:

[script:configure_oauth]
match=/configure_oauth
handlertype = python
handler = configure_oauth.oauth_exchange

This stanza creates the new endpoint configure_oauth.
Note that we used the [script:<uniquename>] form for the stanzas rather than the [admin:<uniqueName>] form. This was settled upon after some experimentation, where we observed that the more direct [script:<uniquename>] method resulted in more reliable script execution.
The relevant portion of the setup.js file is shown here. This code attempts to post to the custom REST endpoint (/services/configure_oauth), and exchanges the validation code for the refresh token:

setup.js Part 2

$("#saveAuth").click(function() {
        var client_id = $("#clientId").val()
        var client_secret = $("#clientSecret").val()
        var auth_code = $("#authCode").val()
        if(auth_code.length > 0) {
            // Creating OAuth2 object for key exchange
            var oauth2_record = {
                "auth_code": auth_code,
                "client_id" : client_id,
                "client_secret" : client_secret
            }

            // Attempting to exchange auth code for refresh token via call to custom RESTful endpoint.
            // Details are located in restmap.conf
            var service = mvc.createService();
            service.post("/services/configure_oauth", oauth2_record,
                function(err, response) {
                    if(null!=response) {
                        $("#codeEntry").addClass('hide');
                        $("#gAuthSuccess").removeClass('hide');
                        $("#gAuthError").addClass('hide');
                    } else {
                        sendDevLog("Token exchange error: " + err.status + ". Message: " + err.error);
                        $("#gAuthError").removeClass('hide');
                        $("#gAuthSuccess").addClass('hide');
                    }
                });
            $("#authEntryError").addClass('hide');
        } else {
            $("#authEntryError").removeClass('hide');
            $("#gAuthSuccess").addClass('hide');
        }
    });

In addition to the configure_oauth endpoint, in the same restmap.conf file, we created the new endpoint configure_oauth/status:

[script:configure_oauth_status]
 match=/configure_oauth/status
 handlertype = python
 handler = configure_oauth.oauth_status

This endpoint was created so that we could build a status indicator for the PAS app's Setup dashboard that indicated whether the Google Drive OAuth 2.0 credentials had been successfully generated. This portion of the setup.js file is the code that posts to that endpoint:

setup.js Part 3

// Determines whether or not the Google Drive OAuth2
    // credentials have been generated and shows UI element
    // indicating credential status
    function isOauthConfigured() {
        var service = mvc.createService();
        service.get('/services/configure_oauth/status?check=configured', "",
            function(err, response) {
                if(JSON.parse(response.data).configured==true) {
                    $('#gAuthNotConfigured').addClass('hide');
                    $('#gAuthConfigured').removeClass('hide');
                } else {
                    $('#gAuthConfigured').addClass('hide');
                    $('#gAuthNotConfigured').removeClass('hide');
                }
            });
    }

What did we learn?

Here are the key lessons we learned while creating alerts and building custom alert actions:

  • A new version of Splunk Enterprise means a brand new journey: extending the functionality of our sample app with the enhanced features of the platform.
  • Thinking ahead by making the app modular meant that the transition to our new main developer went smoothly.
  • We decided to rework the Google Drive add-on's setup process, as the previous workflow put the burden on the user to run a Python script from the command line.
  • To improve the add-on's setup process, we considered recommending using a service account, modified to impersonate a user. However, we decided against this because it undermines OAuth 2.0 security and is discouraged by Google.
  • Instead, we decided to modify the PAS reference app's Setup dashboard to include Google Drive module logic.
  • Requesting access to the user's Google Drive data, obtaining a validation code, and exchanging the validation code for OAuth 2.0 refresh tokens are now all done in the PAS app's Setup dashboard.
  • We were able to do the bulk of the code work in the app's setup.js page, with minimal tweaks to the app's setup.xml file and two Python scripts.
  • The final challenge in getting this new workflow operational was making the user-entered validation code accessible to the existing Python script so that it could be exchanged for a refresh token from the client side.
  • The first possibility was quickly determined a no-go: obtaining the OAuth 2.0 refresh token through JavaScript calls on the client side would have required sensitive information to be stored on the client.
  • Instead, we created a new custom REST endpoint that the add-on calls once the user enters the validation code. Calling the endpoint runs the script that does the server-side refresh token exchange.