Managing and creating your apps_Learning Splunk Web Framework-QQ阅读女生青春网

上QQ阅读APP看书，第一时间看更新

Managing and creating your apps

When we first log on to Splunk, we are presented with the home page, which provides us with a list of all our apps down the left-hand side of the screen and we are able to order it to our liking. By clicking on the Splunk logo at the top left of the screen, we are brought to the home page http://localhost:8000/en-GB/app/launcher/home .

The following screenshot shows the welcome page that is presented when the user logs on to Splunk. The left panel running down the side of the screen allows you to drag and order the Splunk apps, allowing you to place the most important apps up the top of the screen ready for you to use. Above the list of Splunkapps, you will also see the icon of a cog, next to the word Apps. By clicking on this icon you are taken to the Manage Apps screen.

If you are not on the main home screen in Splunk, you can also access the Manage Apps screen by clicking on the Apps menu, which is positioned in the top left-hand corner, next to the Splunk icon. If you are an Administrator of a Splunk environment, you may have made use of the interface that Splunkprovides to allow you to manage, configure, and create your apps.

The Manage Apps configuration page allows you to see all of the Splunk apps within your current environment and allows you to perform a number of different tasks to manage your Splunk apps, including the following:

Search for prebuilt apps: Access the Splunk base to search, download, and install prebuilt and approved Splunk apps and Add-ons. Refer to https://splunkbase.splunk.com/.
Install a Splunk app from a file: Not just Splunk apps and Add-ons from the Splunk base, but any packaged Splunk app that you or someone else may have created and are ready to deploy in your environment can be installed.
Enable and disable Splunk apps: Any Splunk app that is currently installed in your environment can be enabled or disabled at any time. There is no way to delete a Splunk app from the Management console, but this can be achieved from the command line, and we will discuss this later in the chapter.
Manage permissions of your Splunk apps: Change and configure which users can access and make changes to any of the Splunk apps installed on your environment. You can also change whether or not it is visible to users.
Configure your Splunk apps: Make adjustments to the configurations that your Splunk app utilizes and change the specific objects the Splunk app has access to.
Create new Splunk apps: This is what we will do next.

Creating our first Splunk app

We are going to discuss designing applications later in the chapter, but for now it's time to get stuck in our first example Splunk app using our NASA website data. For now, we are going to create the most basic Splunk app to get things started. To create our first application, you can use the following process:

As shown in the following screenshot, go to the top of the screen and click on the Create App button:
You will then be presented with the following page, asking for the details of the Splunk app that you are going to create:
Use the following details to fill in this form:
- Name: This is the text presented on the Splunk web interface as the app. Our new app will be called NASA Squid and Web Data.
- Folder name: This is the name of the directory that will be created on the server and can be accessed via the command line. Enter the folder name of nasa_squid_web_data.
- Version: As our first iteration of our new Splunk app, we enter the value 0.1.
- Visible: This is if you want to make your new Splunk app visible from the point of creation. In this instance, it is Yes.
- Author: This is the person who is developing the Splunk app, so enter your name.
- Description: Provide a brief description of the Splunk app you are creating.
- Template: This is the type of Splunk app that you want to create. At this point in time, we only have a barebones or a Simple template. In this instance, we will use the barebones template. In later chapters, we will create our own template from existing Splunkapps to create some consistency across our development.
- Upload asset: Please leave this blank, but if you need to ever load static images, scripts, or CSS that your Splunk App uses, this is where you put it.
Then click on the Save button at the bottom of the screen to complete the Splunk App creation.

That's it! You have created your first Splunk app. You will see the new Splunk app listed in the Manage Apps screen. You will also be able to see the new Splunk app listed on the left-hand side of the home screen. By clicking on the NASA Squid and Web Data app, you will be taken to your new Splunk app. And from what you can see, it looks similar to the original Search and Reporting screen, with the basic Splunk search bar, and a basic menu, including Search, Pivot, Reports, Alerts, and Dashboards.

It might feel like we still have not come very far, but we are laying the groundwork to ramp things up shortly.

Alternative ways to create a Splunk app

You don't always have to use the web interface to create your Splunk apps. There are two other ways by which we can create our Splunk apps: with a Splunk command on the server or by creating the basic directory structure.

To create a Splunk app from the command line, first access the Splunk server and run the following command:

$SPLUNK_HOME/bin/splunk create app splunk_app_name -template template_name

Here is the description of the preceding command:

$SPLUNK_HOME: This is the location of your Splunk installation on your server.
splunk_app_name: This will be the name of the Splunk app you want to create. As you may remember, in our example, we named our Splunk app NASA Squid and Web Data.
template_name: This will be either barebones or sample_app.

When you are accessing the command line of your server, you will be able to see the location of all the Splunk apps on your environment. The location is as follows:

$SPLUNK_HOME/etc/apps

Looking through the directory, you should be able to see all your environment Splunk apps, including the new one you have just created; in this instance, it should be the same name as the folder name you provided in the Create Apps form as nasa_squid_web_data. Although we advise against creating a Splunkapp straight from the filesystem, the specific files that you need to create to create a new Splunk app are listed as follows:

$SPLUNK_HOME/etc/apps/splunk_app_name/default/app.conf
$SPLUNK_HOME/etc/apps/splunk_app_name/local
$SPLUNK_HOME/etc/apps/splunk_app_name/metadata/default.meta
$SPLUNK_HOME/etc/apps/splunk_app_name/default/data/ui/views/

Tip

You need to have your XML data files in the views directory, and we will be working on these files in the next chapter.

Adding your new Splunk app to Git

In some situations, you may want to create one repository for all Splunk apps, but I always recommend that you have one repository for each of your Splunk apps. Although it may be annoying in a case where you are trying to recover a deleted directory because you may have numerous Splunk apps to retrieve, it will be a lot more efficient when you are updating or releasing Splunk apps on your environment. You will only need to deploy or update one Splunk app instead of the entire set of Splunk apps you have in your environment. To create a new repository for our new NASA data and add our Splunk app to the repository, we perform the following steps:

Create the repository with the name nasa_squid_web_data and use the description NASA Squid and Web Data Splunk App.
Log on to your Splunk server and change to the directory of the new app you have created:
```
 cd $SPLUNK_HOME/etc/apps/nasa_squid_web_data 
```
Initialize the new repository with all the current files in there:
```
 git init 
```
Add all the content to the repository with a full stop (.):
```
 git add . 
```

Commit all your changes to the repository:

 git commit -m "Our first Splunk App Commit"

Set up the remote location of your repository:

 git remote add origin git@github.com: <account>/nasa_squid_web_data.git

Now push all your changes back to GitHub:
```
 git push -u origin master 
```

Deleting Splunk apps

If there is ever a situation in which you would like to completely delete a Splunk app from your environment, the only way you can do this is by deleting the directory from the filesystem on your Splunk server. With our new NASA Web Data App, we can delete it with the following command:

rm -rf $SPLUNK_HOME/etc/apps/nasa_squid_web_data

To activate this change, you will also need to restart the Splunk server. The awesome thing is that if you happened to make this change by accident and had a repository set up in GitHub, all you would need to do to restore this Splunk app is log on to the Splunk server and run the following Git command from the command line:

git clone git@github.com:<account>/nasa_squid_web_data.git $SPLUNK_HOME/etc/apps

The Splunk app directory structure

Now that we are on our Splunk server, it is a good time to run through the different files and directories in our Splunk apps.

As we stated earlier in the chapter, there are four basic directories and files that need to be set up for you to have a Splunk app. The following is a more detailed list of the directory structure and files included.

appserver: The appserver directory includes some of the files that are used as part of the inner workings of your Splunk app. Within this directory, you will see the static directory that will include your CSS, JavaScript, and other files required to configure your app.
bin: The bin directory, as with normal server directory structures, contains binary files, including shell scripts and Python scripts used in delivering your Splunk apps.
default: When you eventually publish your Splunk app ready for production, all your configuration, search, and display data will be moved into the default directory. You will notice that when you make changes to your Splunk apps through the web interface, the changes will be added to the local directory, leaving the default directory pristine.
local: As we stated earlier, any changes that are made to searches, views, or other configurations by the local user are added to the local directory.
lookups: These lookups are specific to your Splunk app searches, which will use CSV files to enhance the data that you currently have indexed in Splunk.
metadata: All objects that are used in Splunk have permissions, and the metadata file includes all the permissions for these objects.

Designing Splunk apps for your audience

When creating your Splunk apps, the first thing you need to realize is that you need to understand the audience that you are creating an app for, so that you can specifically meet their needs. We can do amazing things with the Splunk Web Framework, but the apps you create will only be valuable if someone is actually using them or consuming information.

When it comes to designing Splunk apps, I like to brainstorm as many ideas as possible, as the best thing about the Splunk Web Framework is that you can rapidly prototype ideas. Also, when it comes to designing Splunk apps, I like to work with the four Ds to Discover, Define, Develop, and Deliver. This process is used to help try and think of multiple ideas, try to expand as many ideas as possible, and relate these ideas back to what your user needs. Then try to narrow the focus of the ideas and prioritize what you think is the best thing to deliver.

When we use the four Ds, we:

Discover: This is where we discuss what the user actually needs and try and gather as many different ideas that specifically address this.
Define: This is where we try to define the business objectives with what we are designing and try to align them with what the user actually needs.
Develop: This is where we get to create and develop different solutions to what the user needs, in iterative stages, and test the functionality of what we are creating.
Deliver: The is the final iterative delivery of the application, where we provide further testing in the production environment and gather feedback from the user and the business to further enhance what we have provided.

We can use the four Ds to do a simple exercise with our example of the NASA Squid and Web Data Splunk app. We should have probably done this before we even created our basic Splunk app, but we at least knew that we needed to make this Splunk app. We can also make a number of assumptions with our design process to allow us to move forward, but in real life, you should be able to go directly to your users to gather initial information. If we follow the four Ds once again for our example, we can see the following:

Discover: The user needs to see the squid and web data in a visual format. We will have both technical and non-technical users, and in this instance, we will only create one Splunk app for both of these users. We can also assume that not all our technical users will be comfortable working with Splunk, Splunk searches, and reports. In this part of the process, we would start to brainstorm on how we can set up our interface to present the information to the user.
Define: As the data that we have for our NASA website is specifically squid logs, we will make some assumptions that our non-technical users are more interested in seeing traffic and popularity of certain pages, whereas the more technical users will want information related to errors, throughput, and return codes from the web server. At this point, we would want to brainstorm as many ideas as possible and try to get them all down on paper, as thinking differently can sometimes give us some more interesting ideas. The following section has a drawing of our first dashboard, which will provide details on the following:
- Return codes from the web server
- The top 10 popular pages being accessed
- The average hits per day
Develop: This is the fun part, and we are going to kick off our development shortly to complete our first iteration of our Splunk app.
Deliver: This will be when we communicate the release of our Splunk app with our users, perform training on how to use it, and ask for feedback.

As you can see in the preceding diagram, your initial hand-drawn designs do not need to be perfect but only need to convey the idea you are trying to get across. The idea you choose to develop will then be taken forward and worked on.

Creating a dashboard

It's now time to add some useful interface dashboards and panels to our new Splunk app. So we open our NASA Squid and Web Data Splunk App, and click on the dashboard menu at the top of the screen to get things started:

At the top-right side of the page, you will see the Create New Dashboard button. Click on the button and you will be presented with the following form:
Fill in the form with the following details:
- Title: NASA Web Usage
- Description:Summary page for our new Splunk Dashboard
- Permissions:Shared in App
Then click on the Create Dashboard button.

You have created an empty dashboard, which should like what is shown in the following screenshot. The dashboard is already in Edit mode, allowing us to continue to use the web interface to create panels to display the sets of data that we discussed in the design phase.

Within the page, you can see we have three buttons that we can use to control how we edit the dashboard that we have created for our Splunk app:

Add Panel: This allows us to add a new or existing panel that has been shared across our environment. If we had a large existing environment, we would be able to search for any of the visualizations that have been shared across our different Splunk apps.
Add Input: This allows our users to interact with our dashboard and assist them to customize the information that is presented to the user. We will be covering this in later chapters.
Edit Source: This allows us to directly configure and script the SimpleXML code that underlies our Splunk web interface. This will be covered in later chapters as well.

Field extraction for our NASA data

Before we set up our new panels, we need to set up some field extraction to allow us to simplify our Splunk searches that we use. We are not using advanced searches within our examples as we are focusing more on the Splunk web framework, but this will allow our searches to be more efficient as we only need to search for a field instead of adding extra parameters to our search.

The following is a breakdown of the field extraction we are using with an example output. From the squid log event:

home100.nj.nec.com - - [27/Jul/1995:23:59:59 -0400] "GET /images/WORLD-logosmall.gif HTTP/1.0" 200 669

We have set up the following field extraction:

From: home100.nj.nec.com
SquidDateTime: [27/Jul/1995:23:59:59 -0400]
RequestType: GET
Request: /images/WORLD-logosmall.gif
ReturnCode: 200
ReturnSize: 669

The specific regex we have used as our field extraction is listed as follows:

^(?P<From>[^ ]+)\s+\-\s+\-\s+(?P<SquidDateTime>\[\d+/\w+/\d+:\d+:\d+:\d+\s+\-\d+\])[^ \n]* "(?P<RequestType>\w+)\s+(?P<Request>[^ ]+)[^"\n]*"\s+(?P<ReturnCode>\d+)\s+(?P<ReturnSize>\d+)

Finally, to set up an average of values per month, we have also extracted the day for the month with a second field extraction. Using the preceding example: MonthDay: 27, the specific regex we have used as our field extraction is listed as follows:

^(?:[^\-\n]*\-){2}\s+\[(?P<MonthDay>\d+)

Tip

If you are having trouble with the field extraction, we have included this in the example code that can be downloaded as part of this chapter. The field extraction will be set up as part of the props.conf file in the default directory.

Adding panels to our dashboard

Let's start by setting up a pie chart that will display our return codes from our squid logs for the NASA website. If you are no longer in the edit screen for the NASA Web Usage dashboard, click on the Dashboard menu of your Splunk app, click on Edit Action, and select Edit Panel:

Click on the Add Panel button, select New, and click on the Pie option. You will then be presented with the following form:
For the form, set the following values:
- Set the time range to All Time
- Give the panel a title, such as Web Site ReturnCode
Add the search string that Splunk will use to create your visualization. In this instance, we are going to use the following search:
```
      index=main sourcetype=NASASquidLogs
      | stats count by ReturnCode
```
Click on the Add to Dashboard button.
Then click on the Done button, and you will have your new panel added to your dashboard:

Let's keep going and add a Single Value panel to display the average hits to the website per day:

Get into the Edit screen again for our dashboard. If you still have the preceding screen, displaying the new pie chart, you would have noticed by now that there is an Edit button directly on the dashboard.
Click on the Edit button and select Edit Panel.
To add a Single Value panel, click on Add Panel and select New.
From the list, select Single Value.
When you are presented with the form to add your new panel, enter the following information:
- Set the time range to All Time
- Add Content Title as Average Hits Per Day

Add the new search to be used for the Single Value panel as follows:

      index=main sourcetype=NASASquidLogs
      | stats count by MonthDay
      | stats avg(count) AS AvgHitsPerDay

Click on Add to Dashboard.
Finally, we want to add a Statistics Table to our dashboard to display the top 10 requested items within the squid logs.
You should still be in edit panel mode, but if you are not, click on the Edit button and select Edit Panel.
Click on Add Panel, select New, and select Statistics Panel from the list.
Set Content Title as Top 10 Website Requests.
Add the new Splunk search that will be used for the table to the following values:
```
      index=main sourcetype=NASASquidLogs
      | top 10 Request
```
Click on Add to Dashboard.
As you have created all the visualizations for our iterations, click on the Done button.

So, we're done. The first iteration of our new Splunk app has all the panels and functionality added, but it doesn't really look the way we wanted it to look when we drew our rough sketch during the design phase. There seems to be a lot of wasted space and I think we can make things a little nicer.

Editing existing dashboards

We have created all of the data visualizations and panels for our first iteration of our NASA Squid and Web Data Splunk App, but we want to make the interface a little clearer to the user. We have used the edit feature previously to add extra panels, but in this instance, we will make some changes to the way the dashboard displays. Again, click on the Edit button in the top right of the dashboard screen and select Edit Panels.

When you select Edit Panel, you will notice that you are still presented with the graph or chart as you would normally see it in the dashboard, but you have a number of extra options available on the web interface. The top right of each of the panels gives you full editing control of the panel with the following features:

The X and grey line beside it allow you to quickly delete or reposition the panel around your dashboard
The cog allows you to rename the panel, delete it, or convert it to a prebuilt panel to speed up searches
The magnifying glass lets you change the name of the panel and reconfigure aspects of the Splunksearch being used
The smaller chart image allows you to change the type of visualisation being used
The paint brush then refers to the way that your visualization is presented on the dashboard

With our new dashboard, we want to have Average Hits Per Day and Web Site ReturnCodes displayed next to each other:

To make this change, position your mouse pointer on the grey line at the top of the Web Site ReturnCodes panel, and it will turn into a four-pointed cross.
When you hold your mouse button down, you will see that you can drag the panel around the screen.
Move the panel to the right-hand side of the Average Hits Per Day panel.
The web interface will then rearrange your panels to allow them to fit next to each other.
We now want to change our Average Hits Per Day Single Value panel, to have our search result only provide two decimal places, and add a label underneath the value.
Click on the magnifying glass for the Single Value panel and select Edit Search String.
Add the following line to the end of your search string:
```
      | eval AvgHitsPerDay=round(AvgHitsPerDay,2)
```
Then click on the paint brush icon and add Hits Per Day as an Under Label.
Click on the Done button. Your changes will be saved and your dashboard should look like the following screenshot:

In a small amount of time, I think your users will be pretty impressed with the dashboard we have set up for them, handing over the first iteration of our Splunk app. We have been able to create a functional dashboard that gives useful insights into what their data is providing. At this point in time, there is no way to use the web interface to change what page or dashboard is provided by default for our Splunk app. We will show you in later chapters how to use SimpleXML code to configure the page that is displayed by default to the user when they open the Splunk app. We can, however, display our dashboard in the Splunkhome screen and we will make this change next.

Set your dashboard on the Splunk home screen

Whenever we want to return to our home page in Splunk, unless we have previously changed the default page being displayed, it will provide a blank page as we saw earlier in this chapter. Setting our new NASA Web Usage dashboard as the default dashboard on our home screen can be achieved with the following steps:

For the NASA Web Usage dashboard, click on the Edit option and select Set as Home Dashboard.
Click on the Dashboards menu at the top of the Splunk app.
Access the NASA squid and Web Data Splunk App.
So, every time your users go to the home page, your new dashboard that you have created will be displayed: http://localhost:8000/en-GB/app/launcher/home.

Viewing and saving changes to GitHub

We've now made a few changes to our Splunk app and this should be added to our repository. We can view changes made by running the Git status command from the command line. We can also see a log of all the changes we have made with the Git log command. So, to view the changes we have made and to add our new dashboard to our repository, we perform the following steps:

Log on to the Splunk server and change to your Splunk app directory:
```
 cd $SPLUNK_HOME/etc/apps/nasa_squid_web/
```

Check the status of your current filesystem in relation to Git:

 git status On branch master Your branch is up-to-date with 'origin/master'. Untracked files: (use "git add <file>..." to include in what will be committed) local/data/ nothing added to commit but untracked files present (use "git add" to track)

You will notice that the local/data directory is not currently being tracked. To add this to your repository, use the add command as we did previously:
```
 git add .
```

Now commit the changes to your repository:

 git commit -m "Adding our first dashboard"

Finally, push the changes back to GitHub:
```
 git push -u origin master
```

So far, we are using the master branch, which does not really follow our development process. In our next chapter, we will start to create branches of our repository so that we can start to allow development work to continue across multiple branches without affecting our master code.