Splunk Intermediate

Splunk Tutorial
---------------

https://docs.splunk.com/Documentation/Splunk/latest/Search/ViewsearchjobpropertieswiththeJobInspector


Booleans
AND OR NOT

Fields
--
status=400
status=50*
status!=300

sourcetype=access_combined | fields clientip, action

Table
--
sourcetype=access_combined | fields clientip, action

Rename
--
sourcetype=access_combined | rename clientip as "userip"

Dedup
--
sourcetype=access_combined | dedup clients

sort cmonnad
lookup command

Module 2
--------
Field name are case sensitive
Field values are not case sensitive
Fiend values from a lookup are case sensitive by default
Booleans oparator are case sensitive

time - index - source - host - sourcetype

fast mode - performance
verbose mode - completness
smart mode - combination of fast and verbose mode

Module 3 - Commands for Visualization
-------------------------------------
chart command
--
over - X axis
any stats function can be applied to the chart command

index=web sourcetype=access_combined status>299 | chart count over status
index=web sourcetype=access_combined status>299 | chart count over status by host
index=web sourcetype=access_combined status>299 | chart count by status,host
index=web sourcetype=access_combined status>299 | chart count over host by product_name
index=web sourcetype=access_combined status>299 | chart count over host by product_name usenull=f
index=web sourcetype=access_combined status>299 | chart count over host by product_name useother=f
index=web sourcetype=access_combined status>299 | chart count over host by product_name limit=5
index=web sourcetype=access_combined status>299 | chart count over host by product_name limit=0

Timechart command
-----------------
index=sales sourcetype=vendor_sales | timechart count
index=sales sourcetype=vendor_sales | timechart sum(price) by product_name
index=sales sourcetype=vendor_sales | timechart sum(price) by product_name limit=5
index=sales sourcetype=vendor_sales | timechart span=12hr sum(price) by product_name limit=0

Timewrap Command
----------------
index=sales sourcetype=vendor_sales product_name="Dream Crusher"| timechart span=1d sum(price) by product_name | timewrap 7d

index=sales sourcetype=vendor_sales product_name="Dream Crusher"| timechart span=1d sum(price) by product_name | timewrap 7d
|rename _time as Day | eval Day = strftime(Day,"%A")

Visualization Examples,
----------------------
Line Graph
Formation Option
Chart Overlay
Area Chart
Column Chart
Bar Graph
Pie Chart
Scatter Chart
Bubble Chart
Trellis Layout


https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizDevOverview


Module 4 - Advanced Visualizations
----------------------------------
Use Geographical Info

iplocation command
--
index=security sourcetype=linux_secure action=success src_ip!=10.* | iplocation src_ip

Geostats Command
--
index=sales sourcetype=vendor_sales | geostats latfield=VendorLatitude longfield=VendorLangitude count by product_name globallimit=4
index=security sourcetype=linux_secure action=success src_ip!=10.* | iplocation src_ip | geostats latfield=lat longfield=lon count

Choropleth Map
--
.kmz - Keyhold Markup Language File
Geom command - Adds field with geographical data structures mathing polygons on map.
--
index=sales sourcetype=vendor_sales VendorID>=5000 AND VendorID<=5055 | stats count as Sales by VendorCountry
|geom geo_countries featureidField=VendorCountry

Single Value Visualization
--
index=web sourcetype=access_combined action=purchase | stats sum(price) as total
index=web sourcetype=access_combined action=purchase | timechart sum(price)
index=web sourcetype=access_combined action=purchase | stats sum(price) as total | gauge total 0 30000 600000 700000

Trendline Command - Computes moving avarages of field values.
--
Trendtype:
simple moving average
exponential moving avaerage
weighted moving avarage

index=web sourcetype=access_combined action=purchase status=200 | timechart sum2(price) as sales | trendline wma2(sales) as trendline

Field Formation

Addtotals Command
--
index=web sourcetype=access_combined file=*| chart sum(bytes) over host by file | addtotals
index=web sourcetype=access_combined file=*| chart sum(bytes) over host by file | addtotals col=true label="Total"
index=web sourcetype=access_combined file=*| chart sum(bytes) over host by file | addtotals col=true label="Total" labelfiedl="host" row=false

http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/CustomVizDevOverview

Module 5 - Filtering and Formatting
---------------------------------
Eval command
--
- arthmetic,concatination,boolean valuse supported
- results can be written to new field or replace exising field
- Newly created field values are case sensitive

sourcetype=cisco_wsa_squid s_hostname=* | stats values(s_hostname) by cs_username
sourcetype=cisco_wsa_squid s_hostname=* | stats values(s_bytes) as Bytes by Usage
sourcetype=cisco_wsa_squid s_hostname=* | stats values(s_bytes) as Bytes by Usage | eval bandwidth= Bytes/1024/1024
sourcetype=cisco_wsa_squid s_hostname=* | stats values(s_bytes) as Bytes by Usage | eval bandwidth= round(Bytes/1024/1024,2)
|sort -bandwidth | rename bandwidth as "Bandwidth(MB)" | fields - Bytes


Eval Mathematical Functions
--
index=web sourcetype=access_c* product_name=* action=purchase | stats sum(price) as total_list_price,sum(sale_price) as total_sale_price by product_name
| eval discount= round(((total_list_price - total_sale_price) / total_list_price)*100) | sort - discount
| eval discount = discount."%"


Eval Convert Values
--
Tostring Function - convert numerical values to strings. (cannot sort)
--
index=web sourcetype=access_c* product_name=* action=purchase | stats sum(price) as total_list_price,sum(sale_price) as total_sale_price by product_name
| eval total_list_price = "$" + tostring(total_list_price,"commas")



Fieldformat command - Format values without changing characteristics of underlying values.(can abot to sort)
--
index=web sourcetype=access_c* product_name=* action=purchase | stats sum(price) as total_list_price,sum(sale_price) as total_sale_price by product_name
| eval total_list_price = "$" + tostring(total_list_price,"commas")
| fieldformat total_sale_price = "$"+ tostring(total_list_price,"commas")

Data in the index does not change.

Multiple eval commands
--
index=web sourcetype=access_c* product_name=* action=purchase | stats sum(price) as total_list_price,sum(sale_price) as total_sale_price by product_name
| eval current_discount = round(((list_price - sale_price)/list_price)*100)
| eval new_discount = (current_discount - 5)
| eval new_sale_price = list_price - (list_price * (new_discount/100))
| eval price_change_revenue = (new_sale_price - sale_price)

Eval Command IF Function
--
index=sales sourcetype=vendor_sales
| eval SalesTerritory = if(VendorID < 4000,"North America","Rest of the World")
| stats sum(price) as TotalRevenue by SalesTerritory

Eval Case Function
--
index=web sourcetype=access_combined
| eval httpCategory=case(status>=200 AND stats<300,"Success",status>=300 AND status<400,"Redirect",
status>=400 AND status<500,"Client Error",status>=500,"Server Error",true(),"Something Other")

Eval with Stats
--
index=web sourcetype=access_combined
| stats count(eval(status<300)) as "Success",count(eval(status>=400 AND status<500)) as "Clinet Error",
count(eval(status>500)) as "Server Error"

Search command
--
index=network sourcetype=cisco_wsa_squid usage=Violation
| stats count(usage) as Visits by cs_username | search Visits > 1

Where Commands
--
index=network sourcetype=cisco_wsa_squid
| stats count(eval(usage="Personal")) as Personal,count(eval(usage="Business")) as Business by username
| where Personal > Business | where username!="sie" | sort -Personal

Eval/Where tips
---
_ char to match one
% char for the wildcard

index=web sourcetype=access_combined action=purchase | stats count by product_name
| where product_name like "Worl%"

null and isnotnull
--
index=sales sourcetype=vendor_sales | timechart sum(price) as sales | where isnull(sales)
index=sales sourcetype=vendor_sales | timechart sum(price) as sales | where isnotnull(sales)

Fillnull Command
--
index=sales sourcetype=vendor_sales | chart sum(price) over product_name by VendorCountry
| fillnull value="Nothing here"

Module 6 - Correlating Events
-----------------------------

Transaction Overview
Transaction command
---
index=web sourcetype=access_combined
| transaction clientip
| table clientip,action,product_name

Transaction Definitions
---
maxspan - Allows setting of maximum total time between earliest and latest events.
maxpause - Allowed maximum total time between events.
startswith - Allows forming transaction starting with specified {terms, field values, evaluations}
endswith - Allows forming transaction ending with specified {terms, field values, evaluations}

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Transaction

index=web sourcetype=access_combined
| transaction clientip
 startswith="addtocard" endswith="purchase"
| table clientip,action,product_name

Investigate with Transaction
---
index=network sourcetype=cisco_esa
| transaction mid dcid icid
| search REJECT

Transaction vs Stats
---
transaction
 - Use to see events correlated togather.
 - Use when events need to be grouped on start and end values.
stats
 - Use to see results of a calculation.
 - Use when events need to be grouped on a field value.

index=web sourcetype=access_combined
| transaction  clientip startswith=action="addtocart" endswith=action="purchase"
| table clientip, JSESSIONID, product_name, action, duration, eventcount, price

(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
| search sourcetype=access_combined AND sourcetype=cisco_wsa_squid
| timechart count by status
| addtotals
| search Total>4



Module 7 - Knowledge Objects
----------------------------
Naming conversion - {Group,Type,Platform,Categor,Time,Description}
example - OPS_WFA_Network_Security_na_IPwhoisAction
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Developnamingconventionsforknowledgeobjecttitles

Module 8 - Field Extractions
----------------------------
Field extraction with Regex and Delimiter

Module 9 - Aliases and Calc Fields
----------------------------------
Field Aliases
--

Calculated Fields
 - must be based on extracted or discovered fields
 - Fiedls from a Lookup table or generated from a search command cannot be used


Module 10 - Tags and Event Types
--------------------------------
Tags
- Alllow you to designate descriptive names for key-value pairs
- enable you to search for events that contain particular field value

Tag values are case sensitive

Event Types
-Categorize events based on search strings
-Use tags to organize
-"eventtype" field within a search string
- Time range NOT available

Saved Reports
-Fixed search criteria
-Time range & formatting needed
-share with splunk users
-add to dashborads

(index=web sourcetype=access_combined) OR (index=network sourcetype=cisco_wsa_squid) status> 500
eventtype="web_error"


Module 11 - Macros
------------------
Macros
-Reusable search strings or porttions of search strings
-Useful for frequent searches with complicated search syntax

- Store entire search strings
- Time range independent
- Pass arguments to the search

https://docs.splunk.com/Documentation/CIM/4.15.0/User/Web

index=sales sourcetype=vendor_sales | stats sum(sale_price) as total_sales by Vendor
| eval total_sales = "$" + tostring(round(tatal_sales,2),"commas")

`ConvertUSD` = eval total_sales = "$" + tostring(round(tatal_sales,2),"commas")

Ctrl + Shift + E = Search Explansion Window

index=sales sourcetype=vendor_sales VendorCountry=Germany OR VendorCountry=France OR VendorCountry=Italy
| stats sum(price) as USD by product_name
| eval USD = "$"+tostring(round(USD,2),"commas")

`Europe_sales`

index=sales sourcetype=vendor_sales VendorCountry=Germany OR VendorCountry=France OR VendorCountry=Italy
| stats sum(price) as USD by product_name
| `Europe_sales`

sourcetype=vendor_sales VendorCountry=Germany OR VendorCountry=France OR VendorCountry=Italy
| stats sum(price) as USD by product_name
| eval euro = "€" + tostring(round(USD*0.79,2), "commas"), USD = "$" +tostring(USD, "commas")

stats sum(price) as USD by product_name
| eval $currency$="$symbol$".tostring(round(USD*$rate$,2),"commas"),USD="$" +tostring(USD,"commas")


index= sales sourcetype=vendor_sales VendorCountry=Germany OR VendorCountry=France OR VendorCountry=Italy
|  `convert_sales(euro,€,.79)`

index=sales sourcetype=vendor_sales VendorCountry="United Kingdom"
| `convert_sales(GBP,£,.64)`

index=sales sourcetype=vendor_sales VendorCountry="India"
| `convert_sales(INR,₹,68)`



Module 12 - Workflow Actions
----------------------------
Create links to interact with external resources or narrow search.
GET and POST


Module 13 - Data Models
-----------------------

Data Models consist of: Events searches Transactions
Data Model Framework - Pivot is interface to the data

strftime(_time,"%m-%d %A")


Module 14 - CIM Common Information Model
-----------------------------------------
Maps all data to defined method
Normalizes to common language
Data can be normalized at index time or search time
CIM schema shoud be used for: Field extractions, Event types , Aliases, Tags
Knowledge objects can be shared globally across all apps.