Splunk – Session Notes 1

• What is Splunk?
  • Splunk is an IT operations tool that consumes machine data in real time. Most BI Tools does not analyse data in real time.
  • Security Incident and Event Management Tool
• Splunk main feature is converting unstructured data into structured insights.
• Splunk is used to
  • Investigate
  • Alert
  • build Reports
  • build Dashboards

• Deploying Splunk
  • Deployment Models
    • Splunk has a Cloud Deployment and On Premise deployment options.
    • Splunk Data Pipeline
      • Input
        • Load forwarded data, uploaded data, files etc
      • Parseing
        • Examine the data and adds metadata
      • Indexing
        • Divides the data into events and writes the data into disks in “buckets”.
      • Searching
        • User interaction with the data
    • Splunk Installations
      • Universal Forwarder
        • This is a lightweight splunk agent.
      • Splunk Enterprise Installation
        • This has all the splunk components.
    • Components that make up the splunk
      • Forwarder
      • Heavy Forwarder
      • Search Head
      • Deployment server
      • Master Cluster Node
      • License Server
      • Indexer
      • Deployer
  • Data Storage in disk
    • Index
      • Repository for splunk data
      • Raw data into events and stores then into index buckets with metadata
      • Default index that comes with splunk is called main and also has a _internal that stores the internal logs.
    • Event
      • Single row of data is called event
      • Events have fields which are key value pairs.
      • Splunk adds default fields to all events.
        • Timestamp
        • Host
        • Source
        • Source Type
      • Data is stored into directories called buckets based on age. There are 6 default buckets
        • Hot – Read/Write
        • Warm – Only Read rolled from Hot
        • Cold – Moved to a different location. Data rolled from Warm
        • Frozen – Deleted or Archived data. Data rolled from Cold.
        • Thawed – The data that is restored from Frozen
        • Fished
      • Different configs to be done on bucket
        • Retirement Policy
        • Archive and Restore from archive
        • Backup Data
        • Configure Index Size
        • Partition Index Data
  • Licensing
    • License data injested per data not data stored.
    • Daily indexing volume is measured from midnight to midnight by clock on license master.
    • License Types
      • Standard
      • Enterprise Trial
      • Sales Trial
      • DEV/TEST
      • Free
      • Industrial IOT
      • Forwarder
  • Apps and Addons
    • Apps
      • Visualization
      • Analysis
      • Reports and Dashboard
      • User Interface
    • Add-Ons
      • Data Enrichment
      • Tags
      • Data Models
      • Datasets
    • Apps is a set of configuration files.
    • Addons are subset of apps. This will have a UI.
    • App can be from Splunk or Certified by Splunk by third parties. Apps can be free or premium.
• Configuration Files
  • Text documents that end in .conf extension and stored in /etc. System for global config files and app for application conf file. Splunk determines which conf files to use based on the app. Default directory /etc/app/default contains preconfigured conf files. Do not modify the conf files here. Place the modified conf files in /etc/<app>/local directory.
  • Conf files have [Stanza] and attribute key value pairs.