splunk
Splunk defensive software designed to analyse massive amounts of data from multiple log sources and narrow in on the actually important data. Splunk can help security teams visualize data, make charts, and explore that data with powerful search capabilities.
There are plenty of options to what we can do with splunk, we use the software for security, business analysis, monitor performance on our software systems. As a SIEM ( security information and event management) solution, splunk is extremely efficient and that is the context of what we are going to be using splunk for today.
splunk architecture
Splunk is hosted on a cloud based enviroment, its architecture is split into several components.
fowarders- are responsible for data collection, they forward the raw machine data to the indexers. fowarders will ingest data from; windows event logs, firewall data, web access logs, linux logs, database logs, etc.
indexers- organise the raw machine data they have received from the indexers, they process it and make it searchable. They will also sort the data into directories. Indexers will also process search queries from users and provide the results.
search head- splunks user interface, dispatches search jobs to indexers and allows users to interact with splunk.
additional components
deployment servers, manages instances and configurations across machines,
cluster masters, coordinate data amount distributors in distributed systems
license masters, enforce licenses policies
spl queries (search processing language)
Splunk’s query language is SPL, used to communicate with the UI. Allowing user to search filter and manipulate indexed data. SPL contains over 100 commands, functions, arguments, and clauses. learning the basics of splunk queries is crucial to operating Splunk.
basic syntax of a splunk search
We can narrow down our searches using keywords, boolean operators, and wildcard operators.
boolean operators: combine search terms to refine results
and- returns events that contain all of the specified terms. For example we can combine two keywords using the ‘and’ command such as ‘database’ and ‘error’ so we are given all the searches with both ‘database’ and ‘error’
or- provides search results using at least one of the terms specified. If we use ‘database’ ‘or’ ‘error’ our search results will be provided with at least one of the terms.
not- provides search results excluding the term specified, if we have "‘error not database’ our search results will contain errors but no databases even if their is ‘error’ present in the data.
keywords: simply terms to search for in your data
the basic syntax for searching with keywords looks like:
index=your_index is just the index you are searching in
sourcetype=your_sourcetype is the source you are searching in
then keyword1 and keyword2 are the keywords of your choice.
Wildcard operators: essentially allows you to match patterns with similar text strings not a result that’s exactly the same.
the asterix command () when used with a term such as ‘error*’ will provide search results including ‘errors’ ‘error message’. this is useful if we want to search for all variations of an error message.
fields and comparison operators: when we specify fields, we can use comparison operators to sort fields based on values. The comparison operators are (=,!=,<,>,<=,>=).
=,