Splunk produces software for searching, monitoring, and analyzing machine-generated big data. It turns machine data into valuable insights. With the Splunk log file generated using the Splunk cloud product, helps you to not only track your data over the Splunk cloud environment but also analyze and visualize the data as well.
3. What is splunk ?
● What do we do when you need information about the state of our
application, machine or software? We look at its logfiles. They tell
us the state it is in and what happened recently.
● Splunk started out as a kind of “Google for Logfiles”. It does a lot
more today but log processing is still at the product’s core. It stores
all your logs and provides very fast search capabilities roughly in the
same way Google does for the internet.
4. Brief History...
● Splunk is an American multinational corporation based in San
Francisco, California, which produces software for searching,
monitoring, and analyzing machine-generated big data.
● Splunk (the product) captures, indexes and correlates real-time data
in a searchable repository from which it can generate graphs,
reports, alerts, dashboards and visualizations.
● The company was started in 2003 by co-founders Michael Baum,
Rob Das and Erik Swan.
● With Splunk we can onboard, enrich and analyze machine data faster
than ever before.
5. We see data everywhere. We offers the leading platform for
Operational Intelligence. We look closely at what others ignore
(machine data) and find what others never see: insights that can help
make your company to be more productive, profitable, competitive
and secure.
- Splunk
6. Work ...
Turning Machine Data Into Insights :
Machine-generated data is one of the fastest growing and complex areas of big data.
It's also one of the most valuable, containing a definitive record of all user
transactions, customer behavior, machine behavior, security threats, fraudulent activity
and more. Splunk turns machine data into valuable insights.
Turning Machine Data Into Insights :
Machine-generated data is one of the fastest growing and complex areas of big data.
It's also one of the most valuable, containing a definitive record of all user
transactions, customer behavior, machine behavior, security threats, fraudulent activity
and more. Splunk turns machine data into valuable insights.
8. Why splunk ?
● Index, (no) Schema, Events :
Database requires you to define tables and fields before you can store data whereas
Splunk accepts almost anything immediately after installation. Splunk does not
have a fixed schema. Instead, it performs field extraction at search time. Many log
formats are recognized automatically.
This approach allows for great flexibility. Just as Google crawls any web page
without knowing anything about a site’s layout, Splunk indexes any kind of
machine data that can be represented as text.
● Scalability, (no) Backend :
Splunk stores data directly in the file system.
✔ Installation is superfast
✔ Scalability is easy
✔ No single point of failure
✔ Infinite retention without losing granularity
9. Splunk Products
● Splunk Enterprise
● Splunk Cloud
● Slunk Light
● Hunk
● Splunk Mint
● App for Enterprise Security
● Splunk App for Stream
● Apps and add-ons
10. Splunk Cloud
Splunk Cloud is backed by a 100% uptime SLA, scales to over 10TB/day, and
offers a highly secure environment.
Up to 10x Bursting :
Splunk Cloud supports up to 10x bursting over licensed data volumes.
Reliable Performance :
Dedicated cloud environments help ensure your service performance is not
impacted by the actions of another customer.
100% Uptime SLA :
Splunk Cloud is backed by a 100% uptime SLA.
"Splunk Cloud has widened our user base. Our operations folks use Splunk, our
application people use Splunk and our security people use Splunk. Splunk Cloud
gives you applications that let you get a huge amount of value from your data."
-Sr. Director of Information Security
11. Splunk Cloud installation..
Step-1 : Signup over splunk : https://www.splunk.com
Step-2 : Select Splunk Cloud from Splunk products available.
Step-3 : Go for your own online sandbox. You will get a cloud URL,
click over your cloud URL and start setting up credentials for
your Splunk Cloud.
Step-4 : It will redirect to its dashboard. Now click : settings icon (left
top upper corner) an goto the Universal Forwarder.
Step-5 : Follow the installation steps written and add Splunk Universal
Forwarder.
13. Some useful Splunk Commands...
➢ To Stop the splunk server: $ sudo ./splunk stop
➢ To Start the splunk server: $ sudo ./splunk start
➢ To Check status of splunk server : $ sudo ./splunk status
➢ To Check list of app: $ sudo ./splunk list app
➢ To Remove an app: $sudo ./splunk remove app <appname>
➢ To check list of monitors: $sudo ./splunk list monitor
14. Splunk Sandbox...
The Splunk Online Sandbox is a free and personal online
environment where you can explore the Splunk Enterprise features.
● Using the Splunk Online Sandbox, you can search, analyze, and
visualize your own data or pre-populated data sets. You may index
up to 5GB of data per day,
● up to a total of 28GB of data in your sandbox.
15. API
sp.js provides a common set of tracking methods:
● sp.track(event, properties, fn)
● sp.trackLink(links, event, properties)
● sp.pageview(url)
● sp.identify(userId, userTraits)
16. ● sp.track(event, properties, fn) :
Track a custom event (i.e. user action) along with a set of associated
event properties.
● sp.trackLink(links, event, properties) :
Track link clicks, including outbound links, with a custom event and
custom properties. Tracking occurs before page changes. This
automatically records properties such as the anchor (a) tag's href and
text.
17. ● sp.pageview(url)
Tracks a 'pageview' event including document title and referrer. This
is automatically called by default.
● sp.identify(userId, userTraits)
Associate a user with an ID, and record user-specific traits or
persistent properties. These persistent properties will be
automatically added as properties to any subsequent tracked event.
18. Generating Splunk Logs...
This is a Node.js backend collector for client-side data that is tracked
by sp.js Analytics JavaScript library. All tracked events are collected
in events.log.
● Install with Node.js package manager npm and Start the collector
server by typing:
* You will observe the logs getting tracked in you terminal with every action
you perform over your web-page wherein the logs are placed for tracking.
19. To use sp.js, simply paste the following snippet of code before
the closing </head> tag on your page:
Operational intelligence gives you a real-time understanding of what’s happening across your IT systems and technology infrastructure so you can make informed decisions.
javascript on the server.
Node uses V8 , the virtual machine that powers Google Chrome, for server-side pro-gramming
IT is build on google chrome runtime. It uses v8, the same run time environment, that google chrome uses. Node.js is anothe context. It allows you to run javascript as backend code outside a browser.In order to execute, javascript, that you intend to run in backend, it needs to be well interpreted and well execute , this is what node.js does with the use of v8 virtual macine.
V8 gives a huge boost performance and it is developed by google. Because it prefers straight compilation into native machine code.
Tracking : 1-3 by adding analytic library : sp.js. Which asynchronously fetch the javascript library from global CDN reducing page load time.
Page-Level
Event-Level
Collector : Specify endpoints where tracking to be made.
Collection server (server.js) and collecting events in events.log file.
Analytics & Visualization : is using a splunk forwarder that sends data to existing splunk deployment.
http://blogs.splunk.com/tag/web-analytics/
3rd party web analytics providers such as Google Analytics and Omniture SiteCatalyst. More secure
npm is the official package manager for Node.js. As of Node.js version 0.6.3, npm is bundled and installed automatically with the environment.[1] npm runs through the command line and manages dependencies for an application. It also allows users to install Node.js applications that are available on the npm registry.
npm is written entirely in JavaScript, and runs on the Node.js platform.
The Node Package Manager (npm; https://github.com/isaacs/npm) is a util-
ity that comes bundled with Node. It offers a great deal of functionality, allowing you
to easily install third-party Node modules and globally publish any Node modules
1. After clicking on first Step: You will be redirected to other page where you have to choose the package for your OS.Next install the downloaded file and check your location & Follow these steps and check a new folder with the name “splunkforwarder” will be created in your directory : computer -&gt; opt
2. Click on the second link & download the universal fordwarder credentials and paste that at described location.
3. Now run on terminal: (4th step)
/opt/splunkforwarder/bin/splunk install app /opt/splunkclouduf.spl -auth admin:changeme
4. Now restart the splunk server:
go to the path machine-name:/opt/splunkforwarder/bin
Run these commands (restarting)
sudo ./splunk stop
sudo ./splunk start
5. Now run the 5th step and run the same command from same path (as above in 4rth step)
/opt/splunkforwarder/bin/splunk add monitor -auth admin:changeme /path/to/app/logs/
*here underlined path is location of your log file that is to be monitored over the splunk server.
In the last command as above give the path to your log file (in our case events.log), ie /Documents/Projects/TutorMeDocs/events.log