This document discusses big data architectures using Splunk, Hadoop, and relational databases. It begins with an overview of Splunk's scalability and real-time analytics capabilities. It then discusses Hunk, an analytics platform for Hadoop that provides self-service analytics. The document also examines using structured data in Splunk and connecting to relational databases. A case study examines challenges with the open source Hadoop ecosystem. Finally, it outlines a real-world customer architecture that uses Splunk for machine data, Hadoop for storage, Hunk for analytics, and connects to relational databases.

1. ©
2015
Splunk
Inc.
Real
World
Big
Data
Architecture
–
Splunk,
Hadoop,
RDBMS
Naman
Joshi
– Snr Sales
Engineer

2. Disclaimer
2
During
the
course
of
this
presentation,
we
may
make
forward
looking
statements
regarding
future
events
or
the
expected
performance
of
the
company.
We
caution
you
that
such
statements
reflect
our
current
expectations
and
estimates
based
on
factors
currently
known
to
us
and
that
actual
events
or
results
could
differ
materially.
For
important
factors
that
may
cause
actual
results
to
differ
from
those
contained
in
our
forward-­‐looking
statements,
please
review
our
filings
with
the
SEC.
The
forward-­‐looking
statements
made
in
the
this
presentation
are
being
made
as
of
the
time
and
date
of
its
live
presentation.
If
reviewed
after
its
live
presentation,
this
presentation
may
not
contain
current
or
accurate
information.
We
do
not
assume
any
obligation
to
update
any
forward
looking
statements
we
may
make.
In
addition,
any
information
about
our
roadmap
outlines
our
general
product
direction
and
is
subject
to
change
at
any
time
without
notice.
It
is
for
informational
purposes
only
and
shall
not,
be
incorporated
into
any
contract
or
other
commitment.
Splunk
undertakes
no
obligation
either
to
develop
the
features
or
functionality
described
or
to
include
any
such
feature
or
functionality
in
a
future
release.

3. Agenda
Splunk
Big
Data
Architecture
Alternative
Open
Source
Approach
Real-­‐World
Customer
Architecture
Discussion
Q/A
(Demo)

4. Who’s
This
Dude?
Naman
Joshi
nbjoshi@splunk.com
Senior
Sales
Engineer
• Splunk
user
since
2008
• Started
with
Splunk
in
Feb
2014
• Former
Splunk
customer
in
the
Financial
Services
Industry
• Big
Data
Subject
Matter
Expert

5. Big
Data
Technologies
5

6. 6

7. Splunk
Scalability
7

8. Splunk
Real-­‐Time
Analytics
8

9. Hunk
– Analytics
Platform
for
Hadoop
9

10. HUNK
Unique
Features
10

11. HUNK
Provides
Self-­‐Service
Analytics
For
Hadoop
11

12. HUNK
Provides
Self-­‐Service
Analytics
For
Hadoop
Enterprise
Architect
• Adapt
your
architecture
for
big
data
• Hadoop
shared-­‐service
departments
offer
self-­‐service
analytics
• Data
scientists
can
focus
on
custom
analytics,
not
be
data
butlers
Business
Analyst Developer
• Save
time
by
just
pointing
at
Hadoop
• Avoid
fixed-­‐schemas
and
low-­‐level
tooling
• Answer
questions
iteratively
without
waiting
for
MapReduce
jobs
to
finish
• Build
scalable
big
data
apps
on
top
of
data
in
Hadoop
• Use
the
development
languages
and
tools
you
know
and
like
Pivot
Data
Model
Development
Environment
Interactive
Search
12

13. What
about
Structured
Data?
13

14. Use
Cases
for
Structured
Data
in
Splunk
14

15. Machine
Data
– Delivers
Real-­‐time
insights
15

16. Structured
Data
– Contains
Business
Context
16

17. Splunk
DB
Connect
17

18. Case
Study
–
Open
Source
Alternative

19. Hadoop
Ecosystem
Options
19

20. Hadoop
Advantage/Disadvantage
20

21. Easy
storage
but
hard
analytics:
difficult
to
explore,
analyze,
visualize
Complex
technology:
many
open
source
projects
Hard-­‐to-­‐staff
skills:
must
write
MapReduce
jobs
or
fixed
schemas
21
Hadoop
(MapReduce
&
HDFS)
YARN
DataFu
H
i
v
e
Mahout Pig
Sqoop
Wide
Range
of
Open
Source
Projects
for
Hadoop
Analytics
Azkaban
Getting
Value
from
Hadoop
Data
is
Challenging

22. What
Does
Gartner
Say?
22
TROUGH OF
DISILLUSIONMENT
TECHNOLOGY
TRIGGER
PEAK OF
INFLATED
EXPECTATIONS
SLOPE OF
ENLIGHTENMENT
PLATEAU OF
PRODUCTIVITY
VISIBILITY
TIME
My
most
advanced
Hadoop
clients
are
also
getting
disillusioned
…
The
only
consistent
success,
reported
by
my
clients,
is
with
Splunk.
Svetlana
Sicular,
Gartner
Research
Director,
January
22,
2013
“ “
Many
Hadoop
customers
22

23. Case
Study
–
Real
World
Architecture

24. Summary
Architecture
24

25. Splunk
Deployment
Architecture
25

26. Hadoop
Architecture
26

27. Splunk
+
HUNK
=
All
The
Data
27

28. DB
Connect
Architecture
28

29. Summary
Architecture
29

30. Discussion

31. ©
2015
Splunk
Inc.
Thank
You
www.splunk.com/bigdata