This document discusses Splunk's HTTP Event Collector, which allows sending event data to Splunk via a token-based JSON API. Some key points covered include: - The HTTP Event Collector provides a simple way to send events from anywhere to Splunk using HTTP and tokens. - Events can be sent directly using HTTP requests or via supported logging libraries for languages like .NET, Java and JavaScript. - The presentation demonstrates configuring and using the HTTP Event Collector via the CLI, as well as with CURL and Node.js. It also discusses scaling, high availability, and third party integrations.

1. Copyright
©
2015
Splunk
Inc.
Glenn
Block
(@gblock)
–
Principal
Product
Manager
Jian
Lee
–
Senior
SoFware
Engineer
Splunk
Developer
PlaKorm
&
Core
HTTP
Event
Collector,
Simplified
Developer
Logging
Andrew
Phillips
Senior
SE,
Splunk

2. Disclaimer
2
During
the
course
of
this
presentaUon,
we
may
make
forward
looking
statements
regarding
future
events
or
the
expected
performance
of
the
company.
We
cauUon
you
that
such
statements
reflect
our
current
expectaUons
and
esUmates
based
on
factors
currently
known
to
us
and
that
actual
events
or
results
could
differ
materially.
For
important
factors
that
may
cause
actual
results
to
differ
from
those
contained
in
our
forward-­‐looking
statements,
please
review
our
filings
with
the
SEC.
The
forward-­‐
looking
statements
made
in
the
this
presentaUon
are
being
made
as
of
the
Ume
and
date
of
its
live
presentaUon.
If
reviewed
aFer
its
live
presentaUon,
this
presentaUon
may
not
contain
current
or
accurate
informaUon.
We
do
not
assume
any
obligaUon
to
update
any
forward
looking
statements
we
may
make.
In
addiUon,
any
informaUon
about
our
roadmap
outlines
our
general
product
direcUon
and
is
subject
to
change
at
any
Ume
without
noUce.
It
is
for
informaUonal
purposes
only
and
shall
not,
be
incorporated
into
any
contract
or
other
commitment.
Splunk
undertakes
no
obligaUon
either
to
develop
the
features
or
funcUonality
described
or
to
include
any
such
feature
or
funcUonality
in
a
future
release.

3. HTTP
Event
Collector
3
• A
new
token-­‐based
JSON
API
for
events
• Send
events
directly
from
anywhere
(servers,
mobile
devices,
IOT)
• Easy
to
configure
/
works
out
of
the
box.
• Easy
to
secure
• Highly
performant,
scalable
and
available

4. How
you
use
• Enable
HTTP
Event
Collector
• Create/Get
a
token
• Send
events
to
Splunk
using
the
token
– Use
HTTP
Directly
ê Create
a
POST
request
and
set
the
Auth
header
with
the
token
ê POST
JSON
in
our
event
format
to
the
collector
– Use
logging
libraries
ê Support
for
.NET,
Java
and
JavaScript
loggers
4

5. Demo
Configuring
HTTP
Event
Collector

6. Demo
Using
the
HTTP
Event
Collector
With
CURL

7. Sending
data
//send
with
curl
curl
-­‐k
https://localhost:8088/services/collector
-­‐H
'Authorization:
Splunk
46931F1C-­‐352C-­‐4DF6-­‐820C-­‐
F2689CF88494'
-­‐d
'{"event":"Hello
Event
Collector"}'
7

8. Overriding
defaults
8

9. Demo
Using
the
HTTP
Event
Collector
With
nodejs

10. ./splunk_hhpinput/local/inputs.conf
Global
Stanza
Token
Stanza
Token
Name
Enable/Disable
the
collector
Auth
token
Enable/Disable
the
token
Default
metadata
Default
metadata
Default
index
Allowed
indexes

11. ./splunk_hhpinput/defaults/inputs.conf
Default
port
SSL
Enabled
by
default
Distributed
deployment
disabled

12. Event
Collector
CLI
12
./bin/splunk
hhp-­‐event-­‐collector
help

13. Permissions
and
delegaUon
HTTP
Event
Collector
requires
the
edit_token_h7p
cap.
You
can
delegate
token
admin
to
devops
/
eng
Token
admins
can
only
manage
the
feature,
they
do
not
have
any
other
admin
permissions
in
Splunk
13

14. A
few
Ups
Create
tokens
per
app,
department,
component,
service.
etc.
Not
per
user
or
device
especially
if
you
are
talking
about
a
large
number
(>
10000)
Consider
parUUoning
tokens
to
different
indexes.
This
will
speed
up
searches
and
make
it
easy
to
archive
Consider
delegaUng
token
management
to
devops/eng
Explicitly
set
allowed
indexes
on
the
token.
If
not
set,
the
token
can
send
data
to
any
index.
Use
HTTP
over
HTTPS
when
you
can.
You
can
get
about
a
30%
performance
gain.
Ask
your
devs
to
batch
events.
It
greatly
improves
throughtput.
14

15. 15

16. Scale
and
High
Availability
16
Indexers
Search
Head
/
Deployment
Server

17. Scale
and
High
Availability
17
Event
Collectors
Indexers
Search
Heads

18. Distributed
deployment
HTTP
Event
Collector
can
scale
to
meet
your
needs!
• Built
into
splunkd,
nothing
special
to
install
• Run
directly
on
the
indexer
• Or
run
on
a
dedicated
Collector
instance
and
forward
to
an
indexer
• Uses
Deployment
Server
to
sync
tokens
across
the
Collector
instances
18

19. How
to
setup
a
DS
client
splunk
set
deploy-­‐poll
[host]:8088
splunk
enable
deploy
server
splunk
restart
19

20. Demo
Distributed
deployment

21. Demo
TroubleshooUng/
Monitoring

22. 3rd
party
integraUons
22

23. Send
your
container
logs
DIRECTLY
to
Splunk
23

24. How
it
works
A
new
log
driver
capture
container’s
stdout
and
pushes
to
Splunk
Currently
it
is
in
development,
but
should
be
out
of
the
box
soon.
We’re
contribuUng
to
Docker!!!!!!!
docker
run
-­‐-­‐log-­‐driver=splunk
-­‐-­‐log-­‐opt
splunk-­‐
token=F81DD289-­‐863D-­‐45EF-­‐B9CE-­‐A7D3514AF2C7
-­‐-­‐log-­‐opt
splunk-­‐
url=h7ps://10.20.17.169:8088
-­‐-­‐log-­‐opt
splunk-­‐
insecureskipverify=true
hello-­‐world
24

25. And
finally,
some
useful
resources:
Developer
page
for
HEC:
– hhp://dev.splunk.com/view/event-­‐collector/SP-­‐CAAAE6M
nodejs
logger
–
hhps://www.npmjs.com/package/splunk-­‐bunyan-­‐logger
JS
Logging
resources:
– hhp://dev.splunk.com/view/splunk-­‐logging-­‐javascript/SP-­‐CAAAE6U
HTML5
code
for
shake
demo
– hhps://github.com/splunk/parallel-­‐piper
25

26. Demo:
Docker
driver
26

27. Next
steps?
27
Breakouts
Ø Liberate
Your
ApplicaUon
Logging
More
informaUon
Ø docs.splunk.com,
see
"Gevng
Data
In"
Ø dev.splunk.com
Come
by
the
Developer
Booth
and
say
hi
/
ask
quesSons!
Related
breakout
sessions
and
acUviUes…

28. THANK
YOU