Copyright © 2015 Splunk Inc.
Glenn Block (@gblock) – Principal Product Manager
Jian Lee – Senior Software Engineer
Splunk Developer Platform & Core
Liberate Your Application Logging!
Disclaimer
2
During the course of this presentation, we may make forward looking statements regarding future events
or the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results
could differ materially. For important factors that may cause actual results to differ from those contained
in our forward-looking statements, please review our filings with the SEC. The forward-looking
statements made in the this presentation are being made as of the time and date of its live presentation.
If reviewed after its live presentation, this presentation may not contain current or accurate information.
We do not assume any obligation to update any forward looking statements we may make.
In addition, any information about our roadmap outlines our general product direction and is subject to
change at any time without notice. It is for informational purposes only and shall not, be incorporated
into any contract or other commitment. Splunk undertakes no obligation either to develop the features
or functionality described or to include any such feature or functionality in a future release.
3
“Sending
events to
Splunk from
my apps is
hard!”
A disgruntled
developer:
Your apps should be freed to be Splunked!
4
Free from
requiring a
local forwarder
Free from
complex
configuration
Free from scale and
performance
bottlenecks
Free to be sent
from any client
In Splunk 6.3, they can be liberated!In Splunk 6.3, they can be liberated!
Introducing
HTTP Event
Collector
6
HTTP Event Collector
A new token-based JSON API for events
Send events directly from anywhere
(servers, mobile devices, IOT)
Easy to configure / works out of the box.
Easy to secure
Highly performant, scalable and
available
7
Demo:
HTTP Event Collector
8
Sending data
//send with curl
curl -k https://localhost:8088/services/collector
-H 'Authorization: Splunk 46931F1C-352C-4DF6-
820C-F2689CF88494' -d '{"event":"Hello Event
Collector"}'
9
How you use
• Enable HTTP Event Collector
• Create/Get a token
• Send events to Splunk using the token
– Use HTTP Directly
 Create a POST request and
set the Auth header with the token
 POST JSON in our event format to the
collector
– Use logging libraries
 Support for .NET, Java and JavaScript loggers
10
Event protocol advanced capabilities
Batching:
{"event":"event 1"}
{"event":"event 2"}
{"event":"event 3"}
11
Metadata:
{
"source":"foo",
"sourcetype":"bar",
"host": "192.168.1.1",
"time": 1441139779
"event":"Hello World"
}
Index selection:
{
"index": "collector"
"event":"Hello World"
}
JSON objects
{
"event": {
"message":"….",
"severity":"warn"
"category":"web"
}
}
Logging libraries
• Provide integrations with common
loggers (log4J, log4net, etc)
• Very friendly to developers and
easy to pull into their applications
• Provide robustness (async, retries,
batching)
• Java , .NET and node.js
12
Demo:
Loggers galore
(java, .NET, Node)
13
Retries
Logger can resend if
it is not able to reach
the server.
Uses an incremental
back-off policy by
default
14
Batching
Events can be
batched up:
– number of events
– number of bytes
– time interval
– manual flush
15
Logging libraries
Supported libraries
• Java – log4j, logback, java.util.logging
• .NET – Trace Listeners, SLAB
• Node.js – Bunyan
More to come!
16
How you get them?
• Pull the logging library of choice into your application
• Configure the logger
• Log away!
17
Scale and High Availability
18
Indexers
3rd party integrations
19
Demo:
Splunk and
AWS Lambda
20
Splunk and AWS Lambda
Monitor your Lambda functions within
Splunk!
Use Lambda to pipe events from services
like Kinesis to Splunk
Configure right in the AWS Console or
use our JavaScript and Java logging
libraries
21
http://dev.splunk.com/goto/awslambda
Liberate
your logs!
HTTP Event
Collector
22
Building Solutions on the Splunk Platform


Splunk Reference Apps
Complete, working real-world Splunk solutions
built together with partners
Splunk Developer Guide
dev.splunk.com/goto/devguide
Next steps?
24
Breakouts
 The HTTP Event Collector, a New Way for Developers to Send Events to Splunk
 Accelerating your Solution Development with Splunk Reference Apps
 Cloud, Wearables, and IoT! How Orion Leverages Splunk Cloud to Ensure Performance
and Gain Analytics from Internet-connected Accessories
More information
 dev.splunk.com/goto/awslambda
 dev.splunk.com
 blogs.splunk.com/dev
Come by the Developer Booth and say hi / ask questions!
Related breakout sessions and activities…
THANK YOU

Liberate your Application Logging

  • 1.
    Copyright © 2015Splunk Inc. Glenn Block (@gblock) – Principal Product Manager Jian Lee – Senior Software Engineer Splunk Developer Platform & Core Liberate Your Application Logging!
  • 2.
    Disclaimer 2 During the courseof this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in the this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.
  • 3.
    3 “Sending events to Splunk from myapps is hard!” A disgruntled developer:
  • 4.
    Your apps shouldbe freed to be Splunked! 4 Free from requiring a local forwarder Free from complex configuration Free from scale and performance bottlenecks Free to be sent from any client
  • 5.
    In Splunk 6.3,they can be liberated!In Splunk 6.3, they can be liberated!
  • 6.
  • 7.
    HTTP Event Collector Anew token-based JSON API for events Send events directly from anywhere (servers, mobile devices, IOT) Easy to configure / works out of the box. Easy to secure Highly performant, scalable and available 7
  • 8.
  • 9.
    Sending data //send withcurl curl -k https://localhost:8088/services/collector -H 'Authorization: Splunk 46931F1C-352C-4DF6- 820C-F2689CF88494' -d '{"event":"Hello Event Collector"}' 9
  • 10.
    How you use •Enable HTTP Event Collector • Create/Get a token • Send events to Splunk using the token – Use HTTP Directly  Create a POST request and set the Auth header with the token  POST JSON in our event format to the collector – Use logging libraries  Support for .NET, Java and JavaScript loggers 10
  • 11.
    Event protocol advancedcapabilities Batching: {"event":"event 1"} {"event":"event 2"} {"event":"event 3"} 11 Metadata: { "source":"foo", "sourcetype":"bar", "host": "192.168.1.1", "time": 1441139779 "event":"Hello World" } Index selection: { "index": "collector" "event":"Hello World" } JSON objects { "event": { "message":"….", "severity":"warn" "category":"web" } }
  • 12.
    Logging libraries • Provideintegrations with common loggers (log4J, log4net, etc) • Very friendly to developers and easy to pull into their applications • Provide robustness (async, retries, batching) • Java , .NET and node.js 12
  • 13.
  • 14.
    Retries Logger can resendif it is not able to reach the server. Uses an incremental back-off policy by default 14
  • 15.
    Batching Events can be batchedup: – number of events – number of bytes – time interval – manual flush 15
  • 16.
    Logging libraries Supported libraries •Java – log4j, logback, java.util.logging • .NET – Trace Listeners, SLAB • Node.js – Bunyan More to come! 16
  • 17.
    How you getthem? • Pull the logging library of choice into your application • Configure the logger • Log away! 17
  • 18.
    Scale and HighAvailability 18 Indexers
  • 19.
  • 20.
  • 21.
    Splunk and AWSLambda Monitor your Lambda functions within Splunk! Use Lambda to pipe events from services like Kinesis to Splunk Configure right in the AWS Console or use our JavaScript and Java logging libraries 21 http://dev.splunk.com/goto/awslambda
  • 22.
  • 23.
    Building Solutions onthe Splunk Platform   Splunk Reference Apps Complete, working real-world Splunk solutions built together with partners Splunk Developer Guide dev.splunk.com/goto/devguide
  • 24.
    Next steps? 24 Breakouts  TheHTTP Event Collector, a New Way for Developers to Send Events to Splunk  Accelerating your Solution Development with Splunk Reference Apps  Cloud, Wearables, and IoT! How Orion Leverages Splunk Cloud to Ensure Performance and Gain Analytics from Internet-connected Accessories More information  dev.splunk.com/goto/awslambda  dev.splunk.com  blogs.splunk.com/dev Come by the Developer Booth and say hi / ask questions! Related breakout sessions and activities…
  • 25.