SlideShare a Scribd company logo
Copyright © 2015 Splunk Inc.
Glenn Block (@gblock) – Principal Product Manager
Jian Lee – Senior Software Engineer
Splunk Developer Platform & Core
HTTP Event Collector,
Simplified Developer Logging
Disclaimer
2
During the course of this presentation, we may make forward looking statements regarding future events
or the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results
could differ materially. For important factors that may cause actual results to differ from those contained
in our forward-looking statements, please review our filings with the SEC. The forward-looking
statements made in the this presentation are being made as of the time and date of its live presentation.
If reviewed after its live presentation, this presentation may not contain current or accurate information.
We do not assume any obligation to update any forward looking statements we may make.
In addition, any information about our roadmap outlines our general product direction and is subject to
change at any time without notice. It is for informational purposes only and shall not, be incorporated
into any contract or other commitment. Splunk undertakes no obligation either to develop the features
or functionality described or to include any such feature or functionality in a future release.
Demo
App Monitoring
HTTP Event Collector
4
• A new token-based JSON API for
events
• Send events directly from anywhere
(servers, mobile devices, IOT)
• Easy to configure / works out of the
box.
• Easy to secure
• Highly performant, scalable and
available
Demo
Configuring
HTTP Event Collector
How you use
• Enable HTTP Event Collector
• Create/Get a token
• Send events to Splunk using the token
– Use HTTP Directly
 Create a POST request and
set the Auth header with the token
 POST JSON in our event format to the
collector
– Use logging libraries
 Support for .NET, Java and JavaScript loggers
6
Sending data
//send with curl
curl -k https://localhost:8088/services/collector
-H 'Authorization: Splunk 46931F1C-352C-4DF6-
820C-F2689CF88494' -d '{"event":"Hello Event
Collector"}'
7
./splunk_httpinput/local/inputs.confGlobal
Stanza
Token
Stanza
Token Name
Enable/Disable
the collector
Auth tokenEnable/Disable
the token
Default metadata
Default metadata
Default index
Allowed indexes
./splunk_httpinput/defaults/inputs.conf
Default port
SSL Enabled by default
Distributed deployment disabled
Event Collector CLI
10
./bin/splunk http-event-collector help
Permissions and delegation
HTTP Event Collector requires the
edit_token_http cap.
You can delegate token admin to
devops / eng
Token admins can only manage the
feature, they do not have any other
admin permissions in Splunk
11
A few tips
Create tokens per app, department, component, service. etc. Not per user
or device especially if you are talking about a large number (> 10000)
Consider partitioning tokens to different indexes. This will speed up
searches and make it easy to archive
Consider delegating token management to devops/eng
Explicitly set allowed indexes on the token. If not set, the token can send
data to any index.
Use HTTP over HTTPS when you can. You can get about a 30% performance
gain.
Ask your devs to batch events. It greatly improves throughtput.
12
13
Scale and High Availability
14
Indexers
Search Head /
Deployment Server
Scale and High Availability
15
Event Collectors Indexers Search Heads
Distributed deployment
HTTP Event Collector can scale to meet your needs!
• Build in to splunkd, nothing special to install
• Run directly on the indexer
• Or run on a dedicated Collector instance and forward to an indexer
• Uses Deployment Server to to sync tokens across the Collector
instances
16
How to setup a DS client
splunk set deploy-poll [host]:8088
splunk enable deploy server
splunk restart
17
Demo
Distributed
deployment
Demo
Troubleshooting/
Monitoring
3rd party integrations
20
Send your container logs DIRECTLY to Splunk
21
How it works
A new log driver capture container’s stdout and pushes to Splunk
Currently it is in development, but should be out of the box soon.
We’re contributing to Docker!!!!!!!
docker run --log-driver=splunk --log-opt splunk-token=F81DD289-
863D-45EF-B9CE-A7D3514AF2C7 --log-opt splunk-
url=https://10.20.17.169:8088 --log-opt splunk-
insecureskipverify=true hello-world
22
Demo:
Docker driver
23
Next steps?
24
Breakouts
 Liberate Your Application Logging
More information
 docs.splunk.com, see "Getting Data In"
 dev.splunk.com
Come by the Developer Booth and say hi / ask questions!
Related breakout sessions and activities…
THANK YOU

More Related Content

What's hot

Continuous Delivery in the Cloud with Bitbucket Pipelines
Continuous Delivery in the Cloud with Bitbucket PipelinesContinuous Delivery in the Cloud with Bitbucket Pipelines
Continuous Delivery in the Cloud with Bitbucket Pipelines
Atlassian
 
Tracking Huge Files with Git LFS
Tracking Huge Files with Git LFSTracking Huge Files with Git LFS
Tracking Huge Files with Git LFS
Atlassian
 
Serverless Orchestration with Azure Durable Functions
Serverless Orchestration with Azure Durable FunctionsServerless Orchestration with Azure Durable Functions
Serverless Orchestration with Azure Durable Functions
Callon Campbell
 
What's New in Puppet Enterprise 2016.4
What's New in Puppet Enterprise 2016.4What's New in Puppet Enterprise 2016.4
What's New in Puppet Enterprise 2016.4
Puppet
 
Scaling Your First 1000 Containers with Docker
Scaling Your First 1000 Containers with DockerScaling Your First 1000 Containers with Docker
Scaling Your First 1000 Containers with Docker
Atlassian
 
IoT Google Cloud Functions with Firebase
IoT Google Cloud Functions with FirebaseIoT Google Cloud Functions with Firebase
IoT Google Cloud Functions with Firebase
Shang Yi Lim
 
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
confluent
 
A Microservices Journey - Susanne Kaiser
A Microservices Journey - Susanne KaiserA Microservices Journey - Susanne Kaiser
A Microservices Journey - Susanne Kaiser
Thoughtworks
 
What's New in Puppet Enterprise 2016.5
What's New in Puppet Enterprise 2016.5What's New in Puppet Enterprise 2016.5
What's New in Puppet Enterprise 2016.5
Puppet
 
Apache Kafka® Delivers a Single Source of Truth for The New York Times
Apache Kafka® Delivers a Single Source of Truth for The New York TimesApache Kafka® Delivers a Single Source of Truth for The New York Times
Apache Kafka® Delivers a Single Source of Truth for The New York Times
confluent
 
Spring 5.0 meets reactive programming
Spring 5.0 meets reactive programmingSpring 5.0 meets reactive programming
Spring 5.0 meets reactive programming
Claudio Eduardo de Oliveira
 
Iot with google actions (optimised)
Iot with google actions (optimised)Iot with google actions (optimised)
Iot with google actions (optimised)
Shang Yi Lim
 
OpenFaaS - zero serverless in 60 seconds anywhere with case-studies
OpenFaaS - zero serverless in 60 seconds anywhere with case-studiesOpenFaaS - zero serverless in 60 seconds anywhere with case-studies
OpenFaaS - zero serverless in 60 seconds anywhere with case-studies
Alex Ellis
 
OSMC 2013 | The future of Nagios by Andreas Ericsson
OSMC 2013 | The future of Nagios by Andreas EricssonOSMC 2013 | The future of Nagios by Andreas Ericsson
OSMC 2013 | The future of Nagios by Andreas Ericsson
NETWAYS
 
Cf summit-2016-monitoring-cf-sensu-graphite
Cf summit-2016-monitoring-cf-sensu-graphiteCf summit-2016-monitoring-cf-sensu-graphite
Cf summit-2016-monitoring-cf-sensu-graphite
Jeff Barrows
 
OpenFaaS 2019 Project Update
OpenFaaS 2019 Project UpdateOpenFaaS 2019 Project Update
OpenFaaS 2019 Project Update
Alex Ellis
 
Introducing Tupilak, Snowplow's unified log fabric
Introducing Tupilak, Snowplow's unified log fabricIntroducing Tupilak, Snowplow's unified log fabric
Introducing Tupilak, Snowplow's unified log fabric
Alexander Dean
 
LabGauge - LRIG Late Night
LabGauge - LRIG Late NightLabGauge - LRIG Late Night
LabGauge - LRIG Late Night
xi2elic
 
Windows Azure PowerShell Cmdlets
Windows Azure PowerShell CmdletsWindows Azure PowerShell Cmdlets
Windows Azure PowerShell Cmdlets
Pavel Revenkov
 
Creating a DDP API with Meteor
Creating a DDP API with MeteorCreating a DDP API with Meteor
Creating a DDP API with Meteor
Lukas van Driel
 

What's hot (20)

Continuous Delivery in the Cloud with Bitbucket Pipelines
Continuous Delivery in the Cloud with Bitbucket PipelinesContinuous Delivery in the Cloud with Bitbucket Pipelines
Continuous Delivery in the Cloud with Bitbucket Pipelines
 
Tracking Huge Files with Git LFS
Tracking Huge Files with Git LFSTracking Huge Files with Git LFS
Tracking Huge Files with Git LFS
 
Serverless Orchestration with Azure Durable Functions
Serverless Orchestration with Azure Durable FunctionsServerless Orchestration with Azure Durable Functions
Serverless Orchestration with Azure Durable Functions
 
What's New in Puppet Enterprise 2016.4
What's New in Puppet Enterprise 2016.4What's New in Puppet Enterprise 2016.4
What's New in Puppet Enterprise 2016.4
 
Scaling Your First 1000 Containers with Docker
Scaling Your First 1000 Containers with DockerScaling Your First 1000 Containers with Docker
Scaling Your First 1000 Containers with Docker
 
IoT Google Cloud Functions with Firebase
IoT Google Cloud Functions with FirebaseIoT Google Cloud Functions with Firebase
IoT Google Cloud Functions with Firebase
 
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
Tackling Kafka, with a Small Team ( Jaren Glover, Robinhood) Kafka Summit SF ...
 
A Microservices Journey - Susanne Kaiser
A Microservices Journey - Susanne KaiserA Microservices Journey - Susanne Kaiser
A Microservices Journey - Susanne Kaiser
 
What's New in Puppet Enterprise 2016.5
What's New in Puppet Enterprise 2016.5What's New in Puppet Enterprise 2016.5
What's New in Puppet Enterprise 2016.5
 
Apache Kafka® Delivers a Single Source of Truth for The New York Times
Apache Kafka® Delivers a Single Source of Truth for The New York TimesApache Kafka® Delivers a Single Source of Truth for The New York Times
Apache Kafka® Delivers a Single Source of Truth for The New York Times
 
Spring 5.0 meets reactive programming
Spring 5.0 meets reactive programmingSpring 5.0 meets reactive programming
Spring 5.0 meets reactive programming
 
Iot with google actions (optimised)
Iot with google actions (optimised)Iot with google actions (optimised)
Iot with google actions (optimised)
 
OpenFaaS - zero serverless in 60 seconds anywhere with case-studies
OpenFaaS - zero serverless in 60 seconds anywhere with case-studiesOpenFaaS - zero serverless in 60 seconds anywhere with case-studies
OpenFaaS - zero serverless in 60 seconds anywhere with case-studies
 
OSMC 2013 | The future of Nagios by Andreas Ericsson
OSMC 2013 | The future of Nagios by Andreas EricssonOSMC 2013 | The future of Nagios by Andreas Ericsson
OSMC 2013 | The future of Nagios by Andreas Ericsson
 
Cf summit-2016-monitoring-cf-sensu-graphite
Cf summit-2016-monitoring-cf-sensu-graphiteCf summit-2016-monitoring-cf-sensu-graphite
Cf summit-2016-monitoring-cf-sensu-graphite
 
OpenFaaS 2019 Project Update
OpenFaaS 2019 Project UpdateOpenFaaS 2019 Project Update
OpenFaaS 2019 Project Update
 
Introducing Tupilak, Snowplow's unified log fabric
Introducing Tupilak, Snowplow's unified log fabricIntroducing Tupilak, Snowplow's unified log fabric
Introducing Tupilak, Snowplow's unified log fabric
 
LabGauge - LRIG Late Night
LabGauge - LRIG Late NightLabGauge - LRIG Late Night
LabGauge - LRIG Late Night
 
Windows Azure PowerShell Cmdlets
Windows Azure PowerShell CmdletsWindows Azure PowerShell Cmdlets
Windows Azure PowerShell Cmdlets
 
Creating a DDP API with Meteor
Creating a DDP API with MeteorCreating a DDP API with Meteor
Creating a DDP API with Meteor
 

Viewers also liked

Prodrugs
ProdrugsProdrugs
Prodrugs
Aman Kumar Naik
 
Prodrug
ProdrugProdrug
Antihypertensives drugs
Antihypertensives drugsAntihypertensives drugs
Antihypertensives drugs
Nikhil Bansal
 
Prodrug strategy
Prodrug strategyProdrug strategy
Prodrug strategy
Anvita Bharati
 
Prodrugs
ProdrugsProdrugs
Prodrugs
Gaurav Kr
 
Prodrugs
ProdrugsProdrugs
Prodrugs
ravipharmabwm
 
Prodrugs - concept & Applications
Prodrugs - concept & ApplicationsProdrugs - concept & Applications
Prodrugs - concept & Applications
Janet Thomas
 
Antihypertensives - drdhriti
Antihypertensives - drdhritiAntihypertensives - drdhriti
Antihypertensives - drdhriti
http://neigrihms.gov.in/
 
Antihypertensive drugs
Antihypertensive drugsAntihypertensive drugs
Antihypertensive drugs
Dipesh Kakadiya
 
Anti Hypertensive Drugs
Anti Hypertensive DrugsAnti Hypertensive Drugs
Anti Hypertensive Drugs
mohammed sediq
 
Antihypertensive drugs
Antihypertensive drugsAntihypertensive drugs
Antihypertensive drugs
pavelbd
 
Antihypertensives
AntihypertensivesAntihypertensives
Antihypertensives
raj kumar
 

Viewers also liked (12)

Prodrugs
ProdrugsProdrugs
Prodrugs
 
Prodrug
ProdrugProdrug
Prodrug
 
Antihypertensives drugs
Antihypertensives drugsAntihypertensives drugs
Antihypertensives drugs
 
Prodrug strategy
Prodrug strategyProdrug strategy
Prodrug strategy
 
Prodrugs
ProdrugsProdrugs
Prodrugs
 
Prodrugs
ProdrugsProdrugs
Prodrugs
 
Prodrugs - concept & Applications
Prodrugs - concept & ApplicationsProdrugs - concept & Applications
Prodrugs - concept & Applications
 
Antihypertensives - drdhriti
Antihypertensives - drdhritiAntihypertensives - drdhriti
Antihypertensives - drdhriti
 
Antihypertensive drugs
Antihypertensive drugsAntihypertensive drugs
Antihypertensive drugs
 
Anti Hypertensive Drugs
Anti Hypertensive DrugsAnti Hypertensive Drugs
Anti Hypertensive Drugs
 
Antihypertensive drugs
Antihypertensive drugsAntihypertensive drugs
Antihypertensive drugs
 
Antihypertensives
AntihypertensivesAntihypertensives
Antihypertensives
 

Similar to HTTP Event Collector, Simplified Developer Logging

Anz summit 2015 http event collector - sydney
Anz summit 2015   http event collector - sydneyAnz summit 2015   http event collector - sydney
Anz summit 2015 http event collector - sydney
Splunk
 
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer LoggingSplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
Splunk
 
Splunk for net developers
Splunk for net developersSplunk for net developers
Splunk for net developers
Glenn Block
 
Getting Started with Splunk Enterprises
Getting Started with Splunk EnterprisesGetting Started with Splunk Enterprises
Getting Started with Splunk Enterprises
Splunk
 
Throw Your Future Self a Lifeline with a Little DevOps Data!
Throw Your Future Self a Lifeline with a Little DevOps Data!Throw Your Future Self a Lifeline with a Little DevOps Data!
Throw Your Future Self a Lifeline with a Little DevOps Data!
merbla
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
Splunk
 
Getting Started with Splunk Enterprise Hands-On
Getting Started with Splunk Enterprise Hands-OnGetting Started with Splunk Enterprise Hands-On
Getting Started with Splunk Enterprise Hands-On
Splunk
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
Splunk
 
Splunk Enterprise 6.3 - Splunk Tech Day
Splunk Enterprise 6.3 - Splunk Tech DaySplunk Enterprise 6.3 - Splunk Tech Day
Splunk Enterprise 6.3 - Splunk Tech Day
Zivaro Inc
 
Webinar: Neuigkeiten zu Splunk Enterprise 6.3
Webinar: Neuigkeiten zu Splunk Enterprise 6.3Webinar: Neuigkeiten zu Splunk Enterprise 6.3
Webinar: Neuigkeiten zu Splunk Enterprise 6.3
Splunk
 
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
BrianFraser29
 
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
Splunk
 
Deploying Splunk on OpenShift – Part2 : Getting Data In
Deploying Splunk on OpenShift – Part2 : Getting Data InDeploying Splunk on OpenShift – Part2 : Getting Data In
Deploying Splunk on OpenShift – Part2 : Getting Data In
Eric Gardner
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
Splunk
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
Shannon Cuthbertson
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
Splunk
 
Lesser known-search-commands
Lesser known-search-commandsLesser known-search-commands
Lesser known-search-commands
pendoo
 
Qwasi Splunk and NCR Integration: Business Analytics
Qwasi Splunk and NCR Integration: Business AnalyticsQwasi Splunk and NCR Integration: Business Analytics
Qwasi Splunk and NCR Integration: Business Analytics
Timur Bagirov
 
Getting Started with Splunk Breakout Session
Getting Started with Splunk Breakout SessionGetting Started with Splunk Breakout Session
Getting Started with Splunk Breakout Session
Splunk
 
Splunk in Nordstrom: IT Operations
Splunk in Nordstrom: IT OperationsSplunk in Nordstrom: IT Operations
Splunk in Nordstrom: IT Operations
Timur Bagirov
 

Similar to HTTP Event Collector, Simplified Developer Logging (20)

Anz summit 2015 http event collector - sydney
Anz summit 2015   http event collector - sydneyAnz summit 2015   http event collector - sydney
Anz summit 2015 http event collector - sydney
 
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer LoggingSplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
SplunkSummit 2015 - HTTP Event Collector, Simplified Developer Logging
 
Splunk for net developers
Splunk for net developersSplunk for net developers
Splunk for net developers
 
Getting Started with Splunk Enterprises
Getting Started with Splunk EnterprisesGetting Started with Splunk Enterprises
Getting Started with Splunk Enterprises
 
Throw Your Future Self a Lifeline with a Little DevOps Data!
Throw Your Future Self a Lifeline with a Little DevOps Data!Throw Your Future Self a Lifeline with a Little DevOps Data!
Throw Your Future Self a Lifeline with a Little DevOps Data!
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Getting Started with Splunk Enterprise Hands-On
Getting Started with Splunk Enterprise Hands-OnGetting Started with Splunk Enterprise Hands-On
Getting Started with Splunk Enterprise Hands-On
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Splunk Enterprise 6.3 - Splunk Tech Day
Splunk Enterprise 6.3 - Splunk Tech DaySplunk Enterprise 6.3 - Splunk Tech Day
Splunk Enterprise 6.3 - Splunk Tech Day
 
Webinar: Neuigkeiten zu Splunk Enterprise 6.3
Webinar: Neuigkeiten zu Splunk Enterprise 6.3Webinar: Neuigkeiten zu Splunk Enterprise 6.3
Webinar: Neuigkeiten zu Splunk Enterprise 6.3
 
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
Brisbane MuleSoft Meetup 2023-03-22 - Anypoint Code Builder and Splunk Loggin...
 
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Sp...
 
Deploying Splunk on OpenShift – Part2 : Getting Data In
Deploying Splunk on OpenShift – Part2 : Getting Data InDeploying Splunk on OpenShift – Part2 : Getting Data In
Deploying Splunk on OpenShift – Part2 : Getting Data In
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Lesser known-search-commands
Lesser known-search-commandsLesser known-search-commands
Lesser known-search-commands
 
Qwasi Splunk and NCR Integration: Business Analytics
Qwasi Splunk and NCR Integration: Business AnalyticsQwasi Splunk and NCR Integration: Business Analytics
Qwasi Splunk and NCR Integration: Business Analytics
 
Getting Started with Splunk Breakout Session
Getting Started with Splunk Breakout SessionGetting Started with Splunk Breakout Session
Getting Started with Splunk Breakout Session
 
Splunk in Nordstrom: IT Operations
Splunk in Nordstrom: IT OperationsSplunk in Nordstrom: IT Operations
Splunk in Nordstrom: IT Operations
 

More from Glenn Block

Api Strat Portland 2017 Serverless Extensibility talk
Api Strat Portland 2017 Serverless Extensibility talkApi Strat Portland 2017 Serverless Extensibility talk
Api Strat Portland 2017 Serverless Extensibility talk
Glenn Block
 
Serverless Extensibility
Serverless ExtensibilityServerless Extensibility
Serverless Extensibility
Glenn Block
 
Deploying web apis on core clr to docker
Deploying web apis on core clr to dockerDeploying web apis on core clr to docker
Deploying web apis on core clr to docker
Glenn Block
 
Islands of Node in a sea of big data
Islands of Node in a sea of big dataIslands of Node in a sea of big data
Islands of Node in a sea of big data
Glenn Block
 
Splunk and node
Splunk and nodeSplunk and node
Splunk and node
Glenn Block
 
Splunk, a smorgasboard for developing big data solutions
Splunk, a smorgasboard for developing big data solutions Splunk, a smorgasboard for developing big data solutions
Splunk, a smorgasboard for developing big data solutions
Glenn Block
 
Splunk's api how we built it
Splunk's api   how we built itSplunk's api   how we built it
Splunk's api how we built it
Glenn Block
 
Dev sum hypemedia talk
Dev sum hypemedia talkDev sum hypemedia talk
Dev sum hypemedia talk
Glenn Block
 
Getting your head around big data
Getting your head around big dataGetting your head around big data
Getting your head around big data
Glenn Block
 
Edge
EdgeEdge
Hypermedia
HypermediaHypermedia
Hypermedia
Glenn Block
 
Realtime Conf - Lightning node.js dev
Realtime Conf - Lightning node.js devRealtime Conf - Lightning node.js dev
Realtime Conf - Lightning node.js dev
Glenn Block
 
Node js - Enterprise Class
Node js - Enterprise ClassNode js - Enterprise Class
Node js - Enterprise Class
Glenn Block
 
Node js on Windows Azure
Node js on Windows AzureNode js on Windows Azure
Node js on Windows Azure
Glenn Block
 
Building systems with rest
Building systems with restBuilding systems with rest
Building systems with rest
Glenn Block
 
Building Extensible RIAs with MEF
Building Extensible RIAs with MEFBuilding Extensible RIAs with MEF
Building Extensible RIAs with MEF
Glenn Block
 

More from Glenn Block (16)

Api Strat Portland 2017 Serverless Extensibility talk
Api Strat Portland 2017 Serverless Extensibility talkApi Strat Portland 2017 Serverless Extensibility talk
Api Strat Portland 2017 Serverless Extensibility talk
 
Serverless Extensibility
Serverless ExtensibilityServerless Extensibility
Serverless Extensibility
 
Deploying web apis on core clr to docker
Deploying web apis on core clr to dockerDeploying web apis on core clr to docker
Deploying web apis on core clr to docker
 
Islands of Node in a sea of big data
Islands of Node in a sea of big dataIslands of Node in a sea of big data
Islands of Node in a sea of big data
 
Splunk and node
Splunk and nodeSplunk and node
Splunk and node
 
Splunk, a smorgasboard for developing big data solutions
Splunk, a smorgasboard for developing big data solutions Splunk, a smorgasboard for developing big data solutions
Splunk, a smorgasboard for developing big data solutions
 
Splunk's api how we built it
Splunk's api   how we built itSplunk's api   how we built it
Splunk's api how we built it
 
Dev sum hypemedia talk
Dev sum hypemedia talkDev sum hypemedia talk
Dev sum hypemedia talk
 
Getting your head around big data
Getting your head around big dataGetting your head around big data
Getting your head around big data
 
Edge
EdgeEdge
Edge
 
Hypermedia
HypermediaHypermedia
Hypermedia
 
Realtime Conf - Lightning node.js dev
Realtime Conf - Lightning node.js devRealtime Conf - Lightning node.js dev
Realtime Conf - Lightning node.js dev
 
Node js - Enterprise Class
Node js - Enterprise ClassNode js - Enterprise Class
Node js - Enterprise Class
 
Node js on Windows Azure
Node js on Windows AzureNode js on Windows Azure
Node js on Windows Azure
 
Building systems with rest
Building systems with restBuilding systems with rest
Building systems with rest
 
Building Extensible RIAs with MEF
Building Extensible RIAs with MEFBuilding Extensible RIAs with MEF
Building Extensible RIAs with MEF
 

Recently uploaded

社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
NABLAS株式会社
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
GeorgiiSteshenko
 
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
agdhot
 
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
yuvarajkumar334
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
Timothy Spann
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
Vietnam Cotton & Spinning Association
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
dataschool1
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
keesa2
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
Vietnam Cotton & Spinning Association
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
osoyvvf
 
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
eoxhsaa
 
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
9gr6pty
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
davidpietrzykowski1
 
Senior Engineering Sample EM DOE - Sheet1.pdf
Senior Engineering Sample EM DOE  - Sheet1.pdfSenior Engineering Sample EM DOE  - Sheet1.pdf
Senior Engineering Sample EM DOE - Sheet1.pdf
Vineet
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
bmucuha
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
uevausa
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
Bisnar Chase Personal Injury Attorneys
 
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service LucknowCall Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
hiju9823
 

Recently uploaded (20)

社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
 
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
一比一原版加拿大麦吉尔大学毕业证(mcgill毕业证书)如何办理
 
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS NOTES FOR MCA
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
 
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
一比一原版多伦多大学毕业证(UofT毕业证书)学历如何办理
 
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
 
Salesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - CanariasSalesforce AI + Data Community Tour Slides - Canarias
Salesforce AI + Data Community Tour Slides - Canarias
 
Senior Engineering Sample EM DOE - Sheet1.pdf
Senior Engineering Sample EM DOE  - Sheet1.pdfSenior Engineering Sample EM DOE  - Sheet1.pdf
Senior Engineering Sample EM DOE - Sheet1.pdf
 
一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
一比一原版加拿大渥太华大学毕业证(uottawa毕业证书)如何办理
 
Drownings spike from May to August in children
Drownings spike from May to August in childrenDrownings spike from May to August in children
Drownings spike from May to August in children
 
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service LucknowCall Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
 

HTTP Event Collector, Simplified Developer Logging

  • 1. Copyright © 2015 Splunk Inc. Glenn Block (@gblock) – Principal Product Manager Jian Lee – Senior Software Engineer Splunk Developer Platform & Core HTTP Event Collector, Simplified Developer Logging
  • 2. Disclaimer 2 During the course of this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in the this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.
  • 4. HTTP Event Collector 4 • A new token-based JSON API for events • Send events directly from anywhere (servers, mobile devices, IOT) • Easy to configure / works out of the box. • Easy to secure • Highly performant, scalable and available
  • 6. How you use • Enable HTTP Event Collector • Create/Get a token • Send events to Splunk using the token – Use HTTP Directly  Create a POST request and set the Auth header with the token  POST JSON in our event format to the collector – Use logging libraries  Support for .NET, Java and JavaScript loggers 6
  • 7. Sending data //send with curl curl -k https://localhost:8088/services/collector -H 'Authorization: Splunk 46931F1C-352C-4DF6- 820C-F2689CF88494' -d '{"event":"Hello Event Collector"}' 7
  • 8. ./splunk_httpinput/local/inputs.confGlobal Stanza Token Stanza Token Name Enable/Disable the collector Auth tokenEnable/Disable the token Default metadata Default metadata Default index Allowed indexes
  • 9. ./splunk_httpinput/defaults/inputs.conf Default port SSL Enabled by default Distributed deployment disabled
  • 10. Event Collector CLI 10 ./bin/splunk http-event-collector help
  • 11. Permissions and delegation HTTP Event Collector requires the edit_token_http cap. You can delegate token admin to devops / eng Token admins can only manage the feature, they do not have any other admin permissions in Splunk 11
  • 12. A few tips Create tokens per app, department, component, service. etc. Not per user or device especially if you are talking about a large number (> 10000) Consider partitioning tokens to different indexes. This will speed up searches and make it easy to archive Consider delegating token management to devops/eng Explicitly set allowed indexes on the token. If not set, the token can send data to any index. Use HTTP over HTTPS when you can. You can get about a 30% performance gain. Ask your devs to batch events. It greatly improves throughtput. 12
  • 13. 13
  • 14. Scale and High Availability 14 Indexers Search Head / Deployment Server
  • 15. Scale and High Availability 15 Event Collectors Indexers Search Heads
  • 16. Distributed deployment HTTP Event Collector can scale to meet your needs! • Build in to splunkd, nothing special to install • Run directly on the indexer • Or run on a dedicated Collector instance and forward to an indexer • Uses Deployment Server to to sync tokens across the Collector instances 16
  • 17. How to setup a DS client splunk set deploy-poll [host]:8088 splunk enable deploy server splunk restart 17
  • 21. Send your container logs DIRECTLY to Splunk 21
  • 22. How it works A new log driver capture container’s stdout and pushes to Splunk Currently it is in development, but should be out of the box soon. We’re contributing to Docker!!!!!!! docker run --log-driver=splunk --log-opt splunk-token=F81DD289- 863D-45EF-B9CE-A7D3514AF2C7 --log-opt splunk- url=https://10.20.17.169:8088 --log-opt splunk- insecureskipverify=true hello-world 22
  • 24. Next steps? 24 Breakouts  Liberate Your Application Logging More information  docs.splunk.com, see "Getting Data In"  dev.splunk.com Come by the Developer Booth and say hi / ask questions! Related breakout sessions and activities…