Making sense of your data to give new insight - Elasticsearch at Findability Day 2014
1. Making sense of your data
to give you actionable insights
Presented by: Dimitri Marx, Sales Engineer
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
!
Team here today:
Sejal Korenromp, Arie Chapman
2. What about Elasticsearch the Company?
!
• Drive the ELK Projects forward, great things to come!
• Commercial products: Marvel to monitor and manage
ELK
• Support 100s of Companies in Production environments
• Training Developers and Ops around the world on ELK
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
2014.
ssttrriiccttllyy
pprroohhiibbiitteedd
3. Who’s using Elasticsearch?
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
CCooppyyrriigghhtt
EEllaassttiiccsseeaarrcchh
22001144..
CCooppyyiinngg,,
ppuubblliisshhiinngg
aanndd//oorr
ddiissttrriibbuuttiinngg
wwiitthhoouutt
wwrriitttteenn
ppeerrmmiissssiioonn
iiss
ssstttrrriiiccctttlllyyy
ppprrrooohhhiiibbbiiittteeeddd
4. A quick history of Search
Who doesn’t like a history lesson :-)
!
!
!
!
!
Disclaimer: includes fire
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
5. Life before Elasticsearch!
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
6. Why Search?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
7. How else would you find this?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
8. Or this?
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
9. Or this?
Eyjafjallajökull
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
10. Kinds of Search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
11. People search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
12. Geographic Search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
13. Product Search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
14. Knowledge Search
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
15. Thanks for listening!
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
16. Something else?
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
17. USS Car Carrier
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
18. Search is more than…
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
!
?q=what+is+the+difference+between
+holland+and+netherlands"
!
Don’t know? Watch this:"
https://www.youtube.com/watch?v=eE_IUPInEuc
19. What if we could ask
questions like…
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
20. Are we the victims of a DDOS attack right now?
What searches are trending right now?
Where are my customers coming from right now?
Are there any anomalies in our data right now?
How long is it taking for my customers to get
through our service process?
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
22. Data From
Any Source
Instantly
Analyze
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
2014.
ssttrriiccttllyy
pprroohhiibbiitteedd
Actionable
Insights
The ELK Stack
Logstash Elasticsearch Kibana
23. Clean & enrich before storage
Mobile
Devices
Financial
Web Trends
Data
Logstash
Collect From Any Source
Game
Data
CRM
e-commerce
Log Data
News
Articles
Social
Data
Internet of
things
Enrich: Geo code IP address, aggregate fields, etc.
Clean: Filter out junk, mask, normalize into JSON
24. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
25. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
26. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
27. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
RESTful
API
28. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
RESTful
API JSON
over HTTP
29. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
RESTful
API
JSON
over HTTP
schema
free
30. Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
RESTful
API
JSON
over HTTP
schema
free
multi
tenancy
31. open-source
Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
distributed
scales
massively
high
availability
RESTful
API
JSON
over HTTP
schema
free
multi
tenancy
32. open-source
Elasticsearch
real time,
search and
analytics engine
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
Lucene
based
distributed
scales
massively
high
availability
RESTful
API
JSON
over HTTP
schema
free
multi
tenancy
33. Kibana: Easily Explore Data
• See the Bigger “Picture” from your data
• Explore from 50K feet down to the ground
• Bring together a variety of Machine and User Data, New
Correlations
• Constant query - not a point in a time - live stream
34. Combine Search and Analytics on structured
and unstructured data in a scalable way
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
35. You can always go back to this
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited
36. It’s showtime!
Copyright
Elasticsearch
2013.
Copying,
publishing
and/or
distributing
without
written
permission
is
strictly
prohibited