Your SlideShare is downloading. ×

Search and Analytics (using Elasticsearch)

4,300

Published on

Slides from Costin Leau's talk on Search and Analytics (using Elasticsearch) at the 18th Big Data London meetup

Slides from Costin Leau's talk on Search and Analytics (using Elasticsearch) at the 18th Big Data London meetup

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,300
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
120
Comments
0
Likes
7
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch and Analytics(using Elasticsearch)Costin Leau
  • 2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedWhy search?
  • 3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – what’s the big deal?Basic/Metadata retrieval“Find banks with more then (x) accounts”
  • 4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – what’s the big deal?Basic/Metadata retrieval“Find banks near my location”
  • 5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – What we’re all about
  • 6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch categoriesBasic/Metadata retrievalFull-text searchHighlightingGeolocationFuzzy search (“did-you-mean”)Natural Language
  • 7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch categoriesBasic/Metadata retrievalFull-text searchHighlightingGeolocationFuzzy search (“did-you-mean”)Natural Languagedata storessearch engines
  • 8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited‘Players’ in the search marketSearch engines- Google/Bing/Yahoo!/Ask.com/Yandex/BaiduOpen-Source- Sphinx- Apache Lucene- Elasticsearch- Solr- SenseiEnterprise Search- Oracle Endeca / MDEX- HP Autonomy- Exalead- IBM Enterprise Search
  • 9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearch
  • 10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big Data
  • 11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big DataLightweight
  • 12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big DataLightweightPopular: >200K downloads/month
  • 13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUsers
  • 14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUsers
  • 15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPlatform Adoptionhttp://www.thoughtworks.com/radar#platforms 2013
  • 16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPlatform Adoptionhttp://www.thoughtworks.com/radar#platforms 2013
  • 17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Text Searchhttps://github.com/blog/1381-a-whole-new-code-search
  • 18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearches 50,000,000 venues every day usingElasticsearchUse Case - Geolocation
  • 19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case – Support/Reporting
  • 20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Centralized Logging
  • 21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Pure Analytics
  • 22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch and Big Data
  • 23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedA Holistic View of a Big Data SystemETLRealTimeStreamsUnstructured Data (HDFS)RT SemistructuredDatabase(hBase,Cassandra,Mongo)Big SQL(Greenplum,AsterData,Etc…)BatchProcessingReal-TimeProcessing(s4, storm)Analytics
  • 24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedA Holistic View of a Big Data SystemETLRealTimeStreamsUnstructured Data (HDFS)RT SemistructuredDatabase(hBase,Cassandra,Mongo)Big SQL(Greenplum,AsterData,Etc…)BatchProcessingAnalyticsReal-TimeProcessing(s4, storm)
  • 25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHadoop eco-systemHadoop Distributed File System (HDFS)Map Reduce Framework (MapRed)
  • 26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHadoop eco-systemHadoop Distributed File System (HDFS)Map Reduce Framework (MapRed)
  • 27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearch + Hadoop0102030405060M/R Pig HiveRaw w/ ES0102030405060M/R Pig HiveRaw w/ ESWriting Reading / Querying
  • 28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExplore data through(Elastic)Search
  • 29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedThank you!@costinlhttp://www.elasticsearch.org/

×