Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Search and Analytics (using Elasticsearch)

6,094 views

Published on

Slides from Costin Leau's talk on Search and Analytics (using Elasticsearch) at the 18th Big Data London meetup

Published in: Technology

Search and Analytics (using Elasticsearch)

  1. 1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch and Analytics(using Elasticsearch)Costin Leau
  2. 2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedWhy search?
  3. 3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – what’s the big deal?Basic/Metadata retrieval“Find banks with more then (x) accounts”
  4. 4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – what’s the big deal?Basic/Metadata retrieval“Find banks near my location”
  5. 5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch – What we’re all about
  6. 6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch categoriesBasic/Metadata retrievalFull-text searchHighlightingGeolocationFuzzy search (“did-you-mean”)Natural Language
  7. 7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch categoriesBasic/Metadata retrievalFull-text searchHighlightingGeolocationFuzzy search (“did-you-mean”)Natural Languagedata storessearch engines
  8. 8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited‘Players’ in the search marketSearch engines- Google/Bing/Yahoo!/Ask.com/Yandex/BaiduOpen-Source- Sphinx- Apache Lucene- Elasticsearch- Solr- SenseiEnterprise Search- Oracle Endeca / MDEX- HP Autonomy- Exalead- IBM Enterprise Search
  9. 9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearch
  10. 10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big Data
  11. 11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big DataLightweight
  12. 12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchOpen-Source Search & Analytics engine- Structured & Unstructured Data- Real Time- Analytics capabilities (facets)- REST basedDistributed- Designed for the Cloud- Designed for Big DataLightweightPopular: >200K downloads/month
  13. 13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUsers
  14. 14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUsers
  15. 15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPlatform Adoptionhttp://www.thoughtworks.com/radar#platforms 2013
  16. 16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPlatform Adoptionhttp://www.thoughtworks.com/radar#platforms 2013
  17. 17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Text Searchhttps://github.com/blog/1381-a-whole-new-code-search
  18. 18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearches 50,000,000 venues every day usingElasticsearchUse Case - Geolocation
  19. 19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case – Support/Reporting
  20. 20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Centralized Logging
  21. 21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUse Case - Pure Analytics
  22. 22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch and Big Data
  23. 23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedA Holistic View of a Big Data SystemETLRealTimeStreamsUnstructured Data (HDFS)RT SemistructuredDatabase(hBase,Cassandra,Mongo)Big SQL(Greenplum,AsterData,Etc…)BatchProcessingReal-TimeProcessing(s4, storm)Analytics
  24. 24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedA Holistic View of a Big Data SystemETLRealTimeStreamsUnstructured Data (HDFS)RT SemistructuredDatabase(hBase,Cassandra,Mongo)Big SQL(Greenplum,AsterData,Etc…)BatchProcessingAnalyticsReal-TimeProcessing(s4, storm)
  25. 25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHadoop eco-systemHadoop Distributed File System (HDFS)Map Reduce Framework (MapRed)
  26. 26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHadoop eco-systemHadoop Distributed File System (HDFS)Map Reduce Framework (MapRed)
  27. 27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearch + Hadoop0102030405060M/R Pig HiveRaw w/ ES0102030405060M/R Pig HiveRaw w/ ESWriting Reading / Querying
  28. 28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExplore data through(Elastic)Search
  29. 29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedThank you!@costinlhttp://www.elasticsearch.org/

×