Your SlideShare is downloading. ×
Make sense of your big data - Pilato
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Make sense of your big data - Pilato

710
views

Published on

Slides from David Pilato talk @ codemotion roma 2014

Slides from David Pilato talk @ codemotion roma 2014

Published in: Technology, Business

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
710
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. #elasticsearch MAKE SENSE OF YOUR (BIG) DATA! David Pilato Technical advocate ! elasticsearch. @dadoonet
  • 2. StartUp #elasticsearch data ?
  • 3. StartUp #elasticsearch
  • 4. StartUp #elasticsearch
  • 5. StartUp #elasticsearch
  • 6. StartUp #elasticsearch
  • 7. StartUp #elasticsearch
  • 8. StartUp #elasticsearch BIG data ?
  • 9. StartUp #elasticsearch BIG data ?
  • 10. StartUp #elasticsearch Source: http://www.csc.com/insights/flxwd/78931-big_data_just_beginning_to_explode 35.000.000.000.000.000 mb
  • 11. StartUp #elasticsearch Source: http://www.thebigdatainsightgroup.com/site/article/big-data-infographic
  • 12. StartUp #elasticsearch search = like % ? SELECT doc.*, country.* FROM doc, country WHERE doc.country_code = country.code AND doc.date_doc > to_date('2011-12', 'yyyy-mm') AND doc.date_doc < to_date('2012-01', 'yyyy-mm') AND lower(country.name) = 'france' AND lower(doc.comment) LIKE ‘%product%' AND lower(doc.comment) LIKE ‘%david%';
  • 13. StartUp #elasticsearch Search engine ?
  • 14. StartUp #elasticsearch elasticsearch ? plug & play REST/JSON scalable Apache 2 license Lucene elasticsearch
  • 15. #elasticsearch Start… $ wget https://download.elasticsearch.org/elasticsearch/ elasticsearch/elasticsearch-1.1.0.tar.gz! $ tar -xf elasticsearch-1.1.0.tar.gz! $ ./elasticsearch-1.1.0/bin/elasticsearch! [INFO ][node ][Ghost Maker] {1.1.0}[5645]: initializing
  • 16. #elasticsearch … and play! $ curl -XPUT localhost:9200/sessions/session/1 -d '{! "title" : "Elasticsearch",! "subtitle" : "Make sense of your (BIG) data !",! "date" : "2014-04-12T14:10:00",! "tags" : [ "elasticsearch", "codemotion", "bigdata" ],! "speakers" : [{! "first_name" : "David", ! "last_name" : "Pilato" ! }]! }'
  • 17. #elasticsearch Search! $ curl http://localhost:9200/sessions/session/_search -d' { "query": { "multi_match": { "query": "elasticsearch codemotion david", "fields": [ "title^3", "tags^2", "speakers.first_name" ] } }, "post_filter": { "range": { "date": { "from": "2014-04-09", "to": "2014-04-13" } } } }'
  • 18. StartUp #elasticsearch Compute?
  • 19. #elasticsearch $ curl http://localhost:9200/sessions/session/_search -d' { "query": { ... }, "aggs": { "by_date": { "date_histogram": { "field": "date", "interval": "day", "format" : "dd/MM/yyyy" } } } }' "by_date": [ { "key_as_string": "03/04/2014", "doc_count": 1 }, { "key_as_string": "12/04/2014", "doc_count": 2 }, { "key_as_string": "16/04/2014", "doc_count": 3 } ] Compute!
  • 20. #mstechdays #elasticsearch StartUp #elasticsearch • logs • twitter • github • marketing data • ... • your data • your big data Let’s make sense of …
  • 21. #mstechdays #elasticsearch StartUp #elasticsearch • logs • twitter • github • marketing data • ... • your data • your big data Let’s make sense of … { "name":"Pilato David", "dateOfBirth":"1971-12-26", "gender":"male", "children":3, "marketing":{ "fashion":334, "music":3363, "hifi":2351 }, "address":{ "country":"France", "city":"Paris", "location": [2.332395, 48.861871] } }
  • 22. démo #mstechdays #elasticsearch StartUp #elasticsearch MAKE SENSE OF YOUR (BIG) DATA! let’s inject some marketing documents…
  • 23. #elasticsearch ELASTICSEARCH
  • 24. StartUp #elasticsearch Distributed indices node 1 orders products 1 2 3 4 1 2 $ curl -XPUT localhost:9200/orders -d '{! "settings.index.number_of_shards" : 4,! "settings.index.number_of_replicas" : 1! }' $ curl -XPUT localhost:9200/products -d '{! "settings.index.number_of_shards" : 2,! "settings.index.number_of_replicas" : 0! }'
  • 25. StartUp #elasticsearch Distributed indices node 1 orders products 1 2 3 4 1 2 node 2 $ bin/elasticsearch! [INFO ][cluster.service][Armageddon] detected_master [Ghost Maker]
  • 26. StartUp #elasticsearch Distributed indices node 1 orders products 1 4 1 node 2 orders products 2 3 2 2 3 1 4 2 3 2
  • 27. StartUp #elasticsearch node 3 Distributed indices node 1 orders products 1 4 1 node 2 orders products 2 3 2 2 3 1 4 $ bin/elasticsearch! [INFO ][cluster.service][Karnak] detected_master [Ghost Maker]
  • 28. StartUp #elasticsearch node 3 products orders Distributed indices node 1 orders products 1 4 1 node 2 orders products 2 33 2 2 3 1 4 3 1 4
  • 29. elasticsearch. elasticsearch kibana logstash Marvel
  • 30. elasticsearch. Training (public and on-site) Development support Production support Marvel
  • 31. @dadoonet questions ? we are hiring! jobs@elasticsearch.com
  • 32. @dadoonet grazie !