Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

dataviz on d3.js + elasticsearch

6,256 views

Published on

dataviz on d3.js + elasticsearch

  1. 1. dataviz on bordeaux open data + elasticsearch + d3js + angular.js + google refinemardi 19 mars 13
  2. 2. goal • full stack dataviz • front js centric with angular.js and sass • d3.js -> best lib for dataviz (a bit complex) • elasticsearch scalable search engine access form js ajax client • grab open data and reformat with google refinemardi 19 mars 13
  3. 3. philosophy • smarter and smarter browsers will grab a huge part of the stack • scalable NoSQL solution talk REST: super easy access • you always can enrich, refine, store, model data from anysource: yes you can !mardi 19 mars 13
  4. 4. go open data ! • go to http://opendata.bordeaux.fr/ • we focus ou capc contemporary museum of bordeaux • http://opendata.bordeaux.fr/content/ collections-du-capc-musee-dart- contemporain • curl https:// bdxconfigogdi.blob.core.windows.net/ converteddata/capc.csv -o capc.csvmardi 19 mars 13
  5. 5. google refine • data many times isn’t perfect: humans input, unsuitable model... • with google refine you can tidy data, enrich with web services call and so on... • i think you should never think you don’t have the good data for the job • be smart and be creativ: you have everything you need, all the time !mardi 19 mars 13
  6. 6. import data in elasticsearch • bulk api import for es is handy • http://www.elasticsearch.org/guide/ reference/api/bulk.html • we use templating in google refine to export data to the correct format • { "index" : { "_index" : "musees", "_type" : "capc"} } [snipp]mardi 19 mars 13
  7. 7. elasticsearch • scalable search engine • adding more power == adding more nodes • sharding • replication • fault tolerantmardi 19 mars 13
  8. 8. elasticsearch • store unstructured document (json) in indexes (NoSQL way) • talk REST (api) • advanced query langage • multiple analyzers (tokens, languages, etc...) • blazing fast ! • no alternativ solutions (to my advice) (and kimshy advice too ;) )mardi 19 mars 13
  9. 9. yeoman • perfect tool for the new browser focus stack coming soon • yeoman init angular • yeoman init angular:route capc • yeoman install d3 • yeoman install jquery • yeoman server -> yeah !!!mardi 19 mars 13
  10. 10. angular.js • mvc framework for browser (js) • by google • rely better on html doc • cleaner controller handling than backbone (to my advice)mardi 19 mars 13
  11. 11. d3.js • best javascript lib for dataviz (to my advice) • slow learning curve • based on svg browser markup • data and dom oriented • generic (go low level with svg easily)mardi 19 mars 13
  12. 12. dataviz • make appears informations from pure data • you should explore, analyse and be creativ to grab the most value possible from the data: go as far as you can • tables (excell way) -> forgive • visualization will reveal your data: data is made to be know to take decision, manage, understandmardi 19 mars 13
  13. 13. open data • today, wide range of datas come to the public domain • again, without computing and vizualisation, data has a poor value • open data without data scientists and data visualization has no futurmardi 19 mars 13
  14. 14. futur • with low bandwith, storage and computing cost, you can grab open data from a lot of sources • you can put them in big data store and make analysis, relations between, with open source technologies • you can then share this to the word with data visualization on your website, blogs...mardi 19 mars 13 • amazing isn’t it ?? !!!!!
  15. 15. questions ? infos ? • https://github.com/mathieue • http://www.mathieu-elie.net/ • https://twitter.com/mathieuel • https://www.facebook.com/mathieuelietechmardi 19 mars 13

×