[DRAFT] Workshop - Technical Introduction to joola.io


Published on

This workshop focuses on hands-on introduction to joola.io

For a complete breakdown of the Workshop itself, refer to the project's wiki @ http://github.com/joola/joola.io/wiki/workshops

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

[DRAFT] Workshop - Technical Introduction to joola.io

  1. 1. joola.io Workshops Technical Review of joola.io Worksh op Details an d In stru ction s
  2. 2. Preface
  3. 3. About the workshop Th is is a h an ds-on worksh op Lab m aterials are available for th is worksh op Th is worksh op an d m aterials are available on ou r GitH u b repo
  4. 4. Workshop Goals U n derstan d joola.io on all of its com pon en ts Be able to deploy joola.io Pu sh an d Qu ery data from th e system Draw basic Visu alization s
  5. 5. Lab Details Du rin g th is worksh op we'll be accessin g an on lin e lab Please SSH with th e followin g $ s s h work s hop@lab-intro.joola.io Password: password * ac c es s to the abov e lab is whitelis ted
  6. 6. Understanding joola.io
  7. 7. What is joola.io? $ npm ins tall joola.io Featu re packed Data An alytics an d Visu alization Fram ework Written in N ode.JS Scalable to deal with h igh volu m es of data an d realtim e qu eries Secu re an d m u lti-ten an t Em bed an d in tegrate data visu alization s in to existin g sites an d apps Open Sou rce
  8. 8. How does it work? Operates an d m an aged via a set of H TTP API en dpoin ts Data pu sh ed u sin g JSON via joola.io's Beacon API Stored in cach in g layer, optim ized for qu ery Qu eries an d Visu alization s provided by Javascript SDK Th at's all in a n u tsh ell!
  9. 9. Developer Friendly By Data Lovers for Developers Ease of u se Exten d Em bed an d In tegrate Site specific au tom ated testin g Rich docu m en tation
  10. 10. IT Friendly M asterless grid Easy in stallation Zero con figu ration Powerfu l CLI Sm all footprin t M on itor, Alerts an d Logs
  11. 11. DBA Friendly Easy Data M odellin g Sch em a-less Flexible Zero down tim e u pdates Com plete con trol
  12. 12. Dispatch
  13. 13. Nodes Everywhere Each n ode is a fu lly fu n ction in g joola.io system Com m u n ication is based on Pu b/Su b A n ode n eeds to qu alify for exeu ction Qu alification can vary an d be exten ded Geo prioritization
  14. 14. Dispatch Routing Every action is a m essage All m essages are distribu ted M essages con tain secu rity con text Defau lt `n otm e` h an dlin g logic Backlog processin g of stu ck m essages
  15. 15. Command the Grid U se REPL, CLI or SDK $ joola.io.c li joola.io # admin@loc alhos t:8080 > joolaio.s y s tem.lis tnodes (); joola.io # admin@loc alhos t:8080 > joolaio.s y s tem.terminate('nodeuid'); Wh en you 're con n ected to on e n ode, you 're con n ected to th e Grid N o action is execu ted on th e local n ode, bu t dispatch ed All n odes su bscribe to con figu ration ch an ges
  16. 16. Node.JS' Event Loop Block M an y con sider as N ode.JS' ach illes h eel, we don 't Th e Grid an d Pu b/Su b approach resolves th e core issu e ELB cou n ters an d m on itors U sin g th e righ t tool for th e job M ade ou r software better
  17. 17. Accessing joola.io Via H TTP API en dpoin ts U se SDK, CLI, cU RL or write you r own Sh ipped with Web Adm in con sole Ou t-of-th e-box fu lly fu n ction in g an alytics site Rich docu m en tation
  18. 18. Pushing Data
  19. 19. Pushing Data is Easy joolaio.beac on.ins ert('c ollec tion-purc has es ', { times tamp: new Date(), Play er: { Us ername: 'A play er', Dev ic e: 'Des k top', Brows er: 'Chrome' }, ip: '', Vis its : 1, Vis itDuration: 123, Clic k s : 23, Purhc as es : { Amount: 123 Currenc y : 'USD', Status : 'Proc es s ed' } }});
  20. 20. Data Modelling Describe you r data as a JSON docu m en t(s) Docu m en ts are stored in a Collection Sch em a-less, docu m en ts can be of differen t stru ctu re Su pport for all data types Su pport for Stron g-Typed collection s
  21. 21. Push What You'll Need Store on ly data relatin g to An alytics Separate in to logical collection s Describe every m etric with as m an y attribu tes as possible Pass raw data, don 't aggregate Pu sh an array of docu m en ts wh en possible
  22. 22. Caching Layer Based on leadin g providers, su ch as M on goDB, Cassan dra an d m ore Exten d to su pport addition al In sert, u pdate an d delete directly from cach e Execu te qu eries directly on cach in g layer N orm alized access to u n derlyin g cach e via Beacon API
  23. 23. Query
  24. 24. Understanding Queries Describe wh at you wish to display, rath er th an h ow to get it (SQL) Su pport for Realtim e (WebSocket) Join data from m u ltiple collection s Relative tim efram es Advan ced filters Dim en sion an d M etric tran sform ation s M an y aggregation operators, in clu din g U n iqu e Cou n t
  25. 25. Basic Query Describe th e requ ired dim en sion s an d m etric, n ot h ow to qu ery th em joolaio.query .fetc h({ timeframe: 'las t_month', interv al: 'day s ', dimens ions : ['times tamp', 'dev ic e'], metric s : ['v is its ', { k ey : 'av gv is itduration', name: 'Av g. Vis it Duration', depends On: 'v is itduration', aggregation: 'av g', s uffix : 'ms .' }] }, func tion(err, res ult){ c ons ole.log(res ult.doc uments ); });
  26. 26. Calculated Metrics Com pose virtu al, calcu lated m etrics on -th e-fly metric s : [{ k ey :'av gamountperplay er', name: 'Av g. Amount per Play er', formula: { depends On:[{ k ey : 'av gamount', depends On: 'Purc has e.Amount', aggregation: 'av g' }, { k ey : 'play erc ount', depends On: 'Play er.Us ername', aggregation: 'uc ount' }], run: 'func tion(av gamount, play erc ount) {return av gamount/play erc ount;}' } }]
  27. 27. Query Filters M u ltiple filters - m u tli-level filters - per m etric filters timeframe: 'las t_month', interv al: 'day s ', metric s : [{ k ey : 'play erc ount', depends On: 'Play er.Us ername', aggregation: 'uc ount', filter: [ ['Play er.Brows er', 'eq', 'Chrome']] }], filter: [ ['Play er.Dev ic e', 'eq', 'Des k top'] ]
  28. 28. Query Results Con tain retrieved docu m en ts an d m eta-data describin g th e qu ery an d its tim in gs. Resu lts con tain both raw an d form atted valu es. res ults : [ { v alues :{ times tamp: '2014-03-09T14:03:34.412Z', av gduration: '123 ms .' }, formattedValues : { times tamp: 'Sun Mar 09 2014 15:03:34 GMT+0100 (CET)', av gduration: 123' } }, {...} ]
  29. 29. Realtime Bu ilt-in in to all visu alization s an d in tegral part of qu ery.fetch joolaio.query .fetc h({ //query params , s uc h as timeframe, metric s , etc ... realtime: true }); First execu tion retu rn s en tire tim efram e Followin g packets retu rn on ly delta Data con sisten cy assu ran ce
  30. 30. Visualization
  31. 31. Plotting Data Part of th e Javascript SDK Easy as copy-paste Sh ape to th e desired look, feel an d beh avior Offers jQu ery plu gin s Rich Docu m en tation an d Playgrou n d
  32. 32. Visualization Types M etric Box Sparklin e Pie Ch art Tim elin e Table
  33. 33. Drawing a Visualization Easy as a copy-pastin g from th e wiki Accepts a Qu ery object an d addition al form attin g option s joolaio.v iz .timeline({ c ontainer: doc ument.getElementBy Id('drawhere'), query : { dimens ions :['times tamp'], metric s : ['v alue'] }); U ses H TM L5, su pports all device types (m obiles, tablets)
  34. 34. Integrate, Embed and Extend
  35. 35. Integrate Design ed an d bu ilt for in tegration Pu sh data from an y data sou rce type wh ile keepin g con trol of logic Su pport for h igh -rate distribu ted, gu aran teed writes JSON collection s provide flexibility an d site specific in tegration Collect data from all parts of th e bu sin ess Leverage joola.io to redu ce overh ead of operation al databases
  36. 36. Embed Seam less drop-in visu alization s in you r site an d apps Even t driven com m u n ication with con tain er page Copy-paste from wiki On ly fron t-en d developers are n eeded, redu ce DBA overh ead Con trol th e tin iest detail of look, feel an d flow Respon sive, H TM L5, CSS3 stan dards
  37. 37. Extend M issin g a featu re? Easy to learn in frastru ctu re an d code Follows best practices gu idelin es an d m eth odology Access to sou rce code, issu es, wiki an d kn ow-h ow Con tribu te to open sou rce project Exten d an yth in g from cach in g layer to visu alization s
  38. 38. Endless Possibilities
  39. 39. Analytics-as-a-Service Offer an alytics an d visu alization s to you r en d-u sers Scalable an d redu n dan t to su pport very large operation s M u lti-ten an t an d secu re Easy bran din g an d clien t in tegration
  40. 40. Web Analytics Easy to setu p an d con figu ration Gath er an d join m u ltiple data sou rces in to a sin gle store joolaio.beac on.ins ert('performanc e', window.performanc e); Setu p alerts on specific u ser activity or th resh old
  41. 41. Ad Serving N o n eed to pre-determ in e au dien ces an d segm en ts Leverage split secon d qu ery tim es Apply advan ced filters an d segm en ts in realtim e Determ in e best m atch by u n lim ited param eters Perform fu ll/partial segm en tation on -th e-fly
  42. 42. Infographics Offer "live" in fograph ics Com bin e realtim e calcu lation s in to static con ten t Segm en ted in fograph ics Secu re/m u lti-ten an t in fograph ics
  43. 43. THANK YOU Visit ou r GitH u b to read m ore abou t joola.io