Lily @ Work Webinar

1,744 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,744
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
35
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Lily @ Work Webinar

  1. 1. Lily @ Workwebinar – july 7th 2011 IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org
  2. 2. Overview» Big Data @ Work: opportunities & challenges» Lily: a smart Big Data solution» Roadmap & status» Use cases» The Lily Adoption Program» Conclusions IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 2
  3. 3. Your host» Steven Noels founder and CEO of Outerthought stevenn@outerthought.org @stevenn ask questions! IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 3
  4. 4. Big data @ Work» a practical view on opportunities » store now, act later » live archives » scaling for data + audience» challenges » schemas with a vengeance » index management & maintenance IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 4
  5. 5. Store Now, act later» no Big Design Upfront» smart & scalable storage» at-leisure indexing & processing » batch, map-reduce-based» flexible and versionable schema to cope with increasing subject-matter understanding IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 5
  6. 6. Live archives» A lot of incumbent EDM/RM solutions focus on scale at the cost of functionality » “you’ll find your data if you know where to look”» Library systems are often under-engineered for Big Data » “we index 10.000 documents per hour ... at night”» General developer’s need for combo of storage + put|get|post|delete (CRUD) + index + search IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 6
  7. 7. Scaling for data + audience» classic architecture : interwebs » different server node types » complex application front-end tier deployment management application logic tier » upgrade nightmare data tier » multiple bottleneck possibilities (web/app/db) IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 7
  8. 8. Scaling for data + audience» ‘webscale’ setup interwebs » (mostly) identical nodes » load spread evenly across front-end tier services CPUs and spindles application logic tier services » move app services between nodes (rather than node data tier services reconfigs) IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 8
  9. 9. Schemas with a vengeance» ‘schema-free’ is not the long-term Big Data selling point » you want APIs/contracted contact surfaces in your app architecture » Big Data is now moving beyond log file analytics → more complex (domain-inspired) data models » ongoing insights → evolving schemas» ... schema flexibility is. IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 9
  10. 10. Index maintenance & management» NOSQL → NO QUERY ? » all index-based access must be defined up-front » increasing need for specialized, real-time indexed data access » structured + full-text (idf) searching» increasing data corpus size = increasing index maintenance cost» need for both interactive index updating + batch index regeneration IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 10
  11. 11. Enter Lily, a solution for big data» smart : copes with large-scale data- and audience sets through a simple yet flexible API and model ×» scale : a fully distributed architecture» easy : empowering enterprise devs with reality-inspired tooling (batteries included: schema, indexing, search) IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 11
  12. 12. LILY The Real-time Platform built for the Age of Data. We manage, track and measure your data and users, and do the mat(c)hmaking in-between: » provide you with business intelligence and analytics » harvest user profiles and learn their interests » dynamically engage your users using quality recommendations IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 12
  13. 13. where would you use lily?» large collections of data » large groups of users » content repositories » e-commerce / retail » library catalogs » news / media » (media) asset management » product catalogs » ‘live’ archives » ... if you want to use big data, but you need easy. IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 13
  14. 14. ns pe ap gic h ma he t re he sw si + thiIIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 14
  15. 15. Roadmap & status» Out now: Lily 1.0.1» Out real soon: » Lily embed - developer-friendly lightweight Lily launching options for use in (unit) tests » Cloudera whirr-based (EC2) cluster deployment tools» Next (Q4 2011): » Even more flexible data model (complex attributes, e.g. tags, keywords, substructures) » Secondary Indexes (pure HBase-based indexes) IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 15
  16. 16. Next up: usage stats» sits in CRUD-path» tracks users ops against records interactions » from both perspectives record user » arbitrary K/V properties: time, location, ... rec» automatically builds user om me nd ati o profiles (as records) ns indexes e tim » tied to records ops » indexed access » time dimension: trending IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 16
  17. 17. from usage stats to recommendations ‘light’ record user» grouping of users based on » shared properties » shared record access» grouping of records based on » shared properties { connections » shared user operations recommendations IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 17
  18. 18. Use cases» Adeo» Fatminds» Media IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 18
  19. 19. Groupe adeo» ADEO Groupe = Nr. 4 DIY retailer worldwide» Project “Wikeo”: data- and services platform to back business-unit-operated e-commerce websites» product data, profiles, pricing info, product manuals ...» Lily as a data aggregation, interactivity & scaling platform IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 19
  20. 20. Fatminds» US startup: “Kayak meets Yelp for continuing education”» originally based on Alfresco, started using Lily as their go-to-web scalability platform» www.fatminds.com IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 20
  21. 21. IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 21
  22. 22. media» USA magazine publisher » Swedish newspaper publisher » newsfeed server » feed aggregation e le iv eb bi ch ... w mo ar » personalization through Solr extended services searches » loading via custom Ruby app core services : user profiles / entitlement content / analytics external (editorial) systems IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 22
  23. 23. Lily Adoption program discover explore» facilitated multi-step process adopt» qualify challenges» explore opportunities deploy» learn + change support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 23
  24. 24. Lily adoption program discover explore» open source: adopt www.lilyproject.org business-friendly Apache license deploy support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 24
  25. 25. Lily adoption program discover explore» 2-day workshop-facilitated exploration of opportunities adopt» creating a true business case » make benefits tangible & measurable deploy » architecture study » prepare proof of concept support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 25
  26. 26. Lily adoption program discover explore» classroom developer training» PoC implementation and validation adopt» project guidance and assistance » professional services deploy » work with integrators » architecture and implementation review support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 26
  27. 27. Lily adoption program discover explore» going live» deployment tools adopt » Lily cluster install tools» cluster mgmt deploy » Lily admin application support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 27
  28. 28. Lily adoption program discover explore» enterprise support adopt» bug fixes, hot fixes» intermediate releases deploy» access to roadmap support IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 28
  29. 29. How to get started» Go and discover !» Exploration workshop -20% » This Summer (until of end of September ’11) » Excl. travel, incl. extensive workshop report ✘ US$ 9,000 € 6.250 ➙ US$ 7,250 ➙ € 5.000 IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 29
  30. 30. conclusions Lily allows you to collect, process and manage your data, all of it, in a developer-friendly packaging, to grow knowledge about your data, and how it is being used. IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 30

×