Operational aspects of big data

390 views

Published on

The presentaion discuss what is big data and best practices for operating big data operation
The speaker is the CTO of myThings. It was presented in June 10, 2014 conference "Best practices for SaaS Operaion" sponsered by MoovingON www.moovingon.com

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
390
On SlideShare
0
From Embeds
0
Number of Embeds
35
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Operational aspects of big data

  1. 1. Operational aspects of Big Data • Yoav Chernobroda - CTO
  2. 2. All copyrights reserved to myThings LTD Few facts about myThings  Online ad retargeting for desktop, mobile web and mobile apps  First on Fast 50 companies  150 employees  R&D in Ramat Hachayal + 15 regional offices  Big data at a scale  20TB / day  300M uniques /month
  3. 3. All copyrights reserved to myThings LTD Personalized retargeting She is tagged with myThings’ smart tag, browse site but leave without completing purchase When she later visits any desktop or mobile site on the ad network, she is targeted with an ad User visits e-commerce site but quits without converting1 2 myThings creates, in real time, a personalized ad– custom-made based on consumer intent data, with product info, image 3A personalized ad is presented4When user clicks she is taken back to product page to complete purchase 5
  4. 4. All copyrights reserved to myThings LTD RTB retargeting e-commerce site RTB Exchange Google ad exchange Consumer DB Media Service Tag Service RTB service myThings platform Content site Visits Reads
  5. 5. All copyrights reserved to myThings LTD The big data challenge
  6. 6. All copyrights reserved to myThings LTD The (sad) truth
  7. 7. All copyrights reserved to myThings LTD Big data is not about large data volumes
  8. 8. All copyrights reserved to myThings LTD Classic definition The 3 V’s • Volume (terra / peta / zeta / … bytes) • Variety • The relational model does not hold • Velocity • Traditional relational db are not scalable enough • Technology is built around linear scalability • Examples: – Predictive analytics – Recommendation engines – Customer retention, churn analysis – Social graph analysis – Fraud detection
  9. 9. All copyrights reserved to myThings LTD My definition Big data Operational view Business intel. view Predictive modeling view Real time decisions
  10. 10. All copyrights reserved to myThings LTD The big data challenge Business value – Do we solve the right problem? – How does it help our business? Data quality – Do we have the right data? Organization roles – Collaboration Culture – Process oriented vs. iterative exploratory – Organizational fit Operational and infrastructure – Will get to it in a moment …
  11. 11. All copyrights reserved to myThings LTD myThings big data architecture
  12. 12. All copyrights reserved to myThings LTD Operational challenges • Cost effective architecture • Real time vs. near RT vs. offline processing • Linear scalability • Data routing infrastructure • Data retention and backup • Open source components • Hadoop, Kafka, Storm, Cassandra, … • Cost monitoring • Skillful devops – the human factor
  13. 13. All copyrights reserved to myThings LTD Recommended reading Nathan Marz Originator of Storm and Cascalog The lambda architecture

×