Leveraging Data:Building a Stable PlatformOphir Cohen, Data Platform Lead, ophirc@liveperson.comAmit Fainer, Data QA Lead,...
Connection before content… 2 Who was the commander of whom in the army? Who met his wife in India?
Agenda 3 Connection before content LivePerson Is… Data platform requirements Quality challenges Architecture Develop...
LivePerson Is…Mission:4Company• Cloud-computing, SaaS pioneer since 1998• IPO April 2000 (Nasdaq: LPSN); debt free• 700+ e...
Enterprise Customer Success & Domain ExpertiseFinanceHigh–TechRetailTelecomTravel5
Requirements 6 Massive Data flow (few TB a day) Different Data types, Different Producers Never Lose Data! Variety lat...
Quality Challenges 7 Large volumes of Data – Automate or Die Bugs yield corrupted Data Produced data stays Forever Con...
Architecture 8KafkaData TierApplication TierStormHadoopPigJava MRHive
Architecture – Persistency Layer 9KafkaData TierApplication TierStormHadoopPigJava MRHiveKafka (by LinkedIn):• Queuing mec...
Architecture – Streaming Processing Layer 10KafkaData TierApplication TierStormHadoopPigJava MRHiveStorm (by Twitter)• Str...
Architecture – Batch Processing Layer 11KafkaData TierApplication TierStormHadoopPigJava MRHiveHadoop (an Apache Project)•...
Develop, Test and Deploy at Scale 12 Automated, Continuously integrated with built-in Performancetesting Satisfying Moni...
Case Study – LivePerson BI Reports 13
Case Study – LivePerson BI Reports 14 Source to target Auditing tool as part of data integrity tests Load tests in real...
Thank You 15LivePerson Hire!Feel free to reach out: ophirc@liveperson.com @ophchu amitfa@liveperson.com
Upcoming SlideShare
Loading in...5
×

Live person under_the_hood_taldor_for_publish

255

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
255
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • We need to update this slide
  • The biggest in the areaAll fields: finance, telecom etc…
  • Live person under_the_hood_taldor_for_publish

    1. 1. Leveraging Data:Building a Stable PlatformOphir Cohen, Data Platform Lead, ophirc@liveperson.comAmit Fainer, Data QA Lead, amitfa@liveperson.comMay, 2013
    2. 2. Connection before content… 2 Who was the commander of whom in the army? Who met his wife in India?
    3. 3. Agenda 3 Connection before content LivePerson Is… Data platform requirements Quality challenges Architecture Development and production processes Case study: LivePerson BI Reports
    4. 4. LivePerson Is…Mission:4Company• Cloud-computing, SaaS pioneer since 1998• IPO April 2000 (Nasdaq: LPSN); debt free• 700+ employees• LivePerson offers an extensive and rapidly-growing partner networkCustomers• 8,500 customers around the globe have chosen LivePerson to create secure,reliable connections with their customers. LivePerson clients include:• 8 of the top 10 Fortune 500 companies•Top 10 of 15 commercial banks (Fortune 500)•Top 4 of 5 telecommunication companies (Fortune 500)•4 of the top 7 of the Forbes Global 2000•5 of the top 6 software and services companies (Forbes 2000)•8 of the top 10 of Interbrands Best Global BrandsService Delivery• 1.8 billion visitors monitored per month• 20 million connections per month• Analyzes over 1.2 million documents and chat transcripts per month.MissionCreatingMeaningfulCustomerConnectionsLive Chat and Click-to-CallVendor 2012
    5. 5. Enterprise Customer Success & Domain ExpertiseFinanceHigh–TechRetailTelecomTravel5
    6. 6. Requirements 6 Massive Data flow (few TB a day) Different Data types, Different Producers Never Lose Data! Variety latency needs – Near real-time through Offline Data is accessible to everyone for Processing, in a standardized,common paradigm, adopted by all consumers and producers
    7. 7. Quality Challenges 7 Large volumes of Data – Automate or Die Bugs yield corrupted Data Produced data stays Forever Consumers need a standardized form to assure data integrity
    8. 8. Architecture 8KafkaData TierApplication TierStormHadoopPigJava MRHive
    9. 9. Architecture – Persistency Layer 9KafkaData TierApplication TierStormHadoopPigJava MRHiveKafka (by LinkedIn):• Queuing mechanism• Persistency layer• High availability layer
    10. 10. Architecture – Streaming Processing Layer 10KafkaData TierApplication TierStormHadoopPigJava MRHiveStorm (by Twitter)• Stream processing• Pluggable framework
    11. 11. Architecture – Batch Processing Layer 11KafkaData TierApplication TierStormHadoopPigJava MRHiveHadoop (an Apache Project)• Reliable, scalable, distributedcomputing framework• Rich eco-system
    12. 12. Develop, Test and Deploy at Scale 12 Automated, Continuously integrated with built-in Performancetesting Satisfying Monitoring and Auditing needs of Tiers 1 through 5 On going production tests Auditing mechanism Scrum Isolated production-mirrored environment for Testing
    13. 13. Case Study – LivePerson BI Reports 13
    14. 14. Case Study – LivePerson BI Reports 14 Source to target Auditing tool as part of data integrity tests Load tests in real data env
    15. 15. Thank You 15LivePerson Hire!Feel free to reach out: ophirc@liveperson.com @ophchu amitfa@liveperson.com
    1. Gostou de algum slide específico?

      Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

    ×