The biggest in the areaAll fields: finance, telecom etc…
Leveraging Data:Building a Stable PlatformOphir Cohen, Data Platform Lead, email@example.comAmit Fainer, Data QA Lead, firstname.lastname@example.orgMay, 2013
Connection before content… 2 Who was the commander of whom in the army? Who met his wife in India?
Agenda 3 Connection before content LivePerson Is… Data platform requirements Quality challenges Architecture Development and production processes Case study: LivePerson BI Reports
LivePerson Is…Mission:4Company• Cloud-computing, SaaS pioneer since 1998• IPO April 2000 (Nasdaq: LPSN); debt free• 700+ employees• LivePerson offers an extensive and rapidly-growing partner networkCustomers• 8,500 customers around the globe have chosen LivePerson to create secure,reliable connections with their customers. LivePerson clients include:• 8 of the top 10 Fortune 500 companies•Top 10 of 15 commercial banks (Fortune 500)•Top 4 of 5 telecommunication companies (Fortune 500)•4 of the top 7 of the Forbes Global 2000•5 of the top 6 software and services companies (Forbes 2000)•8 of the top 10 of Interbrands Best Global BrandsService Delivery• 1.8 billion visitors monitored per month• 20 million connections per month• Analyzes over 1.2 million documents and chat transcripts per month.MissionCreatingMeaningfulCustomerConnectionsLive Chat and Click-to-CallVendor 2012
Requirements 6 Massive Data flow (few TB a day) Different Data types, Different Producers Never Lose Data! Variety latency needs – Near real-time through Offline Data is accessible to everyone for Processing, in a standardized,common paradigm, adopted by all consumers and producers
Quality Challenges 7 Large volumes of Data – Automate or Die Bugs yield corrupted Data Produced data stays Forever Consumers need a standardized form to assure data integrity
Develop, Test and Deploy at Scale 12 Automated, Continuously integrated with built-in Performancetesting Satisfying Monitoring and Auditing needs of Tiers 1 through 5 On going production tests Auditing mechanism Scrum Isolated production-mirrored environment for Testing