The evolution of Business Intelligence at Viadeo                                           Techdays 22/11/2012
Agenda What is Business Intelligence? Key Roles Viadeo Data Technical Solutions : a short history
What is Business Intelligence ?  Application Stack  Awareness Stack  Insights   Application   Awareness   Insights  Action...
Key Roles : the Business Analyst                                             Functional       Simple                      ...
Key Roles : the Big Data Engineer                                             Implement        Real Time    REST/Scala/Jav...
Viadeo data : The Dynamics                  • 45 million members                  • Worldwide presence                    ...
Viadeo data : Graph
Technical solutions : The Beginnings   Phase 1: 2006-2008          Phase 2 : 2008-2010           Mysql                    ...
Technical solutions : A betterarchitecture                     Phase 3: 2010 - 2012                           MySQL       ...
Technical solutions : 2 new internalproductsScala-centric, Play! framework Cross-channel messaging system    Email, Mobile...
Technical solutions : a newarchitecturebased on CQRS pattern
Technical solutions : a newarchitecture• Master dataset :  • Historical data stored in HBase  • Provided as a service by a...
Technical solutions : A betterarchitecture             P            O           O         SQ MySQL
Conclusion• Many scalable data storage solutions• Rapid application development frameworks and low-risk  programming langu...
?
Thanks !           flelay@viadeoteam.com           Tél : 01 75 70 12 93
Upcoming SlideShare
Loading in …5
×

L'évolution de l'infrastructure BI Viadeo par François Le Lay

2,363 views
2,207 views

Published on

http://fr.viadeo.com/fr/profile/francois.lelay

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,363
On SlideShare
0
From Embeds
0
Number of Embeds
1,077
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

L'évolution de l'infrastructure BI Viadeo par François Le Lay

  1. 1. The evolution of Business Intelligence at Viadeo Techdays 22/11/2012
  2. 2. Agenda What is Business Intelligence? Key Roles Viadeo Data Technical Solutions : a short history
  3. 3. What is Business Intelligence ? Application Stack Awareness Stack Insights Application Awareness Insights Actions Actions Act Marketing Actions, Business ion Strategies, Operations s Forecasting, Predicting, Statistics, Competitor Insights Information, AnalysisFeedback Awareness Reports, Dashboards Meta Data, KPI’s, Visual Templates, Application Stack Security, Information Dissemination, Scheduling Plumbing of structured and Data Warehouse & ETL unstructured data, logic to persists data
  4. 4. Key Roles : the Business Analyst Functional Simple Web BI Followup (Challenge (Metrics) BI Dashboards Informatio Product Dashboard Analysis PO) Specification n Access s Specificatio n (Scalars) Technical Complex Proactive (Enforce (Data viz) Direct data quality) (SQL, Datameer)
  5. 5. Key Roles : the Big Data Engineer Implement Real Time REST/Scala/Java APIs Data Awarenes Visualization Data plumbing Expose to Apps s Enforce Batch JDBC/ODBC data quality
  6. 6. Viadeo data : The Dynamics • 45 million members • Worldwide presence • China, India, Russia, Mexico,.. • Mobile App, Web, API • B2B / B2C Mining User Usage Engagement
  7. 7. Viadeo data : Graph
  8. 8. Technical solutions : The Beginnings Phase 1: 2006-2008 Phase 2 : 2008-2010 Mysql Mysql Server name : Peach Server name : Lakitu Internal tool to allow C-Level, Sales,… Access data
  9. 9. Technical solutions : A betterarchitecture Phase 3: 2010 - 2012 MySQL Server name : « Unfied ODS » Mysql Server name : Server name : ODS Live ODS Live Cluster 1 Cluster 2 Server name : Server name : ODS Live ODS Live Cluster 3 Cluster 5
  10. 10. Technical solutions : 2 new internalproductsScala-centric, Play! framework Cross-channel messaging system Email, Mobile, Social Flexible content management Flexible targeting of recipients Content testing strategies : A/B, multivariate Event-driven : web app events, mobile events, ad hoc events Automation, scheduling, frequency capping Analytics Data visualization : based on Javascript D3.js, processing.js etc. Tabular Reports, OLAP navigation Pluggable alerts : business activity monitoring A common requirement : scalability!!! Viadeo data is Big Processing performance is not an option, it is mandatory
  11. 11. Technical solutions : a newarchitecturebased on CQRS pattern
  12. 12. Technical solutions : a newarchitecture• Master dataset : • Historical data stored in HBase • Provided as a service by architects team• Datamarts : • Built on HDFS using MapReduce jobs • MapReduce eased by use of Cascading library and Scala DSL (Scalding) • Pushed to in-memory distributed storage • Elastic Search, Riak
  13. 13. Technical solutions : A betterarchitecture P O O SQ MySQL
  14. 14. Conclusion• Many scalable data storage solutions• Rapid application development frameworks and low-risk programming languages on the JVM• Custom analytics = what we implement is what we use • Analytical needs are very well identified• Blend data stream and batch processing to answer different needs• Pluggable Data mining R&D• Analytics for Viadeo members/recruiters/companies : Social Media Monitoring as a Complex Event Processing topic
  15. 15. ?
  16. 16. Thanks ! flelay@viadeoteam.com Tél : 01 75 70 12 93

×