L'évolution de l'infrastructure BI Viadeo par François Le Lay
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

L'évolution de l'infrastructure BI Viadeo par François Le Lay

on

  • 1,741 views

http://fr.viadeo.com/fr/profile/francois.lelay

http://fr.viadeo.com/fr/profile/francois.lelay

Statistics

Views

Total Views
1,741
Views on SlideShare
969
Embed Views
772

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 772

http://dasini.net 771
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

L'évolution de l'infrastructure BI Viadeo par François Le Lay Presentation Transcript

  • 1. The evolution of Business Intelligence at Viadeo Techdays 22/11/2012
  • 2. Agenda What is Business Intelligence? Key Roles Viadeo Data Technical Solutions : a short history
  • 3. What is Business Intelligence ? Application Stack Awareness Stack Insights Application Awareness Insights Actions Actions Act Marketing Actions, Business ion Strategies, Operations s Forecasting, Predicting, Statistics, Competitor Insights Information, AnalysisFeedback Awareness Reports, Dashboards Meta Data, KPI’s, Visual Templates, Application Stack Security, Information Dissemination, Scheduling Plumbing of structured and Data Warehouse & ETL unstructured data, logic to persists data
  • 4. Key Roles : the Business Analyst Functional Simple Web BI Followup (Challenge (Metrics) BI Dashboards Informatio Product Dashboard Analysis PO) Specification n Access s Specificatio n (Scalars) Technical Complex Proactive (Enforce (Data viz) Direct data quality) (SQL, Datameer)
  • 5. Key Roles : the Big Data Engineer Implement Real Time REST/Scala/Java APIs Data Awarenes Visualization Data plumbing Expose to Apps s Enforce Batch JDBC/ODBC data quality
  • 6. Viadeo data : The Dynamics • 45 million members • Worldwide presence • China, India, Russia, Mexico,.. • Mobile App, Web, API • B2B / B2C Mining User Usage Engagement
  • 7. Viadeo data : Graph
  • 8. Technical solutions : The Beginnings Phase 1: 2006-2008 Phase 2 : 2008-2010 Mysql Mysql Server name : Peach Server name : Lakitu Internal tool to allow C-Level, Sales,… Access data
  • 9. Technical solutions : A betterarchitecture Phase 3: 2010 - 2012 MySQL Server name : « Unfied ODS » Mysql Server name : Server name : ODS Live ODS Live Cluster 1 Cluster 2 Server name : Server name : ODS Live ODS Live Cluster 3 Cluster 5
  • 10. Technical solutions : 2 new internalproductsScala-centric, Play! framework Cross-channel messaging system Email, Mobile, Social Flexible content management Flexible targeting of recipients Content testing strategies : A/B, multivariate Event-driven : web app events, mobile events, ad hoc events Automation, scheduling, frequency capping Analytics Data visualization : based on Javascript D3.js, processing.js etc. Tabular Reports, OLAP navigation Pluggable alerts : business activity monitoring A common requirement : scalability!!! Viadeo data is Big Processing performance is not an option, it is mandatory
  • 11. Technical solutions : a newarchitecturebased on CQRS pattern
  • 12. Technical solutions : a newarchitecture• Master dataset : • Historical data stored in HBase • Provided as a service by architects team• Datamarts : • Built on HDFS using MapReduce jobs • MapReduce eased by use of Cascading library and Scala DSL (Scalding) • Pushed to in-memory distributed storage • Elastic Search, Riak
  • 13. Technical solutions : A betterarchitecture P O O SQ MySQL
  • 14. Conclusion• Many scalable data storage solutions• Rapid application development frameworks and low-risk programming languages on the JVM• Custom analytics = what we implement is what we use • Analytical needs are very well identified• Blend data stream and batch processing to answer different needs• Pluggable Data mining R&D• Analytics for Viadeo members/recruiters/companies : Social Media Monitoring as a Complex Event Processing topic
  • 15. ?
  • 16. Thanks ! flelay@viadeoteam.com Tél : 01 75 70 12 93