Your SlideShare is downloading. ×
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
L'évolution de l'infrastructure BI Viadeo par François Le Lay
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

L'évolution de l'infrastructure BI Viadeo par François Le Lay

1,675

Published on

http://fr.viadeo.com/fr/profile/francois.lelay

http://fr.viadeo.com/fr/profile/francois.lelay

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,675
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The evolution of Business Intelligence at Viadeo Techdays 22/11/2012
  • 2. Agenda What is Business Intelligence? Key Roles Viadeo Data Technical Solutions : a short history
  • 3. What is Business Intelligence ? Application Stack Awareness Stack Insights Application Awareness Insights Actions Actions Act Marketing Actions, Business ion Strategies, Operations s Forecasting, Predicting, Statistics, Competitor Insights Information, AnalysisFeedback Awareness Reports, Dashboards Meta Data, KPI’s, Visual Templates, Application Stack Security, Information Dissemination, Scheduling Plumbing of structured and Data Warehouse & ETL unstructured data, logic to persists data
  • 4. Key Roles : the Business Analyst Functional Simple Web BI Followup (Challenge (Metrics) BI Dashboards Informatio Product Dashboard Analysis PO) Specification n Access s Specificatio n (Scalars) Technical Complex Proactive (Enforce (Data viz) Direct data quality) (SQL, Datameer)
  • 5. Key Roles : the Big Data Engineer Implement Real Time REST/Scala/Java APIs Data Awarenes Visualization Data plumbing Expose to Apps s Enforce Batch JDBC/ODBC data quality
  • 6. Viadeo data : The Dynamics • 45 million members • Worldwide presence • China, India, Russia, Mexico,.. • Mobile App, Web, API • B2B / B2C Mining User Usage Engagement
  • 7. Viadeo data : Graph
  • 8. Technical solutions : The Beginnings Phase 1: 2006-2008 Phase 2 : 2008-2010 Mysql Mysql Server name : Peach Server name : Lakitu Internal tool to allow C-Level, Sales,… Access data
  • 9. Technical solutions : A betterarchitecture Phase 3: 2010 - 2012 MySQL Server name : « Unfied ODS » Mysql Server name : Server name : ODS Live ODS Live Cluster 1 Cluster 2 Server name : Server name : ODS Live ODS Live Cluster 3 Cluster 5
  • 10. Technical solutions : 2 new internalproductsScala-centric, Play! framework Cross-channel messaging system Email, Mobile, Social Flexible content management Flexible targeting of recipients Content testing strategies : A/B, multivariate Event-driven : web app events, mobile events, ad hoc events Automation, scheduling, frequency capping Analytics Data visualization : based on Javascript D3.js, processing.js etc. Tabular Reports, OLAP navigation Pluggable alerts : business activity monitoring A common requirement : scalability!!! Viadeo data is Big Processing performance is not an option, it is mandatory
  • 11. Technical solutions : a newarchitecturebased on CQRS pattern
  • 12. Technical solutions : a newarchitecture• Master dataset : • Historical data stored in HBase • Provided as a service by architects team• Datamarts : • Built on HDFS using MapReduce jobs • MapReduce eased by use of Cascading library and Scala DSL (Scalding) • Pushed to in-memory distributed storage • Elastic Search, Riak
  • 13. Technical solutions : A betterarchitecture P O O SQ MySQL
  • 14. Conclusion• Many scalable data storage solutions• Rapid application development frameworks and low-risk programming languages on the JVM• Custom analytics = what we implement is what we use • Analytical needs are very well identified• Blend data stream and batch processing to answer different needs• Pluggable Data mining R&D• Analytics for Viadeo members/recruiters/companies : Social Media Monitoring as a Complex Event Processing topic
  • 15. ?
  • 16. Thanks ! flelay@viadeoteam.com Tél : 01 75 70 12 93

×