Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

2,643 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

  1. 1. Adding Flexibility and Scalability to Bouygues Telecom with MongoDB Pierre-Alban DEWITTE @__pad__ Bouygues Telecom MongoDB World – 06/24/2014
  2. 2. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Bouygues Telecom 13 M Customers
  3. 3. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Who is speaking ? @__pad__
  4. 4. Thanks !
  5. 5. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda Adding Flexibility and Scalability to Bouygues Telecom with MongoDB 1.Existing problems 2.Design choices 3.Build 4.A few steps back
  6. 6. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems
  7. 7. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Consumer Customers Existing Sytem Compagny Customers LDAP access Micropayment system Identity and Oauth management system Network Connexion system ETL
  8. 8. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Two years delay Schema customization very hard to design and implement Performance issues
  9. 9. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems
  10. 10. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices A.MongoDB B.Tomcat C.Storm
  11. 11. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Schema less data integration High availability Sharding Less expensive data storage allowing duplication
  12. 12. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Team with development and architecture skills Schema design experience Capability to train new actors
  13. 13. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld POC between Tomcat and NodeJS We choose our battle, MongoDB ! Design choices - Tomcat
  14. 14. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
  15. 15. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Distributed real-time computation system
  16. 16. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Consumer Customer MongoDB database Compagny Customers REST Micropayment system Identity and Oauth management system Network Connexion system API STORM
  17. 17. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Build A.Storm topology B.Schema design and refactor C.DevOps
  18. 18. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Customer files are pushed every night Read file line by line 1 … DetectFile Spout ProcessLine Bolt 2 line
  19. 19. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
  20. 20. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Emitting tuple for each set of functional collections ProcessLine Bolt 4 Oauth UpdaterBolt Identity UpdaterBoltTransform a line into a document & check delta 3
  21. 21. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
  22. 22. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
  23. 23. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Message failure handle ack(Object msgId) failed(Object msgId) Take care of system stability
  24. 24. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm We scaled and easily !
  25. 25. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
  26. 26. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Conception User -Name -Compagny -Billcycle -Payment Mode Acces Point -Name -Compagny -Billcycle -Payment Mode 1 1..n Conception
  27. 27. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
  28. 28. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Qualification
  29. 29. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
  30. 30. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor « That works » Qualification Conception
  31. 31. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor StressTest
  32. 32. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
  33. 33. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
  34. 34. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
  35. 35. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development Qualification Conception StressTest
  36. 36. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor PERFECT ! Development Conception StressTes
  37. 37. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Tooling first
  38. 38. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Moving the line between Dev and Ops
  39. 39. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Internal training by development team Redaction of a play-book dedicated to ops Pre production tests played by both team Early write of stress tests, played during qualification tests
  40. 40. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld 6 months later
  41. 41. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Iterations during build 4.A few steps back
  42. 42. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back Apache Tomcat Apache STORM Flexibility and Scale with a 100% Open Source Stack !
  43. 43. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back 1.Choose your battle 2.Use the right tool for the right job 3.Be prepare to fail and to rework 4.To move organization, training is a key
  44. 44. Thank you for listening Questions ? Further talks @__pad__
  45. 45. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Photo credits 100 m start : http://fr.wikipedia.org/wiki/100_m%C3%A8tres_%28athl%C3%A9tisme%29#mediaviewer/Fichier :Mens_100m_finals_British_Champs_and_Olympic_Trials.jpg by Paul Foot from Birmingham, UK Question mark block : https://www.flickr.com/photos/jarbo/9379813470 by Jared Cherup Pelleteuse orange : http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de- construction.html by Stephen Mcsweeny Punaise : https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist- f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p- 6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt- 2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ- 9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3- 9XthVN-kBs5mW-6qSPaF-5cA9Qb/ par dractrain94 Une longue vue qui louche : https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/ par Yann Caradec NASA-Apollo8-Dec24-Earthrise : http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14- 2383HR.jpg par NASA / Bill Anders

×