Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Trading up: Adding Flexibility and Scalability to Bouygues Telecom with MongoDB

686
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
686
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Team like to try new things, why not used node ?
    Go quick
  • Scalable database, Scalable front-end but provisioning part will not scale for the customer intergation part if we keep doing our traditional parallelized batch
  • GO QUICK
    Storm well know now for real time processing but you can also use this for a batch
    It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate.
    Suits for real-time data acquisition treatment but also for batch

  • This is the new design of our system.
    POC tests where what we expected. We fell confident enougth to start the building part
  • Focus on three key parts of our project project
  • Transcript

    • 1. Adding Flexibility and Scalability to Bouygues Telecom with MongoDB Pierre-Alban DEWITTE @__pad__ Bouygues Telecom MongoDB World – 06/24/2014
    • 2. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Bouygues Telecom 13 M Customers
    • 3. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Who is speaking ? @__pad__
    • 4. Thanks !
    • 5. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda Adding Flexibility and Scalability to Bouygues Telecom with MongoDB 1.Existing problems 2.Design choices 3.Build 4.A few steps back
    • 6. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems
    • 7. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Consumer Customers Existing Sytem Compagny Customers LDAP access Micropayment system Identity and Oauth management system Network Connexion system ETL
    • 8. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems Two years delay Schema customization very hard to design and implement Performance issues
    • 9. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Existing problems
    • 10. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices A.MongoDB B.Tomcat C.Storm
    • 11. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Schema less data integration High availability Sharding Less expensive data storage allowing duplication
    • 12. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - MongoDB Team with development and architecture skills Schema design experience Capability to train new actors
    • 13. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld POC between Tomcat and NodeJS We choose our battle, MongoDB ! Design choices - Tomcat
    • 14. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
    • 15. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Distributed real-time computation system
    • 16. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Consumer Customer MongoDB database Compagny Customers REST Micropayment system Identity and Oauth management system Network Connexion system API STORM
    • 17. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Build A.Storm topology B.Schema design and refactor C.DevOps
    • 18. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Customer files are pushed every night Read file line by line 1 … DetectFile Spout ProcessLine Bolt 2 line
    • 19. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
    • 20. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology Emitting tuple for each set of functional collections ProcessLine Bolt 4 Oauth UpdaterBolt Identity UpdaterBoltTransform a line into a document & check delta 3
    • 21. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
    • 22. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Storm topology
    • 23. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm Message failure handle ack(Object msgId) failed(Object msgId) Take care of system stability
    • 24. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Design choices - Storm We scaled and easily !
    • 25. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld
    • 26. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Conception User -Name -Compagny -Billcycle -Payment Mode Acces Point -Name -Compagny -Billcycle -Payment Mode 1 1..n Conception
    • 27. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
    • 28. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Qualification
    • 29. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
    • 30. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor « That works » Qualification Conception
    • 31. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor StressTest
    • 32. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
    • 33. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor ConceptionConception
    • 34. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development
    • 35. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor Development Qualification Conception StressTest
    • 36. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - Schema design and refactor PERFECT ! Development Conception StressTes
    • 37. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Tooling first
    • 38. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Moving the line between Dev and Ops
    • 39. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Build - DevOps Internal training by development team Redaction of a play-book dedicated to ops Pre production tests played by both team Early write of stress tests, played during qualification tests
    • 40. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld 6 months later
    • 41. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Agenda 1.Existing problems 2.Design choices 3.Iterations during build 4.A few steps back
    • 42. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back Apache Tomcat Apache STORM Flexibility and Scale with a 100% Open Source Stack !
    • 43. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld A few steps back 1.Choose your battle 2.Use the right tool for the right job 3.Be prepare to fail and to rework 4.To move organization, training is a key
    • 44. Thank you for listening Questions ? Further talks @__pad__
    • 45. Pierre-Alban DEWITTE @__pad__ #MongoDBWorld Photo credits 100 m start : http://fr.wikipedia.org/wiki/100_m%C3%A8tres_%28athl%C3%A9tisme%29#mediaviewer/Fichier :Mens_100m_finals_British_Champs_and_Olympic_Trials.jpg by Paul Foot from Birmingham, UK Question mark block : https://www.flickr.com/photos/jarbo/9379813470 by Jared Cherup Pelleteuse orange : http://fr.123rf.com/photo_3669950_un-grand-pelleteuse-orange-stationne-a-un-chantier-de- construction.html by Stephen Mcsweeny Punaise : https://www.flickr.com/photos/24362608@N05/3501112978/sizes/l/in/photolist- f5R8g4-6kiVfR-6koahu-6ko7US-6ko8Qh-d9NUKU-nDi2eb-ffg13f-8h56wx-cgAcib-hchCtD-decZ4p- 6kocrL-6kj2J6-5doxaw-gbTPmK-nCFBBq-672Snf-2sCzvw-e1zAjM-8voRbc-c9Rh1m-amFFGt- 2iZNBq-cFSnFU-81Ckk4-2XHZAy-9Ggpfw-m8FfKz-8hW6r5-4xSuxC-awXsp4-5aVn7g-fbXVHZ- 9GAPLX-9qr7uU-fcddgA-cP1EN7-np2q92-6V8oBT-cenSsb-cyzaEd-8h5a2F-6V2b3F-ch1RME-fJ52s3- 9XthVN-kBs5mW-6qSPaF-5cA9Qb/ par dractrain94 Une longue vue qui louche : https://www.flickr.com/photos/la_bretagne_a_paris/3847733265/ par Yann Caradec NASA-Apollo8-Dec24-Earthrise : http://www.hq.nasa.gov/office/pao/History/alsj/a410/AS8-14- 2383HR.jpg par NASA / Bill Anders