NoSQL learnings from the world of Telco
•Download as PPTX, PDF•
1 like•596 views
Being a decent-sized Telecommunications provider, we process a lot of calls (hundreds/second), and need to keep track of all the events on each call. Technically speaking, this is "A Lot" of data - data that our clients (and our own people!) want real-time access to in a myriad of ways. We've ended up going through quite a few NoSQL stores in our quest to satisfy everyone - and the way we do things now has very little to do with where we started out. Join me as I describe our experience and what we've learned, focusing on the Big 4, viz -The "solution-oriented" nature of NoSQL repeatedly changed our understanding of our problem-space - sometimes drastically. - The system behavior , particularly the failure modes, were significantly different at scale -The software model kept getting overhauled - regardless of how much we planned ahead -We came to value agility - the ability to change direction - above all (yes, even at a Telco!)
Recommended
More Related Content
Similar to NoSQL learnings from the world of Telco
Similar to NoSQL learnings from the world of Telco (20)
Recently uploaded
Recently uploaded (20)
NoSQL learnings from the world of Telco
- 1. NoSQL the Telco way { @dieswaytoofast (V.P. Ubiquiti Networks)
- 3. The Business
- 4. Phone services for SMBs The Business
- 5. Phone services for SMBs Hosted Phone services for SMBs The Business
- 6. Phone services for SMBs Hosted Phone services for SMBs Hosted Cloud Communications service for SMBs The Business
- 7. Phone services for SMBs T? Hosted Phone services for SMBs HA Hosted Cloud Communications service for SMBs W H HU The Business
- 13. The Metrics
- 14. Phone Calls per Second The Metrics
- 15. Phone Calls per Second x 1000 The Metrics
- 16. Simultaneous Phone Calls The Metrics
- 17. Simultaneous Phone Calls x 10,000 The Metrics
- 18. HTTP API requests The Metrics
- 19. HTTP API requests ∞ The Metrics
- 20. Self-hosted (kinda) The Infrastructure
- 21. Self-hosted (kinda) Big IP pipes The Infrastructure
- 22. Self-hosted (kinda) Big IP pipes Erlang The Infrastructure
- 23. Self-hosted (kinda) Big IP pipes Erlang Polyglot Persistence The Infrastructure
- 24. Self-hosted (kinda) Big IP pipes Erlang Polyglot Persistence H? HU The Infrastructure
- 25. Domain-specific data stores Polyglot Persistence
- 26. Domain-specific data stores SQL Polyglot Persistence
- 27. NoSQL Domain-specific data stores SQL Polyglot Persistence
- 28. NoSQL Domain-specific data stores SQL Polyglot PersistenceFiles
- 29. NoSQL Domain-specific data stores Text SQL Polyglot PersistenceFiles
- 30. NoSQL Excel Domain-specific data stores Text SQL Polyglot PersistenceFiles
- 31. NoSQL Excel Domain-specific data stores Post-It Text SQL Polyglot PersistenceFiles
- 32. NoSQL Excel Domain-specific data stores Post-It Text SQL Polyglot PersistenceFiles
- 33. Not (necessarily) structured data NoSQL
- 34. Not (necessarily) structured data Solution Oriented NoSQL
- 35. Not (necessarily) structured data Solution Oriented H? HU NoSQL
- 36. SQ L
- 37. No SQ L
- 38. 75 bh p
- 39. What d’you want the data to look like when you fetch it from the database? - Casey Rosenthal Solution Oriented
- 42. Key-Value Column Solution Oriented Object
- 43. Key-Value Document Column Solution Oriented Object
- 44. Key-Value Graph Document Column Solution Oriented Object
- 45. Key-Value Graph is lly Document ns tua nt Column te en Ev Solution Oriented Object Co
- 46. Key-Value d re Graph de Or is lly Document ns tua nt Column te en Ev Solution Oriented Object Co
- 47. Key-Value d re Graph de Or is lly Document ns tua nt y Column te or en em Solution Oriented Ev Object M Co
- 48. lu le Key-Value d Va tip re ns tua e Graph de ul M Or is lly Document nt y Column te or en em Solution Oriented Ev Object M Co
- 49. lu le Key-Value d Va tip re ns tua e Graph de ul M Or is lly Document nt y Column te or en em Solution Oriented Ev Object M Co
- 52. Example please?
- 55. "E VE RY B OD YK NO W S
- 57. EN O GI V E NE R ER IN G!
- 58. Anything else?
- 59. I lied about reports Anything else?
- 60. I lied about reports (kind-of) Anything else?
- 61. Do tell…
- 62. If its easy, people might actually use it - <name withheld> Sad but true…
- 67. Friction - Good... Example please?
- 69. Anything else?
- 70. Scaling Matters
- 71. S W NO YK BOD RY Scaling Matters VE "E
- 72. Scaling is easy
- 73. S W NO YK BOD RY Scaling is easy VE "E
- 79. Automatic Scaling is hard
- 80. Automatic Scaling is hard
- 81. Automatic Scaling is hard
- 82. Automatic Scaling is hard
- 83. Automatic Scaling is hard
- 84. Automatic Scaling is hard
- 85. And the Failure modes!
- 86. And the Failure modes!
- 87. And the Failure modes!
- 88. And the Failure modes!
- 89. And the Failure modes!
- 91. New CFO
- 92. New CEO
- 96. a·gil·i·ty /əˈdʒɪlɪti/ noun the power of moving quickly and easily; nimbleness Agility
- 99. Loose Coupling Agility
- 100. Loose Coupling Hot Upgrades Agility
- 101. Loose Coupling Hot Upgrades Polyglot Persistence Agility
- 105. Move call information into one (per-client) database Redesign
- 106. New CFO
- 107. Preprocess Call information Separate out billing information Redesign
- 108. Preprocess Call information Separate out billing information What d’you want the data to look like when you fetch it from the database? - Casey Rosenthal Redesign
- 109. New CEO
- 110. Move all client info into one Database Redesign
- 111. "E VE RY B OD YK NO W S
- 112. More pre-computations (Date ranges! Argh!) Redesign
- 114. Expiring calls? Argh! Redesign
- 115. And the Failure modes!
- 116. Decouple authentication Mnesia Redesign
- 117. Automatic Scaling is hard
- 118. Caches Caches Everywhere… Mnesia Redesign
- 119. Automatic Scaling is hard
- 120. Mnesia Caches Caches Everywhere… Mnesia Redesign
- 121. Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign
- 122. Mnesia Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign
- 123. Mnesia Mnesia Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign
- 124. Mnesia Mnesia Mnesia Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign
- 125. Mnesia Mnesia Mnesia Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign Mnesia
- 126. Mnesia Mnesia Mnesia Mnesia Caches Caches Everywhere… Mnesia Mnesia Redesign Mnesia Mnesia
- 128. Redesign
- 130. Back Office Systems
- 131. Redesign
Editor's Notes
- Images here!!!
- Images here!!!
- Images here!!!
- Images here!!!
- Images here!!!
- Images here!!!
- Images here!!!
- Images here!!!
- Data model is(more) variable
- Images here
- Images here
- Images here
- Images here
- Images here
- Images here
- Images here
- Images here
- Images here
- And this is an issue. If the problem space changes…
- Images here
- And once the problem has changed, you are left w/ a solution that no longer applies
- Reporting
- The two most dangerous words in Product Development
- I person cares about biling. Another about reporting. No one actually cares about calls over a day old. This stuff is – mostly – irrelevant.
- A solution designed to process massive numbers of calls is useless if no-one wants to process massive numbers of calls
- A solution designed to process massive numbers of calls is useless if no-one wants to process massive numbers of calls
- Still one user, but uses it all the timeOh s**t, people actually use this.
- Which leads, inexporably, to the next point, which is that
- Efficient markets
- Friction can be very useful in avoiding issues
- The two most dangerous words in Product Development
- With callrooms, people want to do voicemail in real time. Worldwide. Ack!
- But I designed for scaleAdd servers, rebalance, etc
- But I designed for scaleAdd servers, rebalance, etc
- You can’t restart servers, ‘cos your clients are “always on”
- You can’t restart servers, ‘cos your clients are “always on”
- You can’t predict crazy spikes (Obama)
- Order of magnitude changes are, well, a PITA
- Server crash – clients reloaded from data store – overload - argh
- Server crash – clients reloaded from data store – overload - argh
- Spike, data being stored, overload, argh
- The bigger they are, the harder they fall
- I thought we build products?
- Anniversary billing. Monthly billing. Billing!!!
- Multiple accounts w/ one single relationship
- Customer service? Really?
- Sure, if this floats your boat, but not the point
- You can’t predict crazy spikes (Obama)
- S**t is going to happenPlanning is goodThe ability to react (well) is gooder
- Why not Postgres?“Just Because”Also, Erlang/JSON
- Provides AutomaticScalingFault Tolerance
- Very few people cared about callsCall info is 99.999…. % of the dataMonster CRUD improvement
- Anniversary billing. Monthly billing. Billing!!!
- Multiple accounts w/ one single relationship
- Don’t need Map/Reduce for multi-client queriesPhone informationBilling information
- I person cares about biling. Another about reporting. No one actually cares about calls over a day old. This stuff is – mostly – irrelevant.
- Message / VM / Fax countsBut need to do it by date-rangeNo-one cares about stuff more than a week oldBut some doLearn to say “No”
- Expiring calls? Argh!
- Break call records into individual databases
- Server crash – clients reloaded from data store – overload - argh
- Specifically for phonesBrief outage causes huge spikesMnesia as K-V store
- You can’t predict crazy spikes (Obama)
- Order of magnitude changes are, well, a PITA
- Customer service? Really?
- Billing flexibilityMove billing its own systemStream data from other stores to itCan rebuild offlineExcel is your friend!
- I thought we build products?
- Moved back-office ops to its own storeSingle Source of TruthTightly controlled