0
NoSQL the Telco way  {   @dieswaytoofast      (V.P. Ubiquiti Networks)
The Business
   Phone services for SMBsThe Business
   Phone services for SMBs       Hosted Phone services for SMBsThe Business
   Phone services for SMBs       Hosted Phone services for SMBs       Hosted Cloud Communications service for SMBsThe B...
   Phone services for SMBs                       T?       Hosted Phone services for SMBs                     HA       H...
The Metrics
   Phone Calls per SecondThe Metrics
   Phone Calls per Second               x 1000The Metrics
   Simultaneous Phone CallsThe Metrics
   Simultaneous Phone Calls            x 10,000The Metrics
   HTTP API requestsThe Metrics
   HTTP API requests                    ∞The Metrics
   Self-hosted (kinda)The Infrastructure
   Self-hosted (kinda)       Big IP pipesThe Infrastructure
   Self-hosted (kinda)       Big IP pipes       ErlangThe Infrastructure
   Self-hosted (kinda)       Big IP pipes       Erlang       Polyglot PersistenceThe Infrastructure
   Self-hosted (kinda)       Big IP pipes       Erlang       Polyglot Persistence                                H?   ...
   Domain-specific data storesPolyglot Persistence
   Domain-specific data stores                SQLPolyglot Persistence
NoSQL        Domain-specific data stores                SQLPolyglot Persistence
NoSQL       Domain-specific data stores               SQLPolyglot PersistenceFiles
NoSQL          Domain-specific data storesText              SQLPolyglot PersistenceFiles
NoSQLExcel          Domain-specific data storesText              SQLPolyglot PersistenceFiles
NoSQLExcel          Domain-specific data stores                                         Post-ItText              SQLPolyg...
NoSQLExcel          Domain-specific data stores                                         Post-ItText              SQLPolyg...
   Not (necessarily) structured dataNoSQL
   Not (necessarily) structured data      Solution OrientedNoSQL
   Not (necessarily) structured data      Solution Oriented                                     H?                      ...
SQ   L
No     SQ          L
75     bh        p
What d’you want the data to look like when you fetch    it from the database?                          - Casey RosenthalSo...
Key-ValueSolution Oriented
Key-ValueSolution OrientedObject
Key-Value              ColumnSolution OrientedObject
Key-ValueDocument               Column Solution Oriented Object
Key-Value                GraphDocument               Column Solution Oriented Object
Key-Value                Graph                   is llyDocument                ns tua                        nt           ...
Key-Value     d    re          Graph  deOr                   is llyDocument                ns tua                        n...
Key-Value            d         re       Graph       de     Or                      is llyDocument                   ns tua...
lu le    Key-Value            d                              Va tip         re                   ns tua e                 ...
lu le    Key-Value            d                              Va tip         re                   ns tua e                 ...
http://techcrunch.com/2012/10/27/big-data-right-now-five-trendy-open-source-technologies/
Example please?
"Everybody Knows"
"E  VE    RY       B   OD             YK               NO                    W                        S
EN O  GI V E    NE R      ER        IN          G!
Anything else?
   I lied about reportsAnything else?
   I lied about reports (kind-of)Anything else?
Do tell…
If its easy, people might actually use it                            - <name withheld>Sad but true…
Friction - Bad...
Friction - Good...
Friction - Good...Really?
Friction - Good...Example please?
Anything else?
Scaling Matters
S                  W             NO           YK        BOD     RYScaling Matters   VE "E
Scaling is easy
S                  W             NO           YK        BOD     RYScaling is easy   VE "E
Automatic Scaling is hard
Automatic Scaling is hard
Automatic Scaling is hard
Automatic Scaling is hard
Automatic Scaling is hard
Automatic Scaling is hard
And the Failure modes!
And the Failure modes!
And the Failure modes!
And the Failure modes!
And the Failure modes!
Back Office Systems
New CFO
New CEO
AGILITYThe Bottom Line
a·gil·i·ty   /əˈdʒɪlɪti/     noun     the power of moving quickly and easily; nimblenessAgility
   Loose CouplingAgility
   Loose Coupling        Hot UpgradesAgility
   Loose Coupling        Hot Upgrades        Polyglot PersistenceAgility
   Move call information into one (per-client)        databaseRedesign
New CFO
   Preprocess Call information           Separate out billing informationRedesign
   Preprocess Call information           Separate out billing information    What d’you want the data to look like when ...
New CEO
   Move all client info into one DatabaseRedesign
"E  VE    RY       B   OD             YK               NO                    W                        S
   More pre-computations (Date ranges! Argh!)Redesign
   Expiring calls? Argh!Redesign
And the Failure modes!
   Decouple authentication             MnesiaRedesign
Automatic Scaling is hard
   Caches Caches Everywhere…            MnesiaRedesign
Automatic Scaling is hard
                       Mnesia        Caches Caches Everywhere…            MnesiaRedesign
                       Mnesia        Caches Caches Everywhere…     Mnesia MnesiaRedesign
MnesiaMnesia        Caches Caches Everywhere…      Mnesia  Mnesia Redesign
Mnesia                        MnesiaMnesia        Caches Caches Everywhere…      Mnesia  Mnesia Redesign
Mnesia  Mnesia          MnesiaMnesia        Caches Caches Everywhere…       Mnesia  Mnesia Redesign
Mnesia  Mnesia           MnesiaMnesia       Caches Caches Everywhere…        Mnesia  Mnesia Redesign  Mnesia
Mnesia  Mnesia           MnesiaMnesia       Caches Caches Everywhere…        Mnesia  Mnesia Redesign  Mnesia     Mnesia
Redesign
Back Office Systems
Redesign
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
NoSQL learnings from the world of Telco
Upcoming SlideShare
Loading in...5
×

NoSQL learnings from the world of Telco

510

Published on

Being a decent-sized Telecommunications provider, we process a lot of calls (hundreds/second), and need to keep track of all the events on each call. Technically speaking, this is "A Lot" of data - data that our clients (and our own people!) want real-time access to in a myriad of ways. We've ended up going through quite a few NoSQL stores in our quest to satisfy everyone - and the way we do things now has very little to do with where we started out. Join me as I describe our experience and what we've learned, focusing on the Big 4, viz

-The "solution-oriented" nature of NoSQL repeatedly changed our understanding of our problem-space - sometimes drastically.
- The system behavior , particularly the failure modes, were significantly different at scale
-The software model kept getting overhauled - regardless of how much we planned ahead
-We came to value agility - the ability to change direction - above all (yes, even at a Telco!)

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
510
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Images here!!!
  • Data model is(more) variable
  • Images here
  • Images here
  • Images here
  • Images here
  • Images here
  • Images here
  • Images here
  • Images here
  • Images here
  • And this is an issue. If the problem space changes…
  • Images here
  • And once the problem has changed, you are left w/ a solution that no longer applies
  • Reporting
  • The two most dangerous words in Product Development
  • I person cares about biling. Another about reporting. No one actually cares about calls over a day old. This stuff is – mostly – irrelevant.
  • A solution designed to process massive numbers of calls is useless if no-one wants to process massive numbers of calls
  • A solution designed to process massive numbers of calls is useless if no-one wants to process massive numbers of calls
  • Still one user, but uses it all the timeOh s**t, people actually use this.
  • Which leads, inexporably, to the next point, which is that
  • Efficient markets
  • Friction can be very useful in avoiding issues
  • The two most dangerous words in Product Development
  • With callrooms, people want to do voicemail in real time. Worldwide. Ack!
  • But I designed for scaleAdd servers, rebalance, etc
  • But I designed for scaleAdd servers, rebalance, etc
  • You can’t restart servers, ‘cos your clients are “always on”
  • You can’t restart servers, ‘cos your clients are “always on”
  • You can’t predict crazy spikes (Obama)
  • Order of magnitude changes are, well, a PITA
  • Server crash – clients reloaded from data store – overload - argh
  • Server crash – clients reloaded from data store – overload - argh
  • Spike, data being stored, overload, argh
  • The bigger they are, the harder they fall
  • I thought we build products?
  • Anniversary billing. Monthly billing. Billing!!!
  • Multiple accounts w/ one single relationship
  • Customer service? Really?
  • Sure, if this floats your boat, but not the point
  • You can’t predict crazy spikes (Obama)
  • S**t is going to happenPlanning is goodThe ability to react (well) is gooder
  • Why not Postgres?“Just Because”Also, Erlang/JSON
  • Provides AutomaticScalingFault Tolerance
  • Very few people cared about callsCall info is 99.999…. % of the dataMonster CRUD improvement
  • Anniversary billing. Monthly billing. Billing!!!
  • Multiple accounts w/ one single relationship
  • Don’t need Map/Reduce for multi-client queriesPhone informationBilling information
  • I person cares about biling. Another about reporting. No one actually cares about calls over a day old. This stuff is – mostly – irrelevant.
  • Message / VM / Fax countsBut need to do it by date-rangeNo-one cares about stuff more than a week oldBut some doLearn to say “No”
  • Expiring calls? Argh!
  • Break call records into individual databases
  • Server crash – clients reloaded from data store – overload - argh
  • Specifically for phonesBrief outage causes huge spikesMnesia as K-V store
  • You can’t predict crazy spikes (Obama)
  • Order of magnitude changes are, well, a PITA
  • Customer service? Really?
  • Billing flexibilityMove billing its own systemStream data from other stores to itCan rebuild offlineExcel is your friend!
  • I thought we build products?
  • Moved back-office ops to its own storeSingle Source of TruthTightly controlled
  • Transcript of "NoSQL learnings from the world of Telco"

    1. 1. NoSQL the Telco way { @dieswaytoofast (V.P. Ubiquiti Networks)
    2. 2. The Business
    3. 3.  Phone services for SMBsThe Business
    4. 4.  Phone services for SMBs  Hosted Phone services for SMBsThe Business
    5. 5.  Phone services for SMBs  Hosted Phone services for SMBs  Hosted Cloud Communications service for SMBsThe Business
    6. 6.  Phone services for SMBs T?  Hosted Phone services for SMBs HA  Hosted Cloud Communications service for SMBs W H HUThe Business
    7. 7. The Metrics
    8. 8.  Phone Calls per SecondThe Metrics
    9. 9.  Phone Calls per Second x 1000The Metrics
    10. 10.  Simultaneous Phone CallsThe Metrics
    11. 11.  Simultaneous Phone Calls x 10,000The Metrics
    12. 12.  HTTP API requestsThe Metrics
    13. 13.  HTTP API requests ∞The Metrics
    14. 14.  Self-hosted (kinda)The Infrastructure
    15. 15.  Self-hosted (kinda)  Big IP pipesThe Infrastructure
    16. 16.  Self-hosted (kinda)  Big IP pipes  ErlangThe Infrastructure
    17. 17.  Self-hosted (kinda)  Big IP pipes  Erlang  Polyglot PersistenceThe Infrastructure
    18. 18.  Self-hosted (kinda)  Big IP pipes  Erlang  Polyglot Persistence H? HUThe Infrastructure
    19. 19.  Domain-specific data storesPolyglot Persistence
    20. 20.  Domain-specific data stores SQLPolyglot Persistence
    21. 21. NoSQL  Domain-specific data stores SQLPolyglot Persistence
    22. 22. NoSQL  Domain-specific data stores SQLPolyglot PersistenceFiles
    23. 23. NoSQL  Domain-specific data storesText SQLPolyglot PersistenceFiles
    24. 24. NoSQLExcel  Domain-specific data storesText SQLPolyglot PersistenceFiles
    25. 25. NoSQLExcel  Domain-specific data stores Post-ItText SQLPolyglot PersistenceFiles
    26. 26. NoSQLExcel  Domain-specific data stores Post-ItText SQLPolyglot PersistenceFiles
    27. 27.  Not (necessarily) structured dataNoSQL
    28. 28.  Not (necessarily) structured data  Solution OrientedNoSQL
    29. 29.  Not (necessarily) structured data  Solution Oriented H? HUNoSQL
    30. 30. SQ L
    31. 31. No SQ L
    32. 32. 75 bh p
    33. 33. What d’you want the data to look like when you fetch it from the database? - Casey RosenthalSolution Oriented
    34. 34. Key-ValueSolution Oriented
    35. 35. Key-ValueSolution OrientedObject
    36. 36. Key-Value ColumnSolution OrientedObject
    37. 37. Key-ValueDocument Column Solution Oriented Object
    38. 38. Key-Value GraphDocument Column Solution Oriented Object
    39. 39. Key-Value Graph is llyDocument ns tua nt Column te en Ev Solution Oriented Object Co
    40. 40. Key-Value d re Graph deOr is llyDocument ns tua nt Column te en Ev Solution Oriented Object Co
    41. 41. Key-Value d re Graph de Or is llyDocument ns tua nt y Column teor en em Solution Oriented Ev ObjectM Co
    42. 42. lu le Key-Value d Va tip re ns tua e Graph de ul M Or is llyDocument nt y Column teor en em Solution Oriented Ev ObjectM Co
    43. 43. lu le Key-Value d Va tip re ns tua e Graph de ul M Or is llyDocument nt y Column teor en em Solution Oriented Ev ObjectM Co
    44. 44. http://techcrunch.com/2012/10/27/big-data-right-now-five-trendy-open-source-technologies/
    45. 45. Example please?
    46. 46. "Everybody Knows"
    47. 47. "E VE RY B OD YK NO W S
    48. 48. EN O GI V E NE R ER IN G!
    49. 49. Anything else?
    50. 50.  I lied about reportsAnything else?
    51. 51.  I lied about reports (kind-of)Anything else?
    52. 52. Do tell…
    53. 53. If its easy, people might actually use it - <name withheld>Sad but true…
    54. 54. Friction - Bad...
    55. 55. Friction - Good...
    56. 56. Friction - Good...Really?
    57. 57. Friction - Good...Example please?
    58. 58. Anything else?
    59. 59. Scaling Matters
    60. 60. S W NO YK BOD RYScaling Matters VE "E
    61. 61. Scaling is easy
    62. 62. S W NO YK BOD RYScaling is easy VE "E
    63. 63. Automatic Scaling is hard
    64. 64. Automatic Scaling is hard
    65. 65. Automatic Scaling is hard
    66. 66. Automatic Scaling is hard
    67. 67. Automatic Scaling is hard
    68. 68. Automatic Scaling is hard
    69. 69. And the Failure modes!
    70. 70. And the Failure modes!
    71. 71. And the Failure modes!
    72. 72. And the Failure modes!
    73. 73. And the Failure modes!
    74. 74. Back Office Systems
    75. 75. New CFO
    76. 76. New CEO
    77. 77. AGILITYThe Bottom Line
    78. 78. a·gil·i·ty /əˈdʒɪlɪti/ noun the power of moving quickly and easily; nimblenessAgility
    79. 79.  Loose CouplingAgility
    80. 80.  Loose Coupling  Hot UpgradesAgility
    81. 81.  Loose Coupling  Hot Upgrades  Polyglot PersistenceAgility
    82. 82.  Move call information into one (per-client) databaseRedesign
    83. 83. New CFO
    84. 84.  Preprocess Call information  Separate out billing informationRedesign
    85. 85.  Preprocess Call information  Separate out billing information What d’you want the data to look like when you fetch it from the database? - Casey RosenthalRedesign
    86. 86. New CEO
    87. 87.  Move all client info into one DatabaseRedesign
    88. 88. "E VE RY B OD YK NO W S
    89. 89.  More pre-computations (Date ranges! Argh!)Redesign
    90. 90.  Expiring calls? Argh!Redesign
    91. 91. And the Failure modes!
    92. 92.  Decouple authentication MnesiaRedesign
    93. 93. Automatic Scaling is hard
    94. 94.  Caches Caches Everywhere… MnesiaRedesign
    95. 95. Automatic Scaling is hard
    96. 96.  Mnesia Caches Caches Everywhere… MnesiaRedesign
    97. 97.  Mnesia Caches Caches Everywhere… Mnesia MnesiaRedesign
    98. 98. MnesiaMnesia  Caches Caches Everywhere… Mnesia Mnesia Redesign
    99. 99. Mnesia MnesiaMnesia  Caches Caches Everywhere… Mnesia Mnesia Redesign
    100. 100. Mnesia Mnesia MnesiaMnesia  Caches Caches Everywhere… Mnesia Mnesia Redesign
    101. 101. Mnesia Mnesia MnesiaMnesia  Caches Caches Everywhere… Mnesia Mnesia Redesign Mnesia
    102. 102. Mnesia Mnesia MnesiaMnesia  Caches Caches Everywhere… Mnesia Mnesia Redesign Mnesia Mnesia
    103. 103. Redesign
    104. 104. Back Office Systems
    105. 105. Redesign
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×