5. Facebook, Foursquare vs Twitter
• Facebook, Foursquare: Word or Mouth
dC = k1.C.f.k2
dT
• Kfacebook > kfoursquare
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
6. Twitter
• Celebrity Interest Model
dC = k(Celebrity) => C = KT [Linear Growth]
dT
• Revolution: Word of Mouth, followed by
exponential decay [ C0e^(-kt) ]
• Force exponential on celebs (Difficult)
• Half Life of Topic
– Religion, Politics: Long half life
– Calamity of scale: Long half life
7. Model Fertility
• Trick look for phenomenon where rate is
proportionate to the base
• Rabbits/Revolutionaries: dR = kR => R = R0
e^(kt)
dT
• Same as Flipkart Customers
• Orders is a scale of Customers, usually all
needs of a category from a customer go to 1
shop.
8. Bacteria, Fungus, Mosquitos
• Bacteria in your mouth:
dB = kB => B = B0 e^(kt) [Listerine will time shift
dT this curve]
– Morning After smell is the integral– exponential
– Listerine will time shift but not change k if you rinse
• Fungus growing on your bread
• Mosquitos or Dust allergens with height:
dM = -kM => M = M0 e^(-kh)
dh
11. Consequence of a delay/Pressure
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
Series2
Run rate over time is the gap between the 2 curves
Delta = ekt – ek(t –delay) = ekt [ 1 – e-delay]
If delay is constant = The gap is growing exponentially
Total financial loss = Area between the curves
Integrate and even this is exponential
Every 3 months if things double:
1 hour delay now = 16 hours a year later
= 2 full days a year later
You will never catch up the delay
The bacteria in your mouth never catch up the Listerine
Effect!!
12. Curse of Exponential Curves
– Operations will grow exponentially
• Customer Support
• Warehouse
• Procurement
• Logistics (may get second order effect of density)
– Dev Ops load will grow exponential
• Databases
• boxes/processes
• Design the systems to be logarithmic
• Self similar (love Hadoop)
– Cannot hire exponentially even if you wish to
– Technology needs to do something on Cost and Operations
to spend non-exponentially
13. Database Scaling
• Reads/Writes (# per request)
• Buffering of writes
• Write unstructured, Read/Filter structured
• Dictionary
• Workflows and State Transitions
– Batch Transitions
• Buffered Writes
• Async Writes
• Idempotent Skips
14. Database Scaling (OLTP)
• SOA (DB partitioned by Domain, SSO, Messaging,
2pc)
• Archive (Time window only in OLTP)
• Sharding
– Query Decomposition, Parallel Execution
,Result Recomposition
• NoSQL fine if Lookups/Range Queries
• Pre-Rollup and Staleness + Overlay counter
• Hadoop (OLAP)
15. Message Bus Scaling
• Messaging on DB
– Fragmentation
– Insert/Select/Delete profile
• Sharding (File system, distributed, SSD)
– Message Groups and Ordering (Partition or Replica)
– Distributed Queues
• Node Failures, Recovery, Split Brain
– Sidelining of Messages
– Can files act as queues?
– Does system need to be transactional?
• Anomaly Detection and Idempotent
• Delayed ack
17. NoSQL
• Compressed objects (Memory vs CPU)
– Secret of Dremel (LZO77)
• SIMD (Block Oriented Computing)
• Disk Striping, compressed data
• HDFS: File Triplicate
• Hadoop: Job Scheduler (Fault Tolerance)
18. Management
• As Organization doubles, the 1 pizza table
communication changes to 2 tables, to 4 tables
• Communication distance (Communication needs
grow combinatorially)
• Hallway chatters need move to
– Structured 1-1s
– Group Meetings
– Emails (therefore the good/bad/ugly)
– Management by 1-1s => Management by walking
about, office hours
19. When does exponential end
• Disease Modelling [Coursera: Model Thinking]
• dw = cK (w)(N-w) – r (w) = (w) [cK – r]
• Dt N N N N
ck– r> 0, cK/r>1] Vaccinate = vR, immunize
against competitor
Measles, Mumps
21. Exponential World After All
• It’s a small world after all. Walt Disney lied
• Exponential growth
– From rabbits to revolutions
– From bacteria in your mouth to fungus on white bread
– To the growth of Facebook to a Flipkart
– To the number of Databases to the # of pages
– To the chain reaction of an atom bomb
• Exponential Decay
– To the decay of your infatuation
– To the decay of your sadness and bereavement
– To the bites of mosquitos on your body with height
– To the # of allergens in atmosphere
– To the decay of radioactive material
22. Exponential World After All
• To Flipkart’s revenue
• To Flipkart’s costs
• To Flipkart’s staffing in Ops and Dev Ops
• To the cost of a delay by 1 hour today being
the same as 2 days 1 year later
• Optimize, Analyze, Criticize, Visualize, Just
open your eyes