SlideShare a Scribd company logo
1 of 22
Fungus on White Bread
Ashok Banerjee
Explain Exponential
• Double life
– Constant time double
dy = ky
dx
• Half Life
– Constant time halving
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
0
20
40
60
80
100
120
1 2 3 4 5 6 7
Series1
Word of Mouth & Exponential Growth
• Word of Mouth Driven Model
dC = k1.C.f.k2 – k1
’.C.f.k2
’
dT
C = C0 e^(k1.f.k2 – k1
’.f.k2
’)
• Net Promoter Score = (k1 – k1
’) [Not =
Exponent]
Exponential Growth
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
C = C0 e^(k1.f.k2 – k1
’.f.k2
’)
What if non-exponential?
1. NPS = 0 ?
2. k2
’ >> k2 ?
3. f=0
1. Lonely people
2. Ashamed to share
4. Society
1. findia > fuk
2. fcollege > fprofessional
3. Fwomen > fmen
Facebook, Foursquare vs Twitter
• Facebook, Foursquare: Word or Mouth
dC = k1.C.f.k2
dT
• Kfacebook > kfoursquare
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
Twitter
• Celebrity Interest Model
dC = k(Celebrity) => C = KT [Linear Growth]
dT
• Revolution: Word of Mouth, followed by
exponential decay [ C0e^(-kt) ]
• Force exponential on celebs (Difficult)
• Half Life of Topic
– Religion, Politics: Long half life
– Calamity of scale: Long half life
Model Fertility
• Trick look for phenomenon where rate is
proportionate to the base
• Rabbits/Revolutionaries: dR = kR => R = R0
e^(kt)
dT
• Same as Flipkart Customers
• Orders is a scale of Customers, usually all
needs of a category from a customer go to 1
shop.
Bacteria, Fungus, Mosquitos
• Bacteria in your mouth:
dB = kB => B = B0 e^(kt) [Listerine will time shift
dT this curve]
– Morning After smell is the integral– exponential
– Listerine will time shift but not change k if you rinse
• Fungus growing on your bread
• Mosquitos or Dust allergens with height:
dM = -kM => M = M0 e^(-kh)
dh
Infatuation
• 100 slices of
infatuation
• New Hottie takes
randomly slice
• dI = -kI
• Dt
• Sadness, Memory
decays exponentially
0
20
40
60
80
100
120
1 2 3 4 5 6 7
Series1
Really Flipkart is Exponential?
Weekly Stats
Monthly Stats
Consequence of a delay/Pressure
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Series1
Series2
Run rate over time is the gap between the 2 curves
Delta = ekt – ek(t –delay) = ekt [ 1 – e-delay]
If delay is constant = The gap is growing exponentially
Total financial loss = Area between the curves
Integrate and even this is exponential
Every 3 months if things double:
1 hour delay now = 16 hours a year later
= 2 full days a year later
You will never catch up the delay
The bacteria in your mouth never catch up the Listerine
Effect!!
Curse of Exponential Curves
– Operations will grow exponentially
• Customer Support
• Warehouse
• Procurement
• Logistics (may get second order effect of density)
– Dev Ops load will grow exponential
• Databases
• boxes/processes
• Design the systems to be logarithmic
• Self similar (love Hadoop)
– Cannot hire exponentially even if you wish to
– Technology needs to do something on Cost and Operations
to spend non-exponentially
Database Scaling
• Reads/Writes (# per request)
• Buffering of writes
• Write unstructured, Read/Filter structured
• Dictionary
• Workflows and State Transitions
– Batch Transitions
• Buffered Writes
• Async Writes
• Idempotent Skips
Database Scaling (OLTP)
• SOA (DB partitioned by Domain, SSO, Messaging,
2pc)
• Archive (Time window only in OLTP)
• Sharding
– Query Decomposition, Parallel Execution
,Result Recomposition
• NoSQL fine if Lookups/Range Queries
• Pre-Rollup and Staleness + Overlay counter
• Hadoop (OLAP)
Message Bus Scaling
• Messaging on DB
– Fragmentation
– Insert/Select/Delete profile
• Sharding (File system, distributed, SSD)
– Message Groups and Ordering (Partition or Replica)
– Distributed Queues
• Node Failures, Recovery, Split Brain
– Sidelining of Messages
– Can files act as queues?
– Does system need to be transactional?
• Anomaly Detection and Idempotent
• Delayed ack
Cache Scaling
• Distributed HashTables
• Consistent Hashing
• Memory Mapping (Page Cache vs Row Cache)
NoSQL
• Compressed objects (Memory vs CPU)
– Secret of Dremel (LZO77)
• SIMD (Block Oriented Computing)
• Disk Striping, compressed data
• HDFS: File Triplicate
• Hadoop: Job Scheduler (Fault Tolerance)
Management
• As Organization doubles, the 1 pizza table
communication changes to 2 tables, to 4 tables
• Communication distance (Communication needs
grow combinatorially)
• Hallway chatters need move to
– Structured 1-1s
– Group Meetings
– Emails (therefore the good/bad/ugly)
– Management by 1-1s => Management by walking
about, office hours
When does exponential end
• Disease Modelling [Coursera: Model Thinking]
• dw = cK (w)(N-w) – r (w) = (w) [cK – r]
• Dt N N N N
ck– r> 0, cK/r>1] Vaccinate = vR, immunize
against competitor
Measles, Mumps
Q&A
e
e e
e e
e e
e e
e e
e e
e e
e e
e e
Exponential World After All
• It’s a small world after all. Walt Disney lied
• Exponential growth
– From rabbits to revolutions
– From bacteria in your mouth to fungus on white bread
– To the growth of Facebook to a Flipkart
– To the number of Databases to the # of pages
– To the chain reaction of an atom bomb
• Exponential Decay
– To the decay of your infatuation
– To the decay of your sadness and bereavement
– To the bites of mosquitos on your body with height
– To the # of allergens in atmosphere
– To the decay of radioactive material
Exponential World After All
• To Flipkart’s revenue
• To Flipkart’s costs
• To Flipkart’s staffing in Ops and Dev Ops
• To the cost of a delay by 1 hour today being
the same as 2 days 1 year later
• Optimize, Analyze, Criticize, Visualize, Just
open your eyes

More Related Content

Viewers also liked

E-Commerce Basics
E-Commerce BasicsE-Commerce Basics
E-Commerce BasicsAl Muzahid
 
Continuous deployment-at-flipkart
Continuous deployment-at-flipkartContinuous deployment-at-flipkart
Continuous deployment-at-flipkartPankaj Kaushal
 
Continuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsContinuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsMike Brittain
 
Continuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesContinuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesRoss Snyder
 
Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureDan McKinley
 
Flipkart Logistic & Supply chain management
Flipkart Logistic & Supply chain managementFlipkart Logistic & Supply chain management
Flipkart Logistic & Supply chain managementSagar Sawant
 
Flipkart - Project
Flipkart - ProjectFlipkart - Project
Flipkart - ProjectAnuraga Jha
 

Viewers also liked (7)

E-Commerce Basics
E-Commerce BasicsE-Commerce Basics
E-Commerce Basics
 
Continuous deployment-at-flipkart
Continuous deployment-at-flipkartContinuous deployment-at-flipkart
Continuous deployment-at-flipkart
 
Continuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsContinuous Deployment: The Dirty Details
Continuous Deployment: The Dirty Details
 
Continuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two ApproachesContinuous Deployment at Etsy: A Tale of Two Approaches
Continuous Deployment at Etsy: A Tale of Two Approaches
 
Etsy Activity Feeds Architecture
Etsy Activity Feeds ArchitectureEtsy Activity Feeds Architecture
Etsy Activity Feeds Architecture
 
Flipkart Logistic & Supply chain management
Flipkart Logistic & Supply chain managementFlipkart Logistic & Supply chain management
Flipkart Logistic & Supply chain management
 
Flipkart - Project
Flipkart - ProjectFlipkart - Project
Flipkart - Project
 

Similar to Fungus on White Bread

GoshawkDB: Making Time with Vector Clocks
GoshawkDB: Making Time with Vector ClocksGoshawkDB: Making Time with Vector Clocks
GoshawkDB: Making Time with Vector ClocksC4Media
 
MLSD18. Unsupervised Learning
MLSD18. Unsupervised LearningMLSD18. Unsupervised Learning
MLSD18. Unsupervised LearningBigML, Inc
 
Text Mining, Association Rules and Decision Tree Learning
Text Mining, Association Rules and Decision Tree LearningText Mining, Association Rules and Decision Tree Learning
Text Mining, Association Rules and Decision Tree LearningAdrian Cuyugan
 
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2Yo Halb
 
Fast Single-pass K-means Clusterting at Oxford
Fast Single-pass K-means Clusterting at Oxford Fast Single-pass K-means Clusterting at Oxford
Fast Single-pass K-means Clusterting at Oxford MapR Technologies
 
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)Matthew Lease
 
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...Databricks
 
Sparkling Random Ferns by P Dendek and M Fedoryszak
Sparkling Random Ferns by  P Dendek and M FedoryszakSparkling Random Ferns by  P Dendek and M Fedoryszak
Sparkling Random Ferns by P Dendek and M FedoryszakSpark Summit
 
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)Matthew Lease
 
Protocols of Interaction: Best Current Practices
Protocols of Interaction: Best Current PracticesProtocols of Interaction: Best Current Practices
Protocols of Interaction: Best Current PracticesC4Media
 
Paris data-geeks-2013-03-28
Paris data-geeks-2013-03-28Paris data-geeks-2013-03-28
Paris data-geeks-2013-03-28Ted Dunning
 
2015 9-30-sbc361-research methcomm
2015 9-30-sbc361-research methcomm2015 9-30-sbc361-research methcomm
2015 9-30-sbc361-research methcommYannick Wurm
 
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVC
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVCCreating Presentations That Matter - A 1-day workshop (May 4th) at SVC
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVCAshley Bright
 
Storing Time Series Metrics With Cassandra and Composite Columns
Storing Time Series Metrics With Cassandra and Composite ColumnsStoring Time Series Metrics With Cassandra and Composite Columns
Storing Time Series Metrics With Cassandra and Composite ColumnsJoe Stein
 
(7) Lesson 3.5 - Divide Integers
(7) Lesson 3.5 - Divide Integers(7) Lesson 3.5 - Divide Integers
(7) Lesson 3.5 - Divide Integerswzuri
 
Classification & Clustering.pptx
Classification & Clustering.pptxClassification & Clustering.pptx
Classification & Clustering.pptxImXaib
 

Similar to Fungus on White Bread (19)

GoshawkDB: Making Time with Vector Clocks
GoshawkDB: Making Time with Vector ClocksGoshawkDB: Making Time with Vector Clocks
GoshawkDB: Making Time with Vector Clocks
 
MLSD18. Unsupervised Learning
MLSD18. Unsupervised LearningMLSD18. Unsupervised Learning
MLSD18. Unsupervised Learning
 
Text Mining, Association Rules and Decision Tree Learning
Text Mining, Association Rules and Decision Tree LearningText Mining, Association Rules and Decision Tree Learning
Text Mining, Association Rules and Decision Tree Learning
 
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2
[YIDLUG] Programming Languages Differences, The Underlying Implementation 1 of 2
 
Fast Single-pass K-means Clusterting at Oxford
Fast Single-pass K-means Clusterting at Oxford Fast Single-pass K-means Clusterting at Oxford
Fast Single-pass K-means Clusterting at Oxford
 
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 7: Data-Intensive Computing for Text Analysis (Fall 2011)
 
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...
Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...
 
Sparkling Random Ferns by P Dendek and M Fedoryszak
Sparkling Random Ferns by  P Dendek and M FedoryszakSparkling Random Ferns by  P Dendek and M Fedoryszak
Sparkling Random Ferns by P Dendek and M Fedoryszak
 
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)
Lecture 6: Data-Intensive Computing for Text Analysis (Fall 2011)
 
Protocols of Interaction: Best Current Practices
Protocols of Interaction: Best Current PracticesProtocols of Interaction: Best Current Practices
Protocols of Interaction: Best Current Practices
 
fundamentals.ppt
fundamentals.pptfundamentals.ppt
fundamentals.ppt
 
Paris data-geeks-2013-03-28
Paris data-geeks-2013-03-28Paris data-geeks-2013-03-28
Paris data-geeks-2013-03-28
 
fundamentals.ppt
fundamentals.pptfundamentals.ppt
fundamentals.ppt
 
2015 9-30-sbc361-research methcomm
2015 9-30-sbc361-research methcomm2015 9-30-sbc361-research methcomm
2015 9-30-sbc361-research methcomm
 
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVC
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVCCreating Presentations That Matter - A 1-day workshop (May 4th) at SVC
Creating Presentations That Matter - A 1-day workshop (May 4th) at SVC
 
ACM 2013-02-25
ACM 2013-02-25ACM 2013-02-25
ACM 2013-02-25
 
Storing Time Series Metrics With Cassandra and Composite Columns
Storing Time Series Metrics With Cassandra and Composite ColumnsStoring Time Series Metrics With Cassandra and Composite Columns
Storing Time Series Metrics With Cassandra and Composite Columns
 
(7) Lesson 3.5 - Divide Integers
(7) Lesson 3.5 - Divide Integers(7) Lesson 3.5 - Divide Integers
(7) Lesson 3.5 - Divide Integers
 
Classification & Clustering.pptx
Classification & Clustering.pptxClassification & Clustering.pptx
Classification & Clustering.pptx
 

Recently uploaded

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 

Recently uploaded (20)

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort ServiceHot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
Hot Sexy call girls in Panjabi Bagh 🔝 9953056974 🔝 Delhi escort Service
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 

Fungus on White Bread

  • 1. Fungus on White Bread Ashok Banerjee
  • 2. Explain Exponential • Double life – Constant time double dy = ky dx • Half Life – Constant time halving 0 20 40 60 80 100 120 140 160 1 2 3 4 5 Series1 0 20 40 60 80 100 120 1 2 3 4 5 6 7 Series1
  • 3. Word of Mouth & Exponential Growth • Word of Mouth Driven Model dC = k1.C.f.k2 – k1 ’.C.f.k2 ’ dT C = C0 e^(k1.f.k2 – k1 ’.f.k2 ’) • Net Promoter Score = (k1 – k1 ’) [Not = Exponent]
  • 4. Exponential Growth 0 20 40 60 80 100 120 140 160 1 2 3 4 5 Series1 C = C0 e^(k1.f.k2 – k1 ’.f.k2 ’) What if non-exponential? 1. NPS = 0 ? 2. k2 ’ >> k2 ? 3. f=0 1. Lonely people 2. Ashamed to share 4. Society 1. findia > fuk 2. fcollege > fprofessional 3. Fwomen > fmen
  • 5. Facebook, Foursquare vs Twitter • Facebook, Foursquare: Word or Mouth dC = k1.C.f.k2 dT • Kfacebook > kfoursquare 0 20 40 60 80 100 120 140 160 1 2 3 4 5 Series1
  • 6. Twitter • Celebrity Interest Model dC = k(Celebrity) => C = KT [Linear Growth] dT • Revolution: Word of Mouth, followed by exponential decay [ C0e^(-kt) ] • Force exponential on celebs (Difficult) • Half Life of Topic – Religion, Politics: Long half life – Calamity of scale: Long half life
  • 7. Model Fertility • Trick look for phenomenon where rate is proportionate to the base • Rabbits/Revolutionaries: dR = kR => R = R0 e^(kt) dT • Same as Flipkart Customers • Orders is a scale of Customers, usually all needs of a category from a customer go to 1 shop.
  • 8. Bacteria, Fungus, Mosquitos • Bacteria in your mouth: dB = kB => B = B0 e^(kt) [Listerine will time shift dT this curve] – Morning After smell is the integral– exponential – Listerine will time shift but not change k if you rinse • Fungus growing on your bread • Mosquitos or Dust allergens with height: dM = -kM => M = M0 e^(-kh) dh
  • 9. Infatuation • 100 slices of infatuation • New Hottie takes randomly slice • dI = -kI • Dt • Sadness, Memory decays exponentially 0 20 40 60 80 100 120 1 2 3 4 5 6 7 Series1
  • 10. Really Flipkart is Exponential? Weekly Stats Monthly Stats
  • 11. Consequence of a delay/Pressure 0 20 40 60 80 100 120 140 160 1 2 3 4 5 Series1 Series2 Run rate over time is the gap between the 2 curves Delta = ekt – ek(t –delay) = ekt [ 1 – e-delay] If delay is constant = The gap is growing exponentially Total financial loss = Area between the curves Integrate and even this is exponential Every 3 months if things double: 1 hour delay now = 16 hours a year later = 2 full days a year later You will never catch up the delay The bacteria in your mouth never catch up the Listerine Effect!!
  • 12. Curse of Exponential Curves – Operations will grow exponentially • Customer Support • Warehouse • Procurement • Logistics (may get second order effect of density) – Dev Ops load will grow exponential • Databases • boxes/processes • Design the systems to be logarithmic • Self similar (love Hadoop) – Cannot hire exponentially even if you wish to – Technology needs to do something on Cost and Operations to spend non-exponentially
  • 13. Database Scaling • Reads/Writes (# per request) • Buffering of writes • Write unstructured, Read/Filter structured • Dictionary • Workflows and State Transitions – Batch Transitions • Buffered Writes • Async Writes • Idempotent Skips
  • 14. Database Scaling (OLTP) • SOA (DB partitioned by Domain, SSO, Messaging, 2pc) • Archive (Time window only in OLTP) • Sharding – Query Decomposition, Parallel Execution ,Result Recomposition • NoSQL fine if Lookups/Range Queries • Pre-Rollup and Staleness + Overlay counter • Hadoop (OLAP)
  • 15. Message Bus Scaling • Messaging on DB – Fragmentation – Insert/Select/Delete profile • Sharding (File system, distributed, SSD) – Message Groups and Ordering (Partition or Replica) – Distributed Queues • Node Failures, Recovery, Split Brain – Sidelining of Messages – Can files act as queues? – Does system need to be transactional? • Anomaly Detection and Idempotent • Delayed ack
  • 16. Cache Scaling • Distributed HashTables • Consistent Hashing • Memory Mapping (Page Cache vs Row Cache)
  • 17. NoSQL • Compressed objects (Memory vs CPU) – Secret of Dremel (LZO77) • SIMD (Block Oriented Computing) • Disk Striping, compressed data • HDFS: File Triplicate • Hadoop: Job Scheduler (Fault Tolerance)
  • 18. Management • As Organization doubles, the 1 pizza table communication changes to 2 tables, to 4 tables • Communication distance (Communication needs grow combinatorially) • Hallway chatters need move to – Structured 1-1s – Group Meetings – Emails (therefore the good/bad/ugly) – Management by 1-1s => Management by walking about, office hours
  • 19. When does exponential end • Disease Modelling [Coursera: Model Thinking] • dw = cK (w)(N-w) – r (w) = (w) [cK – r] • Dt N N N N ck– r> 0, cK/r>1] Vaccinate = vR, immunize against competitor Measles, Mumps
  • 20. Q&A e e e e e e e e e e e e e e e e e e e
  • 21. Exponential World After All • It’s a small world after all. Walt Disney lied • Exponential growth – From rabbits to revolutions – From bacteria in your mouth to fungus on white bread – To the growth of Facebook to a Flipkart – To the number of Databases to the # of pages – To the chain reaction of an atom bomb • Exponential Decay – To the decay of your infatuation – To the decay of your sadness and bereavement – To the bites of mosquitos on your body with height – To the # of allergens in atmosphere – To the decay of radioactive material
  • 22. Exponential World After All • To Flipkart’s revenue • To Flipkart’s costs • To Flipkart’s staffing in Ops and Dev Ops • To the cost of a delay by 1 hour today being the same as 2 days 1 year later • Optimize, Analyze, Criticize, Visualize, Just open your eyes