Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Life BEYOND
the ILLUSION
of PRESENT
Jonas Bonér
CTO Typesafe
@jboner
"Time is what prevents everything from happening at once."
- John Archibald Wheeler
Newton's
PHYSICS
THIS SIMPLIFIED MODEL IS
VERY APPEALING TO US
von Neumann
ARCHITECTURE
BACK THEN
LIFE WAS GOOD
THEN, ALONG CAME
CONCURRENCY
MADE LIFE MISERABLE
Jim Gray's
TRANSACTIONS
SAVE THE DAY
WELL, ALONG CAME
DISTRIBUTION
MADE LIFE MISERABLE, again...
But don't be surprised
UNFORTUNATELY,
THIS IS NOT
HOW THE WORLD WORKS
"The future is a function of the past."
- A. P. Robertson
"The (local) present is a merge function of
multiple concurrent pasts."
— Me
val newLocalPresent = observedPasts.
foldLeft(oldLocalPresent) { _ merge _ }
WE NEED TO EXPLICITLY MODEL THE
LOCAL PRESENT AS
FACTS DERIVED FROM THE
MERGING OF MULTIPLE
CONCURRENT PASTS
INFORMATION IS ALWAYS FROM THE PAST
THE TRUTH
IS CLOSER TO
EINSTEIN'S
PHYSICS
INFORMATION HAS
LATENCY
THE COST OF MAINTAINING
THIS ILLUSION IS INCREASED
CONTENTION &
COHERENCY
AS LATENCY GETS HIGHER, THE
ILLUSION CRACKS EVEN MORE
DISTRIBUTED SYSTEMS
EVERYWHERE
THE NETWORK IS RELIABLE...NAT
"If a tree falls in a forest and no one is around to hear it, does it make a
sound?"
— Charles Riborg Mann
Information CAN (and will) GET LOST
HOW DO WE DEAL WITH
INFORMATION LOSS
IN REAL LIFE?
WE USE A SIMPLE PROTOCOL OF
Confirm, Wait & Repeat
We fill in THE BLANKS
...AND IF WE ARE WRONG, WE TAKE
COMPENSATING ACTIONAPOLOGY-ORIENTED PROGRAMMING - PAT HELLAND (IN MEMORIES, GUESSES, AND A...
The bottom line:
WE CAN'T FORCE THE WORLD INTO A
SINGLE GLOBALLY CONSISTENT
PRESENT
Should we just GIVE UP?
I BELIEVE THAT THERE IS A
PATH FORWARD
WE NEED TO TREAT
TIME AS A
FIRST CLASS CONSTRUCT
WHAT IS TIME, really?
TIME IS THE
SUCCESSION OF
CAUSALLY RELATED EVENTS
How can we
MANAGE
TIME?
Think in FACTS
What is a
FACT?
"A Fact is something that truly exists or happens: something that has
actual existence, a true piece of information."
— Me...
IMMUTABILITY
is a requirement
So, do variables
HAVE A PURPOSE IN LIFE?
"The assignment statement is the von Neumann bottleneck of
programming languages and keeps us thinking in word-at-a-time t...
MUTABLE STATE
NEEDS TO BE
CONTAINED
Ok, but how should we
MANAGE FACTS?
Functional
PROGRAMMING
Logic
PROGRAMMING
Dataflow
PROGRAMMING
NEVER
DELETE
FACTS
"When bookkeeping was done with clay tablets or paper and ink,
accountants developed some clear rules about good accountin...
CRUD
"Database is a cache of a subset of the log."
— Pat Helland (2007)
Store facts in an
EVENT LOG
The log allows
TIME
TRAVEL
Can we REWRITE THE PAST?
Allows us to shift our focus from
DATA AT REST, to
DATA IN MOTION
Stream Processing
CONSTRUCTING A SUFFICIENTLY CONSISTENT
LOCAL PRESENT
MEANS EMPLOYING
CONSISTENCY MECHANISMS
Consistency
WHAT?
WHY?
WHEN?
WE NEED TO DECOMPOSE THE SYSTEM USING
CONSISTENCY BOUNDARIES
INSIDE DATA: OUR CURRENT PRESENT
OUTSIDE DATA: BLAST FROM THE PAST
BETWEEN SERVICES: HOPE FOR THE FUTURE
— PAT HELLAND (DA...
MicroSERVICE
AGGREGATE Root
WITHIN THE CONSISTENCY BOUNDARY
EVENT
Sourcing
BETWEEN THE
Consistency
Boundaries
IT'S A ZOO
Decoupling in
TIME / SPACE
STRONG
CONSISTENCY
The wrong default
Here, we are living in the
LOOMING SHADOW OF
IMPOSSIBILITY
THEOREMS
FLPCONSENSUS IS IMPOSSIBLE
PROTOCOLS CLIMB THE LADDER OF KNOWLEDGE
Cɸ: Common Knowledge (infinite number of i)
Eiɸ: (Everyone knows * i) ɸ
E3ɸ: (Ever...
COMMON KNOWLEDGE
IS NOT ATTAINABLE VIA PROTOCOL- JOSEPH HALPERN
CAPCONSISTENCY IS IMPOSSIBLE
Dissecting
CAP
"The first principle of successful scalability is to batter the consistency
mechanisms down to a minimum."
– James Hamilton
EVENTUAL
CONSISTENCY
What does it really mean?
Tracking TIME is tracking CAUSALITY
RELYING ON
TIMESTAMPS
IS A BAD IDEA
Instead, rely on
LOGICAL TIME
Lamport
CLOCKS
GLOBAL CAUSAL ORDERING BETWEEN
Vector
CLOCKSPARTIAL CAUSAL ORDERING BETWEEN EVENTS
Causal
CONSISTENCY
What
CONSISTENCY
DO YOU REALLY NEED AND
when?
ACID 2.0ASSOCIATIVE
COMMUTATIVE
IDEMPOTENT
DISTRIBUTED
CONFLICT-FREE REPLICATED DATA TYPES
DISORDERLY PROGRAMMING
CALM THEOREM
WE ARE JUST GETTING STARTED
WE HAVE A LONG ROAD AHEAD OF US...
Thanks
FOR LISTENING
Life BEYOND
the ILLUSION
of PRESENT
Jonas Bonér
CTO Typesafe
@jboner
Life Beyond the Illusion of Present
Life Beyond the Illusion of Present
Upcoming SlideShare
Loading in …5
×

Life Beyond the Illusion of Present

55,048 views

Published on

Video: https://www.parleys.com/tutorial/life-beyond-illusion-present

Summary: The idea of the present is an illusion. Everything we see, hear and feel is just an echo from the past. But this illusion has influenced us and the way we view the world in so many ways; from Newton’s physics with a linearly progressing timeline accruing absolute knowledge along the way to the von Neumann machine with its total ordering of instructions updating mutable state with full control of the “present”. But unfortunately this is not how the world works. There is no present, all we have is facts derived from the merging of multiple pasts. The truth is closer to Einstein’s physics where everything is relative to one’s perspective.

As developers we need to wake up and break free from the perceived reality of living in a single globally consistent present. The advent of multicore and cloud computing architectures meant that most applications today are distributed systems—multiple cores separated by the memory bus or multiple nodes separated by the network—which puts a harsh end to this illusion. Facts travel at the speed of light (at best), which makes the distinction between past and perceived present even more apparent in a distributed system where latency is higher and where facts (messages) can get lost.

The only way to design truly scalable and performant systems that can construct a sufficiently consistent view of history—and thereby our local “present”—is by treating time as a first class construct in our programming model and to model the present as facts derived from the merging of multiple concurrent pasts.

In this talk we will explore what all this means to the design of our systems, how we need to view and model consistency, consensus, communication, history and behaviour, and look at some practical tools and techniques to bring it all together.

Published in: Technology, Internet, Software

Life Beyond the Illusion of Present

  1. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe @jboner
  2. "Time is what prevents everything from happening at once." - John Archibald Wheeler
  3. Newton's PHYSICS
  4. THIS SIMPLIFIED MODEL IS VERY APPEALING TO US
  5. von Neumann ARCHITECTURE
  6. BACK THEN LIFE WAS GOOD
  7. THEN, ALONG CAME CONCURRENCY MADE LIFE MISERABLE
  8. Jim Gray's TRANSACTIONS SAVE THE DAY
  9. WELL, ALONG CAME DISTRIBUTION MADE LIFE MISERABLE, again...
  10. But don't be surprised UNFORTUNATELY, THIS IS NOT HOW THE WORLD WORKS
  11. "The future is a function of the past." - A. P. Robertson
  12. "The (local) present is a merge function of multiple concurrent pasts." — Me
  13. val newLocalPresent = observedPasts. foldLeft(oldLocalPresent) { _ merge _ }
  14. WE NEED TO EXPLICITLY MODEL THE LOCAL PRESENT AS FACTS DERIVED FROM THE MERGING OF MULTIPLE CONCURRENT PASTS
  15. INFORMATION IS ALWAYS FROM THE PAST
  16. THE TRUTH IS CLOSER TO EINSTEIN'S PHYSICS
  17. INFORMATION HAS LATENCY
  18. THE COST OF MAINTAINING THIS ILLUSION IS INCREASED CONTENTION & COHERENCY
  19. AS LATENCY GETS HIGHER, THE ILLUSION CRACKS EVEN MORE
  20. DISTRIBUTED SYSTEMS EVERYWHERE
  21. THE NETWORK IS RELIABLE...NAT
  22. "If a tree falls in a forest and no one is around to hear it, does it make a sound?" — Charles Riborg Mann
  23. Information CAN (and will) GET LOST
  24. HOW DO WE DEAL WITH INFORMATION LOSS IN REAL LIFE?
  25. WE USE A SIMPLE PROTOCOL OF Confirm, Wait & Repeat
  26. We fill in THE BLANKS
  27. ...AND IF WE ARE WRONG, WE TAKE COMPENSATING ACTIONAPOLOGY-ORIENTED PROGRAMMING - PAT HELLAND (IN MEMORIES, GUESSES, AND APOLOGIES)
  28. The bottom line: WE CAN'T FORCE THE WORLD INTO A SINGLE GLOBALLY CONSISTENT PRESENT
  29. Should we just GIVE UP?
  30. I BELIEVE THAT THERE IS A PATH FORWARD
  31. WE NEED TO TREAT TIME AS A FIRST CLASS CONSTRUCT
  32. WHAT IS TIME, really?
  33. TIME IS THE SUCCESSION OF CAUSALLY RELATED EVENTS
  34. How can we MANAGE TIME?
  35. Think in FACTS
  36. What is a FACT?
  37. "A Fact is something that truly exists or happens: something that has actual existence, a true piece of information." — Merriam Webster
  38. IMMUTABILITY is a requirement
  39. So, do variables HAVE A PURPOSE IN LIFE?
  40. "The assignment statement is the von Neumann bottleneck of programming languages and keeps us thinking in word-at-a-time terms in much the same way the computer's bottleneck does." — John Backus (Turing Award lecture 1977)
  41. MUTABLE STATE NEEDS TO BE CONTAINED
  42. Ok, but how should we MANAGE FACTS?
  43. Functional PROGRAMMING
  44. Logic PROGRAMMING
  45. Dataflow PROGRAMMING
  46. NEVER DELETE FACTS
  47. "When bookkeeping was done with clay tablets or paper and ink, accountants developed some clear rules about good accounting practices. One never alters the books; if an error is made, it is annotated and a new compensating entry is made in the books. The books are thus a complete history of the transactions of the business. Update-in-place strikes many systems designers as a cardinal sin: it violates traditional accounting practices that have been observed for hundreds of years." — Jim Gray (1981)
  48. CRUD
  49. "Database is a cache of a subset of the log." — Pat Helland (2007)
  50. Store facts in an EVENT LOG
  51. The log allows TIME TRAVEL
  52. Can we REWRITE THE PAST?
  53. Allows us to shift our focus from DATA AT REST, to DATA IN MOTION
  54. Stream Processing
  55. CONSTRUCTING A SUFFICIENTLY CONSISTENT LOCAL PRESENT MEANS EMPLOYING CONSISTENCY MECHANISMS
  56. Consistency WHAT? WHY? WHEN?
  57. WE NEED TO DECOMPOSE THE SYSTEM USING CONSISTENCY BOUNDARIES
  58. INSIDE DATA: OUR CURRENT PRESENT OUTSIDE DATA: BLAST FROM THE PAST BETWEEN SERVICES: HOPE FOR THE FUTURE — PAT HELLAND (DATA ON THE INSIDE VS DATA ON THE OUTSIDE)
  59. MicroSERVICE
  60. AGGREGATE Root
  61. WITHIN THE CONSISTENCY BOUNDARY
  62. EVENT Sourcing
  63. BETWEEN THE Consistency Boundaries IT'S A ZOO
  64. Decoupling in TIME / SPACE
  65. STRONG CONSISTENCY The wrong default
  66. Here, we are living in the LOOMING SHADOW OF IMPOSSIBILITY THEOREMS
  67. FLPCONSENSUS IS IMPOSSIBLE
  68. PROTOCOLS CLIMB THE LADDER OF KNOWLEDGE Cɸ: Common Knowledge (infinite number of i) Eiɸ: (Everyone knows * i) ɸ E3ɸ: (Everyone knows * 3) ɸ E2ɸ: Everyone knows Everyone knows ɸ E1ɸ: Everyone knows ɸ Sɸ: Someone knows ɸ
  69. COMMON KNOWLEDGE IS NOT ATTAINABLE VIA PROTOCOL- JOSEPH HALPERN
  70. CAPCONSISTENCY IS IMPOSSIBLE
  71. Dissecting CAP
  72. "The first principle of successful scalability is to batter the consistency mechanisms down to a minimum." – James Hamilton
  73. EVENTUAL CONSISTENCY What does it really mean?
  74. Tracking TIME is tracking CAUSALITY
  75. RELYING ON TIMESTAMPS IS A BAD IDEA
  76. Instead, rely on LOGICAL TIME
  77. Lamport CLOCKS GLOBAL CAUSAL ORDERING BETWEEN
  78. Vector CLOCKSPARTIAL CAUSAL ORDERING BETWEEN EVENTS
  79. Causal CONSISTENCY
  80. What CONSISTENCY DO YOU REALLY NEED AND when?
  81. ACID 2.0ASSOCIATIVE COMMUTATIVE IDEMPOTENT DISTRIBUTED
  82. CONFLICT-FREE REPLICATED DATA TYPES
  83. DISORDERLY PROGRAMMING CALM THEOREM
  84. WE ARE JUST GETTING STARTED
  85. WE HAVE A LONG ROAD AHEAD OF US...
  86. Thanks FOR LISTENING
  87. Life BEYOND the ILLUSION of PRESENT Jonas Bonér CTO Typesafe @jboner

×