Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
ScyllaDB, profile picture
Uploaded byScyllaDB
13 views

Working on Complex Systems by Teiva Harsanyi

Throughout my career, I’ve worked in many complicated environments. Yet nothing comes close to the intricacy of my experience at Google, and a few years there have refined my perception of complexity. In this talk, we'll break down complexity and explain why complex differs from complicated. Next, we'll step back to understand what makes some environments complex, and then explore patterns for navigating these systems effectively.

Technology◦
Download to read offline
A ScyllaDB Community Working on Complex Systems Teiva Harsanyi SRE
What’s a System?
What’s a System? A set of elements interconnected to achieve something
What’s a System? A set of elements interconnected to achieve something
What’s a System? A set of elements interconnected to achieve something
Complicated Complex Complicated vs. Complex
Complicated Complex Complicated vs. Complex
Why Does it Matter? Common solutions to complex systems may not lead to effective results We need to find creative solutions to navigate complex systems
Characteristics of Complex Systems
Emergent Behavior ■ Definition: When you can’t predict how a system will behave just by studying each element alone ■ Impact: Unexpected outcomes
Code change
Code change Staging
Code change Staging Progressive rollout start
Code change Staging Progressive rollout start Progressive rollout end
Code change Staging Progressive rollout start Progressive rollout end
Code change Staging Progressive rollout start Progressive rollout end Incident
Code change Staging Progressive rollout start Progressive rollout end Incident
Delayed Consequences ■ Definition: Actions don't always have immediate effects ■ Impact: False sense of stability
Congestion window size Round trips
■ Definition: Small changes can produce disproportionately large or unpredictable effects ■ Impact: Unpredictable scale of outcomes Nonlinearity
Gatekeeper Service Disruption X on entity Y Yes / No
Capacity Upgrade
Capacity Upgrade
Capacity Upgrade
Local Wins, Global Losses ■ Definition: Optimizing one part doesn’t necessarily improve the whole system ■ Impact: Teams may “hit their numbers” while the whole system stagnates or even degrades
Hysteresis ■ Definition: The system’s current behavior depends on its history ■ Impact: Effects persist after the cause is gone
Patterns to Navigate Complex Systems
Controlled Rollout ■ Feature flags ■ Canary releases / Progressive rollouts ■ Shadow testing
Simulation ■ A story of failure ■ Change simulation ■ Improve decision-making in complex systems
Reversibility One-way door Two-way door
■ Local metrics are helpful but not enough ■ We should also consider metrics to get a holistic view of a system Thinking Beyond Local Metrics
Thinking Beyond Local Metrics A C D B E
Thinking Beyond Local Metrics A C D B E
Thinking Beyond Local Metrics A C D B E
Thinking Beyond Local Metrics A C D B E
Events Patterns and trends System structure Values and beliefs System Thinking
Innovation ■ An impossible story ■ Think out of the box and embrace innovative solutions
Conclusion
Conclusion Shift your thinking Not binary What about your system?
Stay in Touch