SlideShare a Scribd company logo
©2024 Andrea Goulet. All Rights Reserved.
Entropy, Software
Quality, and Innovation
Princeton Plasma Physics Laboratory
PRESENTED BY: ANDREA GOULET
Entropy, Software
MAY 9, 2024
©2024 Andrea Goulet. All Rights Reserved.
About the Presenter
Andrea Goulet is on a mission to embed empathy into the
software industry to help humanity collaborate on the
complex problems we face today. She is a sought-after
international keynote speaker, experienced software
entrepreneur, and award-winning industry leader. Her
expertise centers on applying empathy as a technical skill
to improve organizational effectiveness and modernize
legacy and mission-critical software systems.
Through her online courses, Andrea has taught over
75,000 students how to level up their empathy and
communication skills to create better software. She is the
author of the forthcoming book, Empathy-Driven Software
Development, and the founder of Empathy in Tech and
Legacy Code Rocks, two an online communities where
code and compassion connect.
● LinkedIn: https://www.linkedin.com/in/andreamgoulet
● Email: andrea@andreagoulet.com
● Website: https://www.andreagoulet.com
©2024 Andrea Goulet. All Rights Reserved.
Abstract
In his book, The Nature of the Physical World, Sir Arthur Eddington commented that “We have to
appeal to the one outstanding law — the second law of thermodynamics — to put some sense into the
world.” This sense-making goes beyond the physical world, too. Entropy is also essential in the fields of
information and communication theory.
During this lecture, we’ll discuss the application of entropy-related concepts in two communication
systems: software and collaborative teams. We’ll examine how concepts that help us understand
systemic statistical disorder, such as ergodic systems, Lyapunov exponents, Kolmogorov-Sinai entropy,
and Shannon-entropy can help us optimize for both software quality and innovation. We will also look at
domain-specific models: Lehman’s Laws and Conway’s Law for software, as well as new models from
the presenter’s own research that relate to entropy and innovation.
Entropy helps us understand the world and achieve great things. There is an underlying beauty in its
principles that we can use to advance scientific discovery. When we understand the subtleties related
to balancing surprise and structure, we increase our chances for effective collaboration and finding
novel solutions to complex problems.
©2024 Andrea Goulet. All Rights Reserved.
“We have to appeal to the one
outstanding law — the second
law of thermodynamics — to put
some sense into the world.”
Sir Arthur Eddington
The Nature of the Physical World, 1929
©2024 Andrea Goulet. All Rights Reserved.
©2024 Andrea Goulet. All Rights Reserved.
Entropy is both beautiful…
©2024 Andrea Goulet. All Rights Reserved.
…and frustrating
©2024 Andrea Goulet. All Rights Reserved.
Knowledge
Cascade of Science
Communication
1. Relevance
why does this matter?
2. Accessibility
general understanding
3. Transferability
associative learning
4. Quality Assurance
shared learning with nuance
Jensen, Eric A., and Alexander Gerber. “Evidence-Based Science Communication.” Frontiers in Communication 4 (January 23, 2020). https://doi.org/10.3389/fcomm.2019.00078.
©2024 Andrea Goulet. All Rights Reserved.
“When one meets the concept of
entropy in communication theory,
he has a right to be rather excited—a
right to suspect that one has hold of
something that may turn out to be
basic and important.”
Warren Weaver
The Mathematical Theory of Communication, 1963
©2024 Andrea Goulet. All Rights Reserved.
©2024 Andrea Goulet. All Rights Reserved.
Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication. Illini Books edition, 1963. Urbana: Univ. of Illinois Press, 1949.
SOURCE TRANSMITTER
CHANNEL
RECEIVER DESTINATION
MESSAGE
(encoding)
MESSAGE
(decoding)
What is entropy in a communication system?
“…[logarithmic measure] of various probabilities involved—those of getting
to certain stages in the process of forming messages, and the probabilities
that, when in those stages, certain symbols be chosen next.”
Claude Shannon: Information Theory
NOISE
©2024 Andrea Goulet. All Rights Reserved.
INFERENCE EXPRESSION
INFORMATION CHANNEL
RELATIONSHIP CHANNEL
SENSES INTERPRETATION
ENCODING DECODING
REAPPRAISAL
INTENTION
INFERENCE EXPRESSION
Warren Weaver: Communication Theory
A. Technical - How accurately can the symbols of communication be transmitted? (Information Theory)
B. Semantic - How precisely do the transmitted symbols convey the desired meaning?
C. Effectiveness - How effectively does the received meaning affect conduct in the desired way?
©2024 Andrea Goulet. All Rights Reserved.
Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication. Illini Books edition, 1963. Urbana: Univ. of Illinois Press, 1949.
©2024 Andrea Goulet. All Rights Reserved.
Non-Linear System
Entropy*
• Ergodic Systems
• Lyapunov Exponents
• Kolmogorov-Sinai (metric
entropy)
• Shannon (information entropy)
* The application of these concepts in this presentation are intended as abstractions to aid in conceptual accessibility and transferability. More discourse and
research is likely needed to explore the effectiveness of them as direct mappings.
©2024 Andrea Goulet. All Rights Reserved.
Shannon entropy
(information)
HOW CAN WE PREDICT UNCERTAINTY IN
A MESSAGE SENT BETWEEN POINTS?
● Information = freedom of choice in selecting a
message at a source
● Redundancy = structure of message governed
by system
● Signal = channel capacity
● Coding = signal/information conversion
● Noise = average uncertainty in a message
when signal is known
©2024 Andrea Goulet. All Rights Reserved.
Ergodic Systems
WHAT’S THE LONG-TERM BEHAVIOR
OF A DYNAMIC SYSTEM?
• Observing a single trajectory over
time allows you to infer the statistical
properties of the whole system.
• Statistical behavior of the system is
independent of initial conditions.
Trajectory (collaboration & software quality)
How fast are we identifying problems and
deploying effective solutions?
©2024 Andrea Goulet. All Rights Reserved.
Lyapunov Exponents
HOW FAST DO POINTS MOVE
AWAY FROM EACH OTHER?
Humans = collaboration
Software = developer productivity*
CHAOTIC - fast frustration
STABLE - slow frustration
* There are several new methods to measure developer productivity: DORA, SPACE, and most recently DevEx (cognitive load, feedback loops & flow state)
©2024 Andrea Goulet. All Rights Reserved.
Lyapunov Exponents in Collaboration
● Experience
● Knowledge
● Cognitive Processing
● Values
● Motivation
● Sensitivity
● Power
● Regulation
● Access
● Trust
● Language
● Affect
● Ability
● Energy
● Culture
● Belonging
©2024 Andrea Goulet. All Rights Reserved.
Lyapunov Exponents in Software
● Complexity
● Duplication
● Churn
● Test Coverage
● Dependencies
● Tech Stacks
● Compatibility
● Domain Models
● Customer Knowledge
● Tooling
● Documentation
● Bureaucracy
● Org Goals
● System Access
● Security
● Legibility
©2024 Andrea Goulet. All Rights Reserved.
Kolmogorov-Sinai
entropy (metric)
HOW MUCH DISORDER IS IN A
SYSTEM OVER TIME?
Metric entropy for the system can be
determined by looking at the sum of
positive Lyapunov Exponents.
Chaotic LE → Chaotic system
Stable LE → Stable system
©2024 Andrea Goulet. All Rights Reserved.
Balancing
Structure &
Surprise
©2024 Andrea Goulet. All Rights Reserved.
Hey, Scott.
Got a sec?
©2024 Andrea Goulet. All Rights Reserved.
Finding & Fixing Frustration Points
Andrea Scott
Negotiated
LE Score*
(0 = stable
10 = chaotic)
Sales Software 8
Extrovert Introvert 6
Maker Mender 5
Context Switch Deep Focus 10
Top-Down Bottom-Up 7
Female Male 3
1. Decompose system frustration
into its Lyapunov Exponents
2. Isolate the highest contributing
factor
3. Explore underlying reasons of
each person’s frustration
(empathy)
4. Co-create a protocol to add
redundancy (system defined
message structure)
*These scores are highly local, subjective, and relative, since they are determined collaboratively by the people who are directly involved in the
frustrating situation. Therefore, they may not align with population-level averages. Inductive reasoning determines a pattern from a population, but it
cannot be used to infer that an individual within that population conforms to the average.
©2024 Andrea Goulet. All Rights Reserved.
Co-Created Structure Aids Collaboration
©2024 Andrea Goulet. All Rights Reserved.
Entropy &
Innovation
©2024 Andrea Goulet. All Rights Reserved.
A New Paradigm
of Empathy
1. Collect - Build your personal
capacity, emotional regulation, and
self-awareness
2. Connect - Collect quality data and
attune to another perspective
3. Communicate - Anticipate
miscommunications and optimize
accordingly.
©2024 Andrea Goulet. All Rights Reserved.
Innovation Emerges from
Productive Conflict
● Comprehension - effort is made to clarify
misunderstandings
● Shared Purpose - introduces a boundary condition that
enables us to use entropy principles effectively
● Prosocial Behavior - personal wellbeing and system
goals are regularly balanced
● Self-Organization - individuals feel empowered to adapt
and optimize their local conditions
● Experimentation - new methods can be tested and
learnings are shared
● Co-Creation - people create together what no single
person could create alone.
©2024 Andrea Goulet. All Rights Reserved.
Entropy &
Code Quality
©2024 Andrea Goulet. All Rights Reserved.
Lehman’s Laws Of
Software Evolution
● S-Type: system is static as both the
problem and solution are clearly
defined. (ex: compute area of circle)
● P-Type: system models a real-world
problem. As the world changes, the
system must change to remain valid.
(ex: weather prediction)
● E-Type: system is embedded in the
world it models, making it the most
complex type (ex: air traffic control)
Lehman, M.M., J.F. Ramil, P.D. Wernick, D.E. Perry, and W.M. Turski. “Metrics and Laws of Software Evolution-the Nineties View.” In Proceedings Fourth International Software Metrics
Symposium, 20–32. Albuquerque, NM, USA: IEEE Comput. Soc, 1997. https://doi.org/10.1109/METRIC.1997.637156.
Lehman, M.M., J.F. Ramil, P.D. Wernick, D.E. Perry, and W.M. Turski. “Metrics and Laws of Software Evolution-the Nineties View.” In Proceedings Fourth International Software Metrics
1. Continuing Change - Systems must evolve. If there is not
continuous adaptation, the system will become less satisfactory.
2. Increasing Complexity - Entropy matters. Complexity increases
unless work is done to maintain or reduce it.
3. Self-Regulation - Systems are dynamic. We can use statistically
determinable trends to self-regulate if entropy is stable.
4. Conservation of Organizational Stability - The stability between
businesses and software systems are strongly linked.
5. Conservation of Familiarity - People need to understand the
system they are working on. Confusion increases complexity.
6. Continuing Growth - Systems grow to satisfy the needs of its
users. As functionality increases, so does size and complexity.
7. Declining Quality - Without concerted and rigorous efforts to
maintain a system, its quality will decline over time.
8. Feedback System - Input from a variety of stakeholders is
needed, which requires processes that enable feedback loops.
©2024 Andrea Goulet. All Rights Reserved.
Conway’s Law
Any organization that designs a system will inevitably
produce a design whose structure is a copy of the
organization's communication structure.
Communication structures (ex: protocols) have a direct
impact on software maintainability. When the entropy
balance of communication systems is not maintained, this
will inevitably lead to problems in the code, too.
Conway, Melvin E. “How Do Committees Invent?” Datamation Magazine, April 1968, 28–31. https://www.melconway.com/Home/Committees_Paper.html.
©2024 Andrea Goulet. All Rights Reserved.
Key Takeaways
©2024 Andrea Goulet. All Rights Reserved.
Key Takeaways
• Principles of entropy are essential to both human
collaboration and software quality
• Frame frustration points as Lyapunov Exponents.
Co-create structure through productive conflict.
• Reducing entropy requires effort. You have to
constantly work on the system to minimize disorder.
• Too much cohesion can be just as detrimental as too
much chaos. Balance structure and surprise.
©2024 Andrea Goulet. All Rights Reserved.
Appendix
©2024 Andrea Goulet. All Rights Reserved.
Follow Up to Discussion
After the presentation, a Q&A session took place. During that time, a key question
emerged: What definition of entropy should be used? This appendix provides
additional context to help answer that question in greater detail.
Entropy as a concept started in the field of thermodynamics. Over the course of
150 years, many different types of entropy have been named based on specific
contexts. The following slides provide a historical overview of these terms,
including the date they were introduced and a brief synopsis.
The author used ChatGPT-4o to aid her analysis. You can find more detail about
specific contributors, computations, and citations in her chat, which is available
here: https://chatgpt.com/share/11e3eadb-2a0b-4a29-b5e2-99dcdd4a8faf
©2024 Andrea Goulet. All Rights Reserved.
General Entropy
Entropy is a mathematical concept that quantifies uncertainty, or the amount of statistical
disorder within a system. Key characteristics across most types of entropy include:
1. Logarithmic functions
2. Probability distributions
Entropy plays a key role in understanding:
• Physical states
• Dynamical systems
• Information systems
• Networks
• Social systems
• Software systems, including machine learning
©2024 Andrea Goulet. All Rights Reserved.
Historical Overview
Type of Entropy Year Context
2nd Law of
Thermodynamics
(Clausius-entropy)
1865
Entropy as a term was coined by Rudolf Clausius to describe mathematically
how energy is dispersed in a thermodynamic system, which was theorized in
the Carnot cycle. This kind of entropy is later formalized as the 2nd law of
thermodynamics.
Boltzmann-entropy 1877
Uses system mechanics to mathematically relate microscopic states of a
thermodynamic system to its macroscopic properties.
Gibbs-entropy 1902
Generalizes Boltzmann’s work so entropy is described by a probability
distribution and can be applied to more complex systems.
von Neumann-entropy 1927 Measures entropy within quantum systems.
Shannon-entropy
(Information entropy)
1948
Establishes information theory based on mathematical models of entropy in
the context of communication systems.
©2024 Andrea Goulet. All Rights Reserved.
Historical Overview (cont.)
Type of Entropy Year Context
Kolmogorov-Sinai-entropy
(metric entropy)
1959
Extends entropy to explain the behavior of chaotic systems by measuring
the amount of complexity and unpredictability of dynamical systems.
Rényi-entropy 1961
Introduces a parameter to Shannon-entropy that allows for more
generalized and flexible cases of uncertainty.
Hawking-Bekenstein
entropy
1972
Demonstrates how black holes follow the laws of thermodynamics by
developing a framework that connects quantum mechanics,
thermodynamics, and general relativity.
Social entropy 1956
Applies work from general systems theory to human ecology, providing a
theory for how entropy could be used to determine disorder and complexity
in social systems.
©2024 Andrea Goulet. All Rights Reserved.
Historical Overview (cont.)
Type of Entropy Year Context
Topological entropy 1965
Builds on dynamical systems theory to measure the rate at which distinct
trajectories of the system diverge over time.
Software entropy 1980
Uses entropy to determine practices that reduce the complexity of a
software system so that it is easier to modernize and maintain.
Path entropy 2001
Determines how entropy impacts message routing through a network by
quantifying the diversity of paths.
Degree Distribution entropy 2003
Compares the complexity of different networks, which provides
mechanisms for detecting structural changes in dynamic networks.
Node entropy 2005 Measures the distribution of connections among nodes in a network.
Edge entropy 2005
Determines how the strength or capacity of a connection between nodes
influences the complexity of a system.

More Related Content

Similar to Entropy, Software Quality, and Innovation (presented at Princeton Plasma Physics Laboratory)

Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
mysociety
 
Diff6b eng
Diff6b engDiff6b eng
Higher Ed Exec Forum 2009 Sep
Higher Ed Exec Forum 2009 SepHigher Ed Exec Forum 2009 Sep
Higher Ed Exec Forum 2009 Sep
cdrowell
 
Rsqrd AI - Challenges in Deploying Explainable Machine Learning
Rsqrd AI - Challenges in Deploying Explainable Machine LearningRsqrd AI - Challenges in Deploying Explainable Machine Learning
Rsqrd AI - Challenges in Deploying Explainable Machine Learning
Alessya Visnjic
 
Futures Thinking . Media & entertainment
Futures Thinking . Media & entertainmentFutures Thinking . Media & entertainment
Futures Thinking . Media & entertainment
Jane Vita
 
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
gjhouben
 
Do No Harm: Do Technologists Need a Code of Ethics?
Do No Harm: Do Technologists Need a Code of Ethics?Do No Harm: Do Technologists Need a Code of Ethics?
Do No Harm: Do Technologists Need a Code of Ethics?
Thoughtworks
 
Technology Enabled Business Transformation
Technology Enabled Business TransformationTechnology Enabled Business Transformation
Technology Enabled Business Transformation
Mikkel Brahm
 
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
Ann Davidson
 
Building an Equitable Tech Future - By ThoughtWorks Brisbane
Building an Equitable Tech Future - By ThoughtWorks BrisbaneBuilding an Equitable Tech Future - By ThoughtWorks Brisbane
Building an Equitable Tech Future - By ThoughtWorks Brisbane
Thoughtworks
 
Web 2.0: trendy nonsense?
Web 2.0: trendy nonsense?Web 2.0: trendy nonsense?
Web 2.0: trendy nonsense?
University of Newcastle, NSW.
 
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexityBA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
BA and Beyond
 
2013 leading change for the future doig jansen day shared slides
2013 leading change for the future doig jansen day shared slides2013 leading change for the future doig jansen day shared slides
2013 leading change for the future doig jansen day shared slides
Chris Jansen
 
Testing slides
Testing slidesTesting slides
Testing slides
RuthBeresford
 
AI, Productivity, Innovation, and Sustainability
AI, Productivity, Innovation, and SustainabilityAI, Productivity, Innovation, and Sustainability
AI, Productivity, Innovation, and Sustainability
Robin Teigland
 
Learning Networks: e-Learning 3.0
Learning Networks: e-Learning 3.0Learning Networks: e-Learning 3.0
Learning Networks: e-Learning 3.0
Neil Rubens
 
Sensor Networks and Ambiente Intelligence
Sensor Networks and Ambiente IntelligenceSensor Networks and Ambiente Intelligence
Sensor Networks and Ambiente Intelligence
Rui M. Barreira
 
Communication and technology in the future
Communication and technology in the futureCommunication and technology in the future
Communication and technology in the future
Melissa Herberger
 
Digital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes versionDigital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes version
Marilyn Herie
 
Future forecasting
Future forecastingFuture forecasting
Future forecasting
Kimmo Haapea
 

Similar to Entropy, Software Quality, and Innovation (presented at Princeton Plasma Physics Laboratory) (20)

Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
Making responsible tech the new normal (Sam Brown and Stav Bar-Shany - Doteve...
 
Diff6b eng
Diff6b engDiff6b eng
Diff6b eng
 
Higher Ed Exec Forum 2009 Sep
Higher Ed Exec Forum 2009 SepHigher Ed Exec Forum 2009 Sep
Higher Ed Exec Forum 2009 Sep
 
Rsqrd AI - Challenges in Deploying Explainable Machine Learning
Rsqrd AI - Challenges in Deploying Explainable Machine LearningRsqrd AI - Challenges in Deploying Explainable Machine Learning
Rsqrd AI - Challenges in Deploying Explainable Machine Learning
 
Futures Thinking . Media & entertainment
Futures Thinking . Media & entertainmentFutures Thinking . Media & entertainment
Futures Thinking . Media & entertainment
 
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
UMAP 2013 - Link, Like, Follow, Friend: The Social Element in User Modeling a...
 
Do No Harm: Do Technologists Need a Code of Ethics?
Do No Harm: Do Technologists Need a Code of Ethics?Do No Harm: Do Technologists Need a Code of Ethics?
Do No Harm: Do Technologists Need a Code of Ethics?
 
Technology Enabled Business Transformation
Technology Enabled Business TransformationTechnology Enabled Business Transformation
Technology Enabled Business Transformation
 
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
A Proposed Research Methodology To Conduct Studies About Ubiquitous Learning ...
 
Building an Equitable Tech Future - By ThoughtWorks Brisbane
Building an Equitable Tech Future - By ThoughtWorks BrisbaneBuilding an Equitable Tech Future - By ThoughtWorks Brisbane
Building an Equitable Tech Future - By ThoughtWorks Brisbane
 
Web 2.0: trendy nonsense?
Web 2.0: trendy nonsense?Web 2.0: trendy nonsense?
Web 2.0: trendy nonsense?
 
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexityBA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
BA and Beyond 19 Sponsor spotlight - Namahn - Beating complexity with complexity
 
2013 leading change for the future doig jansen day shared slides
2013 leading change for the future doig jansen day shared slides2013 leading change for the future doig jansen day shared slides
2013 leading change for the future doig jansen day shared slides
 
Testing slides
Testing slidesTesting slides
Testing slides
 
AI, Productivity, Innovation, and Sustainability
AI, Productivity, Innovation, and SustainabilityAI, Productivity, Innovation, and Sustainability
AI, Productivity, Innovation, and Sustainability
 
Learning Networks: e-Learning 3.0
Learning Networks: e-Learning 3.0Learning Networks: e-Learning 3.0
Learning Networks: e-Learning 3.0
 
Sensor Networks and Ambiente Intelligence
Sensor Networks and Ambiente IntelligenceSensor Networks and Ambiente Intelligence
Sensor Networks and Ambiente Intelligence
 
Communication and technology in the future
Communication and technology in the futureCommunication and technology in the future
Communication and technology in the future
 
Digital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes versionDigital Communication Power Tools: Speakers Notes version
Digital Communication Power Tools: Speakers Notes version
 
Future forecasting
Future forecastingFuture forecasting
Future forecasting
 

More from Andrea Goulet

The Marriage of Communication and Code: How Strategists and Technicians Can E...
The Marriage of Communication and Code: How Strategists and Technicians Can E...The Marriage of Communication and Code: How Strategists and Technicians Can E...
The Marriage of Communication and Code: How Strategists and Technicians Can E...
Andrea Goulet
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Andrea Goulet
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Andrea Goulet
 
Should I Estimate That? Complicated VS Complex Projects
Should I Estimate That? Complicated VS Complex ProjectsShould I Estimate That? Complicated VS Complex Projects
Should I Estimate That? Complicated VS Complex Projects
Andrea Goulet
 
Understanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and FoundersUnderstanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and Founders
Andrea Goulet
 
Communication Artifacts: What's Your Code's Legacy?
Communication Artifacts: What's Your Code's Legacy?Communication Artifacts: What's Your Code's Legacy?
Communication Artifacts: What's Your Code's Legacy?
Andrea Goulet
 
Makers and Menders: Putting the Right Developers on the Right Projects
Makers and Menders: Putting the Right Developers on the Right ProjectsMakers and Menders: Putting the Right Developers on the Right Projects
Makers and Menders: Putting the Right Developers on the Right Projects
Andrea Goulet
 
Makers and Menders
Makers and MendersMakers and Menders
Makers and Menders
Andrea Goulet
 
Vulnerability: The Key to Successful Agile Adoption
Vulnerability: The Key to Successful Agile AdoptionVulnerability: The Key to Successful Agile Adoption
Vulnerability: The Key to Successful Agile Adoption
Andrea Goulet
 
Why I Type In Dvorak (And What The Heck That Is)
Why I Type In Dvorak (And What The Heck That Is)Why I Type In Dvorak (And What The Heck That Is)
Why I Type In Dvorak (And What The Heck That Is)
Andrea Goulet
 
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
Andrea Goulet
 
Audience Analysis: Getting the Right Message to the Right People
Audience Analysis: Getting the Right Message to the Right PeopleAudience Analysis: Getting the Right Message to the Right People
Audience Analysis: Getting the Right Message to the Right People
Andrea Goulet
 
Copywriting for Startups and Entrepreneurs
Copywriting for Startups and EntrepreneursCopywriting for Startups and Entrepreneurs
Copywriting for Startups and Entrepreneurs
Andrea Goulet
 
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
Andrea Goulet
 
How to Build a Culture of Content in Your Business
How to Build a Culture of Content in Your BusinessHow to Build a Culture of Content in Your Business
How to Build a Culture of Content in Your Business
Andrea Goulet
 
Content Personalities: Working with Others without Going Insane
Content Personalities: Working with Others without Going InsaneContent Personalities: Working with Others without Going Insane
Content Personalities: Working with Others without Going Insane
Andrea Goulet
 
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
Andrea Goulet
 

More from Andrea Goulet (17)

The Marriage of Communication and Code: How Strategists and Technicians Can E...
The Marriage of Communication and Code: How Strategists and Technicians Can E...The Marriage of Communication and Code: How Strategists and Technicians Can E...
The Marriage of Communication and Code: How Strategists and Technicians Can E...
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
 
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your CareerBecoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
Becoming Technical: Overcome Your Fear of Coding and Supercharge Your Career
 
Should I Estimate That? Complicated VS Complex Projects
Should I Estimate That? Complicated VS Complex ProjectsShould I Estimate That? Complicated VS Complex Projects
Should I Estimate That? Complicated VS Complex Projects
 
Understanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and FoundersUnderstanding Technical Debt: A Primer for Product Owners and Founders
Understanding Technical Debt: A Primer for Product Owners and Founders
 
Communication Artifacts: What's Your Code's Legacy?
Communication Artifacts: What's Your Code's Legacy?Communication Artifacts: What's Your Code's Legacy?
Communication Artifacts: What's Your Code's Legacy?
 
Makers and Menders: Putting the Right Developers on the Right Projects
Makers and Menders: Putting the Right Developers on the Right ProjectsMakers and Menders: Putting the Right Developers on the Right Projects
Makers and Menders: Putting the Right Developers on the Right Projects
 
Makers and Menders
Makers and MendersMakers and Menders
Makers and Menders
 
Vulnerability: The Key to Successful Agile Adoption
Vulnerability: The Key to Successful Agile AdoptionVulnerability: The Key to Successful Agile Adoption
Vulnerability: The Key to Successful Agile Adoption
 
Why I Type In Dvorak (And What The Heck That Is)
Why I Type In Dvorak (And What The Heck That Is)Why I Type In Dvorak (And What The Heck That Is)
Why I Type In Dvorak (And What The Heck That Is)
 
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
So a Baby Walks Into a Tech Conference: Confronting Maternal Bias in the Soft...
 
Audience Analysis: Getting the Right Message to the Right People
Audience Analysis: Getting the Right Message to the Right PeopleAudience Analysis: Getting the Right Message to the Right People
Audience Analysis: Getting the Right Message to the Right People
 
Copywriting for Startups and Entrepreneurs
Copywriting for Startups and EntrepreneursCopywriting for Startups and Entrepreneurs
Copywriting for Startups and Entrepreneurs
 
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
Culture of Content: Bridging the Gap Between Content Leadership and Creative ...
 
How to Build a Culture of Content in Your Business
How to Build a Culture of Content in Your BusinessHow to Build a Culture of Content in Your Business
How to Build a Culture of Content in Your Business
 
Content Personalities: Working with Others without Going Insane
Content Personalities: Working with Others without Going InsaneContent Personalities: Working with Others without Going Insane
Content Personalities: Working with Others without Going Insane
 
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
How to Source, Organize, and Publish Quality Social Media Content (Without Lo...
 

Recently uploaded

SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
GohKiangHock
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 
Preparing Non - Technical Founders for Engaging a Tech Agency
Preparing Non - Technical Founders for Engaging  a  Tech AgencyPreparing Non - Technical Founders for Engaging  a  Tech Agency
Preparing Non - Technical Founders for Engaging a Tech Agency
ISH Technologies
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Green Software Development
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
TaghreedAltamimi
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
YousufSait3
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Liberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptxLiberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptx
Massimo Artizzu
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
Peter Muessig
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
kalichargn70th171
 

Recently uploaded (20)

SQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure MalaysiaSQL Accounting Software Brochure Malaysia
SQL Accounting Software Brochure Malaysia
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 
Preparing Non - Technical Founders for Engaging a Tech Agency
Preparing Non - Technical Founders for Engaging  a  Tech AgencyPreparing Non - Technical Founders for Engaging  a  Tech Agency
Preparing Non - Technical Founders for Engaging a Tech Agency
 
GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Liberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptxLiberarsi dai framework con i Web Component.pptx
Liberarsi dai framework con i Web Component.pptx
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...
 

Entropy, Software Quality, and Innovation (presented at Princeton Plasma Physics Laboratory)

  • 1. ©2024 Andrea Goulet. All Rights Reserved. Entropy, Software Quality, and Innovation Princeton Plasma Physics Laboratory PRESENTED BY: ANDREA GOULET Entropy, Software MAY 9, 2024
  • 2. ©2024 Andrea Goulet. All Rights Reserved. About the Presenter Andrea Goulet is on a mission to embed empathy into the software industry to help humanity collaborate on the complex problems we face today. She is a sought-after international keynote speaker, experienced software entrepreneur, and award-winning industry leader. Her expertise centers on applying empathy as a technical skill to improve organizational effectiveness and modernize legacy and mission-critical software systems. Through her online courses, Andrea has taught over 75,000 students how to level up their empathy and communication skills to create better software. She is the author of the forthcoming book, Empathy-Driven Software Development, and the founder of Empathy in Tech and Legacy Code Rocks, two an online communities where code and compassion connect. ● LinkedIn: https://www.linkedin.com/in/andreamgoulet ● Email: andrea@andreagoulet.com ● Website: https://www.andreagoulet.com
  • 3. ©2024 Andrea Goulet. All Rights Reserved. Abstract In his book, The Nature of the Physical World, Sir Arthur Eddington commented that “We have to appeal to the one outstanding law — the second law of thermodynamics — to put some sense into the world.” This sense-making goes beyond the physical world, too. Entropy is also essential in the fields of information and communication theory. During this lecture, we’ll discuss the application of entropy-related concepts in two communication systems: software and collaborative teams. We’ll examine how concepts that help us understand systemic statistical disorder, such as ergodic systems, Lyapunov exponents, Kolmogorov-Sinai entropy, and Shannon-entropy can help us optimize for both software quality and innovation. We will also look at domain-specific models: Lehman’s Laws and Conway’s Law for software, as well as new models from the presenter’s own research that relate to entropy and innovation. Entropy helps us understand the world and achieve great things. There is an underlying beauty in its principles that we can use to advance scientific discovery. When we understand the subtleties related to balancing surprise and structure, we increase our chances for effective collaboration and finding novel solutions to complex problems.
  • 4. ©2024 Andrea Goulet. All Rights Reserved. “We have to appeal to the one outstanding law — the second law of thermodynamics — to put some sense into the world.” Sir Arthur Eddington The Nature of the Physical World, 1929
  • 5. ©2024 Andrea Goulet. All Rights Reserved. ©2024 Andrea Goulet. All Rights Reserved. Entropy is both beautiful…
  • 6. ©2024 Andrea Goulet. All Rights Reserved. …and frustrating
  • 7. ©2024 Andrea Goulet. All Rights Reserved. Knowledge Cascade of Science Communication 1. Relevance why does this matter? 2. Accessibility general understanding 3. Transferability associative learning 4. Quality Assurance shared learning with nuance Jensen, Eric A., and Alexander Gerber. “Evidence-Based Science Communication.” Frontiers in Communication 4 (January 23, 2020). https://doi.org/10.3389/fcomm.2019.00078.
  • 8. ©2024 Andrea Goulet. All Rights Reserved. “When one meets the concept of entropy in communication theory, he has a right to be rather excited—a right to suspect that one has hold of something that may turn out to be basic and important.” Warren Weaver The Mathematical Theory of Communication, 1963
  • 9. ©2024 Andrea Goulet. All Rights Reserved. ©2024 Andrea Goulet. All Rights Reserved. Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication. Illini Books edition, 1963. Urbana: Univ. of Illinois Press, 1949. SOURCE TRANSMITTER CHANNEL RECEIVER DESTINATION MESSAGE (encoding) MESSAGE (decoding) What is entropy in a communication system? “…[logarithmic measure] of various probabilities involved—those of getting to certain stages in the process of forming messages, and the probabilities that, when in those stages, certain symbols be chosen next.” Claude Shannon: Information Theory NOISE
  • 10. ©2024 Andrea Goulet. All Rights Reserved. INFERENCE EXPRESSION INFORMATION CHANNEL RELATIONSHIP CHANNEL SENSES INTERPRETATION ENCODING DECODING REAPPRAISAL INTENTION INFERENCE EXPRESSION Warren Weaver: Communication Theory A. Technical - How accurately can the symbols of communication be transmitted? (Information Theory) B. Semantic - How precisely do the transmitted symbols convey the desired meaning? C. Effectiveness - How effectively does the received meaning affect conduct in the desired way? ©2024 Andrea Goulet. All Rights Reserved. Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication. Illini Books edition, 1963. Urbana: Univ. of Illinois Press, 1949.
  • 11. ©2024 Andrea Goulet. All Rights Reserved. Non-Linear System Entropy* • Ergodic Systems • Lyapunov Exponents • Kolmogorov-Sinai (metric entropy) • Shannon (information entropy) * The application of these concepts in this presentation are intended as abstractions to aid in conceptual accessibility and transferability. More discourse and research is likely needed to explore the effectiveness of them as direct mappings.
  • 12. ©2024 Andrea Goulet. All Rights Reserved. Shannon entropy (information) HOW CAN WE PREDICT UNCERTAINTY IN A MESSAGE SENT BETWEEN POINTS? ● Information = freedom of choice in selecting a message at a source ● Redundancy = structure of message governed by system ● Signal = channel capacity ● Coding = signal/information conversion ● Noise = average uncertainty in a message when signal is known
  • 13. ©2024 Andrea Goulet. All Rights Reserved. Ergodic Systems WHAT’S THE LONG-TERM BEHAVIOR OF A DYNAMIC SYSTEM? • Observing a single trajectory over time allows you to infer the statistical properties of the whole system. • Statistical behavior of the system is independent of initial conditions. Trajectory (collaboration & software quality) How fast are we identifying problems and deploying effective solutions?
  • 14. ©2024 Andrea Goulet. All Rights Reserved. Lyapunov Exponents HOW FAST DO POINTS MOVE AWAY FROM EACH OTHER? Humans = collaboration Software = developer productivity* CHAOTIC - fast frustration STABLE - slow frustration * There are several new methods to measure developer productivity: DORA, SPACE, and most recently DevEx (cognitive load, feedback loops & flow state)
  • 15. ©2024 Andrea Goulet. All Rights Reserved. Lyapunov Exponents in Collaboration ● Experience ● Knowledge ● Cognitive Processing ● Values ● Motivation ● Sensitivity ● Power ● Regulation ● Access ● Trust ● Language ● Affect ● Ability ● Energy ● Culture ● Belonging
  • 16. ©2024 Andrea Goulet. All Rights Reserved. Lyapunov Exponents in Software ● Complexity ● Duplication ● Churn ● Test Coverage ● Dependencies ● Tech Stacks ● Compatibility ● Domain Models ● Customer Knowledge ● Tooling ● Documentation ● Bureaucracy ● Org Goals ● System Access ● Security ● Legibility
  • 17. ©2024 Andrea Goulet. All Rights Reserved. Kolmogorov-Sinai entropy (metric) HOW MUCH DISORDER IS IN A SYSTEM OVER TIME? Metric entropy for the system can be determined by looking at the sum of positive Lyapunov Exponents. Chaotic LE → Chaotic system Stable LE → Stable system
  • 18. ©2024 Andrea Goulet. All Rights Reserved. Balancing Structure & Surprise
  • 19. ©2024 Andrea Goulet. All Rights Reserved. Hey, Scott. Got a sec?
  • 20. ©2024 Andrea Goulet. All Rights Reserved. Finding & Fixing Frustration Points Andrea Scott Negotiated LE Score* (0 = stable 10 = chaotic) Sales Software 8 Extrovert Introvert 6 Maker Mender 5 Context Switch Deep Focus 10 Top-Down Bottom-Up 7 Female Male 3 1. Decompose system frustration into its Lyapunov Exponents 2. Isolate the highest contributing factor 3. Explore underlying reasons of each person’s frustration (empathy) 4. Co-create a protocol to add redundancy (system defined message structure) *These scores are highly local, subjective, and relative, since they are determined collaboratively by the people who are directly involved in the frustrating situation. Therefore, they may not align with population-level averages. Inductive reasoning determines a pattern from a population, but it cannot be used to infer that an individual within that population conforms to the average.
  • 21. ©2024 Andrea Goulet. All Rights Reserved. Co-Created Structure Aids Collaboration
  • 22. ©2024 Andrea Goulet. All Rights Reserved. Entropy & Innovation
  • 23. ©2024 Andrea Goulet. All Rights Reserved. A New Paradigm of Empathy 1. Collect - Build your personal capacity, emotional regulation, and self-awareness 2. Connect - Collect quality data and attune to another perspective 3. Communicate - Anticipate miscommunications and optimize accordingly.
  • 24. ©2024 Andrea Goulet. All Rights Reserved. Innovation Emerges from Productive Conflict ● Comprehension - effort is made to clarify misunderstandings ● Shared Purpose - introduces a boundary condition that enables us to use entropy principles effectively ● Prosocial Behavior - personal wellbeing and system goals are regularly balanced ● Self-Organization - individuals feel empowered to adapt and optimize their local conditions ● Experimentation - new methods can be tested and learnings are shared ● Co-Creation - people create together what no single person could create alone.
  • 25. ©2024 Andrea Goulet. All Rights Reserved. Entropy & Code Quality
  • 26. ©2024 Andrea Goulet. All Rights Reserved. Lehman’s Laws Of Software Evolution ● S-Type: system is static as both the problem and solution are clearly defined. (ex: compute area of circle) ● P-Type: system models a real-world problem. As the world changes, the system must change to remain valid. (ex: weather prediction) ● E-Type: system is embedded in the world it models, making it the most complex type (ex: air traffic control) Lehman, M.M., J.F. Ramil, P.D. Wernick, D.E. Perry, and W.M. Turski. “Metrics and Laws of Software Evolution-the Nineties View.” In Proceedings Fourth International Software Metrics Symposium, 20–32. Albuquerque, NM, USA: IEEE Comput. Soc, 1997. https://doi.org/10.1109/METRIC.1997.637156. Lehman, M.M., J.F. Ramil, P.D. Wernick, D.E. Perry, and W.M. Turski. “Metrics and Laws of Software Evolution-the Nineties View.” In Proceedings Fourth International Software Metrics 1. Continuing Change - Systems must evolve. If there is not continuous adaptation, the system will become less satisfactory. 2. Increasing Complexity - Entropy matters. Complexity increases unless work is done to maintain or reduce it. 3. Self-Regulation - Systems are dynamic. We can use statistically determinable trends to self-regulate if entropy is stable. 4. Conservation of Organizational Stability - The stability between businesses and software systems are strongly linked. 5. Conservation of Familiarity - People need to understand the system they are working on. Confusion increases complexity. 6. Continuing Growth - Systems grow to satisfy the needs of its users. As functionality increases, so does size and complexity. 7. Declining Quality - Without concerted and rigorous efforts to maintain a system, its quality will decline over time. 8. Feedback System - Input from a variety of stakeholders is needed, which requires processes that enable feedback loops.
  • 27. ©2024 Andrea Goulet. All Rights Reserved. Conway’s Law Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure. Communication structures (ex: protocols) have a direct impact on software maintainability. When the entropy balance of communication systems is not maintained, this will inevitably lead to problems in the code, too. Conway, Melvin E. “How Do Committees Invent?” Datamation Magazine, April 1968, 28–31. https://www.melconway.com/Home/Committees_Paper.html.
  • 28. ©2024 Andrea Goulet. All Rights Reserved. Key Takeaways
  • 29. ©2024 Andrea Goulet. All Rights Reserved. Key Takeaways • Principles of entropy are essential to both human collaboration and software quality • Frame frustration points as Lyapunov Exponents. Co-create structure through productive conflict. • Reducing entropy requires effort. You have to constantly work on the system to minimize disorder. • Too much cohesion can be just as detrimental as too much chaos. Balance structure and surprise.
  • 30. ©2024 Andrea Goulet. All Rights Reserved. Appendix
  • 31. ©2024 Andrea Goulet. All Rights Reserved. Follow Up to Discussion After the presentation, a Q&A session took place. During that time, a key question emerged: What definition of entropy should be used? This appendix provides additional context to help answer that question in greater detail. Entropy as a concept started in the field of thermodynamics. Over the course of 150 years, many different types of entropy have been named based on specific contexts. The following slides provide a historical overview of these terms, including the date they were introduced and a brief synopsis. The author used ChatGPT-4o to aid her analysis. You can find more detail about specific contributors, computations, and citations in her chat, which is available here: https://chatgpt.com/share/11e3eadb-2a0b-4a29-b5e2-99dcdd4a8faf
  • 32. ©2024 Andrea Goulet. All Rights Reserved. General Entropy Entropy is a mathematical concept that quantifies uncertainty, or the amount of statistical disorder within a system. Key characteristics across most types of entropy include: 1. Logarithmic functions 2. Probability distributions Entropy plays a key role in understanding: • Physical states • Dynamical systems • Information systems • Networks • Social systems • Software systems, including machine learning
  • 33. ©2024 Andrea Goulet. All Rights Reserved. Historical Overview Type of Entropy Year Context 2nd Law of Thermodynamics (Clausius-entropy) 1865 Entropy as a term was coined by Rudolf Clausius to describe mathematically how energy is dispersed in a thermodynamic system, which was theorized in the Carnot cycle. This kind of entropy is later formalized as the 2nd law of thermodynamics. Boltzmann-entropy 1877 Uses system mechanics to mathematically relate microscopic states of a thermodynamic system to its macroscopic properties. Gibbs-entropy 1902 Generalizes Boltzmann’s work so entropy is described by a probability distribution and can be applied to more complex systems. von Neumann-entropy 1927 Measures entropy within quantum systems. Shannon-entropy (Information entropy) 1948 Establishes information theory based on mathematical models of entropy in the context of communication systems.
  • 34. ©2024 Andrea Goulet. All Rights Reserved. Historical Overview (cont.) Type of Entropy Year Context Kolmogorov-Sinai-entropy (metric entropy) 1959 Extends entropy to explain the behavior of chaotic systems by measuring the amount of complexity and unpredictability of dynamical systems. Rényi-entropy 1961 Introduces a parameter to Shannon-entropy that allows for more generalized and flexible cases of uncertainty. Hawking-Bekenstein entropy 1972 Demonstrates how black holes follow the laws of thermodynamics by developing a framework that connects quantum mechanics, thermodynamics, and general relativity. Social entropy 1956 Applies work from general systems theory to human ecology, providing a theory for how entropy could be used to determine disorder and complexity in social systems.
  • 35. ©2024 Andrea Goulet. All Rights Reserved. Historical Overview (cont.) Type of Entropy Year Context Topological entropy 1965 Builds on dynamical systems theory to measure the rate at which distinct trajectories of the system diverge over time. Software entropy 1980 Uses entropy to determine practices that reduce the complexity of a software system so that it is easier to modernize and maintain. Path entropy 2001 Determines how entropy impacts message routing through a network by quantifying the diversity of paths. Degree Distribution entropy 2003 Compares the complexity of different networks, which provides mechanisms for detecting structural changes in dynamic networks. Node entropy 2005 Measures the distribution of connections among nodes in a network. Edge entropy 2005 Determines how the strength or capacity of a connection between nodes influences the complexity of a system.