Sand Piles
                 and
              Software

Zach Dennis
Part I
Bak-Tang-Wiesenfeld




Sand Pile Model
macro
 from the
    micro
What eventually
happens if you keep
 dropping grains of
       sand?
It
     la
       nd
        sl
       id
          e
              s.
Systems can only sustain
    so much stress.
self-organized criticality
 A property of a system that has a critical
      state (point) as an attractor.
Complexity




                                  critical point




                 Functionality

Software is attracted to its critical point.
software is not linear




       X
Complexity




                                  critical point




                 Functionality

Software is attracted to its critical point.
software is non-linear
Software feeds back into itself
           Sx = ... (Sx - 1)
0.2 compared to 0.2000000001
Complexity




                             critical point




             Functionality
What we don’t want.
Balanced and Distributed
Sand Piles and Sand Boxes
Complexity




                                       critical point




             The Vicious Stop n Go.
                       Functionality

               Vicious Stop / Go Cycle
Complexity




                                         critical point




             The Vicious Stop n Go.
                       Functionality

                More effort initially.
small app                large app
Complexity




                                          critical point




             The Vicious Stop n Go.
                          Functionality

             Smaller apps get away with it.
Performance =   Complexity(Process)   * Team * Tools
bad processes amplify complexity

   good processes dampen it
How do we
keep software
away from its
critical point?
Part II
Optimized
decision making
   pathways
def solve_problem(problem, knowledge)
 if quick_fix(problem).possible?
   quick_fix.go!
 else
   solve_problem problem, learn(problem)
 end
end
Problem                               Good
                  None       Little                  A lot
Understanding                          Amount




                Random                Intentional   Nailed it
  Solution                 Quick fix
                Attempts
If you do ______________ today,
you’ll likely do ___________ tomorrow.


If you don’t do ______________ today,
you’ll likely won’t do ___________ tomorrow.
Our brains don’t compute probabilities
and then choose the best possible outcome
           when it can avoid it.
Dependencies

                                                  Cheeps
                                                   Data
                                                 Structure



File System



                                                      Person   Subscription
                   DailyCheeper
                     #cheep!




Mobile Push
 Library      - know about where and how to load cheeps
              data
              - know about how person is related to - know
              about subscriptions
              - know about time zone conversion
              - know about about to use mobile push library
              - localizations???
Dependencies after balancing responsibilities




                                                 Person                   Subscription


                       DailyCheeper
                         #cheep!                - knows how to determine eligibility based
                                                on subscription



               - knows how to use cheeps data
 Mobile Push   - knows how to use Person         Cheeps                    File System
  Library      interface to find eligible
               subscribers

               - knows how to use mobile push     - message_for_day
               library                            - localization for accessing cheeps
Let’s improve our decision pathways
Domain complexity

Conceptual complexity

Code complexity
Properties and concepts
for practical application

Cohesion             Exposing Crisp Abstractions /         Responsibility /
Coupling               Concepts                            Dependencies /
Connassence          Testability                           Collaborators /
Entanglement                                                 Boundaries


        Naming, Concepts                     Composition
        Shared Understanding                 Inheritance
        Pattern Language                     Mixins


                               Reflection
Cohesion
 the focus of an individual software component
      given its function or purpose to exist


 Array#push    Cheeper#delivery_daily    Cheeps#load
 Array#pop     Cheeper#delivery_weekly   Cheeps#add
 Array#[]      Cheeper#deliver_monthly   Cheeps#save
 Array#count                             Cheeps#remove
 Array#<<
Not very cohesive
Separating out concepts
Coupling
 The relationship between software components, the
extent to which a component of our software depends
                on other components.
                               Cheeps
                                Data
                              Structure
File System



                                          Person   Subscription
               DailyCheeper
                 #cheep!




Mobile Push
 Library
Connassence
 a single measure combining coupling and cohesion,
measured in terms of strength or weakness (preferred)
 weaker      Name
              Type
            Meaning
            Position           http://en.wikipedia.org/wiki/
                            Connascent_software_components
            Algorithm
            Execution           http://vimeo.com/10837903
              Timing
                        http://onestepback.org/articles/connascence/
             Identity                    index.html
 stronger
Being explicit
  Making concepts and abstractions in your application
    (technical or domain related) first-class citizens.
Extracting a Concept
w/a distinct responsibility
Explicit abstractions




https://github.com/raganwald/homoiconic/blob/master/
              2009-10-08/metalinguistic.md
Rake
       ActiveModel::Validations
       ActiveRecord::Relation
ActionController filters (orthogonality)
           Whenever (gem)
              Capybara
Directed Acyclic Graph




http://en.wikipedia.org/wiki/Directed_acyclic_graph
Testability
Testable components are typically simpler components.
Testability
       - simpler objects

       - simpler interfaces

       - lower number of unnecessary dependencies

       - push external system interaction to the edge,
       isolate when possible

       - remove unnecessary conditionals when
       possible

       - promotes simpler code and more distributed,
       balanced set of responsibilities

       - stub less, mock less, spy only when necessary
Testability
Responsibility / Dependencies /
Collaborators / Boundaries

   - Write a statement describing the responsibility a class or
   module. If there are a bunch of ANDs consider it
   suspicious.

   - Identify “what” your object needs to fulfill that
   responsibility

   - Identify your collaborators to meet those needs

   - Identify boundaries for primary dependencies and
   everything else
Good statement:
#
# The DailyCheeper is responsible for sending mobile notifications via SMS
# to people have subscribed to them.
#




Suspicious statement:
#
# The DailyCheeper is responsible for sending mobile notifications via SMS
# to people have subscribed to them, building reports around cheeps statistics,
# subscribing users to cheeps, and updating/saving cheeps.
#
Dependencies list:
     - find people who are eligible and have subscribed to
     receiving cheeps for given time

     - take into account timezones

     - find out if locales/translations are necessary

     - get/retrieve access to today’s cheep to send

     - mobile library for sending out cheeps
       - access to device_id (can get off from person)
Identify collaborators                                       Load
                                                         Cheeps from
                                                          FileSystem

                                                Cheeps




         Mobile Push             DailyCheeper
          Library




                                                Person




                                                         Subscription
       Primary collaborators
       Secondary collaborators
Identify boundaries                                    Load
                                                   Cheeps from
                                                    FileSystem

                                          Cheeps




        Mobile Push        DailyCheeper
         Library




                                          Person




                                                   Subscription

      Unit testing scope
Identify boundaries                                    Load
                                                   Cheeps from
                                                    FileSystem

                                          Cheeps




        Mobile Push        DailyCheeper
         Library




                                          Person




                                                   Subscription

      Unit testing scope
Naming, Concepts

- Semantic                      CustomersController#create

- Appropriate for level of
abstraction                          Customer#create

- Classes, modules, methods,    ConnectionPool#connection
variables

- Avoid redundancy, ambiguity    MySQLAdapter#insert_sql
Shared Understanding /
Pattern Languages



- Make conventions and patterns an explicit part of your
vocabulary

- Make them up, or re-use useful patterns, etc you’ve found

- Evaluate, share, teach, learn
A few we use:
 - Ruby module pattern for inheritance (for chaining behavior in
 a loosely coupled way, ie: ActiveRecord uses this pattern a lot when for
 #save)

 - Constructor injection (for providing defaults but allowing object to
 be more easily testable)

 - Boundaries (for creating a clear distinction between one part of the
 application and another, ie: Api, Kernel, and Border)

 - Splatting/flattening conditionals (to indicate a conditional,
 branch of code, should not exist, investigate ways to remove it)

 - Everything’s private until it’s public

 - No class variables (they should be banished)
Ruby module pattern example
Boundaries example (API/Library)



 Intuit::Api       Intuit::Kernel          Intuit::Border
 Customer           Customer                 Customer
 Employee           Employee                 Employee
 Vendor             Vendor                   Vendor
 Configuration       QueryDSL                 Connection
 ...                ...                      QBQL
                                             ...

                   Mapping, additional
  Public API for                           Barebones Intuit Definitions
                   functionality, Intuit
   Consumers
                          fixes
Boundaries example (Web App, generic)


  Application                Domain
   Delivery




                                              ORM
                  Domain
                  Services

    Application
     Services
                                      Infrastructure
Request comes in to find an Order


  Application                Domain
   Delivery




                                              ORM
                  Domain
                  Services

    Application
     Services
                                      Infrastructure
Request comes into to process batch of
orders

  Application                 Domain
   Delivery




                                               ORM
                  Domain
                  Services

    Application
     Services
                                       Infrastructure
Request comes into to process batch of orders,
then notifies customer who placed order.

  Application                 Domain
   Delivery




                                               ORM
                  Domain
                  Services

    Application
     Services
                                       Infrastructure
Classes, Mixins, Inheritance, and
Composition


 - Classes are great for representing entities and things which
 have instances (Well-Grounded Rubyist)

 - Mixins are great for characteristics or properties of entities,
 cross-cutting concerns, explicit grouping of functionality

 - Composition is a often under-used. You create a new class/
 object and pass in an existing one as a collaborator/dependency.
Composition



- full control over your object and API

- reduces the ripple effects of change if the API of the
object you’re composing changes

- removes dependencies on parent objects you may or
may not own

- simplifies testing because you own it and its
dependencies
Reflection


 - Give yourself time to reflect

 - Personal growth

 - Make mid-course corrections

 - Keep complexity down

 - if you’re only focused on add, add, add, then you don’t give
 yourself time to do that
Values over
P r a c t i c e s
If you actively seek
ways to exploit your
values, practices will
    come naturally.
<insert here>


good      bad
Practices change more often
than values.
Complexity




                                         critical point




             The Vicious Stop n Go.
                       Functionality

                More effort initially.
May the landscape of your
software be smooth rolling hills.

 twitter: @zachdennis
 github: zdennis
 mutuallyhuman.com
                        Sand Piles & Software
                        Article
                        in April PragPub

Sand Piles and Software - Madison Ruby Conference

  • 1.
    Sand Piles and Software Zach Dennis
  • 3.
  • 4.
  • 5.
  • 7.
    What eventually happens ifyou keep dropping grains of sand?
  • 8.
    It la nd sl id e s.
  • 9.
    Systems can onlysustain so much stress.
  • 10.
    self-organized criticality Aproperty of a system that has a critical state (point) as an attractor.
  • 11.
    Complexity critical point Functionality Software is attracted to its critical point.
  • 12.
  • 13.
    Complexity critical point Functionality Software is attracted to its critical point.
  • 14.
  • 15.
    Software feeds backinto itself Sx = ... (Sx - 1)
  • 16.
    0.2 compared to0.2000000001
  • 17.
    Complexity critical point Functionality
  • 18.
  • 19.
  • 20.
    Sand Piles andSand Boxes
  • 21.
    Complexity critical point The Vicious Stop n Go. Functionality Vicious Stop / Go Cycle
  • 22.
    Complexity critical point The Vicious Stop n Go. Functionality More effort initially.
  • 23.
    small app large app Complexity critical point The Vicious Stop n Go. Functionality Smaller apps get away with it.
  • 24.
    Performance = Complexity(Process) * Team * Tools
  • 25.
    bad processes amplifycomplexity good processes dampen it
  • 26.
    How do we keepsoftware away from its critical point?
  • 27.
  • 28.
  • 29.
    def solve_problem(problem, knowledge) if quick_fix(problem).possible? quick_fix.go! else solve_problem problem, learn(problem) end end
  • 30.
    Problem Good None Little A lot Understanding Amount Random Intentional Nailed it Solution Quick fix Attempts
  • 31.
    If you do______________ today, you’ll likely do ___________ tomorrow. If you don’t do ______________ today, you’ll likely won’t do ___________ tomorrow.
  • 32.
    Our brains don’tcompute probabilities and then choose the best possible outcome when it can avoid it.
  • 36.
    Dependencies Cheeps Data Structure File System Person Subscription DailyCheeper #cheep! Mobile Push Library - know about where and how to load cheeps data - know about how person is related to - know about subscriptions - know about time zone conversion - know about about to use mobile push library - localizations???
  • 38.
    Dependencies after balancingresponsibilities Person Subscription DailyCheeper #cheep! - knows how to determine eligibility based on subscription - knows how to use cheeps data Mobile Push - knows how to use Person Cheeps File System Library interface to find eligible subscribers - knows how to use mobile push - message_for_day library - localization for accessing cheeps
  • 39.
    Let’s improve ourdecision pathways
  • 40.
  • 41.
    Properties and concepts forpractical application Cohesion Exposing Crisp Abstractions / Responsibility / Coupling Concepts Dependencies / Connassence Testability Collaborators / Entanglement Boundaries Naming, Concepts Composition Shared Understanding Inheritance Pattern Language Mixins Reflection
  • 42.
    Cohesion the focusof an individual software component given its function or purpose to exist Array#push Cheeper#delivery_daily Cheeps#load Array#pop Cheeper#delivery_weekly Cheeps#add Array#[] Cheeper#deliver_monthly Cheeps#save Array#count Cheeps#remove Array#<<
  • 43.
  • 44.
  • 45.
    Coupling The relationshipbetween software components, the extent to which a component of our software depends on other components. Cheeps Data Structure File System Person Subscription DailyCheeper #cheep! Mobile Push Library
  • 48.
    Connassence a singlemeasure combining coupling and cohesion, measured in terms of strength or weakness (preferred) weaker Name Type Meaning Position http://en.wikipedia.org/wiki/ Connascent_software_components Algorithm Execution http://vimeo.com/10837903 Timing http://onestepback.org/articles/connascence/ Identity index.html stronger
  • 50.
    Being explicit Making concepts and abstractions in your application (technical or domain related) first-class citizens.
  • 51.
    Extracting a Concept w/adistinct responsibility
  • 53.
  • 54.
    Rake ActiveModel::Validations ActiveRecord::Relation ActionController filters (orthogonality) Whenever (gem) Capybara
  • 55.
  • 56.
    Testability Testable components aretypically simpler components.
  • 57.
    Testability - simpler objects - simpler interfaces - lower number of unnecessary dependencies - push external system interaction to the edge, isolate when possible - remove unnecessary conditionals when possible - promotes simpler code and more distributed, balanced set of responsibilities - stub less, mock less, spy only when necessary
  • 58.
  • 59.
    Responsibility / Dependencies/ Collaborators / Boundaries - Write a statement describing the responsibility a class or module. If there are a bunch of ANDs consider it suspicious. - Identify “what” your object needs to fulfill that responsibility - Identify your collaborators to meet those needs - Identify boundaries for primary dependencies and everything else
  • 60.
    Good statement: # # TheDailyCheeper is responsible for sending mobile notifications via SMS # to people have subscribed to them. # Suspicious statement: # # The DailyCheeper is responsible for sending mobile notifications via SMS # to people have subscribed to them, building reports around cheeps statistics, # subscribing users to cheeps, and updating/saving cheeps. #
  • 61.
    Dependencies list: - find people who are eligible and have subscribed to receiving cheeps for given time - take into account timezones - find out if locales/translations are necessary - get/retrieve access to today’s cheep to send - mobile library for sending out cheeps - access to device_id (can get off from person)
  • 62.
    Identify collaborators Load Cheeps from FileSystem Cheeps Mobile Push DailyCheeper Library Person Subscription Primary collaborators Secondary collaborators
  • 63.
    Identify boundaries Load Cheeps from FileSystem Cheeps Mobile Push DailyCheeper Library Person Subscription Unit testing scope
  • 64.
    Identify boundaries Load Cheeps from FileSystem Cheeps Mobile Push DailyCheeper Library Person Subscription Unit testing scope
  • 65.
    Naming, Concepts - Semantic CustomersController#create - Appropriate for level of abstraction Customer#create - Classes, modules, methods, ConnectionPool#connection variables - Avoid redundancy, ambiguity MySQLAdapter#insert_sql
  • 66.
    Shared Understanding / PatternLanguages - Make conventions and patterns an explicit part of your vocabulary - Make them up, or re-use useful patterns, etc you’ve found - Evaluate, share, teach, learn
  • 67.
    A few weuse: - Ruby module pattern for inheritance (for chaining behavior in a loosely coupled way, ie: ActiveRecord uses this pattern a lot when for #save) - Constructor injection (for providing defaults but allowing object to be more easily testable) - Boundaries (for creating a clear distinction between one part of the application and another, ie: Api, Kernel, and Border) - Splatting/flattening conditionals (to indicate a conditional, branch of code, should not exist, investigate ways to remove it) - Everything’s private until it’s public - No class variables (they should be banished)
  • 68.
  • 69.
    Boundaries example (API/Library) Intuit::Api Intuit::Kernel Intuit::Border Customer Customer Customer Employee Employee Employee Vendor Vendor Vendor Configuration QueryDSL Connection ... ... QBQL ... Mapping, additional Public API for Barebones Intuit Definitions functionality, Intuit Consumers fixes
  • 70.
    Boundaries example (WebApp, generic) Application Domain Delivery ORM Domain Services Application Services Infrastructure
  • 71.
    Request comes into find an Order Application Domain Delivery ORM Domain Services Application Services Infrastructure
  • 72.
    Request comes intoto process batch of orders Application Domain Delivery ORM Domain Services Application Services Infrastructure
  • 73.
    Request comes intoto process batch of orders, then notifies customer who placed order. Application Domain Delivery ORM Domain Services Application Services Infrastructure
  • 74.
    Classes, Mixins, Inheritance,and Composition - Classes are great for representing entities and things which have instances (Well-Grounded Rubyist) - Mixins are great for characteristics or properties of entities, cross-cutting concerns, explicit grouping of functionality - Composition is a often under-used. You create a new class/ object and pass in an existing one as a collaborator/dependency.
  • 75.
    Composition - full controlover your object and API - reduces the ripple effects of change if the API of the object you’re composing changes - removes dependencies on parent objects you may or may not own - simplifies testing because you own it and its dependencies
  • 76.
    Reflection - Giveyourself time to reflect - Personal growth - Make mid-course corrections - Keep complexity down - if you’re only focused on add, add, add, then you don’t give yourself time to do that
  • 77.
    Values over P ra c t i c e s
  • 78.
    If you activelyseek ways to exploit your values, practices will come naturally.
  • 79.
  • 80.
    Practices change moreoften than values.
  • 81.
    Complexity critical point The Vicious Stop n Go. Functionality More effort initially.
  • 82.
    May the landscapeof your software be smooth rolling hills. twitter: @zachdennis github: zdennis mutuallyhuman.com Sand Piles & Software Article in April PragPub