Your SlideShare is downloading. ×
Software development effort reduction with Co-op
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Software development effort reduction with Co-op

696
views

Published on

This talks explains the motivations for the Co-op technology: what are the challenges it addresses, in particular focusing on reducing accidental complexity, where it comes from, and a general vision …

This talks explains the motivations for the Co-op technology: what are the challenges it addresses, in particular focusing on reducing accidental complexity, where it comes from, and a general vision on how to resolve it. Then we continue to show practical application of Co-op, including experience figures from large-scale application of a previous generation of this technology. Show a little bit about its realization, and conclude with an evaluation of the technology.

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
696
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Venture Lab panel presentation Lodewijk Bergmans
  • RoI factors: 400-4000% 70-1000% 150-700% -500% (c) 2010 steX bv – www.stexbv.com
  • quote: “7 th . Law of Computer programming: Program complexity grows until it exceeds the capability of the programmer who must maintain it.” 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Many composition mechanisms... includes domain-specific ones such as exemplified by many design patterns! Note that these are all *well-motivated* with (specific) examples where the specific technique is superior to other ones... so, why so many??  5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Inheritance vs delegation static vs dynamic, sharing structure vs sharing state Smalltalk inheritance vs Beta inheritance robustness/predictability vs. flexibility/extensibility inheritance & aggregation vs. aspects local reasoning vs. global reasoning & maintainability so what do ‘real engineers’ do?  5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • if you look at the construction of houses, cars or even much smaller devices, a lot of different construction techniques are combined because each incurs trade-off: easy-to-use, fast, cheap, strong, can be undone, .. 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • just a handful! it appears as if only nails & rope are sufficient to build the complex systems that we do (AspectJ is a notable exception) typically these include message invocations, object aggregation and inheritance with specific (always fixed) semantics we need to address this issue  5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Many composition mechanisms... includes domain-specific ones such as exemplified by many design patterns! Note that these are all *well-motivated* with (specific) examples where the specific technique is superior to other ones... so, why so many??  5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Time is ready for designing languages without fixed & built-in composition mechanisms: Hypothesis: Prog. languages of the (some) future will support extensible/tailorable composition mechanisms. 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • old news: even Darwin said it 1,5 century ago. According to Darwin in “the origin of species” In fact it is not an exact quote, but a paraphrase of –probably- Megginson in 1963 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • -- And later, about Fortress, in “a growable language”: “ The result is that we have a fairly complicated language for library writers that enables them to write libraries that present a relatively simple set of interfaces to the application programmer.” and in Guy Steele talks about language design in the January Doctor Dobb's. “ I'm not saying we should throw structured programming out the window. I am saying that the trade-offs have changed and are likely to keep changing.” 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Piumarte & Warth “Open extensible object models” in Hirschfeld, rose (eds) Self-sustainable systems (LNCS) 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • composition mechanisms must be composable within the same application also on same abstractions depending on semantics! (inherent incompatabilities) e.g. combining beta- and smalltalk inheritance in same hierarchy makes no sense. closure: composition mechanisms are first-class citizens -> scalable model 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • show a before and after code view!! wher eit is used show/flash the Observe co-operator code?o rewrite in Magik syntax?? Venture lab panel presentation: background material 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute
  • multiple inheritance (reuses single inheritance) tracing abstraction (with & without PA reuse) 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • not all instances are shown! 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • It is about a suitable abstraction level for representing composition operators; this shows the outline of our abstraction. we explain it by showing the computation model/ execution semantics (in fact we believe that in an ever more dynamic world, it is important to have a computation model, so things can be made dynamic, whenever required) 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • evaluate binding: - evaluate constraints to determine a partial ordering and evaluate the first match(es) binding of context variables between incoming and outgoing event 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • dynamic languages flexible, expressive compositions DSLs (also: polyglot & multi-paradigm programming) right abstractions for right job Model Driven Engineering separate (implementation details) and logic, avoid boilerplate code meta-modelling & meta-programming allow for separation of concerns, modularizing common solutions in a transparent manner conclusions: we believe dynamic languages fit more natural wih a computation model, rather than a transformational approach. choose the appropriate language abstractions for expressing a program where possible design domain-specific abstractions once and reuse we are more used to jumping up and down meta-levels and handling higher-level abstractions today 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • e.g. CLOS: the notion of parents is fixed in the design of the MOP 5/20/10 (c) 2009-2010 stex BV, confidential: please do not distribute Venture lab panel presentation: background material
  • Transcript

    • 1. Software development effort reduction with Co-op Dr.ir. Lodewijk Bergmans [lbergmans@acm.org]
    • 2. About me
      • software composition
        • Objects
        • Aspects
        • ...
      • Industry-as-a-laboratory
        • ASML, Oce, Siemens, ..
      • European Network of Excellence on AOSD
      • Assistant Prof., University of Twente
      • Independent consultant (1994-1997, 2008-)
        • teach & mentor SE
        • software architecture
      • e.g. Philips Medical Systems, Ernst & Young MC, Ordina Panfox
      • Ericsson Mobile Communications (Lund)
      • Independent consultant, steX
      • Research
      • Industry
    • 3. Message of this talk
      • Development and evolution cost are high:
        • for frameworks, applications & customizations
      • Cost can be reduced by:
        • reducing accidental complexity
        • improved standardization
        • easier reuse of design solutions (idioms & patterns)
      • Co-op is an enabling technology
    • 4. Contents
      • Challenges in software development
      • Understanding the problems
      • A solution approach  Co-op
      • Application of the Co-op solution technology
      • Practical applicability: previous experiences
      • Technology benefits
    • 5. Challenges in software development
    • 6. Some major issues in SE
      • managing complexity
        • we build increasingly (inherently) complex systems
        • which carry more and more accidental complexity
      • continuous change
        • change starts on day one..
        • causes large maintenance costs (60%-90%)
    • 7. Experience: complexity vs project risk source: Wallace, Keil & Rai, “Understanding software project risk: a cluster analysis”, Information & Management 42 (2004)
    • 8. Improvement potential source: Walker Royce, “Improving Software Economics”: IBM white paper (2009) costs (per person year) 10-35% 5-10% <5% 200-1000% 25-100% 15-35% 5-25% impact (productivity) 25-50%
    • 9. Technical challenges
      • modularization of software
        •  avoid accidental complexity
        • but how to keep concerns clean and well-separated
          • How to retain the design within the actual implementation
      • architectual integrity
        • Does implementation conform to the architecture?
      • optimal usage of skills
        • advanced solutions may not be effective for all developers...
        • Can we encapsulate design solutions and make them easy to (re-)use ?
    • 10.
      • Where does accidental complexity come from?
      Understanding the problem
    • 11. Key technique in SE:
      • divide and conquer:
        • decompose into modules
        • that are again composed into a system
      • hence modules/abstractions and corresponding composition techniques:
        • are a key element in computer science
        • mark each paradigm shift in the history of sw.dev.
        • continously new proposals appear...
      A composition technique (‘composition operator’) combines the behavior of two or more program elements.
    • 12. Result: subroutine co-routine aggregation inheritance (Smalltalk style) inheritance (C++ style) inheritance (BETA style) multiple inheritance dynamic inheritance associative inheritance delegation predicate dispatch mixin inheritance traits point-cut-advice composition inter-type declarations roles contracts composition filters dependency injection parameterization actors hyperspaces layered object model
    • 13.
      • Why? Each technique involves trade-offs
      • e.g. Smalltalk  C++ inheritance
    • 14. Composition techniques in the physical world
    • 15.
      • very few
      • fixed by language
      • thus often not the best trade-off!
      • Composition techniques in software systems
    • 16. Result: accidental complexity
      • c.f. extra glue, nails & screws:
        • ugly and hard to attach, understand, and modify
      • because the composition in the problem domain cannot be expressed adequately in the implementation:
        • extra ‘glue’ code to implement relations
        • cannot properly separate concerns
          • Or need additional design patterns ->more code & complexity
        •  much harder to maintain!
    • 17. The solution approach
    • 18. programming languages should not fix composition mechanisms, but make them extensible and tailorable.
    • 19. “ it is not the strongest of the species that will survive, or the most intelligent. It is the most adaptable to change.” 1859 Darwin (Megginson)
    • 20. “ We need to put tools for language growth in the hands of the users.” 1998 Steele
    • 21. “ ... lets [the programmers] [..] express concise solutions and free the original language designer from ever having to say &quot;I'm sorry&quot; ” 2008 Piumarte & Warth
    • 22. Co-op (‘ Co mposition -Op erator’) solution
      • language enhancement that:
      • allows defining wide range of compositions
        • including domain-specific ones
        • including solution patterns
      • compositions are composable
      • compositions are first-class citizens
        • so they can be part of the solution domain
        • to support reuse, extension and adaptation (scalability)
    • 23.
      • or: how application developers can easily (re)use architectural strategies and tactics
      Technology Application
    • 24. Usage of Co-Op
      • Compositions are first-class abstractions
        • provided by standard library, or
        • custom developed by third party, or
        • custom developed within organisation
      • Using them is comparable to instantiating and initializing objects..
        • CompOperator1 newBetween: objectA and: objectB;
        • CompOperator2 newBetween: ClassA and: ClassB;
        • CompOperator3 newBetween: objectA and: objectB and: objectC;
        • ...
    • 25. Example of using Co-Op Patterns: Observer Pattern
      • Library provides simple Observe composition:
      • Creates a new observer (action) that is to be invoked whenever the methods show and hide have executed on object window:
      • Observe newSubject: window messages: ( ((List new) add: &quot;show”) add: &quot;hide”) observerAction: [ Console write: &quot;!!! window visibility = “; Console writeln: (window isVisible) ] ;
    • 26. A quick view on the definition of the operator
      • module Observe {
      • var @observerBehavior;
      • init(subject, listOfOperations, observerBehavior) {
      • var callsToSubject;
      • var sendToObserver;
      • var binding;
      • var constraint;
      • @observerBehavior = observerBehavior;
      • callsToSubject = [Selector new:
      • { return [[[event target] isSameObject: subject] and: [listOfOperations contains: [event selector]]]; } ];
      • sendToObserver = [Selector new: { return [OperationRef new: &quot;Observe&quot;, &quot;notify&quot;, &quot;inner&quot;, this]; } ];
      • binding = [Binding new: callsToSubject, sendToObserver, { return [Dictionary new]; }];
      • constraint = [PreConstraint new: defaultCallBinding, binding];
      • [constraint activate];
      • [binding activate];
      • }
      • notify() {
      • [Console writeln: &quot;this=&quot;, this];
      • [@observerBehavior execute];
      • }
      • }
    • 27. Example of using Co-Op Patterns: State Pattern
      • fsm = StatePattern newWithContext: this initState: closedState;
      • fsm addTransitionFrom: closedState action: &quot;openPort&quot; to: listenState;
      • fsm addTransitionFrom: listenState action: &quot;receiveSyn&quot; to: synReceivedState;
      • ...
    • 28. Summary
      • This illustrates how design level solutions
        • typically involving multiple modules
      • can be defined and standardized once,
      • And can be instantiated directly by application developers
        • in the language that they know
        • without needing to know the complexity behind those solutions
        • In other words: they become empowered to use advanced techniques
    • 29. Practical applicability
    • 30. Large-scale industrial case
        • 4 concepts ‘crosscut’ the system (-30% of code)
        • error-prone, hard to maintain, ...
      int get_kng(KNG_struct* KNG_ptr) { const char* func_name = &quot;get_kng&quot;; int result = OK; timing_handle timing_hdl = NULL; TIMING_IN; trace_in(mod_data.tr_handle, func_name); if (result == OK) { /* Retrieve current KNG */ *KNG_ptr = mod_data.KNG; } HE(result, &quot;GET_KNG FAILED&quot;); trace_out(mod_data.tr_handle, func_name, result); TIMING_OUT; return result; } primary functionality error handling
    • 31. Industrial case: approach
      • capture the essence of the 4 concepts
        • separate these into independent abstractions
      KNG_struct get_kng() { return ( mod_data.KNG); }
    • 32. Industrial case: numbers
      • We did a test migration for a sample module
        • on three aspects:
          • code reduction of over 20% for a sample module
      • Company estimated potential benefits (COCOMO):
        • 7-10% effort reduction on software
        • 3% lead time reduction on software
      • Later controlled experiments for the tracing aspect :
        • initial programming effort reduction: 6%
        • (severity of) error reduction: 77%
    • 33. Technology realization
    • 34. Co-op Proof-of-concept
      • Based on well-structured event reflection
      • Implemented on Java
        • with Eclipse integration (including debugger) 
      • And two prototypes in Miranda resp. Haskell
      • Library with sample composition operators:
        • single inheritance (3)
          • BETA inheritance
        • multiple inheritance
        • delegation (2)
        • point-cut – advice (2)
        • traits (cf. mix-ins)
      • tracing – abstraction
      • memoization – abstraction
      • subject – observer
      • state – abstraction
    • 35. IDE integration sample
    • 36. Execution model perspective
    • 37. Composition Model primitives
      • Module
      • Event
        • properties: event name, sender, target, lookup type, event annotation, parameters
      • Module specifications (may) include:
        • Binding:
          • Event Selector
          • Action Selector
          • Context Mapping of events
        • Constraints
          • between bindings
    • 38. A helicopter view
      • The event dispatch process
      filter bindings match bindings apply constraints & evaluate bindings
    • 39. Trends in sw. development
      • Upcoming technologies:
        • dynamic languages
        • DSLs (also: polyglot & multi-paradigm programming)
        • Model-Driven Engineering
        • meta-modelling & meta-programming
      • Common theme: separation of concerns
        • (re-)use of tailored (application specific) abstractions that can be parameterized
        • avoid boilerplate code (generate)
    • 40. About related work
      • LOTS!
      • several frameworks that support variety of composition operators.
        • few that unify aspect and object composition in one mechanism.
      • Several approaches for extensible languages
        • we focus on extending with new composition operators (only)
      • MOPs:
        • our approach is compatible with a MOP approach
        • different from existing MOPs:
          • no assumptions about (fixed) composition operators
          • unify aspect & object composition mechanisms
          • allow composition of composition operators
    • 41. Evaluation
    • 42. Benefits per stakeholder audience benefits application developer - straightforward application of canned solutions - write less code (=> less bugs) software designer - designs are explicit in code, and localized - more robust to change - can choose optimal solutions (instead of a few fixed ones) senior SE/sw. architect - more opportunities to (guarantee) use of standard solutions - more opportunities to keep code and design consistent - better tools to manage complexity Sw. manager/project leader - better consistency (i.e. quality) - managing complexity  more control over project - empower all team members to build better software easier. end-user/ product owner - software that has fewer problems - software that is flexible: easier/cheaper to maintain and evolve
    • 43. Co-op: an agile enabler
      • value code over documentation:
        • design patterns & standards can be expressed in software
        • everything is expressed in same language
      • simplicity:
        • co-op helps to keep the software simple
        • co-op helps to reduce the amount of code
      • Embrace change
        • better designs and code that matches the design enables fluent change
    • 44. Conclusion: Co-op application
      • Managing complexity of large software
        • technology is a good match for agile development
      • Offering better means for reuse and evolution
        • helps with customization and version issues
      • Common solutions can be
        • standardized, reused and/or enforced
        • can lead to substantial code reductions
        •  better software quality
        •  empowering application programmers
    • 45. Thanks for your attention
      • questions?
      • Suggestions?
      assistant prof., SE groep / EEM CS University of Twente, The Netherlands email: lbergmans@acm.org phone: +31-53-4894271 web: trese.cs.utwente.nl/~bergmans software engineering specialist steX bv, The Netherlands email: lodewijk@stexbv.com cell phone: +31-651100838 web: www.stexbv.com Dr.ir. Lodewijk Bergmans