The document provides an overview of Agile and SOA (Service Oriented Architecture) methodologies. It discusses the limitations of traditional waterfall development that led to the rise of Agile. Key aspects of XP, Scrum and Lean are summarized. SOA is presented as a means to reduce coupling between legacy systems and improve business agility. Challenges of combining Agile and SOA are explored, with recommendations to start small, avoid unnecessary complexity, and maintain a long term vision.

1. Buzzword Deathmatch: Agile vs SOA formerly “ Agile Development with SOA”

2. About me 10 years experience in IT, mainly as a consultant Took part in many large scale projects Government(s) Banking Insurances A foot in the process, the other one in the architecture. My blog: http://ziobrando.blogspot.com

3. Agenda Starting from the dinosaurs… The Agile Landscape The SOA Landscape

4. The Agile Rationale Waterfall software development proved itself inefficient and unsatisfactory Waterfall is “traditionally” associated with high cost, long development time Result unpredictability and low success ratio Difficulties in managing and accommodating change If asked, nobody is doing waterfall anymore (…or so they think)

5. Unified Process The unified process changed the scenario Iterations as a fundamental part of the process Fine grained roles and artefact definition UML as the official representation language A comprehensive definition of all development process activities Unfortunately, it also set the ground for A family of UML modelling tools A lot of documentation templates

6. The Dark side of Unified Process The tools became more and more important, ultimately twisting the process Analysis and design turned into solo activities Paper, paper, paper, and more paper

7. Developers Developers were considered “interchangeable resources” whose only goal was to implement the specification, though iteratively .

8. Roles and responsibilities The fine grained roles framework eventually turned into a slow down factor: People took over only “the right tasks” (implicit waterfall) Slower response Bottlenecks Blame game ...

9. Rebel forces gathered

10. The three flavours of Agility XP made a breakthrough focusing on extreme software development practices Scrum defined a framework in which Agile practices could be applied within an organization Lean software development introduced concepts and principles from the manufacturing industry into software development (defining also a theoretical background)

11. eXtreme Programming User Stories Less formal than Use cases, act like placeholder for a real discussion Frequent small releases Iterations are shorter and targeted to production, Frequent planning and estimation Each iteration is re-planned according to the currently available information No anticipated development No frameworks or layered architecture Test first Test suites run preserving application integrity, and producing better interfaces Customer availability Real feedback is the key to make the right thing No code ownership Continuous integration The whole system is frequently integrated and tested Pair programming Programmers work in pairs, in coding sessions No overtime ...

12. XP Feedback seems to be the driving factor for many of the proposed practices From the code From peers From the customer From the project From the team The team is largely empowered and encouraged to propose original solutions Process itself might be modified or improved

13. Scrum Scrum does not refer strictly to software development, but provides a framework for adaptive project management Unlike UP, only 3 roles are defined in Scrum The Team The Product Owner The Scrum Master

14. Development team landscape

15. The agile team Small units Cross-functional The team is free to take any design decision In Scrum, the team reports only to the product owner A well defined ceremony and set of actions to prevent the team to drift in dangerous directions

16. Dealing with several stakeholders The Product Owner is the ONE and ONLY responsible for the development team.

17. XP vs Scrum XP Frequent iterations of working software Frequent status update and re-planning Focus on the development team internal organization and practices Scrum Frequent iterations of working software Frequent status update and re-planning Focus on the relations between the development team and the organization

18. XP and Scrum The two perspectives are largely complementary: XP does not say much about what happens outside of the development team Scrum lets the team free to organize, and XP might be a sensible choice Scrum has definitely a better marketing

19. Lean software development principles Eliminate waste Everything that is not delivering any value to the user Amplify learning Developing is discovery, Decide as late as possible Avoid irreversible decisions or take them only when you have as much information as needed to decide Deliver as fast as possible Fast development cycles help feedback. Speed is better than quality Empower the team The team is or will become the maximum expert on the matter Build integrity in Software must be useful, used and right for the job See the whole Failing to see the whole picture will lead to local optimizations

20. Waste Instances of waste that can show up in different shapes Unnecessary documentation Anticipated design Over engineering Waiting Communication leaks Defects Handoff Complexity Blame Quality (?) ...

21. The lo-fi approach As a consequence, some tools were deprecated, favouring a non-tech approach: Paper Post-it Whiteboards Information radiators took advantage of physical proximity to allow a more efficient exchange of meaningful information (where possible) Some tools were basically banned (es. MSProject), others entered the show (es. Wikis)

22. The bottom-up revolution Agile sets the ground for interesting proposal to emerge from the team The team learns and focuses on a given problem space eventually turning into the best available expert on the matter

23. Breeding collaboration Isolation happens seldom Many activities are performed in groups, or pairing, or in a clearly visible way Transparency enables trust and a more effective form of collaboration Information exchange happens in both ways

24. The Ideal Agile scenario Not every condition is optimal to turn to agile: to fully benefit of agile’s potential we should (ideally) Be employed by a company in whose top priority is software development. Be located in the same place Have access to customers (whatever this means) Be free to grow as a team and assume responsibilities Free to arrange logistics, hardware, etc.

25. Agenda Setting the Background The Agile Landscape The SOA Landscape

26. SOA Rationale Large organizations were burdened with the weight of Different legacy systems Mergers and acquisitions Necessity to integrate heterogeneous systems Duplicate and redundant systems High (unnecessary) complexity Operation costs Evolution costs Extremely slow reaction times, or ...basically being stuck, or failing to deliver value ...Does all this sound familiar?

27. Pursuing uniformity Standards, frameworks, rules and tight integration failed to deliver the needed business agility, resulting in a heavier burden to take into account, before even designing a possible solution to a given problem.

28. Service Oriented Architecture SOA aims to dramatically reduce coupling between applications in a given landscape: Language coupling  XML Protocol coupling  WS, SOAP Network coupling  ESB Availability coupling  messages Standards and low coupling defined an enterprise-level architecture made up of replaceable components

29. Low coupling Services are meant to talk to each other, with the lowest possible reciprocal knowledge Enterprise Service Bus

30. The ideal goal SOA was a tool to Allow the long awaited necessary cleanup Allow IT to become a driving factor for the business instead of a burden “ It’s a mess” “ I’ll schedule it for 2010” “ We have no time right now” “ We can do that” “ Why not doing that instead?” “ ...We have an idea” Allow enterprises to reduce vendor lock-in recovering control on IT budget.

31. Put in another way SOA is a tool to allow large enterprises to achieve business agility Shorter products release cycles Getting feedback straight from the market Experimenting new ways to make business SOA is not a way to recreate an existing structure with a newer technology

32. The “classical” SOA scenario The agile greenfield Be employed by a company in whose top priority is software development. Be located in the same place Have access to customers (whatever this means) Be free to grow as a team and assume responsibilities Free to arrange logistics, hardware, etc. The “classical” SOA Company’s top priority is generally not software development Consultants, contractors (and sub-contractors) are the rule Development takes often place in multiple locations , remote and offshore teams are possible. Smaller incentives to grow and assume responsibilities Low control over logistics, hardware, etc.

33. Unleashing freedom Loose coupling and language neutral standards offered the possibility to develop applications in the most appropriate technology Enterprise Service Bus

34. ... Maybe we’re still coupled...? Technical coupling was only one of the many coupling factor on the stage: Licence budget Operations & Support Standards, Rules and existing prescriptions HR strategies Architecture Corporate culture

35. A new Jargon UML was not enough for SOA needs A new Jargon eventually emerged as well as new notations, new diagrams, new stencils, new layers ...

36. Agenda Setting the Background The Agile Landscape The SOA Landscape Putting things together

37. Pairing Agile and SOA “ Enterprises experience more success with SOA when they eschew big top-down delivery projects and instead get down in the trenches with an evolutionary approach. Agile processes provide a basic structure for this kind of incremental delivery.” Carey Schwaber, Forrester Research Interestingly, not so many articles tell how Agility benefits from SOA.

38. Agile as a Cooperative Game Software development might be classified as a finite, goal-directed, cooperative game (Cockburn) Carpet wrestling Jazz Tennis Software Development Dolls Competitive Cooperative Organization Survival Career management Finite, Non-goal-directed Finite, goal-directed Infinite

39. Software development as a Game Finite: a project starts and ends (somehow) Goal directed: es. deliver on time Collaborative : we’re playing together within a team … but we’re not only doing that: We’re playing career game Playing family game Etc. etc.

40. A successful team

41. Some key issues The Team Best of breed selection Team goals before individual ones High motivation The goal Clearly defined goal Time-framed experience Measurable outcome

42. A “not so successful” team

43. Key issues The team Best of breed? Individual goals before common goals The Goal Not so clearly defined Random time-frame Non measurable outcome (only history will tell)

44. Limited resources game A Software Development scenario is bounded on several key areas: Budget Time Expertise Talent Manageable Information

45. SOA game? SOA is generally a different type of game played at a different level: SOA’s goal might be of a longer term strategy Mid term results might not be easily observable and measurable by the team. Es. Budget Suppression of an external system

46. The SOA Game A common critic to Agile methods is that they focus on a short-term strategy, in a given project scope. SOA aims to see “the whole picture” focusing on somewhat obscure long-term goals

47. The dominant culture Enterprise culture might be far from agility SOA might have management’s sponsorship but agile might not. Careers, roles and specialities might be put under pressure

48. Project Size SOA scope calls for many development teams at a time.

49. A more realistic scenario

50. Team’s reward Individual bonuses might be counterproductive or out of control Consultants and contractors might not be reached We’ve got to work on something else Increase the team’s knowledge Improve working conditions Bring (honest) positive feedback ...

51. Top – Down reprise Some tools and artifacts re-introduce a top-down philosophy Role based game Tool driven design Specification based development Another vicious effect is that ... Bottom up feedback is discouraged Possible good ideas get lost

52. Starting Agile and SOA? Though “orthogonal”, two revolutions at the same time are probably too much for any team But ...agile performs well where a lot of explorations and uncertainty is involved: Adaptive process spikes For a good Agile team SOA is “Just another Technical Hassle”

53. Read the scenario Different roles within the organization read buzzwords differently: Team might value agility more (and blame SOA if something goes wrong) Architects might value SOA more (and blame agility if something goes wrong) Management couldn’t care less And value only results

54. Set up the scene Don’t raise expectations you cannot fulfil Keep management aware of potential emerging problems Transitions to agile practices (especially Scrum and lean) stress the organization, exposing problems that lived long “under the carpet”. Problems have always being there, but pointing them out might result impolite .

55. Choose a close target first Can’t wait months to prove that choices were good. Look to close targets Achieve them Build on this Confidence Team jellying Reputation

56. Travel light Availability of tools to manage SOA-related complexity does not mean that complexity has to be encouraged The dark side of SOA would not be better than it was before

57. Adhere to standards As a corollary to “ Decide as late as possible ”: Develop on standards whenever possible Generally better documentation More easily replaceable ... Think long term Avoid temptations from vendor-specific features Keep deviation from standards under control Don’t buy anything until proves necessary.

58. Rethink waste Some obvious form of waste “doing the same thing twice” might be preferable to “document what you’ve been doing” Large scale changes priority Company and location boundaries impact Communication cost Handoff cost

59. The cost of communication Information does not propagate with documentation, but by knowing what others are doing. Keep multiple teams in sync Scrum of Scrums Information radiators Proximity End of iteration demo When written words are better, favour Wikis and on-demand documentation.

60. Share the big plan Preventing developers to see the whole picture severely harms the possibility of a bottom-up contribution Only local optimizations are possible 

61. High level planning SOA is a long term transformation process Every step changes the scenario More knowledge Different weight Different opportunities Frequent high-level re-planning is necessary to achieve the goals (not necessarily the original ones) Measure , don’t expect Assess risks , don’t make predictions Don’t initiate anything that’s not needed now

62. Testing a SOA SOA overall design is based on Design by Contract principles Clean interface definition based upon standards “ Official” expected service behaviour Let’s test on the contract!

63. Testing a SOA: challenges Language independent test suite  Mocks and Stubs implementing services  Selection of key areas  Test environment management 

64. Environment definition Staging environment might be hard to afford in certain context Keep continuous integration tools running and testing the app Extend your integration capabilities as far as you can get Find a balance between correctness and manageability Keep the tests updated

65. Put in two words... Don’t let the “old bad habits” strike back Be prepared to compromises, in the short term (it’s a limited resource game) Keep track the price you pay Be ready to change them if opportunities arise Always aim to your long term goals Involve the right sponsors Communicate! Be ready for a rollercoaster ride!

66. References Agile Software Development – Alistair Cockburn (Addison Wesley) Extreme Programming Explained – Kent Beck (Addison Wesley) The Unified Software Development Process – Ivar Jacobson, Grady Booch, James Rumbaugh (Addison Wesley) Agile Modeling – Scott Ambler (Wiley)

67. References Lean Software Development: An Agile Toolkit – Mary Poppendieck, Tom Poppendieck (Addison Wesley) User Stories Applied – Mike Cohn (Addison Wesley) Enterprise SOA: Service-Oriented Architecture Best Practices – Dirk Krafzig, Karl Banke, Dirk Slama (Prentice Hall) Service-Oriented Architecture: A Field Guide to Integrating XML and Web Services – Thomas Erl (Prentice Hall)