Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ITARC15 Workshop - Architecting a Large Software Project - Lessons Learned


Published on

Improving on a previous version of this session delivered in Lisbon, this deck describes the real experiences in architecting and developing a large software project that took 3 years to go live. It was presented at a 3,5hr ITARC2015 workshop in Stockholm, Sweden.

Published in: Software
  • Be the first to comment

ITARC15 Workshop - Architecting a Large Software Project - Lessons Learned

  1. 1. Architecting a Large Software Project – Lessons Learned João Pedro Martins ITARC 2015 IASA WORLD SUMMIT
  2. 2. Ground Rules and Expectations ∃ Break Interaction expected – share your experiences Software Architecture level (yes, technology is included)
  3. 3. João Pedro “jota” Martins CTO @ |create|it| Software Architect TechEd 2006 – “Iron Architect” Winner BizTalk Server MVP 2006-2011 Co-founder - GASP + APPU Windows Azure Insider
  4. 4. What’s this session about? Architecting a Large Software Project – Lessons Learned
  5. 5. Overview Internal Banking application for corporate credit allocation. 3000+ users. Dev started Jun-2011, R1 installed Jul-2014. Currently on progressive rollout with updates. Web-based, running inside a Windows Forms shell, developed on Microsoft technologies. Scrum - 56 sprints, ~15 dev-years Core team: 14 (5 devs) + ~10 in other teams Integrates with 11 systems THEPROJECT
  6. 6. Current statistics Applicationcode- LOCs Presentation: 63,664 Backend:116,145 Business Rules(xml):36,473 Database:~4,000 Automatedtests:157,586 And also… GeneratedCodeLOCs 81VisualStudioProjects 24Services with213Operations 86Presentation views 1,481classes,96databasetables 1,755AutomatedTests,~80%coverage of backendcode. 100%StyleCopcompliance >15libraries andpackages~800,000 THEPROJECT Storiesand Issues ~550userstories 1,064issues:98%closed,78%bugs.0.04% showstoppers. Inlast5sprints… 56%timenewdevelopments,44% fixes/enhancements Production updatesevery4days
  8. 8. Architecture design: from Theoretical toPragmatictoImplementable Theoretical Implementable ESB/Integration Bus BizTalk Server SOAP Web Services with bank’s custom ServiceHost Enterprise Rules Engine BizTalk Server BRE / Excel Services NxBRE (OSS) – XML Rules Engine Distributed Cache VelocityCTP3 / Windows AppFabric Cookies + Web Server Afinity Load Tests VS2010 Test with distributed agents VS2010 single server load Application Lifecycle Management (ALM) TFS SVN + Issue Tracker + Excel Product/Sprint Backlogs Increasing levels of contraints limit the choices: - Cost, time, available IT, risks, governance, security, skillsets, …
  9. 9. Enough context Now, what we learned…
  10. 10. Secrets forthe success Scrum [Dev] Agility Retrospectives Frequent releases/user demos Notion of Progress User Interface Motivated Team Focus Usable and simple Aesthetic Innovative (not what you would expect in a bank) Team principles - quality Continuous improvement Individual initiative&strengths Redundancy in skills Very low turnover Work-life balance Challenges Latest&Greatest technology Physical workspace Great relationships OVERVIEW
  11. 11. Classeswithtoomanyresponsibilities LEARNING#1 Service Implementation Business Manager Data Access Layer Operation Contracts Data Contracts Business Manager Thesegottoolarge  WCF Physical Hosting (.svc’s)
  12. 12. How toworkaround this? SOLID principles - Single Responsibility Domain Driven Design – vertical slices of behavior Separation of Concerns – partial classes + smaller classes When in doubt: • Create another class • Use interfaces LEARNING#1
  13. 13. Logging Do you have detailed logging enabled in your production environments? LEARNING#2 Is this useful? 
  14. 14. Logging Instead of this (the “Laconic Logging” Anti-Pattern)… … do something like this… LEARNING#2
  15. 15. Logging-hints Log operation details vs Security/Privacy Use end-to-end correlation. Define your logging policy. Beware of impact on performance. Beware of storage space required. DO cleanup/archive. LEARNING#2
  16. 16. Interfaces vs Inheritance “Why extends is evil - Improve your code by replacing concrete base classes with interfaces” (Allen Holub) LEARNING#3  Dependency injection interfaces. Very limited use of OO inheritance - Base Data Contracts with minimum property set (e.g., id + name) - About ~10 uses in total
  17. 17. Dependency Injection Indirection when instantiating objects: - Container builds and reuses objects - Flexibility and reduced coupling - Supports mocking for automated tests Interception: - Cache - Logging - Profiling - Exception Shielding Fully configurable. LEARNING#4  Caller Target Interceptor – Call handler
  18. 18. Dependency Injection "All problems in computer science can be solved by another level of indirection“ (David Wheeler) LEARNING#4 Config If asked for IServiceA  create ServiceA instance creates&uses A
  19. 19. Dependency Injection pitfalls Initial setup can be demanding (skills+time) Programming configurations (complex debugging) Impact on runtime performance Productivity (F12 goes to interface and not implementation) LEARNING#4
  20. 20. Cache Cache transparentlyviainterception + configuration. Cache before accessing the network. Initially designed for 3freshness configs. Idempotence as a side-benefit. LEARNING#5 Presentation Layer  Whenever this assembly is used … and a method with this name is called … apply this interceptor with this configuration
  21. 21. Cache: ooops! Business information presented must be accurate – and data is not stable in external systems. Very little external reference data. User authorization. Transparent, configuration-based cache, is convenient - however, you can’t selectively expire contents  LEARNING#5
  22. 22. Transparent caching pitfals Per operation call, not per business entity E.g.,“method_1_2_3”ascache_key,insteadof“client_1” You don’t control cache keys on the runtime (general purpose cache-key: hash generator of input parameters) Hard Impossible to track dependencies in complex business models. => Cache invalidation is all-or-nothing LEARNING#5
  23. 23. Automated tests Requirement: 80% coverage by automated tests in service layer. Team principle: the AGILE team is not afraid to change any piece of code for fear of breaking something. Approach: service-level, end-to-end tests - Visual Studio Tests framework - Not unit tests  integration, end-to-endtests - Depend on external data - Sprint Backlog: One service operation  one test set LEARNING#6 
  24. 24. Automated tests – mmmm… Test suite takes too long to run (~2-3h) : - SQL scripts  SQL Database Snapshots - Service layer tests  Business layer tests External data not stable – mocks But: how to test complex business cases dependent on external data of which we can’t be sure? LEARNING#6
  25. 25. Automated tests – moremmmm… Have a Test King in the team to nurture and run tests VS2012test runner worse/slower than VS2010 (!) Smart asserts can help improve code-coverage: Tool recommendation: SSMS to generate T-SQL from data LEARNING#6
  26. 26. Code Conventions Agree on coding conventions and stylecop compliance at start of project. Architect/Dev Lead name all the main artifacts: service contracts, database artifacts, etc. Strive for consistency. LEARNING#7 
  27. 27. Code Conventions notes Focus on code legibility: - Comment your code (take special care with algorithms) - Don’t use var for non-anonymous types - Don’t overdo Linq statements Mistakes will happen, and rename refactors will be needed (mixing PTwith EN is frequent). Standardize verbs in services/methods, db naming (ex: List vs GetAll) Do NOT argue tab size. When in doubt, use defaults. LEARNING#7
  28. 28. Negotiation…withyourteam,andwiththecustomer Always voice your opinion, focusing on what you think is the best for the project architecture-wise. Create a trust relationship. When your recommendation is not followed, and you are sure you are right, present objective arguments – don’t be emotional. Argue for as long as you must, but no longer. And don’t say “I told you so”. Accept defeat, make compromises. LEARNING#8 
  29. 29. Negotiation…somemorenotes Be attentive of the other’s possible hidden motivations, but be careful in exposing them. Consensus is not always possible. Your options will be questioned, and sometimes you will be wrong. Remember the 3 views of architecture: theoretical, pragmatic, implementable. Ask open questions. LEARNING#8
  30. 30. Functional team/domain experts They are your peers, and part of the team. You depend on well written and clear user stories. Domain experts that understand Scrum, priorities and constraints make the difference. Rely heavily on them and their tools. When you don’t understand, ask questions until you do. LEARNING#9 
  31. 31. Functional Team do’s and don’ts Sometimes the way a story is written crystalizes a way of implementation. Some stories will be hard to understand and decompose into tasks. Ask for clarifications and don’t implement blindly. Business context is sometimes missing. Tendency to “follow the old ways”. Tendency to abide to single-user/hierarchical requests.
  32. 32. Use Your Brain:design elegantly You are not paid to write code, you are paid to think and communicate. Think things through before committing to a solution. Try to isolate and design autonomous and change- tolerant components. Step back, look at the larger picture. As an Architect, you DO NOT have to be a technical expert in everything: focus on capabilities and structure. LEARNING#10 
  33. 33. Impediments to using your brain Interruptions, background noise, phones, no whiteboard, lack of natural light, music on headphones, time or budget pressure, too much coffee, personal problems, lack of sleep, … What’s your style: collaborate then design, or design and then collaborate? Isolate yourself to design. Make drawings, and then document your proposed solution. *Thinking is hardwork.
  34. 34. Wireframes Create and discuss mockups pre- implementation. Ps: if you have an UI expert in your team, don’t let business experts create them. Tool recommendation: BalsamiqMockups LEARNING#11 
  35. 35. UsabilityTests Usability tests are simple! Just looking at users during training uncovered both problems and ideas for improvement. LEARNING#12 
  36. 36. Revisiting 4 technical choices KnockoutJS or MVC3? Took time to decide and spike, there was an initial setback with KO and adoption was reversed. 2nd attempt and investment proved correct. NxBRE Rules Engine QuickGraph.net3.6 DistributedCache XML-based rules engine DLL. XML file can be replaced without recompilation. Works fine and is fast, but hard to code and read. Jury is still out. Formal Architectural feedback was tacitally dismissed as non- pragmatic, and package was used. VelocityCTP3 was refused as non- supported. AppFabric not available in Windows 2003. Oracle Coherence never provided.
  37. 37. Lightning round
  38. 38. Your team is an extension of your body.
  39. 39. Use diagrams to communicate and structure your ideas. NOT MY SKETCH
  40. 40. Use an issue tracker, designate someone(s) to do the triage, and configure mail alerts, your pages/modules, team, and sprints. Teach the client how to use it for bugs & enhancements.
  41. 41. Know thy user’s pc: @start, 1GB RAM, IE8, WinXp, 1024px Javascript + IE8 + 1GB RAM  recipe for disaster.
  42. 42. 4 layers & no distributed cache mean no real-time features
  43. 43. Be lazy. Don’t waste time coding your own, special, data- access layer/library/.... Scavenge codeplex, github, nugget, etc. for assets&tools to reuse or buy.
  44. 44. Use extension methods – don’t pollute your classes with auxiliary methods (ex: finders in collections) * and kill those «helper» classes, too
  45. 45. Just 3 more…  Humans make mistakes. Scripts don’t. Use scripts and obsessively automate repetitive tasks or installations. Know your branches, merges, shelves, labels, versioning (just use best practices, don’t invent). Innovating and surprising your customer, and the cherry on the cake, makes a world of difference.
  46. 46. Closing message It’s an architect job to address the clients’ needs and deliver quality products. Three stone cutters were asked abouttheir jobs. The first one replied, “I’m paidto cutstones.” The second replied, “I usespecialtechniquesto shapestones inan exceptional way,hereletme showyou.” He proceeded to demonstrate. The third just smiled and said, “I buildcathedrals.” -Ricardo Semler Hope I helped!
  47. 47. João Pedro “jota” Martins (+351) 96 782 5537 ITARC 2015 IASA WORLD SUMMIT Thanks! Questions?Thoughts?
  48. 48. |create|it| Started 2001 @ Lisboa, Portugal Systems Integrator Team of 26 Microsoft Gold Certified Partner Azure BizTalk Office365 SharePoint Umbraco NopCommerce