Downloaded 26 times
Uploaded byDeborah Schalm
Is a Monolith Standing in the Way of Your Digital Transformation? Refactor for Better Agility
The document discusses the refactoring of monolithic applications, detailing their pain points and the business drivers that motivate such changes. It emphasizes the advantages of transitioning to a microservices architecture for improved agility, flexibility, and maintainability, while outlining various approaches and IBM's tools to facilitate refactoring. Key points include the importance of governance, regression testing, and leveraging APIs for better integration in modern development environments.
More Related Content
Similar to Is a Monolith Standing in the Way of Your Digital Transformation? Refactor for Better Agility
Is a Monolith Standing in the Way of Your Digital Transformation? Refactor for Better Agility
- 1. © 2017 IBMCorporation© 2017 IBM Corporation Refactoring Monolithic Applications Bill Alexander, Architect - IBM Developer for z Systems Paul Pilotto, Solution Architect ADDI
- 2. © 2017 IBMCorporation 2 Disclaimer IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here
- 3. © 2017 IBMCorporation 3 Discussion Topics Definition of a Monolithic Application Why refactor Approaches to refactoring Detailed walkthrough of scenarios
- 4. © 2017 IBMCorporation 4 Business Drivers Motivating People to Refactor Business Agility – Speed of change is faster with a more modular architecture – micro-services architecture is a popular approach Accelerate Digital Transformation – Provide more flexibility in what capabilities to reuse. Facilitate onboarding of new people – Reduce learning curve by providing a more understandable architecture Enable rewrite (e.g. java, node) – Incremental approaches centered on smaller components radically reduces risk.
- 5. © 2017 IBMCorporation 5 Monolithic Applications Monolithic Applications are self-contained applications in which functionally distinguishable aspects (for example presentation logic, business logic, and data access) are all interwoven components. All code exists in a single tier (all combined) and deploys as a single unit. Business Logic Data Access LogicPresentation LogicData Access Logic
- 6. © 2017 IBMCorporation 6 What are the pain points of Monolithic Applications Lack of Understanding – Makes simple changes cumbersome – Next generation programmers have no insight – Business Logic exponentially grows by copy/paste • Fear of making mistakes duplication of logic • Inability to leverage business logic Lack of Agility – Small code changes require test and package of the entire application – Slow response to API Economy – Difficult integration with new technology (Mobile, Cloud, Continuous DevOps) – Monoliths were designed assuming infrequent deployment
- 7. © 2017 IBMCorporation 7 The Danger of the Monolithic App https://devops.com/danger-monolithic-app/
- 8. © 2017 IBMCorporation 8 Complementary Approaches to Be Used to Refactor API enable transactions and other resources • Enables these resources to be easily reused inside and outside IBM Z • Moves the needle towards a micro services architectures Break big applications into smaller applications, incrementally • Localizes change impact, enabling agile / DevOps styles of development • Provides a more fine-granular opportunity for reuse and API enablement Rewrite business logic into business rules managed in a decision management tool • Enables rules to be updated by SMEs without programming • Facilitates reuse by ensuring business rule is only captured once All of these approaches needs to be complemented by • Governance of the refactoring effort to ensure clear goals are set and managed to • Regression testing to ensure function and performance are not compromised
- 9. © 2017 IBMCorporation 9 API enable transactions and other assets Application Discovery and Delivery Intelligence (ADDI) Discover And Understand Your z/OS Assets Refactor applications to leverage APIs and test as needed Application Delivery Foundation for z Systems (ADFz) Manage and control the API’s IBM z/OS Connect EE Create business API’s for what you discovered IBM API Connect
- 10. © 2017 IBMCorporation 10 Break big applications into smaller applications Application Discovery and Delivery Intelligence (ADDI) Identify code that appears to be somewhat loosely coupled. Leverage refactoring wizard to refactor module into independent program. Unit test the code. Application Delivery Foundation for z Systems (ADFz) IBM z/OS Connect EE If appropriate, create a REST-ful API for the recently refactored application. Application Discovery and Delivery Intelligence (ADDI) Optimize test effort and determine whether acceptable code coverage of change code has been achieved.
- 11. © 2017 IBMCorporation 11 Rewrite business logic into business rules Application Discovery and Delivery Intelligence (ADDI) Capture business information, including business terms. Identify Candidate Business rules and author them as Business Rules (FUTURE) Refactor code to replace the candidate rules with calls to Business rules in a business rules engine like ODM for management of business logic outside the Application logic. Application Delivery Foundation for z Systems (ADFz) Business Rule Engine Invoke and manage business rules Application Discovery and Delivery Intelligence (ADDI) Optimize test effort and determine whether acceptable code coverage of changed code has been achieved.
- 12. © 2017 IBMCorporation 12 Refactor to fit in a DevOps strategy Smaller modules have a more agile impact on DevOps
- 13. How ADDI andADFz help you in this Refactoring Process “The GPS for your Mainframe Developers” 13
- 14. Increase Agility by Simplifyingand Standardizing Work Item Estimation • Use standardized Metrics (such as Maintainability Index or Halstead Complexity) to make more realistic estimations • Callgraphs, Usage Reports and other Reports help planning application modernization and make application portfolio decisions • Dead-Code Reports give indications about possible dead-code to remove 14 Accelerate Transformation
- 15. Navigate easily inside largeand complex programs • Program Flow provides a quick overview about the parts of a specific program • Find code lines accessing specific resources or interacting with other programs 15 Accelerate Transformation
- 16. Navigate easily inside largeand complex programs • Flow-Charts show an in-detail view of the inner workings of programs 16 Accelerate Transformation
- 17. Accelerate Refactoring • Reduceguesswork about side-effects while changing applications • Navigate the code more effectively • Understand unknown code faster • Adopt a more structured approach to changes using ADDI as a guidance 17 Accelerate Transformation
- 18. © 2017 IBMCorporation 18 IBM Application Discovery and Delivery Intelligence V5.0.3 (Available Now) • Graphical artifact interdependency analysis • Web based search • Extensibility improvements • Preprocessors • Custom communication layers • Additional coding rules • Additional language translations • Performance, Scalability and Usability improvements • Several customer RFEs
- 19. © 2017 IBMCorporation 19 A central link in an integrated and flexible DevOps toolchain IBM Developer for z Systems V14.1 (Available Now) Issue Tracker Source Code Repository Delivery Pipeline Insights and Analysis • Refactoring support for monolithic programs • API Enablement • Agility through modularization • Improve Maintainability • Integration with Application Discovery • Integration with SonarQube • Improved Software Analyzer reports • Enhanced SCM integration • Day 1 support for IBM Z compilers & z/OS • Several customer RFEs https://developer.ibm.com/mainframe/2017/09/08/whats-new-idz-v14-1/
- 20. © 2017 IBMCorporation 20 Leverage existing investments in new ways Create new reusable programs from existing logic Invoke the new program from the original source Creating opportunities for new APIs
- 21. © 2017 IBMCorporation 21 IBM z/OS Connect Enterprise Edition Create new business opportunities Map RESTful APIs to existing mainframe services and data Single endpoint for service access
- 22. © 2017 IBMCorporation 22 Improve testability Program functionality can be easily tested in isolation Testing a Monolith Testing individual components
- 23. © 2017 IBMCorporation 23 Test automation for z/OS applications Automating regression tests becomes easy using tools – z/OS Automated Unit Testing Framework (zUnit feature in IDz) – Rational Test Workbench – XaTester (Xact Consulting A/S) – MF-Test (MOST Technologies)
- 24. © 2017 IBMCorporation 24 Optimize Test Case Execution Detect which regression tests cases maps to changed code to reduce test efforts in early iterations by a magnitude or more Optimize test case execution by identifying which test cases may be redundant, so low value tests can be reduced Close test gaps before exiting test phase by showing code coverage of changed and unchanged code. 24 Increase Productivity
- 25. © 2017 IBMCorporation 25 Improve agility Multiple developers can work concurrently Only deploy what has been modified Underlying technologies can change over time Parallel component development Linear development of a monolith
- 26. © 2017 IBMCorporation 26 Reduce program complexity Program logic is easier to understand by humans and tools
- 27. © 2017 IBMCorporation 27 Summary Refactoring monolithic applications into smaller components has many potential benefits, including… – Reuse – APIs – Continuous Testing – Continuous Feedback – Continuous Delivery – Maintainability IBM has tools that can help, including… Application Delivery Foundation for z Systems Application Discovery and Delivery Intelligence z/OS Connect Enterprise Edition
- 28. © 2017 IBMCorporation © 2017 IBM Corporation Thank You
Editor's Notes
- #2 Good morning, I am Paul Pilotto and I am a Solution Architect focusing on ADDI and DevOps on System z. I have 30+ years of experience in the Enterprise Modernization space and had the opportunity to work with tons of mainframe customers from all over the world. I will focus today on what Monolithic Applications are, what their bottlenecks are and how ADDI can help to break them down. And I am Bill Alexander, an IBMer working in the Architecture and Development team for Application Delivery Foundation for z Systems (ADFz). I will discuss some new tooling we recently announced that will help you refactor those monoliths into modular components.
- #4 The agenda of today’s webcast is to define what we understand to be a monolithic application and how they, in a lot of cases, were created. Once we understand the complexity of a monolithic application Bill and I will walk you through the different reasons and scenarios why refactoring becomes a need and we will show you some approaches on how our tooling can help with that.
- #5 What are the different Business Drivers that motivate companies to refactor. First there is business agility, the speed that our marketing and business expects from IT to change faster their solutions and respond to faster to this disruptive business needs. Micro-services, which is in contradiction with monolithic applications, is today the answer to respond faster to these needs because often the core business processes don’t have to change but have to integrate with these new digital business needs By accelerating the digital transformation you can provide more flexible and reusable capabilities to these needs. Another important business driver is to get new peoples, the millennials or Generation Y, up to speed with our core business and give them the opportunity to learn faster what we have by providing them a more understandable architecture. It also enables you to eventually rewrite into modern languages in an incremental approach based on smaller components to reduce the risk and the cost of an eventual rewrite.
- #6 Before client-server or browser based architectures became trend for our day-to-day IT thinking, Monolithic Applications were commonly used and accepted, not only in the Mainframe spectrum but common for all platforms. The need for separating Presentation Logic from the Business Logic and separating Data Access logic from the Business Logic was not there and it was much more easier for a developer, in function of Logical Unit of Work or for ease of deployment, the logic that belonged to one screen was handled in one program. Over the years these monolithic programs got changed over and over again because of new business needs, new business products, acquisitions and mergers, business regulations, year2K and so on. Also often the original developer retired or moved to another job, within the company or left the company and these monolithic applications just grew and grew in size and functionality.
- #7 Successor developers were so afraid of making mistakes or breaking the spaghetti code, that they preferred to copy and paste portions of the logic and made their changes in the copied source and often left the original code inside the program which then became dead code This resulted in difficult to maintain masterpieces to which a successor developer, millennials or outsourcers or even the original developer, was afraid to touch it because it functionally did what it had to do but became a monster to maintain. Another problem came along when the Tester community popped up, they had to write testing scripts for these little monsters and had to understand how the code worked in order to guarantee an acceptable Code Coverage percentage. And lets not even think about creating an automation in testing these code icebergs, it is doable but the test case will be a monolith as well. There is absolutely no agility in monolithic applications and every little code change involves a compile, involves full testing and involves a packaging of the full functioning monolith which all costs lots of CPU consumption, just because the impact of the change is hard to predict. And let’s not forget the pain they cost when new business needs popup, for example to open some of the logic as a public API or integrate the business critical functionality with a new Mobile solution. Or even, when there is no business need, if IT wants to move forward with an hybrid DevOps solution … these monoliths are a left over from when teams deployed infrequently and they remain a pain.
- #8 If you don’t do anything about it
- #9 When we want to move forward into the API economy or we want to integrate mainframe development and deployment within our continues DevOps process, if we want to automate our testing process and make sure that during this testing, code coverage percentages get guaranteed. Above all this we implement agility in the development process because we can parallelize development and deployment and testing we can conclude that REFACTORING is the solution to these monolith programs problems.
- #10 There are several business drivers for refactor, one of them is that we want to move forward into an API economy Using Application Discovery and Delivery Intelligence to understand the monolithic application and use Application Delivery Foundation to refactor and thus make it possible to use z/OS Connect to make a public, or on premise used, API to please our business in order to make our valuable business critical logic available to our partners or other in house projects
- #11 A first scenario in this API journey is to break big applications into smaller ones Like said earlier, ADDI can easily identify the code that is somewhat loosely coupled. By the use of ADFz we provide some refactoring wizards that make these loosely coupled code into an independent callable program that is more easy to unit test and to turn into a RESTfull API by running it through the z/OS connect wizards, passing it the interfaces such as a CICS Commarea for example. Application Delivery Intelligence will optimize the testing efforts by recommending which and how many testcases have to be executed when code is changed.
- #12 Another important business driver is the demand for creating and govern business rules. ADDI will capture business information, including business terms such as “a contract”, “a claim”, ”a financial transaction” etc. In the current roadmap and near future ADDI will identify the candidate business rules. Again ADFz will provide the refactoring wizards to turn those candidates Business Rules into manageable and executable logic through fore example IBM’s ODM
- #13 And last but not least we better refactor our monolithic applications because we want to integrate our valuable legacy applications into our hybrid DevOps process because smaller units are more agile and easier to maintain and manager and thus also easier run through testing automation tools. Code coverage percentages can also be guaranteed and followed up and code review can be made much agile.
- #14 This is the point in the refactoring process where ADDI and ADfz come in: they are basically the “GPS for your mainframe source-code” AD will takes you through your applications without getting lost in the complexity of monolithic applications IDz will help with new features of refactoring And Delivery Intelligence will help in keeping you on track when it comes to agility in testing and making testing recommendations without going in all the testing details. However, you still need to know where you want to go! Tools are not the „self-driving-car“ equivalent!
- #15 In AD we offer different reporting facilities which use industry standard metrics and algorithms. It is proven that these metrics make it more easy to make more realistic estimations to make a change We can produce A McCabe Complexity Report, which presents Cyclomatic complexity in the number of linearly-independent paths through a program module. The Halstead Complexity Report which can be used to compare the complexity between two programs or two applications. The Heuristic complexity Report determines the complexity of an application by assigning values to each type of statement available A Maintainability Index Report is a measurement report that is intended to track maintainability to indicate when it becomes cheaper/or less risky to rewrite code instead of changing it. Callgraphs and all kinds of Usage Reports should make the discovery process easier for a person who has to make refactoring decisions And Dead-code Reports should remove the code that has being copied and paste over time for reducing the fear-factor of a developer who had to touch a working program by making a small change and who opted to copy the original code and made his change in the copied statements
- #16 Once we decided which programs are the typical Monoliths we want to refactor through running the reports, we can then go into discovering the programs individually by looking at them using graphs within the AD Analyze client as demonstrated yesterday by Omer. The Program Flow graph presents the internal structure of a program, including all referenced files, SQL tables, screens, and other programs called. It visualizes the flow of Performs, GOTO’s, Calls, PL1 Procs etc.
- #17 By analyzing the Program Flow Chart we better understand the structure of the program on a statement and decision tree level. This gives the programmer a better understanding of the relationships within the logic of a monolithic program.
- #18 All these features help to accelerate the refactoring process because it reduces the guesswork about the side-effects that might be the result of a change. AD has lots of features to navigate through the monolithic program more effectively and gives the person who analyses a better and faster understanding of the unknown. AD offers a more structured approach for preparing the change because of all the visualization features it has.
- #19 Data captured at runtime, during testing, and static analysis data are combined together to produce an application composition map. This diagram is annotated with visual indicators highlighting programs which are violating the code complexity, maintainability and code coverage thresholds previously defined by your organization. This allows you to rapidly assess and plan changes to the problematic programs and also assess a high-level impact of making the program changes. This graphical analysis allows you to size the changes and the overall maintenance effort, thereby reducing the risk in your DevOps process.
- #20 On Sept 8, 2017 besides the release of ADDI v5.0.3, we also released v14.1 of IBM Developer for z Systems. Besides the Refactoring tools that I will discuss in more detail on the next slide, this new release of IDz also has: - Additional integration features when Application Discovery is also installed, including in context analysis for CICS Transactions, Database tables, and MVS datasets - Support for open DevOps toolchain products such as SonarQube through on-the-fly analysis of COBOL or PLI source code when the SonarLint plugins are added to IDz - Day 1 support for the new COBOL and PLI compilers - And many Requests For Enhancement submitted by our user community If you would like to learn more about all the features in this new release I would recommend reading the blog located at the URL you see at the bottom.
- #21 After gaining a complete understanding of the program logic by using the impact analysis reports and visualization tools in Application Discovery, a developer can use the new Refactoring capabilities for COBOL source included in IBM Developer for z Systems version 14.1. Once the desired logic is selected the available options include: Creating a new program Creating a new copybook Or extracting the logic into a new paragraph By creating a new program from the chosen existing logic , the functionality provided by this logic is rapidly made available for use in new ways. The original monolith will still be able to invoke the new program thus preserving existing functionality, but the new program can also be exposed as a service. It does not matter whether you plan to restrict the use of this new service to internal applications only, or whether you intend to expose this new service as an external API. Creating individual functional services allows organization to quickly adapt to changing business requirements.
- #22 Too often business logic that could provide companies with a competitive advantage if it was reusable, is instead locked inside a monolithic application. Attempting to duplicate this logic on another platform or in a different language, can lead to errors and redundancy. Instead, refactoring the logic into a reusable module can create new business opportunities, as well as continue to support the existing application. IBM z/OS Connect Enterprise Edition provides a single, common way of invoking services for modern Mobile, Web and Cloud applications. The z/OS Connect API Editor can be used to define RESTful APIs, and map the API requests to traditional backend interfaces such as COBOL copybooks. It also allows for the assignment of static values or the removal of unwanted fields in order to simplify the API for the consumer.
- #23 Traditionally monolithic programs contain many different function paths each requiring different input data in order to produce their expected results. This scenario causes lengthy manual testing schedules even for small changes. It also makes the creation of automated test cases difficult if not impossible. Refactoring program logic into individual components, each providing discrete functionality, makes it easy to create automated regression test cases as the input and expected output becomes well known. This often reduces overall testing time and improves the testability and amount of code covered during testing.
- #24 Automated test cases can uncover defects early in the software development life cycle as you make modifications to your programs. The earlier these defects can be found the less costly they are to fix. There are several DevOps tools available to assist with test case creation and automation. Some of these offerings include: zUnit which is a feature built directly into the developers IDE when using IBM Developer for z Systems Rational Test Workbench which not only provides API testing, but also service virtualization XaTester which is a Unit testing focused offering from IBM Business Partner Xact Consulting and MF-Test which is an automated testing offering from IBM Business Partner MOST Technologies Regardless of the tools you choose to use, creating and running automated test cases becomes easier after a monolithic application has been refactored into smaller components. Test automation improves the confidence that software changes can be made without causing regressions. This confidence allows for more frequent software delivery, accelerating the time to value for application changes.
- #25 On the last slide we discussed multiple tools that can assist with defining and running test cases. Another key benefit to refactoring is the ease with which code coverage results can be obtained and examined. ADFz provides a code coverage collector which can be used in conjunction with any of the tools I mentioned previously, or even in cases where you are conducting manual tests. The code coverage results collected during test case execution can then be combined with the cognitive capabilities of Application Delivery Intelligence to produce trend information. ADI has the intelligence to guide the tester in deciding which test cases to execute based on the lines of code that were changed. This visualization clearly identifies gaps in your test cases which can prevent you from leaving the testing phase without properly testing the code changes that were just delivered. The guidance from ADI can also identify potentially redundant test cases allowing the same programs to be tested with close to or equal to a code coverage score as previous test runs without having the need to execute all the test cases. This knowledge can lead to a reduction in overall testing time while keeping the same high quality code that you desire.
- #26 Besides the opportunity to create APIs, or to improve testing, another benefit provided by refactoring is increased agility. Separating a monolith into smaller components allows multiple developers the opportunity to work concurrently. Parallel development can occur for each module that requires changing, rather than linear development where programmers are waiting for the single monolithic program to become available for change. Deployment flexibility can also be achieved as often times only the component program that was modified is required to be deployed. Of course, this depends on the nature of the change, as sometimes the monolithic program which invokes the component program may also need to be deployed, but this will not always be the case. In addition, as technologies change over time these loosely coupled components provide the flexibility for implementation changes without disruption to their consumers. For example, if an application was using a VSAM file previously and the data access logic has now been separated out to it’s own module, it becomes easier to replace the data store in the future if desired.
- #27 Mostly thus far I have been talking about creating component programs from a monolithic program. However, even simply creating copybooks for common code, or extracting logic into paragraphs to improve program flow and readability, can reduce the complexity of a program. Using these Refactor options in IDz can make the code easier to read by humans, and sometimes these techniques can also improve industry standard measurements. Maintainability Index is a calculated measurement based primarily on Cyclomatic Complexity (which was introduced by Thomas J. McCabe, Sr back in 1976) and Halstead Volume (which was introduced by Maurice Howard Halstead in 1977). Application Delivery Intelligence (ADI) tracks trends in various metrics, including the Maintainability Index, Delivered Bugs, Lines of Code, and others. This information allows teams to quickly understand just how complex, or easy to maintain, their application code actually is.
- #28 In Summary, Refactoring is the process of improving the structure of software without changing its external behavior. And refactoring monolithic software into smaller functional components can improve the extensibility of a function by making it reusable either inside or outside of the current application. Refactoring can also assist in adopting DevOps practices like Continuous Testing by making it easier to create automated test cases. These automated tests when combined with code coverage results can provide Continuous Feedback giving developers confidence in the changes they have delivered. Having a high level of code quality and a separation of functional components can also allow development teams to deploy more rapidly. This differs from the more traditional infrequent deployments we have seen in the past with monolithic software. And of course any developer who has taken over support for a pile of spaghetti code agrees that well structured code is much easier to understand, maintain and evolve as both functional and nonfunctional requirements change. Whether your goal is creating APIs, or improving testing, or improving maintainability IBM tools like ADFz, ADDI and z/OS Connect EE can assist.