QUality Assurance in EAI Projects


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

QUality Assurance in EAI Projects

  1. 1. Quality Assurance in EAI projects WHITE PAPER Author: Pradeep Vengayil This document is intended to describe general understanding of basic testing, overview of enterprise application integration describing various levels of integration, testing challenges in enterprise application integration projects, typical enterprise application integration QA framework and test automation. In the testing challenges section an attempt is made to evaluate the existing process and the suggestion to overcome the challenges in EAI testing with a brief explanation about testing methods. Even though the testing methodology may vary based on specific project or application requirements, a commonly used enterprise application integration testing framework is given in the section- Typical QA framework for EAI testing.
  2. 2. WHITE PAPER Quality Assurance in EAI projects Table of Contents Introduction......................................................................................................................... 3 Software Testing ................................................................................................................. 3 Enterprise Application Integration (EAI) ........................................................................... 3 Testing challenges in EAI projects ..................................................................................... 5 Typical QA framework for EAI testing .............................................................................. 9 Test automation in EAI..................................................................................................... 11 Reference .......................................................................................................................... 12 Conclusion ........................................................................................................................ 12 About the Author .............................................................................................................. 13 About Wipro Technologies............................................................................................... 13 © Wipro Technologies Page 2 of 13
  3. 3. WHITE PAPER Quality Assurance in EAI projects Introduction Testing is one of the critical stages of software development life cycle and is mandatory to improve and evaluate the functionality and performance of complex software systems. Traditionally, the testing of such systems proceeds in a series of successive phases, known as unit, integration and system tests. This step-wise approach on one hand allows testers to master the inherent complexity of the task, by subdividing the test object into smaller, more manageable pieces, and, on the other hand, enforces a disciplined procedure. Enterprise Application Integration is a widely accepted model for integrating various applications in the enterprise. Even though participating applications are system tested well before integration, most of the issues come out from the interfaces. To avoid major reworks or design changes in the enterprise systems it is a good practice to have proper testing methodologies well in advance. Software Testing Basic software testing has two important aspects: 1. Ensure that the system meets all the customer requirements and 2. Ensure that system will not misbehave when an unexpected input is received. Good testers think technically, creatively, critically, and practically: Technical thinking: The ability to model technology and understand causes and effects. This includes things like knowledge of relevant technical facts and the ability to use tools and predict the behavior of the systems. Creative thinking: The ability to generate ideas and see possibilities. You will test only in ways that you can imagine testing. You will look only for problems that you imagine can exist. Critical thinking: The ability to evaluate ideas and make inferences. This includes the ability to detect and eliminate errors from your thinking, to relate product observations to quality criteria, and to build a compelling case for a particular belief or suggested course of actions. Practical thinking: The ability to generate ideas that are practical and put these ideas into practice. Enterprise Application Integration (EAI) EAI is the process of coordinating the operation of various applications across an enterprise. Typically, an enterprise has existing legacy applications and databases and wants to continue to use them while adding or migrating to a new set of applications that makes use of the Internet, e- commerce, extranet, CRM, Billing and other new technologies based on requirements. EAI may involve developing a new total view of an enterprise's business and its applications, seeing how existing applications fit into the new model, and then devising ways to efficiently reuse what already exists while adding new applications and data. .Now a days EAI is commonly used while defining new enterprise wide systems which makes use of multiple systems (for example CRM, Portals, Billing, Data warehousing etc.), those are proven in the market. © Wipro Technologies Page 3 of 13
  4. 4. WHITE PAPER Quality Assurance in EAI projects Different levels of integration that EAI involves are Business Process Integration (BPI), Application Integration and Data Integration: Business Process Integration (BPI): A business process contains sequence of interactions. For example a customer contacts a supplier to place an order. A sales representative takes the order and passes it on to accounting and manufacturing and so on. Each interaction in a process typically involves the exchange of information and may include human-to-system interactions. When integrating business processes, a corporation must define, enable and manage the processes for the exchange of enterprise information among diverse business systems. This allows organizations to streamline operations, reduce costs and improve responsiveness to customer demands. Elements here include process management, process modeling, and workflow, which involve the combination of tasks, procedures, organizations, required input and output information, and tools needed for each step in a business process. Quality assurance or evaluation of the Business Process Integration is done in a broad level rather than individual unit level. Test scenarios have to be developed by considering the end users and business potentials. Most of the testing required for the integrated solution falls on the end-to-end testing phase or end-to-end system integration testing. Application Integration: At this level of integration, the goal is to bring data or a function from one application together with that of another application that together provide near real-time integration. Application Integration is used for, to name a few, B2B integration, implementing customer relationship management (CRM) systems that are integrated with a company's backend applications, web integration, and building Web sites that leverage multiple business systems. Custom integration development may also be necessary, particularly when integrating a legacy application with a newly implemented ERP application. Most of the time communication between various systems fails due to the mismatch on the expected format or value of data. System Integration testing (SIT) plays a vital role in terms of identifying the interface issues. SIT is one of the most challenging tasks in terms of software testing. This is the starting point of handshaking between various systems and involves lot of coordination and follow up activities between various systems. Data Integration: In order for both Application Integration and Business Process Integration to succeed, the integration of data and database systems must be tackled. Prior to integration, data must be identified (where it is located), cataloged, and a metadata model must be built (a master guide for various data stores). Once these three steps are finished, data can then be shared/distributed across database systems. Usually verification of data integration is done implicitly along with the system integration and end to end testing. Standards of Integration: In order to achieve full Data Integration, standard formats for the data must be selected. Standards of Integration are those that promote the sharing and distribution of information and business data – standards that are at the core of Enterprise Application Integration. These include COM+/DCOM, CORBA, EDI, JavaRMI, and XML. Platform Integration: To complete the system integration, the underlying architecture, software and hardware, the separate needs of the heterogeneous network must be integrated. Platform Integration deals with the processes and tools that are required to allow these systems to © Wipro Technologies Page 4 of 13
  5. 5. WHITE PAPER Quality Assurance in EAI projects communicate, both optimally and securely, so data can be passed through different applications without difficulty. For example, figuring out a way for an NT machine to pass information reliably to a UNIX machine is a large task for integrating an entire corporate system. Testing challenges in EAI projects Even though the success of any project is a collective responsibility of all the members of the team, many times the responsibility of the delivery is in the hands of the Quality Assurance (QA) team. QA team should be having the commanding capacity to accept or delay the delivery based on the quality of the deliverables; hence testing is a challenging task in terms of over all project success. EAI QA is one of the most neglected parts even though EAI methodologies got main stream market acceptance. Most often EAI testing is delayed until the end of the project due to several reasons like less time, unavailability of standard testing strategy, insufficient tools and expertise to accomplish the task. However, delaying QA activities might impact severely due to the difficulties involved in testing a complex integration involving many end systems. Identifying bugs in a later stage is much more expensive than identifying the same in an early stage. It is a known fact that stability of an integrated system lays on the stability and reliability of the middleware platform it is built up on. When the complexity increases, it is essential that all the loopholes have to be closed. Following a traditional QA lifecycle like unit tests, integration tests, systems tests, to acceptance tests will not work for EAI projects easily, since EAI QA involves system-to-system integration. Most of the IT testing tool do not work well for testing EAI projects since the EAI architecture is multi protocol message oriented and simulation of applications are required rather than virtual users. Few key EAI testing challenges are as follows: • Integration project architecture is message based and the failure points are generally in the messages, hence test design should focus on the messages and message failure, which require good understanding of system architecture and the interfaces. Testers should be technically capable to understand the system design and architecture. • Since most of the testing tools are focused on the functionality testing and performance testing of applications, developers need to create stubs or elaborate error-prone traces for testing purpose. Some products provides test stub and test harness exclusively for Integration testing using their products. • Integration projects involve many different applications. This poses both organizational and technology challenges. Organizationally, teams need to think not just about their code, but how it affects both input and output components. In most cases application development teams don't have the vision or the technology to validate their systems operation in the overall enterprise. The entire verification responsibility of EAIis in the hands of testing team and hence the team should be well equipped with the thorough understanding of handshaking applications. • An integration project may also include many different technologies, such as XML, MQ, middleware brokers, TCP/IP and others. Integration test teams need tools that work with © Wipro Technologies Page 5 of 13
  6. 6. WHITE PAPER Quality Assurance in EAI projects many technologies so that they can validate the technology handoffs, as well as the business logic. Test team should be well versed with the messaging standers and the validation process • It is almost impossible in a large integration project to orchestrate end-to-end tests with application test tools. The risk involved in this can be mitigated by performing tests early in development phase. • When the integration solutions streamline development and architecture of an existing solution the complexity, paths and permutations still remain and need to be tested. Testing has to be done for all of the complex business logic, which will be now under another layer of integration-ware after the streamlining process • Other major hurdles in the EAI testing could include components developed in different programming languages; Difficulties in Test Case generation and Test result comparison, Base lining, Generating Test Data, Identifying test bed, proper testing framework and testing tools are not readily available. • When it comes to the actual interface testing, availability of integrating applications for testing is always a big concern. This usually requires lot of co-ordination and follow ups • Transaction testing across multiple systems requires complete understanding of the business requirements in order to verify all the transactions across the interfaces • When the communications between various systems are asynchronous, it is difficult to ensure the guaranteed delivery of the message. Special attention has to be given for the verification of handshaking mechanisms to ensure the proper message delivery • Security related testing is very importance when the transaction that flows through the integration layer is critical. All the error scenarios have to be captured very well along with the possible positive scenarios to avoid most of the security related issues • A minor change in the message set could cause huge impact in related transactions. To avoid this EAI require more regression testing and impact analysis when there is a change in the message sets compared to other applications • Verification of failover mechanisms and recovery mechanisms are of more important in case of integration projects. It is a difficult task to manage and control multiple applications for this purpose • Higher dependency on availability of end applications to perform different levels of testing. • Test cases involving timers, events and re-producing such scenarios is complicated. An effective EAI testing requires a good understanding of the entire systems that are involved (going to involve in future) in the message path as well as the integration architecture. Agile methodology with incremental component development, builds and deployments is a good process for EAI development and testing. Enterprise integration should be broken down into business integration domains to reduce the complexity and risk of the undertaking. This © Wipro Technologies Page 6 of 13
  7. 7. WHITE PAPER Quality Assurance in EAI projects incremental approach requires a matching incremental QA strategy, which places an additional strain on the environmental requirements and dependencies on end system availability – the testing environments are required for a long period of time. Also, the frequency of regression tests increase to ensure that new builds do not break the ones further along in the QA process or in production. In a complex software development scenario, traditional scenarios like sequential or iterative model face problems in accepting continuous changes. Agile methodologies are developed for accepting the changes quickly and smoothly. It is a collection of values, principles and practices that incorporate iterative approach for development, test and feedback for further changes. Here software is developed in incremental, rapid cycles, with each release building on previous functionality. Each release is thoroughly tested, which ensures that all issues are addressed in the next iteration. When we follow this approach for EAI project each components of EAI will be available for multiple round of testing in the early stages of the development cycle. Availability of each component for testing in the early stages will ensure the strength and stability of the application by eliminating the hidden issues in a later stage. Performance testing is also very importance to ensure the system stability. The architecture of the EAI solution needs to provide the infrastructure for performance, as well as the selected enabling technology. A simple way of measuring the performance is, run some tests to see if the system performs well under a variety of conditions and measure the following, How well does it perform at 100 users, 500 users, 1000 users, and even 10,000 users? What about processing load? What happens to performance as you extend the EAI solution over time? A well defined process for quality assurance always help to minimize major risks involved in the EAI testing area. Formulating a strategy based on the type of the project is also very important. Following process in a commonly accepted model can be a good solution to overcome the major challenges • Test requirement analysis • Good estimation • Identification of proper QA strategy well in advance • Establish a test plan. • Proper test cases design considering the various interfaces • Based on test cases; generate test data for each test case • Proper testing bed. Test requirement analysis: This is one of the early stages of the project. Along with the implementation requirements special stress has to be given for the testing considerations. This can be a part of the business requirement analysis by identifying the business critical areas. In a broad way, requirement analysis should cover functional requirements, non-functional requirements, interface requirements, features to be tested, features not to be tested, assumptions, dependencies, limitations, acceptance criteria and system hardware requirements. The study of the existing systems (if any) and the existing business requirements has to be considered during this analysis. Overall analysis should give more importance for the interface requirements since EAI will be the core component of integration. An experienced test requirement analyst can contribute more on the architectural design of the integrating system, since he/she must be having the good understanding of the weaker sections of the system. A test analyst will be considering the whole system as a component while a developer will be focused on to an individual component of the system. It is always a good practice to bring test analyst together with the business analyst during © Wipro Technologies Page 7 of 13
  8. 8. WHITE PAPER Quality Assurance in EAI projects the initial analysis. Dedicated testers bring benefits like, focus on customer usage over technical implementation and focus on uncovering flaws over confirming completeness. Test Estimation: An estimate should accurately predict and guide the projects future. It should be realistic by including all the tasks that can be reasonably anticipated. It forecast what, based on the current knowledge, is most likely to happen and it should reveal the risk of the test area, so that mitigation plan can be done properly. Estimate should be actionable, which gives clear ownership of tasks by committed individual contributors and to see assigned resources and known dependencies. One more important aspect of estimate is that, it should be flexible, what if the deadline and resources constraints are immovable? The estimate must accommodate project realities. Work breakdown structure is a good way of estimation. It is a hierarchical decomposition of the project into stages activities and tasks. Stages for the test estimation are Planning, Staffing (if applicable), Test environment acquisition and configuration, Test development and Test execution (including find/fix/retest cycles). Identifying the actual duration of each task require good background knowledge and experience. It is always better to involve the team for a better estimation. Feed back and suggestion from each member has to be considered for finalizing a good estimate. QA Strategy: Defining a proper test strategy is one importance success factor in the project. Strategy has to be defined by considering various factors including the type, schedule and the budget. Test strategy should prove that the system requirements and architecture can be validated by the defined tests. A proper test methodology has to be defined here. It is very essential to prove that the team acquired a thorough understanding of the existing system or the new system before formulating the strategy. Usually a brief explanation of the overall architecture has to be presented in the strategy and each interfacing requirements has to be explained with the main verification points. In terms of individual components and integration of each component to develop the system, strategy should be defined to explain what type of test is most suitable? How many cycles of test have to be carried out? What are the different phases of tests? And how the overall progress of testing happens? Test plan: A test plan is particularly important because of the difficulty in testing an EAI solution. Most of the source and target systems are business critical, and therefore has to be considered here. It should bring more visibility on the test design and execution based on the defined strategy. Test plan should give consideration for the following points. • Describe and justify the test strategy (including proposed test coverage) in relation to Technical requirements and technical risk • Describe and justify any special requirements or entry criteria that must be met in order for the test project to proceed, as well as any exit or process for determining when to stop testing • Support the initiation and organization of the test project, including preparations, staffing, delegation of responsibilities, facility acquisition, task planning, and scheduling. • Support daily management and evaluation of the test project and test strategy • Identify and manage any risks or issues that may impact the project • Specify the deliverables of the test project, and the delivery process. © Wipro Technologies Page 8 of 13
  9. 9. WHITE PAPER Quality Assurance in EAI projects For designing the test case with respect to the individual component, knowledge of message format and interface contracts are essential. Test case design will vary based on the type of test. A unit level test case should have a complete mapping with the different message formats that the component is going to handle. When the test phase progress from unit to system and then to integration, the requirement coverage should be taken care with proper mapping. Testing Steps can be described by the following flow diagram. Testing Steps Test Case Identification Test Case Definition Test Case Documentation Test data setup / Test code Test Execution Test Result Test Coverage Analysis Analysis Defect fix and Test Report Re-testing Generation Regression Testing Fig-1 Test cases identification has to be done based on the level of requirement that applies to the type of test. Once the test cases are identified, defining and documentation can be done based on the method of testing. For generating the test input, if the test harness or test stubs are not available appropriate tools have to be developed for the test execution. Test result analysis and test coverage analysis are done to make sure that the test coverage is adequate. Regression testing is a very important part of all EAI testing, since a minor change in the flow or the structure of the message may cause lot of impacts on the interfacing applications. Some basic test cases which cover all the basic functionalities have to be identified for execution after each code drop or modification in the code. Typical QA framework for EAI testing As already mentioned, when a proper quality assurance process is in place, testing related activities can be integrated into the project lifecycle starting from the beginning of the project. This will help to design the test cases based on the actual user requirement and not the design specification that may be defective. Test cases should be part of the requirements that drive © Wipro Technologies Page 9 of 13
  10. 10. WHITE PAPER Quality Assurance in EAI projects system design that are understood by the users, analysts and developers, such as test cases derived from use cases and event flows. When the test cases are driven by requirements, test execution will be more complicated due to the distributed nature of EAI. This requires a dedicated test bed or a QA integration framework throughout the test execution phase. Some of the basic requirements that contribute to the QA integration frameworks are. • Standardized interfaces and messages for testing • Recording and re-playing tools • Simulators or test harness to replace the actual applications • Stubs or loop backs to acknowledge the data output and data input • Validation tools, to validate the expected behavior based on the base lined data A typical integration scenario and corresponding QA framework is explained below. Consider an application takes orders over the web and distributes them to ERP, CRM and Billing systems. The scenario for testing the web-based application in this integration would consist of the following steps. Billing Web Validation (Simulate / Input Reply) Common Message Bus CRM ERP (Simulate / (Simulate / Reply) Reply) Fig 2 • Enter orders into the web application – could be driven based on HTTP messages or from a GUI-based test tool (record/replay). Web application can be replaced with a standard message generator or test harness in the early stages of testing. Messages can be defined based on the requirements. • The web application would publish messages to the middleware common message bus which will be available to the respective target applications like ERP, CRM and billing system • The QA system would simulate the ERP, CRM and Billing systems by replying to the web application’s messages based on a prior recorded session – replaying the messages in the proper sequence. Stubs or loop back can be used to simulate the target application response for each request. • The QA validating system would record the messages sent by the web application and received from the systems participating in the test (simulated or live). This system can be used for the automatic data comparison based on the base line data. © Wipro Technologies Page 10 of 13
  11. 11. WHITE PAPER Quality Assurance in EAI projects • Manual verification can be done if the data comparison is more complex or message format is constantly changing Quality Assurance Integration (QAi): Complexity of an EAI project is greater than other projects knowing the fact that EAI basically integrates different applications together, for the smooth business flow. Importance of specialized testing on integration projects can be a good solution for facing the challenges in EAI. This can be named as QAi (Quality Assurances Integration). Good software testing is a challenging intellectual process. In early days first impacting area of any software budget cut is the testing team, but the situation changed over a period due the underlining fact QUALITY. It is impossible for any product to exist in the highly competitive world with poor quality, so quality assurance is an integral part of anything. An ordinary tester can assure the quality of a product by considering it as a black box. When he understands the internal complexity and the fundamental architecture of the product, we can name the tester as specialized. It is quit obvious that when multiple systems operates in different environments with different languages, interacts using a common middleware, the knowledge of the middleware architecture and the interacting sub-systems are essential, more over the communication between multiple systems are based on messages. Essentials for QAi: When a developer does a testing the attitude will be to pass the test case with out damaging the built up product, so there are chances of hiding instability and loopholes in the product, which is going to fail in production. Dedicated testers bring two benefits: • Focus on customer usage over technical implementation • Focus on uncovering flaws over confirming completeness It is fun to read the word construction by destruction, and this can be seen only in software. Attitude of a real tester is always destructive construction, and this can be developed only when the tester is independent. When the tester acquires knowledge of design and the overall architecture, the result will be a purified product with stability and reliability in the production. Following are the essential requirements for a member in QAi • Capability to understand the business scenarios technical architecture • Thorough knowledge on testing methodologies • Good knowledge on test automation tools • Testing attitude • Good knowledge on various middleware integration (EAI) tools • Basic Unix skills • Basic development experience Test automation in EAI Many testing tools are available in the market for test automation of GUI based applications; most of these tools are following the capture/replay method for the functional verification. © Wipro Technologies Page 11 of 13
  12. 12. WHITE PAPER Quality Assurance in EAI projects Since EAI projects involves various applications talking to each other, a combination of EAI testing framework with GUI based testing tool can be used. In a typical EAI project as shown in Fig 2, when various systems are talking to each other through a common message bus, an input application can be automated using a capture/replay tool and external applications like CRM, ERP, Billing etc has to be simulated using loop backs (program which returns valid reply to the interfacing application based on the input) with all the interface trace logs enabled. Along with the test result, failures and errors in the trace log has to be verified carefully. Since major failures happen with the interface rather than the external errors, it is always suggested to do manual functional verification and automation for the regression test cases. Errors visible in the trace logs can be verified for finding the cause of the failures. Standard testing tool for testing the EAI projects are rare in market. Test automation is a mix of capture/reply tool with the EAI framework. EAI testing can be automated by writing automation scripts for the following activities • Start related components • Inject Message • Open the output file • Compare the output file with expected output file Make a batch file to automate all of the above mentioned steps sequentially. Comparison of the output file with expected output file can be done in the following ways. • Write a test data verifier code using simple assert type methods or even can use Junit™ • Generate a Verifying output flat file containing few passes/fails It is ideal to avoid expensive tools for verifying EAI module in an EAI project Reference 1. World Wide Web 2. Lessons Learned in Software Testing: by Kaner 3. General books on testing methodologies Conclusion As the Information Technology sector is navigating briskly to the Integration Technology line with large number of integration tools getting released in the market, it is quite obvious that there is a huge business potential hiding in the quality assurance area of enterprise application integration sector. A standardized testing framework for integration project which should be able to use with flexibility across platforms and applications is definitely going to help the industry in many ways. © Wipro Technologies Page 12 of 13
  13. 13. WHITE PAPER Quality Assurance in EAI projects About the Author The author, Pradeep Vengayil, posses a graduation with B.E. in Electronics and communication. He has been working with Wipro Technologies (http://www.wipro.com) for the last 5 years. Your comments are welcome to pradeep.vengayil@wipro.com. About Wipro Technologies Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Company globally. Wipro provides comprehensive IT solutions and services (including systems integration, IS outsourcing, package implementation, software application development and maintenance) and Research & Development services (hardware and software design, development and implementation) to corporations globally. Wipro's unique value proposition is further delivered through our pioneering Offshore Outsourcing Model and stringent Quality Processes of SEI and Six Sigma. For further information visit us at: http://www.wipro.com For more whitepapers logon to: http://www.wipro.com/insights © Wipro Technologies Page 13 of 13