ALM describes automated project management.
* Scrum based task management with issue tracking system.
* Contiguous Build
* Regression Test
* V-Model based Test process
* Defect management process
Application lifecycle management (ALM) is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management.
Composed with open source tools and cheap popular commercial tools
Open source tools Reduce learning curves
Less investment Free or cheap commercial tools
Based on practical and agile methodology
Practical ALM use case scenario Developer Perspective
Developer goes to office in morning.
Run eclipse IDE.
In IDE, there are task list that are assigned to him.
He checks tasks with priority and pick up one.
Change a status of the task to In Progress
Check out code from source code repository.
After implementation, he implement unit test with testing framework.
Execute test with standard build script. It automatically generate test result, code coverage rate and run code inspection. Code inspection shows illegal naming convention or potential bugs.
He fix the code and re-run the script.
He checks code coverage rate in eclipse IDE. After confirming the code coverage rate is meet target rate. He commit the code with the task ID.
CI (Contiguous Integration) system get a notify that main source repository has been changed.
It check out the code and run build script.
Compile and deploy the code to staging machine.
Run a test case. If test is broken. It automatically sends a notification to the developer and project manager. And then roll back the code in source repository and staging machine.
If the test is passed. it invokes code inspection and coverage analysis. If there is a problem (cannot match up target test code coverage rate or some static bug.) it will also report the problem.
Developer notify that all of implementation has been done successfully.
He write a his work history in task management system thru eclipse IDE. and change a status to Resolved
Project Manage get a notification the task is resolved. He check up a note of the task and task result etc. And then change a status of the task to Closed
Developer come back home.
Practical ALM use case scenario Project Leader/Manager perspective
Project manager attend on requirement meeting with customer.
He summarize meeting minutes and extract tasks from that with customer's agreement (or confirm)
He creates new tasks with priority.
He plans the task followed by project milestone.
He check up developer's load by using task management system to find who is under lower load.And assign it to appropriate person.
Project manager estimate task implementation time with assigned developers.
Project manager adjust plan with schedule and estimated time.
During the project, project manager tracking the task status and everyday adjust the tasks.
Project manager or senior developer checks project health status with unit testing result and code coverage rate.
In addition senior developer concentrate testing in high code complexity area. It is founded by code complexity analysis report from CI tool's daily build.
Project manager also manage risk by using project status report from task management tool and reports from CI tools. And he check up high priority Opened task every day to control risk.
Practical ALM use case scenario Project Leader/Manager perspective
Customer side PM come to office in the morning.
Execute web browser and go to ALM dash board page.
Each project has one page dash board. It shows how many tasks are opened and closed for each milestone.How many pending tasks are there. How many critical tasks are opened.
He just notify potential risk and project healthy status by using the dash board.
Make a meeting with Project Manager or Project Leader to control the risks.
If he want to track the critical issue (task). He subscribe the task. All of comment and change are automatically reported. So he doesn't need to make a meeting or make a phone call to check up progress of the critical task.
Modules Extract Requirement Task Management Dashboard Source code management Contiguous Build Standard IDE Standard Build Script Testing Process Testing Framework Static Testing Defect Management Wiki based doc mgmt Forum based bi-directional communication Code review
Summarize meeting minutes and post it into internal wiki
Extract action items and get a agreement from customer.
And put that action items into task list.
Google site : gather all meeting minutes into one place with jump start
Atalssian confluence wiki : cheap and powerful.
Customer PL Meeting Minutes Extract Action Item Ex) AI 1. Implement Logging with JMS (JIRA-101) AI 2. Evaluate Logging architecture in Test (JIRA-102 ) Task 1.Log whole of meeting 2. Extract action item in end of meeting. Action item should be agreed between customer and PL WIKI Meeting Task List or Task Management System Register action item to Task Process
Duration 4~6 weeks Requirement Use case (Sub) Requirement Task Task Task Analysis Design Implementation Testing Working Item Documentation Meeting Testing Internal Release QA has to test every internal release based on test plan General Available (Release) High Risk Mandatory Functions Non-Functional Requirement Major UI Low Risk Optional Functions Enhancement Scheduling (Before implementation phase) Wik i Task Management System Tracking in WBS (Link to MS Project) Use numbering in title to track of Requirement Task Task Task Task LINK Release Release Sprint Sprint Sprint Requirement Requirement Requirement Requirement Requirement Requirement Meeting minutes Optional Depends on requirement granularity
Task is real working item to implement requirement
Task has to have traceability to requirement
Task type definition is very important.
Task can have hierarchy.
[!] Requirement can be separated into more detail sub requirement System Task Sub Task Description Requirement N/A Detailed functional requirement. Requirement cannot contain any child requirement. It is leaf node of requirement tree. Name of requirement has to have number to provide traceability. Ex) 2.1.1 Logging inbound transaction WorkingItem Working items for developer. Eg. Coding,debugging etc. Documentation Write deliverables as document on wiki or document Test Develop and execute test of “ Requirement ” . It includes implementing Unit Test,design, execute etc. Meeting Meeting about specific issue Bug N/A Bug reported by somebody(not a author) after every internal release phase.
PL PL PL PL Customer Based on technical use case. Extract Child requirement and register it into JIRA Scheduling child requirement based on iterative development. It is linked to WBS in this phase In Every iteration, when the requirement related sub-task is started, change status to In Progress After finishing all of sub tasks for the child requirement. Change the status to “Resolved”
Open Assigned In Progress Resolved Closed PL Need more Information Postponed PL creates detail task for developer. Task is usually finished 1~4 day. (1~2 days are recommended) Discuss with developer and estimate time. Assign the time to task Developer Developer Developer Developer PL PL Developer discuss with PL PL lets developer to postpone the sub task. The postponed task should be finished in same iteration phase. Or closed
Build Failed Test failed Tagging current version and Roll back SCM to previous tagging version Event Trigger Send SMS/Email/IMS Check out source Build Packaging Deploy Testing & coverage anaysis Inspect code Roll back code Tagging Reporting Send Notification SCM Commit Event Scheduled time
< Verification & Validation > < V-Model > Verification Validation To verify the artifacts that has been produced in each development cycle. Valid & estimate the system Review & Inspection with artifacts from each development step Testing with system Static testing Dynamic testing
※ Integration strategies Level Description Testing Type Responsibility Unit Test Verify software component White box test Lead by Dev Team Use Code coverage Integration Test Verify integration between component. Verify software flow ,interface & interaction White box test Lead by Dev Team Use Mock Object (Test Stub ,Driver) System Test Test system over production environment to verify system itself and production environment Include Functional & Non Functional Test (Availibility, Stablity, Extendability, Performance etc) Black box test Lead by QA Team (Specialized for system test) Acceptance Test Verify customer ’ s requirement (End User) Verify customer ’ s Legal issue (Legal) Verify customer ’ s maintanance issue (SM) Black box test Lead by QA Team or Customer Alpha Test Beta Test Top down Bottom up Big-bang Backbone Desc Integrate from top (use test stub) Integrate from bottom (use test driver) Integrate all in time Most important & high risk component first Adv easily find architectural defect Can test implementation logic Short time Ex. BPM SOA svc Small project
Erik van neneendaal, Risk Based Testing, STAREAST,2006 Likelihood Impact factor factor factor factor Risk item Risk item Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area)
Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area) Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area) ( Low Level Test) ( High Level Test) - Unit Test - Integration Test - System Test - Acceptance Test
Develop TC,Test data,Test stub & driver,Test script,Test suite
Logging expected result, real result
Time & resource usage
Use Bug(Issue) tracking system ( Mantis,Bugzilla,JIRA,Trac) can be integrated with Task Management module in ALM.
Define tracking workflow
TEST CASE TESTING TOOL TEST DATA MODULE MODULE MODULE MODULE TEST SCRIPT TEST SUITE1 TEST SUITE2 TEST SIMULATOR Test Execution Test Report Defects Issue Tracking System Progress Report Test logs TEST DRIVER TEST STUB TARGET SYSTEM
Tools and environment to support executing testing
Unit Test Framework Load Testing Tools UI based Automated Testing Monitoring Test case Management Defect Management Tool Contiguous Integration Tool Unit Test Integration Test System Test UAT Regression Test Regression Test Regression Test Report Defects Testcase Management Execute Test
Test Test Test Test Application Unit Test framework Support J2ee incontainer Testing Validate test itself User Interface Business Logic DAO Interface (RMI,REST,WS*) xUnit xUnit for DBTesting xUnit for Protocol or message Type xUnit for UI In Container Testing Tool Code Coverage Analysis Tool
Very effective and low cost action to keep code quality.
Kind of Code Review
Intend to find a defect. Executed with formal process and team. (For more information. See Fagan’s Code inspection)
Team has four roles. (Mediator – project manager, Reader – understand system to be inspected, Designer/Coder – Inspect and generate recommendation, Tester – Test inspected defect and validate) And Six steps in process.
To execute code inspection, you need to have formal inspection team in your organization or bring a consulting who has highly trained.
Performance of inspection is poor. But effect is very nice. Code Inspection is done after implementation. Recommend to use code inspection in system testing phase.
Less formal than code inspection. But it has process also.
Mediator manage Team review session, and code author explains about his code.
Find a defects or gather idea for enhancement and write it to code review report. (Recommend to use wiki)
The ideas and defects are handled as Task in ALM/Task Management
Recommend one time per week. Or after testing, after internal releasing, after sprint.
Informal review. It can be used to share idea.
Only author has a rule that explains his code. Nobody has a responsibility.
All things are up to the author
Review code on a desk.
Usually 2~3 people can attend on review. Review code over a monitor and share idea.
It is nice to educate junior developer or who doesn’t have experience about product or skills.
Call coordinator Validator Project manager Developer QA Engineer User 1.Create case Filter
Log and screen shout
2.Yank & validate 2-1 Request more information 3.Assign 5.Assign 4. Priories and schedule 6.Fix (Workaround or patch) 7. Provide Solution 8. Confirm 9. Commit to main branch with Defect # Defect Reporter Implementation Team Resolved defects list is combined to release note Defect Management System Source code Management system
Defect management system is integrated with SCM to track change of source code to fix the defect.
Web based SCMbrowser shows changes for defect in a web from Defect management system link
Task Management system can be used as Defect management system as well.
In Big project or if there are so many defects are reported then separate defect management system and integrate it into task management system. It needs to separate organization to implementation and defect handling team
Defect Management System Source code Management system Web based SCM browser Track changes Link Shows change of code Task Management System Defect is task
Process Developer Management (PM/PMO/PL) QA Development Lifecycle Testing Model (Based on V-Model) Agile/Scrum Base Practical methodology base V-Model Based Analysis Design Implementation Testing Unit Testing Integration Testing System Test UAT Task Management Process (Traditional Methodology) Task Management Process (ALM/Scrum based) Task Management Process (ALM/Scrum basder) Build Process Test Process SCM Process Test Process Defect Mgmt Process Task Management Process (ALM/Scrum based))
Reference architecture describes what we need to build up ALM system with tools and software.
Layout of ALM architecture
Build Status Test Result Task Status Defect Status Check out Check in Management (PM/PMO/PL) Developer QA Assign Task Provide guide & process Manage document. Task Mgmt Contiguous Integration Testing Framework Build Script Source Code Management IDE Build Script Testing Framework Defect Management Testing Framework (System& UAT) Dashboard Wiki
Atlassian product is very popular , so easy to get a support and get a resource who can use it.
Atlassian has CI tools, SCM connector, Coverage analysis tools,,Wiki etc. So it provides strong integrity.
But, origin of JIRA is for Bug Tracking System not a Agile based task management system . It needs a customization in process and tool.
Atlassian Confluence Wiki Atlassian JIRA zAgile Portal MS Project Requirement Management Task Management Major Milestone management for mgmt org. (eg. PMO) Personalization, Integrated view Scrum based task management process Scrum in JiRA : http://www.slideshare.net/paulrene/adapting-jira-for-scrum-presentation
Reference Architecture Task Management – Alternative solution “Spira Team”
SpriaTeam has great agile approach . It already contains concept of Release,Iteration,Requirement,Task,Test..
Most excellent thing is it provides traceability from Requirement-Task-Test. It enables track whole things in manner of ALM.
It also provides some connector to integrate with issue tracking system (JIRA,Bugzilla)
But it doesn ’ t support integration with CI tools or coverage analysis tools . It should be implemented with other product.
Company looks new and small, so they seems like have few reference yet. But cheap..
Atlassian Confluence Wiki Infrectra Spira Team Requirement Management Requirement Management, Task Management, Schedule mgmt & Test case management Scrum based task management process LINK
Reference Architecture Task Management – Another alternative
VersionOne ( http://www.versionone.com )
- Agile Oriented
They provide hosted task management tool and sell it as package.
Well support scrum methodology.
Free license for one small team for evaluation. Try it …
Rally Enterprise ( http://www.rallydev.com )
They provide hosted (Saas) Task management tool.
It is based on Agile methodology. And they have great agile knowledge.
But it is not inconvenient for my practical ALM approach. (It doesn ’ t mean their product is poor)
If you want to get a deep consulting about Task management process, I want to recommend this one.
It is traditional pure ALM player .
It is very popular and covers big area of ALM.
It needs high level team maturity.
I just want to recommend this one to Enterprise who really willing to invest ALM and Governance.
It is also very powerful ALM vendor.
Same as serena, it covers whole of ALM scope and IT governance. It supports CMMI level as well and has very rich capabilities .
There is light version- Polarion Track & Wiki (Issue tracking + Wiki + light build mgmt + Eclipse Plug in)
※ This evaluation of ALM tools is up to my personal opinion.
Reference Architecture Test Automation JUnit EasyMock Cactus Selenium DBUnit XMLUnit Japex Basic testing framework Mockup test J2EE In container test UI Test DB Test XML Test Junit based stress test Cobertura Test Code coverage analysis SourceForge Grinder Stress Test SOAPUI Stress Tes t for Webservice,REST Find Bugs Code inspection Atlassian JIRA Defect tracking Functional Testing Framework (Unit testing/Integration Testing) Non functional testing (System Testing) Static Testing Defect mgmt V model
Design principal of Test framework is to support automated testing and regression test.
Test framework covers functional testing includes unit test and integration test. (Integration test can be viewed as coarse grained unit test)
Test framework combines – Functional testing capabilities followed by characteristics of component like DB Testing,UI Testing,XML based messaging and Unit based stress test, coverage analysis and mock up test environment etc.
This test framework covers broad range of testing. It means it is not easy. (concentrated on Unit testing and Integration testing. Partially covers System test and UAT. I recommend to use more matured tools for System test and UAT)
ROI of code inspection is very high. How to managing rule set is most important.
Optionally, I recommend to set up test case management tools from QA team. (ex. Testopia)
Apache ant will not be developed anymore, Maven will replace ant. Maven also support unified library management.
Hudson is now most popular free CI tools.
- It supports a lot of plug ins for JIRA,Junit,Find Bugs etc.
It is very easy to learn.
Functional Testing Framework Test framework Maven Standard Build Script Subversion Source code Management Eclipse Standard IDE Hudson Contiguous Integration Practical methodology ( Joel Spolsky,Kent Beck, Erich Gamma)
Atlassian confluence provides a lot of plug in. Like MS-Office connector etc. It is very useful to sharing your team knowledge.
If you already have collaboration infrastructure in your company, do not extend product.
Atlassian Confluence Wiki Document Management Knowledge Transfer JForum Bi-directional knowledge exchange
Reference Architecture Development environment
In staging environment, highly recommend to use hardware virtualization (CPU,Memory). It is very useful to validate system capacity. (Scalability test result is be basis of capacity planning)
Development Environment Staging Production Responsibility Development Team QA Team SM Team GA version (General Available) Internal Release GA Unit Test Integration Test System Test (Software perspective) System Test (Include validating hardware infrastructure) UAT Virtualization is useful
Organization Role & Responsibility PMO (Project Mgmt Office) Implementation Team (PL) Application Architect Quality Architect ALM Master Extract Requirement Task Management Dashboard Source code management Contiguous Build Standard IDE Standard Build Script Testing Process Testing Framework for Unit,Integration Testing Static Testing Defect Management Wiki based doc mgmt Forum based bi-directional communication Code review Testing Framework for System,UAT
Who has a responsibility to build up process and system?
Mentoring (Very important)
Organization Role & Responsibility Implementation Team (PL) Extract Requirement Task Management Dashboard Source code management Contiguous Build Standard IDE Standard Build Script Testing Framework for Unit,Integration Testing Static Testing Defect Management Wiki based doc mgmt Forum based bi-directional communication Code review
Implementation team uses a lot of things in ALM
It needs a strong mentoring to use it. Mentoring is not a option but a mandatory..!!
Required capabilities : Understand whole ALM architecture & tools and process - May be genius? No. think with customer, act with customer. (Just right understanding of their ALM and help them not to miss a way..)
Help customer to build up their ALM process and system.
Help customer to adapt ALM.
Inspect ALM usage in the team. (Inspect task management system note, Review Wiki usage and structure. Check up Build script and CI status. ) and mentor them to right way.
Convince customer or project manager that no more free lunch.!! (to do something, let them know it needs a time and resource)
Each stakeholder wants to customize ALM for their perspective. In ALM, all of stakeholder has to share their perspective. (Even if they can see different views for their role) – It is liquid flow. So ALM master lead the stakeholder to have same perspective..
Help customer to change their culture slowly,smooth without revolt, awareness.
<Insert Picture Here> Strategy for delivering Practical ALM Framework