Lessons learnt in outsourced project delivery. Test team view from India and CIS
Upcoming SlideShare
Loading in...5
×
 

Lessons learnt in outsourced project delivery. Test team view from India and CIS

on

  • 318 views

What are similarities and differences in outsourced/outstaffed testing for CIS and India? What are main risks and what are the key success factors? ...

What are similarities and differences in outsourced/outstaffed testing for CIS and India? What are main risks and what are the key success factors?
Update: some explanation slides were added to the original presentation. They might seem dull, but they bring the main ideas which were explained during the presentation.

Statistics

Views

Total Views
318
Views on SlideShare
318
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In any case, communication should be a top priority in this relationship, as according to a Price Waterhouse Coopers 2007 global outsourcing survey, 69% of outsourcing deals fail, in whole or part. Actually poor communication and unclear expectations are very frequently the main reason outsourcing deals fail
  • More people mean more lines of communication, more opportunities for miscommunication, and more misunderstandings and mistakes. Mistakes are much harder to see and fix in outsourced projects. First, the parties must clearly agree on the expectations from the outsourcing team, the deliverables and the <>
  • I was advised by my manager that if you want to win trust of people, you need to respond quickly and in such a way that the matter doesn’t need any more communication, if possible. If more communication is required try to answer emails in such time span that matters can be resolved on the same day. For example, first thing in the morning, answer emails from Japan and if you are catching tail end of work somewhere else, like USA from India. This will give them time to respond and you to respond back. Changing your working hours so that you can get some overlap with the clients is also very important esp. if you have 12 hour time zone difference.
  • Most IT organizations experience a 20% decline in productivity during the first year of an agreement, largely due to time spent transferring both technical and business knowledge to the vendor. Many offshore vendors are deploying video conferencing (avoiding travel) and classroom settings (creating one-to-many transfer) to improve the efficacy of knowledge transfer. In addition, employee turnover often places a burden on the IT organization to provide additional information for new team members. <>
  • Your company and your offshore outsourcing partner should have a clear common goal a successful project.
  • Even small incidents can appear as big, relation threatening in absence of trust and big incidents can be handled easily when trust exists. Raising negative information as soon as possible, admitting to mistakes and putting measures in place to avoid repeat behavior is very important for building trust. In one case We found out that our productivity was 1/5 th of what we had initially projected. After trying hard to improve productivity we found that we could not increase it beyond 1/4 th of initial prediction. We called for a face to face meeting with the client, explained the situation, told them about why productivity was low and invited their teams to spend time with us to 1. understand why we were low on productivity and where were the problems. They were upset but reluctantly agreed to spend that time. After spending a week with us they realized that the mistake wasn’t ours. There were a number of reasons – 1. Issues with test cases not being updated consequently our actual results seldom matched the expected results in the test cases resulting in long cycles of clarification. 2. Environment availability 3. Slow speed of their link. From our side we kept two extra resources (low experience, low cost) as back up resources who could take up extra work or deliver more work. We did not charge them anything. These measures helped restore the client confidence in us.>>
  • In one case we found out that the person who was responsible for outsourcing decision was battling political battles in his organization and some people wanted him to fail. We made sure that we deliver so that the trust placed in us by the person is not broken. Once we delivered good results the political opposition died down and sailing was smooth. In other case when our sponsor left the company, we were ill-prepared because we had not interacted at top level with other people. Soon for various reasons we were thrown out of that account. One need not indulge in the politics but one needs to be aware of it and make sure that people in client organization are aware of good work done by you.
  • Skill training + motivation training, making sure that the whole team shares the same goal – delivery of a successful product Good feedback is vital for the team morale. The outsourcing team, esp. the testers, should know that they are valuable contributors to the project. Special attention to managing people, knowing their needs should be paid.
  • Weekly status for project is not sufficient. One needs to engage at multiple levels so that in cases of escalation things can be handled. Small projects but extensive metrics program to capture data result in too much overhead. One needs to sit with the client to understand how do we measure the health of the project, what are the parameters and what is the effort in capturing the data. There are some standard metrics – Milestones, Effort, Defect and Test case based data. Risk is one which is often missed out. Follow top 5/10 risk every week and once a month perform complete risk analysis for long term projects. For short term projects it may be done informally. Customer satisfaction is another thing that needs to be measured at the end of every project and annually covering larger cross-sections of client management.
  • Testing depends on everything, and if the scope creeps or developers fail to deliver the product, time for testing is reduced. Solution: risk-based and prioritized testing. Equipment acquisition includes not only persuading your client to provide you with some specific equipment, but also issues related to customs and delivery. Do not underestimate them!
  • Frequent problem areas Access to application to be automated Application keeps evolving. Customers might expect automation done on a given build to work on more recent builds Number of application instances. If multiple people need to automate, they might need multiple environments, tool licenses, database instances etc. Lack of precisely documented test STEPS Lack of domain knowledge in automation teams and client resources having no time to do handholding, knowledge transfer, resolve queries Not sharing development plans often leaves odd-shore team automating things that need not be automated or are incorrectly automated
  • Work culture – some countries treat work above everything and some treat family first. Some people can’t say no but one has to understand the no How to explain to a German what is the reason for the 9 th of May Holiday? client and service provider may have different norms in terms of speed, style, decision making and organizational structure. Sometimes both organizations can take extreme, inflexible positions that serve to create tension or distrust (e.g, the client taking a position that “I don’t care what the contract says, I’m the customer” and the service provider equally digging in its heels and taking a view that “we are delivering to contract specifications”).
  • Tests executed, cycles done, tests created etc. are common productivity metrics for testing. For automation – scripts converted/created, framework functions created are some others. Outcome based testing is becoming popular where outcomes like number of defects found, tests run, field defects found etc. are used as parameters for testing effectiveness and efficiency.
  • clarify the responsibilities and role of each party determine how often reports (with what level of detail!) are to be made and to whom. some cultures, might not feel comfortable reporting problems, or giving bad news. This is why you should realistically define success, and give clear measurements for it.

Lessons learnt in outsourced project delivery. Test team view from India and CIS Lessons learnt in outsourced project delivery. Test team view from India and CIS Presentation Transcript

  • Lessons learnt in outsourcedproject delivery. Test team view from India and CIS By Vipul Kocher & Kateryna Nesmyelova
  • Vipul KocherVipul Kocher is a Co-founder of PureTesting,a global testing services organization.He has over 17 years of testing experience. Hebuilt and led testing teams at reputedcompanies such as Adobe Systems, AplionNetworks and River Run Software Group.Vipul has presented papers and tutorials atvarious international testing conferencesthroughout the world. He is also the inventorof Q-Patterns, a method of capturing testingknowledge, and writing reusable test casesand Extension to Noun-and-Verb technique –a test design method.Vipul is also President of the Indian TestingBoard, Indian National Board of the ISTQB. By Vipul Kocher & Kateryna Nesmyelova
  • Kateryna Nesmyelova8+ years in software testing. Katerynaspecializes in functional testing and testmanagement. She participated in testing ofvarious projects - from medical systems tothe systems of primary stock market for theworlds largest banks and innovative powermanagement in large data centers.QA Club speaker and trainer, Katerynaconducts a training to prepare for theFoundation Level ISTQB certification and aproject for ISTQB - certification withinQAClub since 2009.Member of USQB, Ukrainian branch ofISTQB, since 2010;owner of the ISTQB FullAdvanced Certificate.By Vipul Kocher & Kateryna Nesmyelova
  • What You Can’t OutsourceBy Vipul Kocher & Kateryna Nesmyelova
  • Topics to be covered1. Knowledge transfer.2. Communication issues, taking into consideration timegap and language issues.3. Team Management (Governance).4. Different goals for the in-house and outsourcing team.5. Team issues (such as insufficient qualification of teammembers, team member leaving the team etc.)6. Peculiar testing issues (test environment and test bedissues etc.)7. Automation.8. Cultural issues. By Vipul Kocher & Kateryna Nesmyelova
  • Reasons to FailBy Vipul Kocher & Kateryna Nesmyelova
  • Reasons to FailIn any case, communication should be a toppriority in this relationship, as according to a PriceWaterhouse Coopers 2007 global outsourcingsurvey, 69% of outsourcing deals fail, in whole orpart.Actually poor communication and unclearexpectations are very frequently the main reasonoutsourcing deals fail By Vipul Kocher & Kateryna Nesmyelova
  • Communication issues- Wrong expectations- Time zones difference- Language problems (insufficient knowledge of the communication language)- Slow reaction to requests- Lack of feedback By Vipul Kocher & Kateryna Nesmyelova
  • Communication issuesMore people mean more lines of communication, more opportunities formiscommunication, and more misunderstandings and mistakes. Mistakes aremuch harder to see and fix in outsourced projects.First, the parties must clearly agree on the expectations from the outsourcingteam, the deliverables and the schedule.<<Vipul – Communication is a two way street. Often the problem arises becausethe client is unable to understand the MEANING of what is being said.In India we have often found that our method of saying No is not as straightforward as in the USA or Europe. We almost never say no to anything. Thiscauses unrealistic expectations. Indians are often reluctant to give bad newsimmediately. This results in late surprises esp. when you can’t read the non-verbalcues.Some people, where their written communication is better than verbal, prefer not tocommunicate in conference calls. This results in impersonal communication.Emails are an impersonal means of communication. Phone calls are better, videocalls even better and in-person meetings are best. Once you can associate a faceto a name then emails, calls all become easier and more effective.>> By Vipul Kocher & Kateryna Nesmyelova
  • Time ZonesIf you work with multiple locations of oneclient how to handle time zones?Golden rule - Respond to those ahead ofyou first and those after you laterFlexibility in working hours required toget some overlap with the clients By Vipul Kocher & Kateryna Nesmyelova
  • Time ZonesThere is a good advice from an experienced manager that ifyou want to win trust of people, you need to respond quicklyand in such a way that the matter doesn’t need any morecommunication, if possible.If more communication is required try to answer emails insuch time span that matters can be resolved on the sameday. For example, first thing in the morning, answer emailsfrom Japan and if you are catching tail end of worksomewhere else, like USA from India. This will give them timeto respond and you to respond back.Changing your working hours so that you can get someoverlap with the clients is also very important esp. if you have12 hour time zone difference. By Vipul Kocher & Kateryna Nesmyelova
  • Knowledge Transfer- Fear of knowledge loss- Insufficient business knowledge- Too little documentations- Afraid to ask questions By Vipul Kocher & Kateryna Nesmyelova
  • Knowledge TransferMost IT organizations experience a 20% decline in productivity during thefirst year of an agreement, largely due to time spent transferring bothtechnical and business knowledge to the vendor. Many offshore vendorsare deploying video conferencing (avoiding travel) and classroom settings(creating one-to-many transfer) to improve the efficacy of knowledgetransfer. In addition, employee turnover often places a burden on the ITorganization to provide additional information for new team members.<<Vipul - One method we have tried to use is recording of knowledgetransfer sessions and then later editing those to make them short andcompact, esp. if we are dealing with complex topics and/or high turnover ofpeople. Wikis also help. Creating FAQ also helps. Asking clients for sameinformation again and again makes for a very bad experience.>>Here I wouldnt agree with Vipul, as my own experience tells me that itsalways better to double-check if you are unsure about something. But I doagree that the clarification shouldnt be extremely bothering. By Vipul Kocher & Kateryna Nesmyelova
  • Different goals- In-house team goal: make a successful product- Outsource/outstaff team goal: sell as many man-hours as possible (independently on the product success).- Different goals for each team member. By Vipul Kocher & Kateryna Nesmyelova
  • Trust• Trust about capabilities, intentions, security of information etc. is built over time.• What is your strategy for building trust?• Some activities: – Face-to-face time of key people from both sides – Doing something “extra” without charging customers – Flexibility, willingness to adjust – Consistent delivery By Vipul Kocher & Kateryna Nesmyelova
  • TrustVipul >> Even small incidents can appear as big, relation threatening in absence oftrust and big incidents can be handled easily when trust exists.Raising negative information as soon as possible, admitting to mistakes and puttingmeasures in place to avoid repeat behavior is very important for building trust.In one case We found out that our productivity was 1/5th of what we had initiallyprojected. After trying hard to improve productivity we found that we could notincrease it beyond 1/4th of initial prediction. We called for a face to face meeting withthe client, explained the situation, told them about why productivity was low andinvited their teams to spend time with us to 1. understand why we were low onproductivity and where were the problems. They were upset but reluctantly agreed tospend that time. After spending a week with us they realized that the mistake wasn’tours. There were a number of reasons: 1. Issues with test cases not being updated consequently our actual results seldommatched the expected results in the test cases resulting in long cycles of clarification.2. Environment availability3. Slow speed of their link.From our side we kept two extra resources (low experience, low cost) as back upresources who could take up extra work or deliver more work. We did not chargethem anything.These measures helped restore the client confidence in us. By Vipul Kocher & Kateryna Nesmyelova
  • Vested InterestOffshoring may – put a fear of Job loss in minds of client’s employees – Some of them may want to see off-shoring failWho is your champion in the clientorganization who would want you tosucceed?Don’t get caught in the politics of yourclient By Vipul Kocher & Kateryna Nesmyelova
  • Vested InterestOn one project the outsource team found out that the personwho was responsible for outsourcing decision was battlingpolitical battles in his organization and some people wanted himto fail.The outsource team made sure that they deliver reliable resultsin timely manner so that the trust placed in the team by thatperson was not broken. Once the team delivered good resultsthe political opposition died down and sailing was smooth.In other case when the team sponsor left the company, the teamwas were ill-prepared because they had not interacted at toplevel with other people. Soon for various reasons the outsourceteam was thrown out of that account.One need not indulge in the politics but one needs to be awareof it and make sure that people in client organization are awareof good work done by you. By Vipul Kocher & Kateryna Nesmyelova
  • Team issues- Lack of management astuteness to understand problems before they happen- Lack of skills/expertise- Hiring wrong people- Process Discipline- Lack of resources- Lack of ownership By Vipul Kocher & Kateryna Nesmyelova
  • Team Management (Governance).- Turnover of key personnel- Too many managers on both sides / No management in Agile development By Vipul Kocher & Kateryna Nesmyelova
  • Team Management (Governance).Skill training + motivation training, making sure that thewhole team shares the same goal – delivery of asuccessful product.Good feedback is vital for the team morale.The outsourcing team, esp. the testers, should know thatthey are valuable contributors to the project.Special attention should be paid to managing people,knowing their needs By Vipul Kocher & Kateryna Nesmyelova
  • GovernanceWhat is your governance strategy and plan? – When it is a small project – When it is a large projectWhat are the parameters that you measureand analyze?Perform Risk analysis periodically – both project riskanalysis and quality risk analysisWhat is the frequency with which youengage with client? At what levels?Engage with the client at multiple levels,going up as high as possible By Vipul Kocher & Kateryna Nesmyelova
  • GovernanceWeekly status for project is not sufficient. One needs to engage at multiplelevels so that in cases of escalation things can be handled.Small projects but extensive metrics program to capture data result in toomuch overhead. One needs to sit with the client to understand how do wemeasure the health of the project, what are the parameters and what is theeffort in capturing the data.There are some standard metrics – Milestones, Effort, Defect and Testcase based data. Risk is one which is often missed out. Follow top 5/10risk every week and once a month perform complete risk analysis for longterm projects. For short term projects it may be done informally.Customer satisfaction is another thing that needs to be measured at theend of every project and annually covering larger cross-sections of clientmanagement. By Vipul Kocher & Kateryna Nesmyelova
  • Peculiar testing issues- High level of dependency- Shared environment- Equipment acquisition By Vipul Kocher & Kateryna Nesmyelova
  • Peculiar testing issuesTesting depends on everything, and if the scope creeps ordevelopers fail to deliver the product, time for testing isreduced.Solution: risk-based and prioritized testing.Equipment acquisition includes not only persuading yourclient to provide you with some specific equipment, but alsoissues related to customs and delivery. Do notunderestimate them! By Vipul Kocher & Kateryna Nesmyelova
  • Automation– Access to application to be automated– Scripts support andcreation– Environmental andtool needs– Lack of preciselydocumented test STEPS– Lack of domain knowledge in automation teams– Not sharing development plans By Vipul Kocher & Kateryna Nesmyelova
  • AutomationFrequent problem areas- Access to application to be automated- Application keeps evolving. Customers might expectautomation done on a given build to work on more recent builds- Number of application instances. If multiple people need toautomate, they might need multiple environments, tool licenses,database instances etc.- Lack of precisely documented test STEPS- Lack of domain knowledge in automation teams and clientresources having no time to do handholding, knowledgetransfer, resolve queries- Not sharing development plans often leaves off-shore teamautomating things that need not be automated or are incorrectlyautomated By Vipul Kocher & Kateryna Nesmyelova
  • Culture IssuesImportant to – understand the culture of the client – help them understand your cultureMost important points related to culture – What is the meaning and method of saying Yes/No in that culture – What are the main festivals and holidays – What are the religious, social sensitivities – What is the work culture? By Vipul Kocher & Kateryna Nesmyelova
  • Culture IssuesWork culture – some countries treat work above everything andsome treat family first.Some people can’t say no but one has to understand the no.For example, how to explain to a German what is the reason forthe 9th of May Holiday?Client and service provider may have different norms in terms ofspeed, style, decision making and organizational structure.Sometimes both organizations can take extreme, inflexiblepositions that serve to create tension or distrust (e.g, the clienttaking a position that “I don’t care what the contract says, I’mthe customer” and the service provider equally digging in itsheels and taking a view that “we are delivering to contractspecifications”). By Vipul Kocher & Kateryna Nesmyelova
  • ProductivityWhat are the client expectations ofproductivity?How are you measuring these?What are the factors that impact yourproductivity?What are you doing to communicatethese to the client? By Vipul Kocher & Kateryna Nesmyelova
  • ProductivityTests executed, cycles done, tests created etc. arecommon productivity metrics for testing.For automation – scripts converted/created,framework functions created are some others. Outcome based testing is becoming popular whereoutcomes like number of defects found, tests run, fielddefects found etc. are used as parameters for testingeffectiveness and efficiency. By Vipul Kocher & Kateryna Nesmyelova
  • Conclusions- Set common goals- Align expectations- Good communication is a key to success- Manage carefully- Monitor and motivate people- Make the common process as clear as possible By Vipul Kocher & Kateryna Nesmyelova
  • Conclusions- Clarify the responsibilities and role ofeach party- Determine how often reports (with whatlevel of detail!) are to be made and towhom.- Some cultures, might not feelcomfortable reporting problems, orgiving bad news. This is why you shouldrealistically define success, and giveclear measurements for it. By Vipul Kocher & Kateryna Nesmyelova
  • Q&ABy Vipul Kocher & Kateryna Nesmyelova
  • Kateryna.Nesmyelova@gmail.com kittyness - Skype By Vipul Kocher & Kateryna Nesmyelova