How to Become A TestingExpertLiang Gao
Typical Interview Question• Do you know programming• Answer: Yes, a little• Which language do you know the best• Answer: I...
Tester’s Dilemma• Testing is not cool• Tester is second class when compared withdeveloper• Developer using me as servant• ...
Top Managements Dilemma也知道, 在招到好的 工程 是有多你 现 测试 师 难- 北京研究所副所华为 长 2008It took Juniper 6 month plus to find a new QAdirector
Career Story of Ting• Graduate at 1990, only job found is the tester at Sun• Join Cisco as tester in 1993, then test autom...
Lesson• You may have to stay long enough to becompetitive.
What is a testing expert?• Ability to find critical bugs in given time frame(hot gun)• Ability to build a comprehensive te...
What is a testing expert Looks like?• Play the video of James Bach
Find Bug that could make News
Bugs in the News• A Cisco Secure Access Control Server (ACS) that isconfigured to use Extensible AuthenticationProtocol-Tr...
Bugs in the News• CSCdv24925 It is possible to read storedconfiguration file from the Storage Routerwithout any authorizat...
Bugs in the News• Versions of the Cisco ACE 4710 ApplicationControl Engine appliance prior to softwareversion A1(8a) use d...
2 Factors of a hot gun• Technical Expertise• Thinking methodology
Hot Gun’s Bug PercentageP4P2P1P3CosmeticFunctionalityMajor FunctionalityCritical Functionality, Crash, Hang10%30%40%20%
Thinking Methodology
Thinking Methodology• Customers• Information• Developerrevelations• Team• Equipment &Tools• Schedule• Test Items• Delivera...
Thinking Methodology• Customers• 客服• 售前• 售后• 市场
Thinking Methodology发布升级 / 安装配置入网运营维护发布流程测试Installation/Upgrade 测试HA你知道客户典型配置么?你知道客户跑什么业务?你知道客户跑什么业务?你知道客户典型应用场景么?客户 1-2 年...
Thinking MethodologyNegative Stress Boundary FeaturesInteractionSecurityFeaturePoint 1FeaturePoint 2FeaturePoint 3FeatureP...
Develop Testing Strategy• Be the review person• 知其然也知其所以然
Develop Testing Strategy• What is your goal first?– Find more bugs?– Find more critical bugs?– Ensure product or feature h...
What is Coverage Strategy?• How to thoroughly test OSPF Hello protocol?– Function points? (tester)– User Scenarios? (test ...
What is Coverage Strategy?• 7 platforms• 6 different line cards• 2 modes (main/aggressive)• AH/ESP• CA/No CA• HA/No HA• Hu...
Types of Testing Covered on Different ReleaseMajorReleaseMinorReleasePatchReleasePlatformReleaseSFR CSPNew Feature Test Fu...
Advise to the New Expert• Practice, Practice, Practice• Don’t confuse experience with Expertise• Don’t trust the folklore ...
An Expert’s Vision• An Expert’s Vision– I can test anything– Under any condition– On any given time frame.
An Expert’s Vision
Top Expert• Release process• Testing organization management
Release process
Release Process• Start from the MRD• End with the FCS (first customer ship)
敏捷开发与持续集成 – 迭代中的持续集成
Test Organization management
Thank you!
Upcoming SlideShare
Loading in …5
×

How to become a testing expert

603 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
603
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • 陈胜吴广为什么不能如期到达目的地。 为什么没有 IPv5 为什么 TCP 是三次握手
  • How to become a testing expert

    1. 1. How to Become A TestingExpertLiang Gao
    2. 2. Typical Interview Question• Do you know programming• Answer: Yes, a little• Which language do you know the best• Answer: I know C a little• How many lines of code have you written• Answer : Umm.., about 300 lines• Do you know Java or .NET,• Answer : No• Do you know Linux• Answer : No• Do you know networking• Answer : I installed Windows XP on my PC….• Ok, you can work for us as testing engineer
    3. 3. Tester’s Dilemma• Testing is not cool• Tester is second class when compared withdeveloper• Developer using me as servant• I don’t see I can make difference in our company• Testers are keep coming, they are young andbetter than me.• 我在吃青春饭, I don’t know where my careerpath is• OK , I need to be a developer and write code!
    4. 4. Top Managements Dilemma也知道, 在招到好的 工程 是有多你 现 测试 师 难- 北京研究所副所华为 长 2008It took Juniper 6 month plus to find a new QAdirector
    5. 5. Career Story of Ting• Graduate at 1990, only job found is the tester at Sun• Join Cisco as tester in 1993, then test automationengineer (design Cisco’s automation framework)• Cisco grows from 2000 to 40000 from 1993 to 1998• Join Netscreen as the 1st testing engineer at 1998• NetScreen went to Nasdaq on 2001• Grow with NetScreen as testing lead, testing manager,testing senior manager and testing director• Found Sigma at 2004, served as CEO as today
    6. 6. Lesson• You may have to stay long enough to becompetitive.
    7. 7. What is a testing expert?• Ability to find critical bugs in given time frame(hot gun)• Ability to build a comprehensive testingstrategy in given time frame (expert)• Ability to manage the release process (topexpert)
    8. 8. What is a testing expert Looks like?• Play the video of James Bach
    9. 9. Find Bug that could make News
    10. 10. Bugs in the News• A Cisco Secure Access Control Server (ACS) that isconfigured to use Extensible AuthenticationProtocol-Transport Layer Security (EAP-TLS) toauthenticate users to the network will allowaccess to any user that uses a cryptographicallycorrect certificate which can be expired, or comefrom an untrusted Certificate Authority (CA) andstill be cryptographically correct.• CSCse58195. The WLC contains a bug whenprocessing WLAN ACLs that causes theWLANvACL configuration to be saved with aninvalid checksum. When the configuration issubsequently reloaded at boot time, thechecksum fails and the WLAN ACLs are notinstalled.
    11. 11. Bugs in the News• CSCdv24925 It is possible to read storedconfiguration file from the Storage Routerwithout any authorization.• CSCdu45417 It is possible to halt the StorageRouter by sending a fragmented packet overthe Gigabit interface.• CSCdv24925 An unauthorized person mayread the configuration of the Storage Router.That may lead to unauthorized access of astorage space.
    12. 12. Bugs in the News• Versions of the Cisco ACE 4710 ApplicationControl Engine appliance prior to softwareversion A1(8a) use default administrator, webmanagement, and device managementaccount credentials. The appliance andmodule do not prompt users to modify systemaccount passwords during the initialconfigurationprocess.• Crafted SSH Packet Vulnerability• Crafted SNMPv2c Packet Vulnerability
    13. 13. 2 Factors of a hot gun• Technical Expertise• Thinking methodology
    14. 14. Hot Gun’s Bug PercentageP4P2P1P3CosmeticFunctionalityMajor FunctionalityCritical Functionality, Crash, Hang10%30%40%20%
    15. 15. Thinking Methodology
    16. 16. Thinking Methodology• Customers• Information• Developerrevelations• Team• Equipment &Tools• Schedule• Test Items• Deliverables StructuresFunctions Data Platform Operation Time Capability Reliability Usability Security Scalability Performance Installability Compatiabilty Supportability Testabilty Measurability Portability Localizations Functiontesting Domaintesting Stress testing Flow testing Scenariotesting Claims testing User testing Automatictesting
    17. 17. Thinking Methodology• Customers• 客服• 售前• 售后• 市场
    18. 18. Thinking Methodology发布升级 / 安装配置入网运营维护发布流程测试Installation/Upgrade 测试HA你知道客户典型配置么?你知道客户跑什么业务?你知道客户跑什么业务?你知道客户典型应用场景么?客户 1-2 年运营的可能遇到情况可维护性测试兼容性?
    19. 19. Thinking MethodologyNegative Stress Boundary FeaturesInteractionSecurityFeaturePoint 1FeaturePoint 2FeaturePoint 3FeaturePoint 4FeaturePoint 5
    20. 20. Develop Testing Strategy• Be the review person• 知其然也知其所以然
    21. 21. Develop Testing Strategy• What is your goal first?– Find more bugs?– Find more critical bugs?– Ensure product or feature has no critical defects– Ensure customers will be ok after the release?• The strategy– Bug oriented?– Coverage oriented?– Customer oriented?
    22. 22. What is Coverage Strategy?• How to thoroughly test OSPF Hello protocol?– Function points? (tester)– User Scenarios? (test expert)– Scalability? (test expert)– Performance?– Security? (test expert)– ………
    23. 23. What is Coverage Strategy?• 7 platforms• 6 different line cards• 2 modes (main/aggressive)• AH/ESP• CA/No CA• HA/No HA• Hub Spoke/Partial Mesh/Full Mesh7x6x2x2x2x3 = 1088
    24. 24. Types of Testing Covered on Different ReleaseMajorReleaseMinorReleasePatchReleasePlatformReleaseSFR CSPNew Feature Test Full Full TBD FullRegression Test Full Partial Partial Partial PartialSystem Test Full Full Full FullInteroperability Test Full TBD TBD TBDPerformance Test Full TBD Full TBDCapacity Test Full Full TBDSecurity Test Full Full Full Full FullAutomation Test(partial regression)Full Full Full Full Full FullSFR – Special Feature Release CSP – Customer SpecificPatch
    25. 25. Advise to the New Expert• Practice, Practice, Practice• Don’t confuse experience with Expertise• Don’t trust the folklore – but learn it anyway• Take nothing on faith, own your methodology• Drive your own education, no one else will• Reputation = Money: Build and protect yourreputation• Relentlessly gather resources, materials and tools.• Associate with demanding colleagues.• Write, speak
    26. 26. An Expert’s Vision• An Expert’s Vision– I can test anything– Under any condition– On any given time frame.
    27. 27. An Expert’s Vision
    28. 28. Top Expert• Release process• Testing organization management
    29. 29. Release process
    30. 30. Release Process• Start from the MRD• End with the FCS (first customer ship)
    31. 31. 敏捷开发与持续集成 – 迭代中的持续集成
    32. 32. Test Organization management
    33. 33. Thank you!

    ×