Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
When Dev met Test
(The future of software testing)
2016.01 by JungGun
home: genycho.blog.me
role : specialist
actor : domain expert
role : generalist
actor : test specialist
Developer
Tester
When dev met test
2015 ...
This is about…
Message 1: Co-Work (3Amigos)
Message 2: Shift-left-testing (early testing activities)
Several cases that te...
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
V. Conclusion
Target projects
Small, Research&Dev project (Trendy technology related)
Video, Data analystics, Network&Security, IoT base...
Test Operation organization
Flexible test resource managing (Shared Service) for small, multiple projects
Each person has ...
Projects information (2014yr~2015yr)
Mobile authentication Security FIDO - external requests
Admin portal(re-usable
asset)...
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, p...
Customized quality education
To prevent defects, project-customized education in the early phase
( An example about outlin...
> Education: sharing the test plan
- what test activities will be done and why
- the whole test cycle, roles&responsibilit...
> Education: Test Design guide
- the difference with Unit & Integration testing
- the template of test scenario and guide
...
> Education: Sharing old defects, cases
- defects and test cases based on the product’s domain, technology
[ defects(Web) ...
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, p...
What Test Design
- The thinking process to test effectively and efficiently.
- because the projects were mainly a kind of ...
※ Wait!, Who is the best person do test design?
- Test Desing should be done by Testers!!!
- They can do anyhow, but do th...
Step 1) Understanding with documents, interviews, everything as possible
Step 2) Sharing what understood and getting feedb...
Step 1) Understanding with things – documents, interviews
Step 2) Sharing what understood and getting feedback
(Customer)
...
>Test Design: Test Coverage reviews
- testers review test coverage (line&branch coverage) on real code
- if needed, they a...
※ (Example) Graphical approach to review
If the documents are insufficient, dev code is the best documents.
First, draw an...
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, p...
Customized Test Code guide
- To minimize the learning curve to write unit-test code
. language - Java, C/C++, JavaScript, ...
※ (Example) Structured Test code
- To easy-maintain, data-driven testing
- Naming, Comments guide
Test Code Structure
Faro...
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, p...
Pair-Testing
- not fix bugs, but fix human mistakes
- developer+tester pair in a sprint testing
- developers learn what, h...
(Example) Pair-testing
- 40minutes pair-testing with 6 develpers
- more than 40 defects and issues (The team needed to def...
※ Test Code Pair-programming
- developers can learn how to write test code in the specific cases.
testers can know better ...
I. Introduction
II. Engineering activities
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, p...
Test Automation plan
- project customized Test automation plan should be established
RDMBS
DAO implementations
DAO interfa...
Test Automation Infra in early phase
Customer,
Other stakeholders
Product
Manager
Dev team
Shared & visualized
Test result...
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Othe...
Test Strategy
(Example 1) Client-Server application
- define ‘Test Levels’ like (1)Unit testing, (2)Integration1(Client, S...
(Example 2) Cascading Framework considered Unit/Intg Testing
- define Unit / Integration testing cosidering target’s archi...
(Example 3) Hardware-Software integratied product
- define various test approach in various dev phase
: Software level Uni...
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Othe...
Test Result Report & Assets
Dev/Quality issues based on test result refine best practices, documents, guides
I. Introduction
II. Engineering activities
III. Strategic activities
1. Test Strategy/Plan
2. Test analysis/report
3. Othe...
[ Refinement about Test requirements ]
- Functional, Non-Functional requirements
- Test Plan for Non-Functional testing
[ ...
FunctionalFunctionalFunctionalFunctional
TestTestTestTest
ReportReportReportReport
Standardized and Reliable Test Service
...
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
1. 3 Amigos
2. Code review
3. Co...
Testing was a communication channel during the develoment
- readable, visible approach as possible to show and get feedbac...
Publish readable TestCase Spec documents with TestCode comments
- in-house tool which parse testcode comments and publish ...
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
1. 3 Amigos
2. Code review
3. Co...
Dev, Test code review (Online)
- Formal process for devcode review is done, testers mainly review testcode
- reviews testc...
Rules and Occurrence
Code Inspection tool drive from the beginning
- Code Inspection tool(PMD) setting in the beginning of...
PMD, security, performance,
Official process, etc
Quality/Test
Shared Service
Dev Team
Shared SA
Fortify, white hacking
~~...
I. Introduction
II. Engineering activities
III. Strategic activities
IV. Other activities
V. Conclusion
1. retrospectives
...
Retrospective 1
person Good Bad
C With PMD, I could learn a lot about coding.
I could have experienced TDD
I failed TDD..
...
Retrospective 2
[ Pair-testing ]
(Good) Lots of bugs could’ve been found. Pair-testing in Web UI was much more effective t...
Final Message
Dev TeamDev Team TestTest CustomerCustomer
테스트 강화로 제품
의 기능& 품질 확보
관련 프로세스,산출
물 가이드로 과제 자
체에 대해 더 집중
가능
코드 리뷰...
Tester’s Job shifting
DEV
TEST
CUST
OMER
Test doesn’t know
how much we are
Burdened (crying.)
Dev doesn’t know what
I want...
Various Test careerpaths
- as many as various work needed, as many as various job needed
methodolgi
st,
SA,TA,DA,…
develo
...
Q & A
Do you think testers can write business test scenario?
Why testers care about Unit-testing?, It is the job for devel...
T.H.A.N.K Y.O.U
Upcoming SlideShare
Loading in …5
×

When develpment met test(shift left testing)

555 views

Published on

Sharing my thoughts and cases about co-work with test and developemnt. Two big approaches.
One is Engineering approach (
1. Early testing education
2. Test design
3. Test code guide
4. Pair-testing, programming
5. Test-Automation),
Second is Strategic activities (
1. Test Strategy/Plan
2. Test analysis/report)

Also, I wanted to mention tester's various career paths.
Thank you.

Published in: Software
  • Be the first to comment

When develpment met test(shift left testing)

  1. 1. When Dev met Test (The future of software testing) 2016.01 by JungGun home: genycho.blog.me
  2. 2. role : specialist actor : domain expert role : generalist actor : test specialist Developer Tester When dev met test 2015 remade in Software testing
  3. 3. This is about… Message 1: Co-Work (3Amigos) Message 2: Shift-left-testing (early testing activities) Several cases that test people co-worked with development team closely Not only about testing, but also about dev quality
  4. 4. I. Introduction II. Engineering activities III. Strategic activities IV. Other activities V. Conclusion
  5. 5. Target projects Small, Research&Dev project (Trendy technology related) Video, Data analystics, Network&Security, IoT based projects Current quality level was low, but the impact was huge Small(~12people) Internal funding Flexible requirements Complex, Trendy tech Young, Highly educated Human resource Testability가 떨어지는 Low quality concept
  6. 6. Test Operation organization Flexible test resource managing (Shared Service) for small, multiple projects Each person has one or two specialized domain area Big Data VA, IoT Security, Network Domain study Test Strategy Test Education Test Design Test Scripting Testware dev, Automation Test Execution Quality Review 0.25 0.25 0.25 0.25 0.5 0.25 0.5 0.25 Project A Project B Project 1 Project 2 a project TM TE TM TE TM TE※ Shared Service ※ Practices & effort
  7. 7. Projects information (2014yr~2015yr) Mobile authentication Security FIDO - external requests Admin portal(re-usable asset) Web application Admin portal 6 external requests projects technology Target domain Dev size (people) challenges, difficulty, etc MES bigdata analytics Data Analytics Manufacture Factory 5 Verifying statistics functions result and the integration with big data platform Crypt/DeCrypt algorithm Security (en/decryption) Encryption/decrypti on 3 Complex security algorithm, Integration with Legacy system Logistics demand prediction engine Data Analytics Logistics 4 complex prediction algorithm based of rules Integration with legacy system IoT based smart factory(Smart OHT) IoT Semiconductor manufacturing 6 Understanding IoT(hardware+software) technology and its Testability political issues with customers
  8. 8. I. Introduction II. Engineering activities 1. Early testing education 2. Test design 3. Test code guide 4. Pair-testing, programming 5. Test-Automation III. Strategic activities IV. Other activities V. Conclusion
  9. 9. Customized quality education To prevent defects, project-customized education in the early phase ( An example about outline ) - what test activities will be done and why - Frequently occurred defects based on similar doman, technology - How to write (User) test scenario & the basic of test design - Intro about sample unit test code with very early dev code - Test Automation introduction with Jenkins
  10. 10. > Education: sharing the test plan - what test activities will be done and why - the whole test cycle, roles&responsibility - be able to get feedback by sharing with dev
  11. 11. > Education: Test Design guide - the difference with Unit & Integration testing - the template of test scenario and guide - basic test design technique - practical guide with other input documents
  12. 12. > Education: Sharing old defects, cases - defects and test cases based on the product’s domain, technology [ defects(Web) ] [ Quality guide for API ] API Development Standards on quality - Logging - Exception handling & error code - API Spec guild - Unit test checklists
  13. 13. I. Introduction II. Engineering activities 1. Early testing education 2. Test design 3. Test code guide 4. Pair-testing, programming 5. Test-Automation III. Strategic activities IV. Other activities V. Conclusion
  14. 14. What Test Design - The thinking process to test effectively and efficiently. - because the projects were mainly a kind of pure algorithm libray, they were so complex and needed to design test cases well. Black-box approach - initial test approach was black-box which design tests with specification - various documents – requirements, algorithm description, Architecture doc etc - because they were so vague or insufficient to understand the algorithm, we focused on visualizing the logic to share the understanding with customers White-box approach - the real code was the best documents - made the domain-specific dev-code checklist - do offline reviews with dev code or unit-testing code (regular pair-programming) ※ Grey-box testing? - basically do blackbox-testing while checking dev code
  15. 15. ※ Wait!, Who is the best person do test design? - Test Desing should be done by Testers!!! - They can do anyhow, but do they know what custmer really wants? Business Analyst Customer Tester “ANALYSTS!!!” (pros): knows well the businees and dev both. Need to be reviewed what they analyzed (cons) : not know about testing technique “TESTERS!!!” (pros) : test experts, experienced in many domain, focused on testing (cons) : need to understand requirements fully “CUSTOMERS!!!” (pros): best know about business (cons): no time, no tes or IT knowledge Who shuld do test design? The answer depends on ‘Test Level’ or ‘type’, but basically everyone should work together.
  16. 16. Step 1) Understanding with documents, interviews, everything as possible Step 2) Sharing what understood and getting feedback (Example) Analyzing old test scenarios find questions to confirm And test conditons Make checklist, diagrams to share with dev team
  17. 17. Step 1) Understanding with things – documents, interviews Step 2) Sharing what understood and getting feedback (Customer) Easy to understand with diagrams (Developers) With the checklists, I was able to self-define the detail parts (Product Manager) Visual diagram can help potential stakeholders to understand our product I can give the team more detailed requirements
  18. 18. >Test Design: Test Coverage reviews - testers review test coverage (line&branch coverage) on real code - if needed, they add more test cases or ask developers to do TC add) need to test server and client different version check test Coverage check) The code is not reachable(double checking) Need to delete
  19. 19. ※ (Example) Graphical approach to review If the documents are insufficient, dev code is the best documents. First, draw any diagrams if it can be helpful, then review the logic, workflow with various stakeholders 소스 코드 구조-Process 처리여부 구조-Log 처리 구조-Exception 처리 입력 파라미터 체크 input table 없는 경우 처리 output table 없는 경우 처리 해당 컬럼명이 없는 경우 처리 로직 상의 if 문 조건 나열 계산식에서 입력값 짝에 따라 분자 or 분모가 0이 되는 경우 코드 내에 형 변환(문자 -> 숫자, 숫자-> 문자, Double- >int 등) 로직 수행 중 오류 발생 시 에러 테이블에 넣어주고 있는지 Ex 1) the sequence of dev code 2) Code review checklist (a) main structure (b) logging (c) exception handling … (f) if conditions … (j) calculation (k) type casting
  20. 20. I. Introduction II. Engineering activities 1. Early testing education 2. Test design 3. Test code guide 4. Pair-testing, programming 5. Test-Automation III. Strategic activities IV. Other activities V. Conclusion
  21. 21. Customized Test Code guide - To minimize the learning curve to write unit-test code . language - Java, C/C++, JavaScript, Scala(Spark) . Specific framwork based test code(Spring fr, MapReduce, Spark, Kura…) Target Guide Type Complex pure algorithm Various Test conditons and cases junit C language program Tool research, Gtest pilot GoogleTest Big Data Data-driven testing using Junit, MRUnit MRUnit(MapRedu ce), ScalaTest(Spark) Complex prediction algorithm Unit/Intg test code with Junit, CascadingUnit, Various test cases Cascading IoT based software unit-testing mocking Hardware parts Eclipse Kura Framework Web application various test cases on various layers Spring Framework Target to Test (TC1) basic workflow [ Basic Test Approach ] (TC2) code branched workflow (TC3) specific business workflow (TC4) exception workflow … [ Projects ]
  22. 22. ※ (Example) Structured Test code - To easy-maintain, data-driven testing - Naming, Comments guide Test Code Structure FaroServiceTestCase - 테스트 수행 전 스프링프레임워크 의 설정 파일 로딩 - 매 테스트 후 테스트 이전 상태로 DB 초기화 - 특정 쿼리문을 직접 수행 가능한 인터페이스 제공 등 SampleServiceTest @Autowired로 객체 주입 후 - 기본적인 테스트 - 다른 흐름 테스트 - 잘못된 키 값 등의 테스트 - 필수입력 누락 등의 테스트 … SampleTestData - 테스트에 사용되는 데이터를 분리하여 다른 테스트에서도 참조할 수 있도록 정의 (추가로 엑셀 또는 DB에서 가져올 수 있게 보완 가능) CodeMngService - 코드 전체 조회 - 코드 검색 - 코드 등록, 수정, 삭제 … AbsractTestData - 테스트 데이터를 DB, 엑셀 등으로 부터 관리할 수 있도록 지원하는 추 상 클래스 Spring configration loading, common variables, utils are in a super Parent class If the configuration(file) is changed, only one class needs to be changed If the data is changed, Only one is to change Data that can be Referred in other tests are stored in a separate class
  23. 23. I. Introduction II. Engineering activities 1. Early testing education 2. Test design 3. Test code guide 4. Pair-testing, programming 5. Test-Automation III. Strategic activities IV. Other activities V. Conclusion
  24. 24. Pair-Testing - not fix bugs, but fix human mistakes - developer+tester pair in a sprint testing - developers learn what, how testers think and test testers learn what the code intended and expected - (Expected) better communication reduced bugs (defect prevention) by learning testers can find more serious bugs the product’s quality is better (Not-Expected) Developers and testers do not trust each other complaint that pair-testing is useless and time-consuming job disappointment about testers, testing
  25. 25. (Example) Pair-testing - 40minutes pair-testing with 6 develpers - more than 40 defects and issues (The team needed to define the standard) Sessi on dev date target Notes (Reminds, Test paths, defects, etc) 1 A dev -2d mornig Role- management [역할관리 - 조회] (1) 화면 리사이즈시 데이터 영역이 잘리는 현상 있음(하단 스크롤로 끝까지 옮겨도 표시되지 않음) (2) (공통-검토필요) 현재 화면 위치를 왼쪽 메뉴에서 초록색으로 표시해 주는데 F5 새로 고침 시 초록색이 표시되지 않음 (3) (권고) 조회조건에 역할의 사용여부로 조회할 수 있는 기능이 있었으면 편할 것으로 보임 (4) 검색조건에 특수문자 입력이 가능하거나 타 화면과 달리 trim 처리가 적용/ 또는 미적용된 경우가 섞여 있음 (5) (검토예정이라고 함) 역할명, 설명등을 주어진 길이만큼 입력하면 목록에서 ... 으로 표시됨. 툴팁이 표시되면 좋을 것으로 보임 * 수정팝업을 띄우면 표시되니 결함보다는 권고사항성입니다~ [역할관리 - 추가] (6) 입력 시 id, 명에 trim 처리가 필요할 것으로 보임.(조회시는 권고이나 등록/수정 시에는 trim이 필요) (7) (다른 화면도 체크 필요) 신규 추가시에 사용여부를 N으로 하고 저장해도 Y로 저장 됨 <- 쿼리에 'Y'로 박혀 있음 [ 사용자 팝업1,2 ] (8) 직급 컬럼 등의 정렬이 타 화면과 일치하지 않음 (가운데 정렬 필요) (9) 조회조건 필드가 특수문자는 막혀 있는데 trim 처리는 하는 등 타 검색 필드와 일관성 필요 (10) “그거”라고 했던 건 - 체크박스를 선택하고 페이지를 이동하면 다른 페이지에서 선택하지 않은 건들이 선택되어 있는 등 이상 동작을 하는 경우가 발생 함 2 B dev -2d mornin g Group- management- popups [업무그룹 메뉴팝업] (1) 팝업 화면 타이틀명에 공통적으로 언더바가 포함되어 있음 (2) 여러 팝업에서 같은 항목의 정렬이 서로 다름 - 길이가 일정한 코드성 데이터는 가운데 정렬, 이메일, 설명 등과 같이 값이 가볍적으로 바뀌는 값들은 좌측 정렬이 맞을 것으로 보임 (3) (확인필요) 화면 및 컬럼 폭 리사이즈가 안 되는데 길이가 길어질 수 있는 항목의 경우 툴팁을 검토할 필요가 있음 (4) 메뉴에 대해 사용여부를 N으로 수정해도 타 화면 팝업에서 메뉴가 조회됨 (5) (권고) 메뉴 추가에서 기존 추가한 메뉴와 새로 추가할 수 있는 메뉴가 구분이 안 됨. 방안 몇가지 얘기했던 것 검토 요청 [사용자 팝업] (6) 사용자를 선택하지 않고 선택 버튼을 클릭하면 메시지는 "한 건 이상을 선택해 주세요" 라고 정상 표시되나 바로 팝업 창을 닫아버림. 선택할 수 있도록 팝업 창 유지 필요 (7) 각 항목들의 좌우 정렬이 타 화면과 다름 (8) 팝업 화면 전체에 공통 발생 - 조회 데이터의 필드들이 모두 불필요하게 editable 함 3 C dev -2d afterno on Group- management [업무그룹관리] (1) 업무그룹 ID에 한글이 들어갈 수 있으나 조건조회 해당 필드에 한글이 입력되지 않음 (2) (확인필요-일관성) 신규 추가후 리프레시 될 때 추가된 건이 전체 페이지의 제일 하단에 표시되는데 타화면과 기준 논의 필요 <- 사용자 추가의 경우 해당 정렬 위치에 추가되었던 것으로 보임 (3) (권고) 조회조건에 업무그룹의 사용여부별로 조회할 수 있었으면 좋겠음 (4) 업무그룹ID 등에 trim처리를 해야 할 것으로 보임 (5) "그거" 라고 했던, a) 목록 조회에서 전체 조회하여 여러 페이지 조회, b) 조회조건 입력 후 조회버튼이 아닌 끝 으로 이동 선택 => 아무 건도 조회되지 않음, c) 조회조건 삭제하여 전체 조회 => 전체조회는 정상적으로 되나 하단 페이지 숫자가 표시되지 않음. 4 D dev -1d Mornin g Login,User- registration [회원가입] (1) 로그인 ID 값에 대해 trim 처리를 하지 않음 (2) ID 입력, 중복체크 한 상태에서 사용자 이름을 입력하지 않고 저장하면 ID 중복 체크를 하라는 메시지가 표시 됨 (3) 비밀번호, 비밀번호 체크에 대해 필수입력 표시 누락 (4) (검토) 이메일, 전화번호에 대한 영문자 입력 등에 대해 체크 로직 적용에 대해 검토 권고 * ID, 비밀번호 생성 규칙은 반영 예정이라고 함 [로그인] - N/A [사용자관리] (5) UI 브라우저 리사이즈 시 사용자 관리만 하단 스크롤이 2개 생기는 현상 있음 (6) 전체 화면이 아닌 경우 하단 footer가 페이지 네비게이션 영역을 덮는 현상 발생 (7) (권고,공통) 왼쪽 메뉴 접기 아이콘이 있는데 접은 상태에서는 화살 방향이 반대여야 펼치는 기능으로 인지하기 쉬울 것으로 보임 (8) 조회 조건의 사용여부 콤보박스 값이 직접 선택 및 수정이 가능함 (9) 브라우저 리사이즈 시 조회 영역은 정상 확장되나, 하단 데이터 영역이 확장되지 않는 현상 발생 (10) 사용자 정보 수정 팝업에서 필수값을 입력하지 않았을 때의 메시지가 ID중복체크를 하지 않았다는 잘못된 메시지 발생 (11) (동시성) A 관리자가 사용자 정보를 수정(팝업)하고 있을 때 B 관리자가 해당 사용자를 삭제. 다시 A 관리자가 수정 저장하면 메시지 상으로는 정상적으로 수정되었다고 표시가 됨(해당 사용자는 삭제) - 서버 상에서 수정하려는 id가 없어서 정상 반환하는 현상 때문으로 보임. 5 E dev -1d afterno on Menu- management [메뉴관리] (1) 메뉴에 대해 사용여부를 N으로 수정해도 타 화면 팝업에서 메뉴가 조회됨 (2) 메뉴 하위의 메뉴페이지 리스트에 대해 N으로 수정해도 Y로 다시 변경되어 조회 됨 (3) 메뉴 하위의 메뉴페이지 신규 추가시 사용여부를 N으로 하고 추가하면 추가는 되었다고 메시지 표시되나 실제로는 저장 안 됨 (4) 메뉴 페이지를 여러개 등록한 후 삭제하면, 삭제가 1,2건만 되고 이후 삭제가 되지 않음 (5) 하나의 메뉴에 대해 동시에 다른 페이지에서 접근하는 테스트에 대해 A가 보고 있는 상태에서 B가 삭제하는 경우 A에서 다른 동작을 하면 "알 수 없는 에러가 발생하였습니다" 라는 상황을 파악하기 어려운 메시지가 표시됨 (6) UI 브라우저 리사이즈 시 조회 영역은 정상 확장되나, 하단 데이터 영역이 확장되지 않는 현상 발생 Early testing, easy to understand and fix Not just finding defects, but also chances to define the dev standards in real time
  26. 26. ※ Test Code Pair-programming - developers can learn how to write test code in the specific cases. testers can know better about dev code - Testers can talk about quality aspects in dev code level Well, at first, let’s write the most basic case which We can debug what we write. after that, everytime you need to check your code or Debug, add more tests cover those cases In someway, we can check the test coverage, it will help us if we should add more tests or not Should I write test code for this part? That is too much and difficult to synch…
  27. 27. I. Introduction II. Engineering activities 1. Early testing education 2. Test design 3. Test code guide 4. Pair-testing, programming 5. Test-Automation III. Strategic activities IV. Other activities V. Conclusion
  28. 28. Test Automation plan - project customized Test automation plan should be established RDMBS DAO implementations DAO interfaces Service implementations Service interfaces Other remote interfaces Data Access Layer Service Layer Presentation Layer Web UI Test Controller REST API UI 55 √ Controller Test Service Test DAO Test RESTful API Test √ 11 22 33 44 √ Test Level Target Tools Unit Test (Sprint) DAO Junit Service Junit REST API SoapUI Web UI Pair-testing Sprint Integration Test Internal Workflow Selenium (+Sikuli) Integration External/Legacy Selenium / Manual Integration Test Internal/External Workflow Manual User Test Internal/External workflow Manual [Example: Test automation strategy ]
  29. 29. Test Automation Infra in early phase Customer, Other stakeholders Product Manager Dev team Shared & visualized Test results Okay. We are coding&testing My tests are ok. Junit test reports Test coverages (Method,Line,Bran ce, etc) Code Inspection (PMD,…) Jekins (+SonarQube?) [Example: Test automation ]
  30. 30. I. Introduction II. Engineering activities III. Strategic activities 1. Test Strategy/Plan 2. Test analysis/report 3. Other Professional services IV. Other activities V. Conclusion
  31. 31. Test Strategy (Example 1) Client-Server application - define ‘Test Levels’ like (1)Unit testing, (2)Integration1(Client, Server each), (3)Integration2(Client-Server) and each level’s purpose, target, role, method User Env Unit Testing Intg Testing 1 Intg Testing 2 Acceptance Testing 각 메소드(또는 단위 기능)에 대해 다양한 예외 상황을 만들어 면밀히 테스트 서버, 클라이언트 각 영역 안에서 여러 단위기능들의 묶음에 대해 기능 흐름을 몇 가지로 정의하고 테스트 서버 - 클라이언트 간의 상호 연동 관점에서 기능 흐름을 몇 가지로 정의하고 테스트 XXXeaXXXea XeaXea XXeaXXea ???? 사용자 관점(사업부)에서 제품을 어떻게 사용할지 흐름을 정의하고 테스트 Client Server Mod A Mod B Mod C Mod D Mod E Mod F Unnit Test Acceptance Testing Intg 1 Intg 1 Intg 2
  32. 32. (Example 2) Cascading Framework considered Unit/Intg Testing - define Unit / Integration testing cosidering target’s architecture and functional roles Unit Test : targets one Pipe mainly Integration Test : targets workflows from Source to Sink(or failed end)
  33. 33. (Example 3) Hardware-Software integratied product - define various test approach in various dev phase : Software level Unit/intg testing, Hardware-Software integrated testing, real environment testing etc Level Software testing On board testing Simulated Env Real Env Description Software Unit/Intg testing for Java programs Application interface testing. Hardware, software integration checking Simulated, down-scaled environment testing Real Envronment Methods Junit With test scenarios With business scenarios With business scenarios In charge Dev team(testers) Dev team(testers) Dev team, Customers Customers/Dev team 어플리케이션 정보 파일 센서, 하드웨어
  34. 34. I. Introduction II. Engineering activities III. Strategic activities 1. Test Strategy/Plan 2. Test analysis/report 3. Other Professional services IV. Other activities V. Conclusion
  35. 35. Test Result Report & Assets Dev/Quality issues based on test result refine best practices, documents, guides
  36. 36. I. Introduction II. Engineering activities III. Strategic activities 1. Test Strategy/Plan 2. Test analysis/report 3. Other Professional aids IV. Other activities V. Conclusion
  37. 37. [ Refinement about Test requirements ] - Functional, Non-Functional requirements - Test Plan for Non-Functional testing [ Customized test strategy ] - Architecture level test plan - Communications with several group(dev, customer, etc) [ Items ] - Purpose, Levels - Input/Output, Entry/Exit Criteria - Test env - Roles and Responsibility - Schedule Test related documents(Plan, Scenario, etc) template, guide and drafts 테스트테스트테스트테스트 계획서계획서계획서계획서 Standized documents template and guide
  38. 38. FunctionalFunctionalFunctionalFunctional TestTestTestTest ReportReportReportReport Standardized and Reliable Test Service at the end of project Total Quality reportTotal Quality reportTotal Quality reportTotal Quality report . Functional Testing result . Performance Testing result . Security Testing result . Software License verification . Code Inspection result …… Regularly
  39. 39. I. Introduction II. Engineering activities III. Strategic activities IV. Other activities 1. 3 Amigos 2. Code review 3. Code Inspection(tool) 4. Standard Process advisor V. Conclusion
  40. 40. Testing was a communication channel during the develoment - readable, visible approach as possible to show and get feedback to customers Customer Ahha, Graphical test case documents Complex workflow to real world
  41. 41. Publish readable TestCase Spec documents with TestCode comments - in-house tool which parse testcode comments and publish test specification on web … Customer Readable, Real-time, Synchronized Test Specification @author @Desc @Step @Input @Expected @Name @TC_ID GenerateGenerate Writing comments Other developer Dev lead, tester
  42. 42. I. Introduction II. Engineering activities III. Strategic activities IV. Other activities 1. 3 Amigos 2. Code review 3. Code Inspection(tool) 4. Standard Process guide V. Conclusion
  43. 43. Dev, Test code review (Online) - Formal process for devcode review is done, testers mainly review testcode - reviews testcase’s purpose, assertion and also work with test-coverage by checking which line or branch covered or not - if necessary, devcode review perfomed(often needed) Test code review example - The test method name should be test{target}_{prupose}_seq. - Each test method should test one purpose. - Copy&paste ‘throws Exception’ to each test method. We don’t need to exception handling (try,catch) in test code - If you want to test the specific Exception occur, just type the exact exception name not general ‘Exception’. And use ‘fail’ statement just after the line you expect for code readers - External resource(File..) should be referenced in relative path not fixed path in your pc Dev Code review example - Not covered toString method in test code. One more thin, Eh,.. It’s better to return at least empty String like “” rather than null…for toString() methods. It can occur NullPointerException if somenone uses and in some case. - This checking code is not covered in testing. Well, you are double checking, so it looks impossible to reach that code. Need to refactor. - This several catch clauses are not reachable… In my opinion, you can replace higher exception class than lower ones - This method will return null, if something’s wrong, but the callers are not prepared. It needs to be considered
  44. 44. Rules and Occurrence Code Inspection tool drive from the beginning - Code Inspection tool(PMD) setting in the beginning of the project - Detailed code guide for Top 10 violations regularly Top 10, Rule explanation and solution, alternatives
  45. 45. PMD, security, performance, Official process, etc Quality/Test Shared Service Dev Team Shared SA Fortify, white hacking ~~ Shared TA PMDPMD SecuritySecurity PerformancePerformance PMD Rule customizing meeting Security testing meeting Performance test meeting . How to measure web client response time(any tools)? . How to test JavaScript? . What is the official process to finish the project? HelpDesk for other quality services - As a quality member, introduce what to do, how to do in quality - explain similar project cases and its pros/cons
  46. 46. I. Introduction II. Engineering activities III. Strategic activities IV. Other activities V. Conclusion 1. retrospectives 2. Final message 3. Career path in testing
  47. 47. Retrospective 1 person Good Bad C With PMD, I could learn a lot about coding. I could have experienced TDD I failed TDD.. D Anyway finished, it’s good!!! With test cases co-writing, I could learn my code myself. Too short deadline Documents synchrozing E Test code, PMD helped my coding skill Too short time schedule G At first, I didn’t believe testers cannot be helpful, but now we together worked for the product’s quality. That is important Tight time scheduling Test Scenario planning was so good, it should be happened in very early phase. H (PM) Well, I have never thought co-working with testers in dev phase. Next time, this approach will be basic. “I thought they would do nothing because they knew nothing about algorithm, but what they did was that helping me understanding my product well”
  48. 48. Retrospective 2 [ Pair-testing ] (Good) Lots of bugs could’ve been found. Pair-testing in Web UI was much more effective than test code programming. It was good to hear other projects, other developers cases (Bad) 30 or 40min is too short, I want to do more. [ Test Scenario(cases) co-review ] (Good) Well, I learned how you approach. It was much easier to think how to test. (Bad) We changed requirements and test cases, that contents are not shred. [ testcode pair-programming ] (Good) It was good to show how Junit test code is written in detail. It was so easy after checking what you made. (Bad) I needed to be prepared. I didn’t know much about Junit itself [ Other words? ] (Good) Time-boxing for every activity (30min, 40min) was really good. (Bad) Well, I think there should be sessions that we talk what we should develop and test together. Only with us, developers, we say similar things with similar views. Let’s work together.
  49. 49. Final Message Dev TeamDev Team TestTest CustomerCustomer 테스트 강화로 제품 의 기능& 품질 확보 관련 프로세스,산출 물 가이드로 과제 자 체에 대해 더 집중 가능 코드 리뷰, 단위테 스트 방안 가이드로 개발 관점의 품질활 동 수행 개발 시작부터 3자 (사용자) 관점에서 지속적인 피드백으 로 협업 다양한 테스트 프랙 티스에 대한 적용 경 험 축적(소규모 과 제) 개발 마지막에만 하 는 테스트가 아니라 개발 초기부터 능동 적이고 적극적으로 하는 테스트 수행 개발팀-고객 사이에 서 가교 역할 기본 품질이 확보된 제품 딜리버리 테스트 내용 (실시 간)공유를 통해 제품 에 대한 상세한 파악 가능 제품에 대한 고객 관 점의 피드백 제공 가 능 win-win win-win Easy Understanding User centric product delivery Easy Understanding User centric product delivery Synergy by working together
  50. 50. Tester’s Job shifting DEV TEST CUST OMER Test doesn’t know how much we are Burdened (crying.) Dev doesn’t know what I want(Crying..) No documents. Everyone’s busy (Crying..) AS-IS DEV TEST CUST OMER TEST Test repsents us to dev (Thnx~) Test came to help me (smile~) My job is helpful for dev(smile~) Became active (real) member To deliver the right product(smile~) TO-BE
  51. 51. Various Test careerpaths - as many as various work needed, as many as various job needed methodolgi st, SA,TA,DA,… develo per PM PL Tool engineer Dev project Tester Test Lead Test Engineer Test Architect Test Manager Tester TM TL Test project Test Strategy Education, guide Manual, Automation Support
  52. 52. Q & A Do you think testers can write business test scenario? Why testers care about Unit-testing?, It is the job for developers. The activities are too many, do you mean we should do them all? Basically, every member(including customers) should participate in writing business test scenario and evolved during the whole project period. Who is in charge to write the draft depends on project’s conditions Well, Unit-testing is the developer’s job. That’s right. However as quality people, we can check anywhere the quality exists, and give them feedback with our profession, quality. Moreover we can learn much about product internally. Of course not. The message is just delivering co-work with dev. Each person(testers) has different background, skills, interests, I just wanted to Introduce various activities and make them as one, each career path
  53. 53. T.H.A.N.K Y.O.U

×