Foreword Testing is an incredibly
complex subject. You won't become fu| |—f| edged testers by sitting here. This is no tutorial for writing tests for your individual WordPress plugin/ theme. Instead, you will be briefly introduced to several relevant topics of software testing. If you have more questions after this talk than you have now, I reached my goal. Sorry. (V)
Reasons for Testing 0 Software
contains defects. They hide. o Defects can cause software failures. 0 Failures can cost money, and even be mortal! 0 Testing assures the quality of the software. 0 Testing accelerates software development. 0 And much, much more.
Ariane—5 0 Inertial navigation software
taken from Ariane—4. Untested. o All other systems thoroughly tested component—by—component. o Ariane—5 had a different trajectory than Ariane 4. 0 Converting 64-bit f| oating—point data into 16-bit unsigned integer values. —> Arithmetic overflow. 0 There was an exception handler for that. It had been disabled. 0 Not even 40 seconds after launch, Ariane—5 literally se| f—destructed. Successfully.
/‘J . g fz, ,,
-‘ t ‘/1 g Inverted Flight r A developer of the US Air Force improved a program for an unmanned rocket: , : r. when crossing the equator, flip the coordinates's leading sign. ,1-’ r The program therefore needed less memory. c The rocket also made a 180—degrees roll. So what? r The program later was used for the autopilot of an F-18 fighterjet. I c When crossing the equator, the pilot certainly was somewhat suprised.
Heartbleed «r Heartbeat Extension of
TLS is used to keep Datagram TLS sessions open. ~: Simple request and response scheme: r "Send me back the following (padded) string which is n bytes long. ,, An attackerjust had to request a long string, while telling it is short. -: Other party responded with short string, then leaked potentially confidential data. if it
Do Software Testing. Right. «r
Testing has to be planned. -' Testing costs easily up to 40 % of a project's budget. c Testing has to be performed in a reasonable way. 6 Testing shall be independent and objective. Testing has to be managed. Software testing is a fundamental part of professional software development!
Testing The process consisting of
all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. —| STOB®
Requirements and Specification The requirements
document is the input to a development phase. The specification document is the ouput of a development phase. The specification document resulting from development phase X serves as requirements document for development phase X+1. Requirements and specification are the same, depending on the usage and point of view. The product is 100 % specification.
Validation The process of evaluating
a system or component during or at the end of the development process to determine whether it satisfies specified requirements. -IEEE Std 610.12—1990 ’$4re we building the right product? ,,
Verification The process of evaluating
a system or component to determine whether the system ofa given development phase satisfies the conditions imposed at the start of that phase. -IEEE Std 610.12—1990 ’$4re we building the product righ t? ,,
Static Analysis 1* Analyze code
(e. g., control flow, data flow), and generated output. -: Typical use cases for static analysis: r syntax checking; n code smell detection; c coding standards compliance. ~? Typical defects found by static analysis: r syntax violations; C unreachable code; r overly complicated constructs.
Reviews r Different types of
reviews: r informal review (no formal process; inexpensive); r walkthrough (train colleagues and users; gain understanding); n technical review (documented, defined process; discuss); r. inspection (formal process; gain metrics). -r Success factors: r clear predefined objectives; r defects found are welcomed and expressed objectively; r application of suitable review techniques; r management supports review process; r emphasis on learning and process improvement.
Black-box Testing «' Specification—based testing.
6 Test functionality by observing external behavior. I: No knowledge of internals (required). -: Different black-box testing techniques: r equivalence partitioning; r boundary value analysis; c decision table testing; F7
White-box Testing ~ Structure—based testing.
it Close examination of procedural level of detail. . Knowledge of internals required. ; Different white-box testing techniques: r statement testing; r decision testing; r. (multiple) condition testing; [7
System Testing Test complete, integrated
system. 7 Evaluate system compliance with specified requirements. Stress, performance, usability etc. testing. . Done by (external) testers. .. In general, black-box tests. Additional white-box tests possible.
Acceptance Testing Test complete, integrated
system. Evaluate system compliance with specified acceptance criteria. v: May be performed at various times during development. c Done by customers/ users. Only black-box tests.
Static Testing -:2 PHP_CodeSniffer: c
Dynamic Testing PHPUHIIZ r unit
Dynamic Testing c Mockery: r
PHP mock object framework for use in unit testing. 6 Prophecy: r object mocking framework for PHP. r Phake: r PHP mocking framework. C WP_Mock: r API mocking framework for unit testing within WordPress. <‘ Brain Monkey: r mocking utility for PHP functions and WordPress plugin API.
Continuous Integration I Automated activities:
r execute static code analysis; c execute unit tests, and check code coverage; F deploy to test environment; I“. execute integration tests; c Benefits: r earlier detection and analysis of conflicting changes; regular feedback on whether the code is working; no big-bang integration; reduces repetitive manual testing activities; F1 F1 F1 71
Travis CI r Hosted continuous
integration server. Single requirement: GitHub account. vi Getting started: r sign in with your GitHub account; r accept GitHub access permissions confirmation; r set up . travis . yml file for your repositories; c enable Travis CI builds for individual repositories; c develop continuously integrated software. .. / ** @see travis-ci. org */
ISTQB® Certified Tester Most widespread
qualification scheme in the world. Syllabi contents and glossary de facto industry reference. Test Management Improving the Testing Process 1-es‘ 5""‘<‘9'< W"-'9°'"“"‘ Automation Security Testing Opcmiiomil Tusl lilipleiliciilinq 1031 Engineering , Fim, M,.0. Z0. 5, Mnmiqonwnt Pmcmc. inupmwnwni ll/ aiiagirig the rim Team Assessing Test F’ioLenes Test Manager Test Analyst Tecxggigigest Agile Model Tester Based Testing II‘. II M 0 Lu k) Z < > D <( ISTQB Glossary Foundation FOUNDAT ON / ** @see www. istqb. org */
Lessoris Learned 0 Software testing
is important! So make it a topic. I Use (the right) tools, such as a decent IDE. 0 Do code reviews. 0 Write unit tests. continue with integration tests. I Automate!