Testen: Je mehr Du es tust,
desto mehr wirst Du es lieben
“I can’t believe we’re still talking about this!”
Testen: Je mehr Du es liebst,
desto mehr wirst Du es tun
“I can’t believe we’re still talking about this!”
Sebastian Bergmann
@s_bergmann
sebastian@thephp.cc
27 years programming experience
19 years PHP experience
Co-Founder of thePHP.cc
Jeffrey A. "jam" McGuire
Evangelist, DevRel Acquia
dev.acquia.com/podcast
@horncologne
jam@acquia.com
horncologne@gmail.com
Acquia Podcast:
dev.acquia.com/podcast
Introduction
Why testing? What’s the problem?
Benefits of testing.
Addressing roadblocks and
objections to testing.
Result!
Outline
Why testing?
“The software team not only has to ensure that it makes a
positive contribution to the business goals; it must also ensure
that it makes as few negative contributions … as possible.” -
Andresen.
Benefits of testing
(business cases)
“Testing solves many problems, the least of which is being
sure the code does what it is supposed to do.”
Benefits of testing
(business cases)
“Testing solves many problems, the least of which is being
sure the code does what it is supposed to do.”
The Phoenix Project: A Novel About
IT, DevOps, and Helping Your
Business Win
Gene Kim, Kevin Behr, George
Spafford
https://en.wikipedia.org/wiki/
The_Phoenix_Project_(novel)
The Four Types of Work
1. Business project
2. IT (Infrastructure/Internal) project
3. Changes (Operational/Maintenance)
4. Unplanned work
TDD & the Four Types of Work
TDD helps focus on delivering
features/value through business and
IT projects.
TDD makes changes (Operational/
Maintenance) faster, more secure
TDD reduces unplanned work
Really?
Yes, really.
Time out!
State-of-play?
Do you even test?
I find your lack of tests disturbing.
Really?
Yes, really.
http://dl.acm.org/citation.cfm?id=952753
SAC '03 Proceedings of the 2003 ACM
symposium on Applied computing
Pages 1135-1139
ACM New York, NY, USA ©2003
ISBN:1-58113-624-2
1
http://dl.acm.org/citation.cfm?id=1159787
ISESE '06 Proceedings of the 2006 ACM/IEEE
international symposium on Empirical software
engineering, Pages 356-363
ACM New York, NY, USA ©2006
ISBN:1-59593-218-6
2
http://dl.acm.org/citation.cfm?id=1070834
IEEE Transactions on Software Engineering
Volume 31 Issue 3, March 2005
Page 226-237
3
http://dl.acm.org/citation.cfm?id=1802420
ISSRE'09 Proceedings of the 20th IEEE
international conference on software reliability
engineering, Pages 81-89
IEEE Press Piscataway, NJ, USA ©2009
ISBN: 978-1-4244-5375-7
4
http://dl.acm.org/citation.cfm?id=776892
ICSE '03 Proceedings of the 25th International
Conference on Software Engineering,
Pages 564-569, IEEE Computer Society
Washington, DC, USA ©2003
ISBN:0-7695-1877-X
5
Pro
1. 18% more functional tests passed
2. More than 2X increase in code quality,

tests served as “auto documentation”
3. more tests == more productivity

more productivity == better quality
4. 20.9% decrease in defects,

decrease in defects found by customers

5. 50% reduction in defect rate,

the test suite “will improve quality over
lifetime of the software system”, “quality
contract” between team members
6. TDD ROI … Show me the money!
Con
1. 16% more development time
2. 15% more development time

3. -

4. 30% more development time

non-iterative testing (improvement
possible)
5. (“minimal development productivity
impact”)



6. -



Summary
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.7.4232
About the Return on Investment of Test-Driven Development (2003)
Matthias M. Müller , Frank Padberg
International Workshop on Economics-Driven Software Engineering
Research EDSER-5
6
Conventional
TDD
Development Quality Assurance
Development
Investment
Life Cycle Benefit
Net Return
Müller / Radberg - TDD Benefit/Cost Ratio Calculation
Photo by CEphoto, Uwe Aranas
“In most cases, a client will not
state how long a comparable
previous system functioned …
Request this information and use it
in relation to test-driven
development.” - Andresen.
Photo by CEphoto, Uwe Aranas
Developers!
This book is your friend!
https://leanpub.com/
SellingTestDrivenProjects
Boehm - Relative cost of a bug-fix
0
40
80
120
160 150X
50X
20X
10X5X1X
Requirements Design Code Developer
Tests
Acceptance
Tests
Operations
Developers!
This book is your friend!
https://leanpub.com/
SellingTestDrivenProjects
Delivering the project
is only the beginning
of delivering value.
“Many software teams present quality assurance as a
separate item in their offer … This gives clients the impression
that it is an element that can be eliminated.” - Andresen.
Sounds convincing … Devs?
Roadblocks,
Questions,
Objections
Getting to testing
Objections &
Questions
1. What can developers get out
testing?
2. How does testing save time and
frustration?
3. But I’m writing the software twice!
That’s twice as much work!
4. What’s the most important tip you
can give about testing?
Objections &
Questions
1. What can developers get out
testing?
2. How does testing save time and
frustration?
3. But I’m writing the software twice!
That’s twice as much work!
4. What’s the most important tip you
can give about testing?
Specification
Documentation
Verification
“Die Hauptaufgabe eines Entwicklers
ist nicht das Schreiben von Code,
sondern das Verstehen eines
Problems. Das Formulieren von Tests,
also der Zielvorgaben, hilft, ein
Problem Schritt für Schritt zu
durchdringen. Die Tests treiben die
Entwicklung also nicht als
Selbstzweck, sondern tun dies, indem
sie die notwendigen Denkprozesse
anstoßen.”
https://thephp.cc/neuigkeiten/2017/02/testen-haelt-mich-von-der-arbeit-ab
Carola
Lilienthal
llsa.de
Carola Lilienthal
class AuctionTest extends PHPUnitFrameworkTestCase
{
private $auction;
protected function setUp()
{
$this->auction = new Auction(
new User(
new Name('Tess Tester'),
new Email('tester@example.com')
),
new Description('...'),
new DateTimeImmutable('2017-04-07'),
new DateTimeImmutable('2017-04-15'),
EUR::fromCents(100)
);
}
// ...
}
class AuctionTest extends PHPUnitFrameworkTestCase
{
// ...
public function testUserCannotBidOnOwnAuction()
{
$this->expectException(
CannotBidOnOwnAuctionException::class
);
$this->auction->bid(
new Bid(
new DateTimeImmutable(‘2017-04-08'),
$this->auction->getUser(),
EUR::fromCents(100)
)
);
}
}
$ phpunit --testdox AuctionTest
PHPUnit 6.0.13 by Sebastian Bergmann and contributors.
Runtime: PHP 7.1.2 with Xdebug 2.5.1
Configuration: /home/sb/example/phpunit.xml
Auction
[x] Has description
[x] Has start date
[x] Has end date
[x] Has initial price
[x] User can bid on auction created by another user
[x] User cannot bid on own auction
[x] User cannot bid on auction that has not yet started
[x] User cannot bid on auction that has already ended
Result! Result!
Introduction
Why testing? What’s the problem?
Benefits of testing.
Addressing roadblocks and
objections to testing.
Result!
Recap
Photo Credits
https://creativecommons.org/licenses/by-nc-sa/2.0/:
Sebastian Bergmann https://www.flickr.com/photos/stuartherbert/
6231499431
Locomotive spec https://www.flickr.com/photos/
12567713@N00/281324130
Steam engine https://www.flickr.com/photos/lofink/445470024
Train maintenance https://www.flickr.com/photos/kheelcenter/
5279863818/
Motorcycle rider https://www.flickr.com/photos/conner395/8339912312
https://creativecommons.org/licenses/by-nc-sa/2.0/:
Köln/Bonn Flughafen night https://www.flickr.com/photos/top10-flickr/
8676295874/
Köln/Bonn Flughafen day https://www.flickr.com/photos/
dirkvorderstrasse/10583451894/
Flughafen Köln/Bonn ceiling https://www.flickr.com/photos/
fridgemonkey/4262843982/
—————
Köln Musical Dome photo by CEphoto, Uwe Aranas https://
commons.wikimedia.org/wiki/File:Cologne_Germany_Musical-
Dome-01.jpg
Developers!
This book is your friend!
https://leanpub.com/
SellingTestDrivenProjects
Carola
Lilienthal
llsa.de
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.7.4232
About the Return on Investment of Test-Driven Development (2003)
Matthias M. Müller , Frank Padberg
International Workshop on Economics-Driven Software Engineering
Research EDSER-5
6
Thank you!
Questions?
Sebastian Bergmann
@s_bergmann

sebastian@thephp.cc

Jeffrey A. "jam" McGuire
@horncologne

jam@acquia.com

horncologne@gmail.com

Testing: the more you do it, the more you'll like it

  • 1.
    Testen: Je mehrDu es tust, desto mehr wirst Du es lieben “I can’t believe we’re still talking about this!”
  • 2.
    Testen: Je mehrDu es liebst, desto mehr wirst Du es tun “I can’t believe we’re still talking about this!”
  • 4.
    Sebastian Bergmann @s_bergmann sebastian@thephp.cc 27 yearsprogramming experience 19 years PHP experience Co-Founder of thePHP.cc
  • 5.
    Jeffrey A. "jam"McGuire Evangelist, DevRel Acquia dev.acquia.com/podcast @horncologne jam@acquia.com horncologne@gmail.com
  • 6.
  • 8.
    Introduction Why testing? What’sthe problem? Benefits of testing. Addressing roadblocks and objections to testing. Result! Outline
  • 9.
    Why testing? “The softwareteam not only has to ensure that it makes a positive contribution to the business goals; it must also ensure that it makes as few negative contributions … as possible.” - Andresen.
  • 10.
    Benefits of testing (businesscases) “Testing solves many problems, the least of which is being sure the code does what it is supposed to do.”
  • 11.
    Benefits of testing (businesscases) “Testing solves many problems, the least of which is being sure the code does what it is supposed to do.”
  • 12.
    The Phoenix Project:A Novel About IT, DevOps, and Helping Your Business Win Gene Kim, Kevin Behr, George Spafford https://en.wikipedia.org/wiki/ The_Phoenix_Project_(novel)
  • 13.
    The Four Typesof Work 1. Business project 2. IT (Infrastructure/Internal) project 3. Changes (Operational/Maintenance) 4. Unplanned work
  • 14.
    TDD & theFour Types of Work TDD helps focus on delivering features/value through business and IT projects. TDD makes changes (Operational/ Maintenance) faster, more secure TDD reduces unplanned work
  • 15.
  • 16.
  • 17.
    I find yourlack of tests disturbing.
  • 18.
  • 19.
    http://dl.acm.org/citation.cfm?id=952753 SAC '03 Proceedingsof the 2003 ACM symposium on Applied computing Pages 1135-1139 ACM New York, NY, USA ©2003 ISBN:1-58113-624-2 1
  • 20.
    http://dl.acm.org/citation.cfm?id=1159787 ISESE '06 Proceedingsof the 2006 ACM/IEEE international symposium on Empirical software engineering, Pages 356-363 ACM New York, NY, USA ©2006 ISBN:1-59593-218-6 2
  • 21.
    http://dl.acm.org/citation.cfm?id=1070834 IEEE Transactions onSoftware Engineering Volume 31 Issue 3, March 2005 Page 226-237 3
  • 22.
    http://dl.acm.org/citation.cfm?id=1802420 ISSRE'09 Proceedings ofthe 20th IEEE international conference on software reliability engineering, Pages 81-89 IEEE Press Piscataway, NJ, USA ©2009 ISBN: 978-1-4244-5375-7 4
  • 23.
    http://dl.acm.org/citation.cfm?id=776892 ICSE '03 Proceedingsof the 25th International Conference on Software Engineering, Pages 564-569, IEEE Computer Society Washington, DC, USA ©2003 ISBN:0-7695-1877-X 5
  • 24.
    Pro 1. 18% morefunctional tests passed 2. More than 2X increase in code quality,
 tests served as “auto documentation” 3. more tests == more productivity
 more productivity == better quality 4. 20.9% decrease in defects,
 decrease in defects found by customers
 5. 50% reduction in defect rate,
 the test suite “will improve quality over lifetime of the software system”, “quality contract” between team members 6. TDD ROI … Show me the money! Con 1. 16% more development time 2. 15% more development time
 3. -
 4. 30% more development time
 non-iterative testing (improvement possible) 5. (“minimal development productivity impact”)
 
 6. -
 
 Summary
  • 25.
    http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.7.4232 About the Returnon Investment of Test-Driven Development (2003) Matthias M. Müller , Frank Padberg International Workshop on Economics-Driven Software Engineering Research EDSER-5 6
  • 26.
    Conventional TDD Development Quality Assurance Development Investment LifeCycle Benefit Net Return Müller / Radberg - TDD Benefit/Cost Ratio Calculation
  • 27.
  • 28.
    “In most cases,a client will not state how long a comparable previous system functioned … Request this information and use it in relation to test-driven development.” - Andresen. Photo by CEphoto, Uwe Aranas
  • 30.
    Developers! This book isyour friend! https://leanpub.com/ SellingTestDrivenProjects
  • 31.
    Boehm - Relativecost of a bug-fix 0 40 80 120 160 150X 50X 20X 10X5X1X Requirements Design Code Developer Tests Acceptance Tests Operations
  • 32.
    Developers! This book isyour friend! https://leanpub.com/ SellingTestDrivenProjects
  • 33.
    Delivering the project isonly the beginning of delivering value. “Many software teams present quality assurance as a separate item in their offer … This gives clients the impression that it is an element that can be eliminated.” - Andresen.
  • 34.
  • 35.
  • 36.
    Objections & Questions 1. Whatcan developers get out testing? 2. How does testing save time and frustration? 3. But I’m writing the software twice! That’s twice as much work! 4. What’s the most important tip you can give about testing?
  • 37.
    Objections & Questions 1. Whatcan developers get out testing? 2. How does testing save time and frustration? 3. But I’m writing the software twice! That’s twice as much work! 4. What’s the most important tip you can give about testing?
  • 38.
  • 39.
  • 40.
  • 41.
    “Die Hauptaufgabe einesEntwicklers ist nicht das Schreiben von Code, sondern das Verstehen eines Problems. Das Formulieren von Tests, also der Zielvorgaben, hilft, ein Problem Schritt für Schritt zu durchdringen. Die Tests treiben die Entwicklung also nicht als Selbstzweck, sondern tun dies, indem sie die notwendigen Denkprozesse anstoßen.” https://thephp.cc/neuigkeiten/2017/02/testen-haelt-mich-von-der-arbeit-ab
  • 42.
  • 43.
  • 44.
    class AuctionTest extendsPHPUnitFrameworkTestCase { private $auction; protected function setUp() { $this->auction = new Auction( new User( new Name('Tess Tester'), new Email('tester@example.com') ), new Description('...'), new DateTimeImmutable('2017-04-07'), new DateTimeImmutable('2017-04-15'), EUR::fromCents(100) ); } // ... }
  • 45.
    class AuctionTest extendsPHPUnitFrameworkTestCase { // ... public function testUserCannotBidOnOwnAuction() { $this->expectException( CannotBidOnOwnAuctionException::class ); $this->auction->bid( new Bid( new DateTimeImmutable(‘2017-04-08'), $this->auction->getUser(), EUR::fromCents(100) ) ); } }
  • 46.
    $ phpunit --testdoxAuctionTest PHPUnit 6.0.13 by Sebastian Bergmann and contributors. Runtime: PHP 7.1.2 with Xdebug 2.5.1 Configuration: /home/sb/example/phpunit.xml Auction [x] Has description [x] Has start date [x] Has end date [x] Has initial price [x] User can bid on auction created by another user [x] User cannot bid on own auction [x] User cannot bid on auction that has not yet started [x] User cannot bid on auction that has already ended
  • 47.
  • 48.
    Introduction Why testing? What’sthe problem? Benefits of testing. Addressing roadblocks and objections to testing. Result! Recap
  • 49.
    Photo Credits https://creativecommons.org/licenses/by-nc-sa/2.0/: Sebastian Bergmannhttps://www.flickr.com/photos/stuartherbert/ 6231499431 Locomotive spec https://www.flickr.com/photos/ 12567713@N00/281324130 Steam engine https://www.flickr.com/photos/lofink/445470024 Train maintenance https://www.flickr.com/photos/kheelcenter/ 5279863818/ Motorcycle rider https://www.flickr.com/photos/conner395/8339912312 https://creativecommons.org/licenses/by-nc-sa/2.0/: Köln/Bonn Flughafen night https://www.flickr.com/photos/top10-flickr/ 8676295874/ Köln/Bonn Flughafen day https://www.flickr.com/photos/ dirkvorderstrasse/10583451894/ Flughafen Köln/Bonn ceiling https://www.flickr.com/photos/ fridgemonkey/4262843982/ ————— Köln Musical Dome photo by CEphoto, Uwe Aranas https:// commons.wikimedia.org/wiki/File:Cologne_Germany_Musical- Dome-01.jpg
  • 50.
    Developers! This book isyour friend! https://leanpub.com/ SellingTestDrivenProjects
  • 51.
  • 52.
    http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.7.4232 About the Returnon Investment of Test-Driven Development (2003) Matthias M. Müller , Frank Padberg International Workshop on Economics-Driven Software Engineering Research EDSER-5 6
  • 53.
    Thank you! Questions? Sebastian Bergmann @s_bergmann sebastian@thephp.cc JeffreyA. "jam" McGuire @horncologne jam@acquia.com horncologne@gmail.com