• Save
12 05 ipc_se_testen_state_of_the_art
Upcoming SlideShare
Loading in...5
×
 

12 05 ipc_se_testen_state_of_the_art

on

  • 273 views

Automatisiertes Testen ist zum Standard in der PHP-Welt avanciert. Aber ist Ihre Test-Suite immer noch State of the Art? Nutzen Sie die neuesten Features, um sich das Testleben einfacher zu machen? ...

Automatisiertes Testen ist zum Standard in der PHP-Welt avanciert. Aber ist Ihre Test-Suite immer noch State of the Art? Nutzen Sie die neuesten Features, um sich das Testleben einfacher zu machen? Sind Ihre Tests verlässlich? Lernen Sie, was en vogue ist in der PHP-Testwelt im Jahr 2012.

http://talks.qafoo.com

Statistics

Views

Total Views
273
Views on SlideShare
273
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

12 05 ipc_se_testen_state_of_the_art 12 05 ipc_se_testen_state_of_the_art Presentation Transcript

  • Testen State of the Art IPC Spring 2012 Manuel Pichler (@manuelp) Tobias Schlitt (@tobySen) 2012-06-04Testen State of the Art 1 / 40
  • About us Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Open source enthusiasts Contributing to various FLOSS projectsTesten State of the Art 2 / 40
  • About us Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Open source enthusiasts Contributing to various FLOSS projectsTesten State of the Art 2 / 40
  • About us Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Open source enthusiasts Contributing to various FLOSS projectsTesten State of the Art 2 / 40
  • About us Co-founders of Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Open source enthusiasts Contributing to various FLOSS projectsTesten State of the Art 2 / 40
  • About us Co-founders of Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Helping people to create Open source enthusiasts high quality web Contributing to various applications. FLOSS projectsTesten State of the Art 2 / 40
  • About us Co-founders of Manuel & Toby Degree in computer sience More than 10 years of Qafoo passion for software quality professional PHP Helping people to create Open source enthusiasts high quality web Contributing to various applications. FLOSS projects http://qafoo.comTesten State of the Art 2 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 3 / 40
  • PHPUnit schematics TestRunner MyComponentSuite TestSuite MyClassTest TestCase MyClassTesten State of the Art 4 / 40
  • PHPUnit schematics TestRunner MyComponentSuite TestSuite MyClassTest TestCase MyClassTesten State of the Art 4 / 40
  • Test suites File system based phpunit.dist.xml / phpunit.xml @group annotationsTesten State of the Art 5 / 40
  • phpunit.xml In the dir you execute $ phpunit phpunit.xml available? phpunit.dist.xml available? $ phpunit -c my-phpunit.xmlTesten State of the Art 6 / 40
  • phpunit.xml 1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ” 2 colors=” true ” 3 convertErrorsToExceptions=” true ” 4 convertNoticesToExceptions= ” true ” 5 convertWarningsToExceptions = ” t r u e ” 6 > 7 < groups> 8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude > 9 < / groups>1011 <t e s t s u i t e s>12 < t e s t s u i t e name= ” A l l Tests ” >13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >14 </ t e s t s u i t e>15 </ t e s t s u i t e s>1617 <filter>18 <w h i t e l i s t>19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >20 < exclude >21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >22 < / exclude >23 </ w h i t e l i s t>24 </ f i l t e r >25 < / p h p u n i t>Testen State of the Art 7 / 40
  • phpunit.xml 1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ” 2 colors=” true ” 3 convertErrorsToExceptions=” true ” 4 convertNoticesToExceptions= ” true ” 5 convertWarningsToExceptions = ” t r u e ” 6 > 7 < groups> 8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude > 9 < / groups>1011 <t e s t s u i t e s>12 < t e s t s u i t e name= ” A l l Tests ” >13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >14 </ t e s t s u i t e>15 </ t e s t s u i t e s>1617 <filter>18 <w h i t e l i s t>19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >20 < exclude >21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >22 < / exclude >23 </ w h i t e l i s t>24 </ f i l t e r >25 < / p h p u n i t>Testen State of the Art 7 / 40
  • phpunit.xml 1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ” 2 colors=” true ” 3 convertErrorsToExceptions=” true ” 4 convertNoticesToExceptions= ” true ” 5 convertWarningsToExceptions = ” t r u e ” 6 > 7 < groups> 8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude > 9 < / groups>1011 <t e s t s u i t e s>12 < t e s t s u i t e name= ” A l l Tests ” >13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >14 </ t e s t s u i t e>15 </ t e s t s u i t e s>1617 <filter>18 <w h i t e l i s t>19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >20 < exclude >21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >22 < / exclude >23 </ w h i t e l i s t>24 </ f i l t e r >25 < / p h p u n i t>Testen State of the Art 7 / 40
  • phpunit.xml 1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ” 2 colors=” true ” 3 convertErrorsToExceptions=” true ” 4 convertNoticesToExceptions= ” true ” 5 convertWarningsToExceptions = ” t r u e ” 6 > 7 < groups> 8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude > 9 < / groups>1011 <t e s t s u i t e s>12 < t e s t s u i t e name= ” A l l Tests ” >13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >14 </ t e s t s u i t e>15 </ t e s t s u i t e s>1617 <filter>18 <w h i t e l i s t>19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >20 < exclude >21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >22 < / exclude >23 </ w h i t e l i s t>24 </ f i l t e r >25 < / p h p u n i t>Testen State of the Art 7 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Useful Annotations @covers @group @ticket @dataProvider @expectedException* @codeCoverageIgnore* @dependsTesten State of the Art 8 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 9 / 40
  • A Minimal File Upload 1 <?php 2 class UploadExample 3 { 4 protected $dest ; 5 public function c o n s t r u c t ( $dest ) 6 { 7 $ t h i s −> d e s t = r t r i m ( $dest , ’ / ’ ) . ’ / ’ ; 8 } 9 public function handle ( $name )10 {11 i f ( i s u p l o a d e d f i l e ( $ FILES [ $name ] [ ’ tmp name ’ ] ) ) {12 move uploaded file (13 $ FILES [ $name ] [ ’ tmp name ’ ] ,14 $ t h i s −> d e s t . $ FILES [ $name ] [ ’ name ’ ]15 );16 }17 }18 }Testen State of the Art 10 / 40
  • A Minimal File Upload 1 <?php 2 class UploadExample 3 { 4 protected $dest ; 5 public function c o n s t r u c t ( $dest ) 6 { 7 $ t h i s −> d e s t = r t r i m ( $dest , ’ / ’ ) . ’ / ’ ; 8 } 9 public function handle ( $name )10 {11 i f ( i s u p l o a d e d f i l e ( $ FILES [ $name ] [ ’ tmp name ’ ] ) ) {12 move uploaded file (13 $ FILES [ $name ] [ ’ tmp name ’ ] ,14 $ t h i s −> d e s t . $ FILES [ $name ] [ ’ name ’ ]15 );16 }17 }18 }Testen State of the Art 10 / 40
  • PHPT PHP Testing Framework (PHPT) Used to test PHP core $ pear run-tests example-test-case.phpt Integrates into PHPUnit Details: http://qa.php.net/phpt_details.phpTesten State of the Art 11 / 40
  • A Simple PHPT Test 1 −−TEST−− 2 Example t e s t case 3 −−FILE −− 4 <?php 5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ; 6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ; 7 −−EXPECT−− 8 int (7) 9 bool ( false )Testen State of the Art 12 / 40
  • A Simple PHPT Test 1 −−TEST−− 2 Example t e s t case 3 −−FILE −− 4 <?php 5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ; 6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ; 7 −−EXPECT−− 8 int (7) 9 bool ( false )Testen State of the Art 12 / 40
  • A Simple PHPT Test 1 −−TEST−− 2 Example t e s t case 3 −−FILE −− 4 <?php 5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ; 6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ; 7 −−EXPECT−− 8 int (7) 9 bool ( false )Testen State of the Art 12 / 40
  • File Uploads with PHPT 1 −−TEST−− 2 Example t e s t e m u l a t i n g a f i l e upload 3 −−POST RAW−− 4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 5 6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ” 8 Content −Type : t e x t / p l a i n 910 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g1112 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”1415 Upload16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−17 −−FILE −−18 <?php19 require DIR . ’ / UploadExample . php ’ ;2021 $upload = new UploadExample ( ’ / tmp ’ ) ;22 $upload −> handle ( ’ f i l e ’ ) ;2324 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;25 ?>26 −−EXPECT−−27 bool ( true )Testen State of the Art 13 / 40
  • File Uploads with PHPT 1 −−TEST−− 2 Example t e s t e m u l a t i n g a f i l e upload 3 −−POST RAW−− 4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 5 6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ” 8 Content −Type : t e x t / p l a i n 910 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g1112 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”1415 Upload16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−17 −−FILE −−18 <?php19 require DIR . ’ / UploadExample . php ’ ;2021 $upload = new UploadExample ( ’ / tmp ’ ) ;22 $upload −> handle ( ’ f i l e ’ ) ;2324 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;25 ?>26 −−EXPECT−−27 bool ( true )Testen State of the Art 13 / 40
  • File Uploads with PHPT 1 −−TEST−− 2 Example t e s t e m u l a t i n g a f i l e upload 3 −−POST RAW−− 4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 5 6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ” 8 Content −Type : t e x t / p l a i n 910 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g1112 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”1415 Upload16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−17 −−FILE −−18 <?php19 require DIR . ’ / UploadExample . php ’ ;2021 $upload = new UploadExample ( ’ / tmp ’ ) ;22 $upload −> handle ( ’ f i l e ’ ) ;2324 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;25 ?>26 −−EXPECT−−27 bool ( true )Testen State of the Art 13 / 40
  • File Uploads with PHPT 1 −−TEST−− 2 Example t e s t e m u l a t i n g a f i l e upload 3 −−POST RAW−− 4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 5 6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn 7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ” 8 Content −Type : t e x t / p l a i n 910 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g1112 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”1415 Upload16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−17 −−FILE −−18 <?php19 require DIR . ’ / UploadExample . php ’ ;2021 $upload = new UploadExample ( ’ / tmp ’ ) ;22 $upload −> handle ( ’ f i l e ’ ) ;2324 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;25 ?>26 −−EXPECT−−27 bool ( true )Testen State of the Art 13 / 40
  • PHPT within PHPUnit 1 <ph punit > 2 <testsuites > 3 < t e s t s u i t e name= ”PHPT” > 4 < d i r e c t o r y s u f f i x = ” . phpt ” > . / < / d i r e c t o r y > 5 </ t e s t s u i t e > 6 </ t e s t s u i t e s > 7 </ p h p u n i t >Testen State of the Art 14 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 15 / 40
  • Behavior Driven Development Behavior Driven Development (BDD) Deeper coupling between Business Analysis Quality Assurance Basic ideas: Use cases as requirements Make use cases executable Verify functionality Advantages: Communication Documentation Short Human readable testsTesten State of the Art 16 / 40
  • Behavior Driven Development Behavior Driven Development (BDD) Deeper coupling between Business Analysis Quality Assurance Basic ideas: Use cases as requirements Make use cases executable Verify functionality Advantages: Communication Documentation Short Human readable testsTesten State of the Art 16 / 40
  • Behavior Driven Development Behavior Driven Development (BDD) Deeper coupling between Business Analysis Quality Assurance Basic ideas: Use cases as requirements Make use cases executable Verify functionality Advantages: Communication Documentation Short Human readable testsTesten State of the Art 16 / 40
  • Behavior Driven Development Behavior Driven Development (BDD) Deeper coupling between Business Analysis Quality Assurance Basic ideas: Use cases as requirements Make use cases executable Verify functionality Advantages: Communication Documentation Short Human readable testsTesten State of the Art 16 / 40
  • The BDD process 1. Define a Domain Specific Language (DSL) 2. Describe Use Cases 3. Implement against Use Case executionTesten State of the Art 17 / 40
  • Behat BDD framework for PHP http://behat.org/ Inspired by Ruby’s Cucumber Getting started: http://docs.behat.org/quick_intro.htmlTesten State of the Art 18 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Defining Features 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 19 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {48 /∗ ∗49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /50 ∗/51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )52 {53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;5455 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;56 $ s e c t i o n C r e a t e −>name = $name ;57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;5859 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e );60 }112 }Testen State of the Art 20 / 40
  • When . . . 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 21 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {69 /∗ ∗70 ∗ @When / ˆ I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /71 ∗/72 public function i D e l e t e S e c t i o n W i t h I d e n t i f i e r ( $ i d e n t i f i e r )73 {74 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;7576 try77 {78 $ s e c t i o n = $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;79 $ s e c t i o n S e r v i c e −> d e l e t e S e c t i o n ( $ s e c t i o n ) ;80 }81 catch ( E x c e p t io n $e )82 {83 $ t h i s −> runtimeData [ ’ e x c e p t i o n ’ ] = $e ;84 }85 }112 }Testen State of the Art 22 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 {69 /∗ ∗70 ∗ @When / ˆ I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /71 ∗/72 public function i D e l e t e S e c t i o n W i t h I d e n t i f i e r ( $ i d e n t i f i e r )73 {74 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;7576 try77 {78 $ s e c t i o n = $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;79 $ s e c t i o n S e r v i c e −> d e l e t e S e c t i o n ( $ s e c t i o n ) ;80 }81 catch ( E x c e p t io n $e )82 {83 $ t h i s −> runtimeData [ ’ e x c e p t i o n ’ ] = $e ;84 }85 }112 }Testen State of the Art 22 / 40
  • Then . . . 1 Feature : 2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API 3 4 Scenario : Admin user can d e l e t e a s e c t i o n 5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ” 6 And I am logged i n as user ” admin ” 7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ” 8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s 910 Scenario : Anonymous user cannot d e l e t e a s e c t i o n11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”12 And I am logged i n as user ” anonymous ”13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”Testen State of the Art 23 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 { 87 /∗ ∗ 88 ∗ @Then / ˆ I expect no s e c t i o n f o r i d e n t i f i e r ” ( [ ˆ ” ] + ) ” e x i s t s $ / 89 ∗/ 90 public function i E x p e c t N o S e c t i o n F o r I d e n t i f i e r E x i s t s ( $ i d e n t i f i e r ) 91 { 92 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ; 93 try 94 { 95 $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ; 96 throw new RuntimeException ( ” I d e n t i f i e r { $ i d e n t i f i e r } s t i l l exists . ” ) ; 97 } 98 catch ( eZ P u b l i s h API R e p o s i t o r y E x c ep t i o ns NotFoundException $e ) 99 {100 }101 }112 }Testen State of the Art 24 / 40
  • Implementing the DSL16 class F ea tu re Co nt ex t extends BehatContext17 { 87 /∗ ∗ 88 ∗ @Then / ˆ I expect no s e c t i o n f o r i d e n t i f i e r ” ( [ ˆ ” ] + ) ” e x i s t s $ / 89 ∗/ 90 public function i E x p e c t N o S e c t i o n F o r I d e n t i f i e r E x i s t s ( $ i d e n t i f i e r ) 91 { 92 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ; 93 try 94 { 95 $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ; 96 throw new RuntimeException ( ” I d e n t i f i e r { $ i d e n t i f i e r } s t i l l exists . ” ) ; 97 } 98 catch ( eZ P u b l i s h API R e p o s i t o r y E x c ep t i o ns NotFoundException $e ) 99 {100 }101 }112 }Testen State of the Art 24 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 25 / 40
  • Automated Acceptance Tests High level system tests Assert user expectations Good start for refactoring Approaches HTTP spider (e.g. Goutte) Browser controller (e.g. Selenium, Sahi) Browser emulator (e.g. Zombie.js)Testen State of the Art 26 / 40
  • Mink Acceptance Test Framework http://mink.behat.org Abstraction for underlying tools Full integration into BehatTesten State of the Art 27 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Testing a Registration Form 1 Feature : R e g i s t r a t i o n 2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r 3 As a normal user 4 I must perform a r e g i s t r a t i o n 5 6 Scenario : Test a d e f a u l t r e g i s t r a t i o n 7 Given I am on ” / r e g i s t e r ” 8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ” 9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”13 And I f i l l in ” register personal password ” with ” sindelfingen ”14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”18 And I f i l l i n ” zipcode ” w i t h ”45886”19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”20 And I s e l e c t ” 2 ” from ” c o u n t r y ”21 And I press ” r e g i s t e r b u t t o n ”22 Then I should see ” Willkommen , Tobias S c h l i t t ”Testen State of the Art 28 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 29 / 40
  • Testing JavaScript You test PHP . . . . . . do you test JavaScript?Testen State of the Art 30 / 40
  • js-test-driver Test runner for JavaScript https://code.google.com/p/js-test-driver/ Test JS inside browsers Run JS tests from command line No HTML requiredTesten State of the Art 31 / 40
  • Simple JS Code 1 var H e l l o = function ( ) { } ; 2 Hello . prototype = { 3 say : function ( who ) { 4 i f ( who === u n d e fi n e d ) { 5 who = ” World ” ; 6 } 7 r e t u r n ” H e l l o ” + who + ” ! ” ; 8 } 9 }Testen State of the Art 32 / 40
  • Simple JS Code 1 var H e l l o = function ( ) { } ; 2 Hello . prototype = { 3 say : function ( who ) { 4 i f ( who === u n d e fi n e d ) { 5 who = ” World ” ; 6 } 7 r e t u r n ” H e l l o ” + who + ” ! ” ; 8 } 9 }Testen State of the Art 32 / 40
  • Simple JS Code 1 var H e l l o = function ( ) { } ; 2 Hello . prototype = { 3 say : function ( who ) { 4 i f ( who === u n d e fi n e d ) { 5 who = ” World ” ; 6 } 7 r e t u r n ” H e l l o ” + who + ” ! ” ; 8 } 9 }Testen State of the Art 32 / 40
  • Simple JS Code 1 var H e l l o = function ( ) { } ; 2 Hello . prototype = { 3 say : function ( who ) { 4 i f ( who === u n d e fi n e d ) { 5 who = ” World ” ; 6 } 7 r e t u r n ” H e l l o ” + who + ” ! ” ; 8 } 9 }Testen State of the Art 32 / 40
  • A Simple Test Case 1 TestCase ( ” H e l l o o b j e c t t e s t s ” , { 2 ” setUp ” : function ( ) { 3 t h i s . h e l l o = new H e l l o ( ) ; 4 }, 5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) { 6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ; 7 }, 8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) { 9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;10 }11 }) ;Testen State of the Art 33 / 40
  • A Simple Test Case 1 TestCase ( ” H e l l o o b j e c t t e s t s ” , { 2 ” setUp ” : function ( ) { 3 t h i s . h e l l o = new H e l l o ( ) ; 4 }, 5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) { 6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ; 7 }, 8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) { 9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;10 }11 }) ;Testen State of the Art 33 / 40
  • A Simple Test Case 1 TestCase ( ” H e l l o o b j e c t t e s t s ” , { 2 ” setUp ” : function ( ) { 3 t h i s . h e l l o = new H e l l o ( ) ; 4 }, 5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) { 6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ; 7 }, 8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) { 9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;10 }11 }) ;Testen State of the Art 33 / 40
  • A Simple Test Case 1 TestCase ( ” H e l l o o b j e c t t e s t s ” , { 2 ” setUp ” : function ( ) { 3 t h i s . h e l l o = new H e l l o ( ) ; 4 }, 5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) { 6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ; 7 }, 8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) { 9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;10 }11 }) ;Testen State of the Art 33 / 40
  • Configuring js-test-driver 1 s e r v e r : h t t p : / / l o c a l h o s t :4223 2 3 load : 4 − src / hello . j s 5 − tests / hello . jsTesten State of the Art 34 / 40
  • Configuring js-test-driver 1 s e r v e r : h t t p : / / l o c a l h o s t :4223 2 3 load : 4 − src / hello . j s 5 − tests / hello . jsTesten State of the Art 34 / 40
  • Running the Tests First shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223 Capture browser http://localhost:4223/capture Second shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l 2 s e t t i n g runnermode QUIET 3 .... 4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms) 5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms) 6 Chrome 18 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)Testen State of the Art 35 / 40
  • Running the Tests First shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223 Capture browser http://localhost:4223/capture Second shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l 2 s e t t i n g runnermode QUIET 3 .... 4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms) 5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms) 6 Chrome 1 8 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)Testen State of the Art 35 / 40
  • Running the Tests First shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223 Capture browser http://localhost:4223/capture Second shell 1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l 2 s e t t i n g runnermode QUIET 3 .... 4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms) 5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms) 6 Chrome 18 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)Testen State of the Art 35 / 40
  • More on JS Testing JavaScript QA: js-test-driver und Sinon.JS richtig nutzen Jakob Westhoff (Qafoo) Wednesday, 14:00, Salon 2Testen State of the Art 36 / 40
  • Outline Unit Testing Testing File Uploads Behavior Driven Development Automated Acceptance Tests Testing JavaScript OutlookTesten State of the Art 37 / 40
  • Unified Build Process https://github.com/manuelpichler/build-commons Framework for build scripts Unified targets, like test package verify deploy 39 supported tool extensions PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . . DB, DBDeploy, PHAR, Pirum, . . . js-test-driver, JSLint, JSHint, . . . ...Testen State of the Art 38 / 40
  • Unified Build Process https://github.com/manuelpichler/build-commons Framework for build scripts Unified targets, like test package verify deploy 39 supported tool extensions PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . . DB, DBDeploy, PHAR, Pirum, . . . js-test-driver, JSLint, JSHint, . . . ...Testen State of the Art 38 / 40
  • Unified Build Process https://github.com/manuelpichler/build-commons Framework for build scripts Unified targets, like test package verify deploy 39 supported tool extensions PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . . DB, DBDeploy, PHAR, Pirum, . . . js-test-driver, JSLint, JSHint, . . . ...Testen State of the Art 38 / 40
  • Continuous Performance Does your app still perform? Find performance regressions Ensure optimizations persist Apache JMeter https://jmeter.apache.org/ Complex user paths Concurrend requests Record on proxy Clustering Integrate into Continuous Integration Immer am Optimum, Continuous Performance Wednesday, 16:15, Salon 2 Kore Nordmann, Manuel PichlerTesten State of the Art 39 / 40
  • Continuous Performance Does your app still perform? Find performance regressions Ensure optimizations persist Apache JMeter https://jmeter.apache.org/ Complex user paths Concurrend requests Record on proxy Clustering Integrate into Continuous Integration Immer am Optimum, Continuous Performance Wednesday, 16:15, Salon 2 Kore Nordmann, Manuel PichlerTesten State of the Art 39 / 40
  • Continuous Performance Does your app still perform? Find performance regressions Ensure optimizations persist Apache JMeter https://jmeter.apache.org/ Complex user paths Concurrend requests Record on proxy Clustering Integrate into Continuous Integration Immer am Optimum, Continuous Performance Wednesday, 16:15, Salon 2 Kore Nordmann, Manuel PichlerTesten State of the Art 39 / 40
  • Continuous Performance Does your app still perform? Find performance regressions Ensure optimizations persist Apache JMeter https://jmeter.apache.org/ Complex user paths Concurrend requests Record on proxy Clustering Integrate into Continuous Integration Immer am Optimum, Continuous Performance Wednesday, 16:15, Salon 2 Kore Nordmann, Manuel PichlerTesten State of the Art 39 / 40
  • Thanks for Listening Please rate this talk at https://joind.in/event/IPC2012 (Slides: http://talks.qafoo.com) Stay in touch Manuel Pichler Tobias Schlitt manuel@qafoo.com toby@qafoo.com @manuelp / @qafoo @tobySen / @qafoo Rent a PHP quality expert: http://qafoo.comTesten State of the Art 40 / 40
  • Thanks for Listening Please rate this talk at https://joind.in/event/IPC2012 (Slides: http://talks.qafoo.com) Stay in touch Manuel Pichler Tobias Schlitt manuel@qafoo.com toby@qafoo.com @manuelp / @qafoo @tobySen / @qafoo Rent a PHP quality expert: http://qafoo.comTesten State of the Art 40 / 40
  • Thanks for Listening Please rate this talk at https://joind.in/event/IPC2012 (Slides: http://talks.qafoo.com) Stay in touch Manuel Pichler Tobias Schlitt manuel@qafoo.com toby@qafoo.com @manuelp / @qafoo @tobySen / @qafoo Rent a PHP quality expert: http://qafoo.comTesten State of the Art 40 / 40