Slides from my Acceptance & Functional Testing with Codeception given at SunshinePHP 2016

1. Acceptance & Functional
Testing with Codeception
Joe Ferguson
February 6th 2015

2. Who Am I?
Joe Ferguson
PHP Developer
Twitter: @JoePFerguson
Organizer of @MemphisPHP
@NomadPHP Lightning Talks
Passionate about Community

3. One late night coding session
of fighting with tests…

4. I found

5. hey, I’m everyone…

6. Example Test
Sample taken from http://codeception.com

7. You gotta be…
…me!

8. Codeception
• Selenium WebDriver integration
• Elements matched by name, CSS, XPath
• Symfony2, Laravel, Yii, Phalcon, Zend Framework
• PageObjects and StepObjects included
• BDD-style readable tests
• Powered by PHPUnit
• API testing: REST,SOAP,XML-RPC
• Facebook API testing
• Data Cleanup
• HTML, XML, TAP, JSON reports
• CodeCoverage and Remote CodeCoverage
• Parallel Execution

9. Types of Testing
Unit
Functional
Acceptance

10. Unit Testing
In computer programming, unit testing is a software
testing method by which individual units of source
code, sets of one or more computer program
modules together with associated control data, usage
procedures, and operating procedures, are tested to
determine whether they are fit for use.
https://en.wikipedia.org/wiki/Unit_testing

11. Functional Testing
Functional testing is a quality assurance (QA) process and a type
of black box testing that bases its test cases on the specifications of
the software component under test. Functions are tested by
feeding them input and examining the output, and internal
program structure is rarely considered (not like in white-box
testing). Functional testing usually describes what the system
does.
Functional testing does not imply that you are testing a function
(method) of your module or class. Functional testing tests a slice
of functionality of the whole system
https://en.wikipedia.org/wiki/Functional_testing

12. Acceptance Testing
In engineering and its various subdisciplines, acceptance testing is
a test conducted to determine if the requirements of a
specification or contract are met.
In software testing the ISTQB defines acceptance as: formal testing
with respect to user needs, requirements, and business processes
conducted to determine whether or not a system satisfies the
acceptance criteria and to enable the user, customers or other
authorized entity to determine whether or not to accept the
system.
Acceptance testing is also known as user acceptance testing (UAT),
end-user testing, operational acceptance testing (OAT) or field
(acceptance) testing.
https://en.wikipedia.org/wiki/Acceptance_testing

13. How I think about types
Unit tests test your functions and methods.
Functional tests test parts of your application.
Acceptance tests test your interactions.

14. Isn’t unit testing enough?

15. It might be, how high is
your coverage?

16. How do you know
methods work together?

17. Functional tests allow you to
test individual scenarios in
your application

18. Acceptance tests allow you to
test the interactions in your
application

19. Where do we start?

20. Install Codeception

21. Eh…wait a minute…

22. Better way to install*
*http://blog.doh.ms/2014/10/13/installing-composer-packages

23. Add to PATH

24. Bootstrap Codeception

25. Configure Codeception

26. Run Codeception

27. Add Laravel5 Helper

28. Codecept build

29. Our First Test

30. Login Form

31. Login Form Template

32. Generate Functional
Test

33. Writing our test

34. Click on what?
Click on ‘Copy CSS Path’

35. How to run tests
codecept run // runs all tests
codecept run functional // run functional tests
codecept run functional tests/functional/
AdminCanLoginCept.php // run single test
Can also add -vv or -vvv for detailed output

36. Run our test

37. What are we testing?
The route ‘auth/login’ shows us a login form
Submitting login form performs some action
Dashboard (after successful login) displays
Some success text exists “Hello admin!”

38. What are we NOT testing?
Navigation on the page
“Remember Me” functionality works
Password Reset works
User is actually an admin

39. Functional testing tests
a slice of functionality
of the whole system

40. We don’t care about
Navigation on the page
“Remember Me” functionality works
Password Reset works
User is actually an admin

41. We care about
$I->wantTo('ensure an admin can log in');

42. What if our test fails?

43. Update our password

44. Run Our test

45. Failed tests output

46. Test will pass when…
the user exists in the database
the routes (get & post) work
login form view renders properly
dashboard view renders properly

47. Framework Modules

48. Using Laravel5 Module
amOnAction is translated to the /auth/login url

49. Test Passes

50. Check the DB for user
seeRecord() will fail if the user does not exist

51. User is in the database

52. Check for errors
Useful for checking that error messages are returned
dontSee() ensures text does not exist on view

53. Testing your API

54. Generate API Suite
api.suite.yml

55. Test API Create User
http://codeception.com/docs/10-WebServices

56. Now we know slices of
our application work

57. Acceptance Testing

58. Testing Real(ish) things

59. Using a real(ish)
browser

60. Add PhantomJS

61. Configure
acceptance.suite.yml

62. Export your Database
php artisan migrate
php artisan db:seed
mysqldump -u homestead -psecret homestead > tests/_data/dump.sql

63. Our 1st acceptance test

64. Test passes

65. That looks a lot like our
functional test!

67. Functional Test runs
via PhpBrowser

68. Acceptance Test runs
via PhantomJS

69. Testing Elements via
XPath

70. Use XPath to submit
form

71. /html/body/div/div/div/div/
div[2]/form/div[4]/div/button

72. Even more specific

73. Checking specific
elements

74. Using XPath for form fields
isn’t very easy to read.
XPath isn’t very easy to
read…

75. When to use XPath?
When you can’t add an ID or Class on an
element
When you want to ensure styles/formatting
(error messages, flash messages, etc)
When testing navigation elements
When testing any dynamic elements

76. Acceptance failures
come with screenshots!

77. Log in and look for
content that doesn’t exist

78. Test Failed!

79. We see our content does
not exist on the page

80. Some issues…
PhantomJS pretty broken on Travis-CI
Acceptance tests are SLOW (and memory hogs)
Acceptance tests aren’t unit tests, it can be hard
to find WHAT is causing the tests to fail
Functional & Acceptance tests aren’t
replacements for unit tests, but can be easier to
write if your app isn’t written to be testable

81. Codeception Resources
http://www.codeception.com
http://chrislema.com/acceptance-testing-
wordpress-codeception
Writing Acceptance Tests:
https://vimeo.com/113466213

82. Testing Resources
https://leanpub.com/grumpy-testinghttps://leanpub.com/grumpy-phpunit
http://grumpy-learning.com

83. Questions?

84. Feedback!
https://joind.in/talk/4c449
Joe Ferguson
Twitter: @JoePFerguson
Email: joe@joeferguson.me
Freenode: joepferguson
Contact Info: