Automated Testing in WordPress, Really?!

Automated Testing in

WordPress,
Really?!
Rate this talk: https://joind.in/10115

#dc4d - Automated Testing in WordPress with @ptahdunbar

Ptah (Pirate) Dunbar

●

Started with WordPress and PHP
in ‘05

●

Contributing developer to
WordPress, BuddyPress, bbPress

●

Full stack Web Developer

●

Architect at LiveNinja.com

●

WPMIA co-organizer and
SoFloPHP member

☠ Became Pirate Dunbar


Agenda

In one hour
● Understand automated testing concepts,
ideas and best practices.
● Learn PHPUnit basics and the WordPress testsuite.
● Resources and homework


WordPress
powers

1 in 5
websites
source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites


WordPress
community

28,510
2,177
source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites


“The result is that a lot of the
plugins are written in poor code
and turn out to be poorly
compatible with other plugins”
— Yoast

http://yoast.com/plugin-future/


Manual Testing

Pull out the tools
●
●
●
●
●

WP_DEBUG
var_dump();
print_r();
error_log();
debug_backtrace();


Manual Testing

Pull out the tools
●
●
●
●
●

WP_DEBUG
var_dump(); Temporary
Ad-hoc &
print_r();
error_log();
debug_backtrace();


Manual Testing

Pull out the tools
●
●
●
●
●

WP_DEBUG
var_dump(); Error Prone
SLOW &
print_r();
error_log();
debug_backtrace();


Manual Testing

Pull out the tools
●
●
●
●
●

WP_DEBUG
var_dump();
Doesn’t scale
print_r();
error_log();
debug_backtrace();


Automated Testing

A scripted process that
invokes your app to test
features and compares the
outcome with expected
results.


Automated Testing

Persistent var_dumps();


Automated Testing

Better than checking the logs


The Bigger Picture

Continuous Integration
vagrant
Phing

Continuous Delivery

BDD

Automated Testing
Scrum
Agile

TDD

Continuous Inspection
Releasing early, releasing often

Automate Testing

Getting started


CHOOSE YOUR FRAMEWORK

There are so many

Frameworks

CHOOSE YOUR FRAMEWORK

PHPUnit
http://phpunit.de/manual/
Sebastian Bergmann


{
"require-dev": {
"phpunit/phpunit": "3.7.*",
"phpunit/phpunit-selenium" : "*",
}
}

http://getcomposer.org

vim composer.json && composer update


PHPUnit

$>./vendor/bin/phpunit


PHPUnit

Terminology


PHPUnit

Terminology
● Test Case
A set of conditions that you set up in order
to assert expected outcome.


PHPUnit

Terminology
● Test Case
● Test Class
A collection of test cases, extends PHPUnit


PHPUnit

Terminology
● Test Case
● Test Class
A collection of test cases, extends PHPUnit
● Test Suite
A collection of test classes


PHPUnit

TEST CLASS
<?php
// test class
class CalTest extends PHPUnit_Framework_TestCase
{
// test case
public function testAddReturnsSumOfTwoPositiveIntegers()
{
// assert stuff.
}
}


PHPUnit
plugin/
loader.php
includes/
admin.php
api.php
…
phpunit.xml
tests/
adminTest.php
ApiTest.php
…
…


PHPUnit
plugin/
loader.php
includes/
admin.php
functions.php
…
phpunit.xml
tests/
integration/
…
acceptance/
…
…


PHPUnit

phpunit.xml - configuration file for PHPUnit
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php">
<testsuites>
<testsuite name="tests">
<directory suffix="Test.php">tests/</directory>
</testsuite>
</testsuites>
</phpunit>


PHPUnit

phpunit.xml
<testsuites>
</testsuite>
</testsuites>
</phpunit>

Configure your test suite location


PHPUnit

phpunit.xml
<testsuites>
<testsuite name="integration">
<directory suffix="Test.php">tests/integration</directory>
</testsuite>
<testsuite name="acceptance">
<directory suffix="Test.php">tests/acceptance</directory>
</testsuite>
</testsuites>
</phpunit>

Configure your test suite location


PHPUnit

phpunit.xml
<testsuites>
</testsuite>
</testsuites>
</phpunit>

Bootstrap file is included before any tests run


PHPUnit

Assertions
Explicitly check expected outcome
agaisnt actual outcome.


PHPUnit

Assertions
Explicitly check expected outcome
agaisnt actual outcome.
$this->assertTrue(condition);


PHPUnit

Arrange, Act, Assert


PHPUnit

function testThatItsTestingTime()
{
1. A
2. A
3. A
}


PHPUnit

{
1. A
2. A
3. Assert (check for the expected behavior)
}


PHPUnit

{
1. A
2. Act (call the method/trigger the action)
}


PHPUnit

{
1. Arrange (the context/dependencies)
2. Act (call the method/trigger the action)
}


PHPUnit

Example


PHPUnit
<?php
{
{
// Arrange

// Act

// Assert
}
}
plugin/tests/unit/calTest.php

PHPUnit
<?php
{
{
// Arrange

// Act

// Assert
$this->assertEquals(3, $result);
}
}

PHPUnit
<?php
{
{
// Arrange

// Act
$result = $calculator->add(1,2);
// Assert
}
}

PHPUnit
<?php
{
{
// Arrange
$calculator = new Calculator();
// Act
// Assert
}
}

./vendor/bin/phpunit

PHPUnit

<?php
{
{
// Arrange
// Act
// Assert
}
}


PHPUnit

<?php
{
{
// Arrange
Time: 148ms, Memory: 2.75Mb
// 1
OK: (1 test, Actassertions)

// Assert

}
}

PHPUnit
<?php
{
{
// Arrange
// Act
// Assert
}
}


PHPUnit

<?php
{
{
// Arrange
// Act
// Assert
}
}


PHPUnit

<?php
{
{
// Arrange

Failed asserting that 4 equals 3

// Act
// Assert
}
}

PHPUnit
<?php
class UserServiceTest extends PHPUnit_Framework_TestCase
{
public function testPersistWithValidUserdataReturnsUserObject()
{
// Arrange

// Act

// Assert
}
//…
}

plugin/tests/unit/LiveNinja/User/ServiceTest.php

PHPUnit
<?php
{
{
// Arrange

// Act

// Assert
$this->assertTrue($user instanceof ‘LiveNinjaUserEntity’);
}
//…
}


PHPUnit
<?php
{
{
// Arrange

// Act
$user = $service->persist($validUserdata);
// Assert
}
//…
}


PHPUnit
<?php
{
{
// Arrange
$service = new LiveNinjaUserService;
$validUserdata = [...];
// Act
// Assert
}
//…
}



PHPUnit

<?php
{
{
// Arrange
// Act
// Assert
}
//…
}



PHPUnit

<?php
{
{
// Arrange


// Act

OK: (1 test, 1 assertions)

// Assert
}
//…
}


PHPUnit
<?php
{
{
// Arrange
// Act
// Assert
$this->assertTrue($user instanceof ‘LiveRacoonsUserEntity’);
}
//…
}



PHPUnit

<?php
{
{
// Arrange
// Act
// Assert
}
//…
}



PHPUnit

<?php
{
{
// Arrange

Failed asserting that false equals true

// Act

// Assert
}
//…
}


PHPUnit

ASSERTIONS
Appendix: http://phpunit.de/manual/3.7/en/appendixes.assertions.html

Use the most specific assertion possible
● assertTrue();

● assertFalse();

● assertEquals();

● assertNotEquals();

● assertContains();

● assertContainsOnly();

● assertGreaterThan();

● assertLessThan();

● assertNotNull();

● assertSame();


PHPUnit

FAIL

There was 1 failure:
1) Tests_Basic::test_readme
readme.html's version needs to be updated to 3.9.
Failed asserting that '3.8' matches expected '3.9'.
/private/tmp/wordpress-tests/tests/phpunit/tests/basic.php:29


FAIL
There was 1 failure:
1) Tests_User_Author::test_get_the_author
Failed asserting that two objects are equal.
--- Expected
+++ Actual
@@ @@
WP_User Object (
'data' => stdClass Object (
'ID' => '3'
'user_login' => 'User 1'
'user_pass' => '$P$BpIqOzMNRGZNy9qxKL/3cCDCMe85o2.'
'user_nicename' => 'user-1'
'user_email' => 'user_2@example.org'
+
'ID' => '2'
+
'user_login' => 'test_author'
+
'user_pass' => '$P$BUdMebxEjJ23.6LbH9ujvVUFBsUuZv/'
+
'user_nicename' => 'test_author'
+
'user_email' => 'user_1@example.org'
'user_url' => ''
'user_registered' => '2013-12-20 15:31:01'
'user_activation_key' => ''
'user_status' => '0'
'display_name' => 'User 1'
+
'display_name' => 'test_author'
)
'ID' => 3
+
'ID' => 2
'caps' => Array (

PHPUnit

PHPUnit
<?php
{
{
// Arrange
// Act
// Assert
$this->assertTrue($user instanceof ‘LiveRacoonUserEntity’);
}
//…
}


PHPUnit
<?php
{
{
// Arrange
// Act
// Assert
$this->assertInstanceOf(‘LiveNinjaUserEntity’, $user);
}
//…
}


<?php
{
{
// Arrange
// Act
// Assert
}
//…
}


PHPUnit

<?php
{
{
// Arrange


// Act


// Assert
}
//…
}


PHPUnit

PHPUnit
<?php
{
public function testPersistWithInvalidUserdataReturnsWPError()
{
// Arrange
$invalidUserdata = [];
// Act
$user = $service->persist($invalidUserdata);
// Assert
$this->assertInstanceOf(‘WP_Error’, $user);
}
//…
}


WordPress Testsuite

WordPress with Tests
http://develop.svn.wordpress.org/trunk/
1858 Tests, 8611 Assertions, 2.59 minutes


WordPress Testsuite


WordPress Testsuite

Getting started


WordPress Testsuite

<phpunit bootstrap="tests/bootstrap-wp.php">
<testsuites>
</testsuite>
</testsuites>
</phpunit>


WordPress Testsuite
<phpunit bootstrap="tests/bootstrap-wp.php">
<testsuites>
</testsuite>
<testsuite name="integration">
<directory suffix="Test.php">integration/</directory>
</testsuite>
</testsuites>
</phpunit>


WordPress Testsuite

<?php
class PluginTest extends PHPUnit_Framework_TestCase
{
// test cases...
}

plugin/tests/integration/PluginTest.php


WordPress Testsuite

<?php
class PluginTest extends WP_UnitTestCase
{
// test cases...
}

plugin/tests/integration/PluginTest.php


WordPress Testsuite

$>./vendor/bin/phpunit


WordPress Testsuite

Run Tests
inside of an isolated

WordPress Environment


WordPress Testsuite
bootstrap.php

Configure

$GLOBALS['wp_tests_options'] = [
'active_plugins' => [
'hello.php',
...
],
'current_theme' => 'kubrick',
...
];

WordPress
Options

WordPress Testsuite

Configure

bootstrap.php

WordPress
Includes

function __muplugins_loaded()
{
// code and stuff.
require_once 'env-debug.php';
}
tests_add_filter('muplugins_loaded', '__muplugins_loaded');


WordPress Testsuite
●

Navigate to site URL (Updates globals)
$this->get_url($url);

●

Test WP_Query for Conditionals (is_page, is_single, is_404)
$this->assertQueryTrue($arg1, $arg2, ...);

●

Test for Errors
$this->assertWPError($thing);

●

Genereate WordPress data fixtures
$this->factory->post->create_and_get();
$this->factory->comment->create_post_comments($pid, 100);
$this->factory->user->create_many(5);
$this->factory->blog->create();
and more…


WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange

// Act

// Assert
}
}

plugin/tests/integration/WPCustomizationTest.php

WordPress Testsuite
<?php
{
// test cases
{
// Arrange

// Act

// Assert
$this->assertQueryTrue( 'is_404' );
}
}


WordPress Testsuite
<?php
{
// test cases
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
// Act
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
}
}



WordPress Testsuite

<?php
{
// test cases
{
// Arrange
// Act
$this->go_to('/2007/');
// Assert
}
}



WordPress Testsuite

<?php
{
// test cases
{
// Arrange

OK: (1// Act 1 assertions)
test,

$this->go_to('/2007/');
// Assert

}
}


WordPress Testsuite
<?php
{
/**
* @dataProvider getRequiredPlugins
*/
function testAllRequiredPluginsAreActive($plugin)
{
// Assert
$this->assertTrue( is_plugin_active($plugin),
sprintf('%s is not activated.', $plugin) );
}
function getRequiredPlugins()
{
return [
[‘hello.php’],
];
}
}


WordPress Testsuite

<?php
{
/**
*/
{
// Assert
}
{
return [
[‘hello.php’],
];
}
}


WordPress Testsuite

<?php
{
/**
*/
{
// Assert
}


{
return [
[‘hello.php’],
];
}
}

WordPress Testsuite
<?php
{
/**
* @dataProvider getWPOptions
*/
function testWPOptionSettingsAreConfigured($option_name, $option_value)
{
// Assert
$this->assertSame($option_value, get_option($option_name));
}
function getWPOptions()
{
return [
[‘home’, ‘http://example.org/wp/’],
[‘siteurl’, ‘http://example.org/’],
];
}
}


WordPress Testsuite

<?php
{
/**
*/
{
// Assert
}
{
return [
];
}
}


WordPress Testsuite

<?php
{
/**
*/
{
// Assert
}


{
return [
];
}
}

http://www.seleniumhq.org/


Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Framework_TestCase
{
protected function setUp()
{

}
public function testUserCanLogInViaTwitter()
{

}
}
plugin/tests/acceptance/ConnectTest.php

Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
{

}
{

}
}

Acceptance Testing
<?php
{
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
{

}
}

Acceptance Testing
<?php
{
{
}
{
$this->open("/");

}
}

Acceptance Testing
<?php
{
{
}
{
$this->open("/");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");

}
}

Acceptance Testing
<?php
{
{
}
{
$this->open("/");
$this->click("css=img[alt="Twitter"]");
}
}

Acceptance Testing
<?php
{
{
}
{
$this->open("/");
$this->assertContains( ‘dashboard’, $this->title() );
}
}

Acceptance Testing
<?php
{
{
}


{
$this->open("/");
$this->assertContains( ‘dashboard’, $this->title() );
}


}

Acceptance
Selenium IDE Plugin

●

Visually navigate throughout your
site and generate a PHPUnit
test case.

●

Download Extension
○ http://www.seleniumhq.
org/projects/ide/

●

Download PHPUnit Formatter
○ https://addons.mozilla.org/enUS/firefox/addon/seleniumide-php-formatters/


How can we be
confident that our tests
cover everything?


Testing boundaries


Testing boundaries

●

(User) Acceptance Testing
○

Verify that all features are done done.

○

Black-box testing, no knowledge of internals.


Testing boundaries

●

○
○

●


Integration Testing
○

Test WordPress settings/configuration;

○

Compatibility between plugins and themes.


Testing boundaries

●

○
○

●


Integration Testing
○
○

●

Test WordPress settings/configuration,
Compatibility between plugins and themes

Unit Testing
○

Test class methods and functions in isolation, zero dependencies

○

Does one “behavoir”


Testing boundaries

●

Verify that all features are done done,
black-box testing, no knowledge of

Acceptance
Testing

internals

●

Integration Testing
Test WordPress settings/configuration,
compatibility between plugins and

Integration Testing

themes

●

Unit Testing

Unit Testing
Test class methods and functions in
isolation, zero dependencies,
does one “behavoir”.


What to tests?

● Test plugin works in various WordPress setups
○ Does it work under multisite?
○ What about a custom content directory?
● Test all code paths in functions and methods
● Test compatiblity between most popular plugins
● Test that default pages exists

#ATWP // Automated Testing in WordPress // @ptahdunbar

What to tests?

● Test for theme support
● Test that post formats contain property elements
● Test any required assets that need to be loaded in
templates
● Test for required elements on a page
● Verify search results template displays search term
● Verify SEO meta tags


What to not tests?

1. WordPress APIs


What to not tests?

1. WordPress APIs
2. PHP language features


What to not tests?

1. WordPress APIs
2. PHP language features
3. Third party vendor code


Getting into the groove



● Build out templates



○ Create HTML/CSS



○ Create HTML/CSS
○ Identify dynamic elements and their data
structure



○ Create HTML/CSS
○ Identify dynamic elements and their data
structure
○ Label them and fill them with dummy data



○ Verbally state your trying to do



○ Verbally explain what the code does



○ Verbally explain what the code does
○ Do this alone or with a fellow dev :)


Get started

“A Walking Skeleton is a tiny implementation of the thinnest
possible slice of real functionality that we can automatically
build, deploy and test end-to-end.”

●

Download WP Skeleton Family
○

https://github.com/ptahdunbar/wp-skeleton-site

○

https://github.com/ptahdunbar/wp-skeleton-plugin

○

https://github.com/ptahdunbar/wp-skeleton-theme


Resources
● Art of Unit Testing (.NET)
○ https://leanpub.com/u/royosherove
○ Udemy Five day course
● #GOOS Book (Java)
● XUnit Test Patterns (Java)
● Grumpy Books (PHP)
○ https://leanpub.com/u/chartjes
● Misko Hevery


Homework!

TODO
● Learn moar PHPUnit features
○ data providers,
○ mocks and stubs
○ wordpress testsuite
● Goal: Write at least 100 assertions!


Automated Testing


Automated Testing
increases your productivity


Automated Testing
facilitates more shipping


Automated Testing
scales with you


Automated Testing
is your professional duty
as a developer


Thank you
Automated Testing in WordPress
Pirate Dunbar
@ptahdunbar
yarr@piratedunbar.com

Rate this talk:
https://joind.in/10115


Automated Testing in WordPress, Really?!

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Automated Testing in WordPress, Really?!

Similar to Automated Testing in WordPress, Really?! (20)

More from Ptah Dunbar

More from Ptah Dunbar (6)

Recently uploaded

Recently uploaded (20)

Automated Testing in WordPress, Really?!