SmokeTests
The what, why and how
Who am I?
Sebastian Thoss
Chapter Lead Backend
Better ventures group
01 02 03 04 05
Agenda
Types of Tests
SmokeTests

What and Why
How do
SmokeTest
work
Recommended
Server
Architecture
SmokeTests 

as a Service
Types Of Tests
Test Pyramid
UNIT TESTS
Class 3
Class 2
Unit Tests
Class 1
Class 4
Mock
Mock
Test Pyramid
INTEGRATION TESTS
UNIT TESTS
Class 4Class 3
Integration Tests
Class 1 Class 2
Mock Mock
Test Pyramid
ACCEPTANCE TESTS
INTEGRATION TESTS
UNIT TESTS
Acceptance Tests
Class 1 Class 2
Class 3 Class 4
Test Pyramid
ACCEPTANCE TESTS
INTEGRATION TESTS
UNIT TESTS
?
SmokeTests: What & Why?
What are SmokeTests?
What are SmokeTests?
In computer programming and
software testing, smoke testing
is preliminary testing to reveal
simple failures severe enough to
reject a prospective software
release.



Source: https://en.wikipedia.org/wiki/Smoke_testing
What are SmokeTests?
1. … be simple
2. … be fast
3. … test URLs and optional parameters too
4. … cover at least all URLs in google index
5. … be extended by every new URL
6. … never forget a URL
7. … use a (manual) maintained list of URLs
SmokeTests should…
What are SmokeTests?
1. Status code
2. Time to first byte
3. If body is provided
4. Correct server
What SmokeTests validate?
What are SmokeTests?
SmokeTest

Client
HTTP 1.1/200 OK

<html>

<head>

<title>Foo</title>

<body>

<div id="bar"><span>foobar</span></div>

</body>

</html>
SmokeTests are NOT Acceptance Tests!
How do SmokeTests work
How do SmokeTests work?
https://confoo.ca/en
<html><body>…</body></html>
TTFB: 65ms
HTTP 1.1/200 OK
SmokeTest

Client
CI Server
Application 

to test
Production Server
How do SmokeTests work?
https://confoo.ca/foo
<html><body>…</body></html>
TTFB: 3ms
HTTP 1.1/404 Not Found
SmokeTest

Client
CI Server Production Server
Application 

to test
class SmokeTest extends TestCase

{

/**

* @dataProvider urlProvider

*/

public function testWebsiteUrl(Url $url)

{

$result = $this->sendGetRequest($url);

$this->assertSame(200, $result->getStatusCode());

$this->assertNotEmpty($result->getBody());

$this->assertLessThanOrEqual(100, $result->getTimeToFirstByteInMilliseconds());

}

public function urlProvider()

{

$urls = [‘http://www….de’, …];

$urlCollection = UrlCollection::fromStrings($urls);

return $urlCollection->asDataProviderArray($urlCollection);

}

…
Logic is in here
Kinda slow,
Mate!
Concurrent SmokeTests
How do SmokeTests work?
SmokeTest

Client
CI Server
Application

to test
Production Server
https://www.confoo.ca/baz
https://www.confoo.ca/bar
https://www.confoo.ca/en
How do SmokeTests work?
https://www.confoo.ca/baz
https://www.confoo.ca/bar
https://www.confoo.ca/en
TTFB: 34ms
SmokeTest

Client
CI Server
Application

to test
Production Server
How do SmokeTests work?
https://www.confoo.ca/foobaz
https://www.confoo.ca/bar
https://www.confoo.ca/en
SmokeTest

Client
CI Server
Application

to test
Production Server
How do SmokeTests work?
TTFB: 65ms
https://www.confoo.ca/foobaz
https://www.confoo.ca/bar
https://www.confoo.ca/en
SmokeTest

Client
CI Server
Application

to test
Production Server
How do SmokeTests work?
https://www.confoo.ca/123
https://www.confoo.ca/foobaz
https://www.confoo.ca/bar
SmokeTest

Client
CI Server
Application

to test
Production Server
How do SmokeTests work?
TTFB: 620ms
https://www.confoo.ca/123
https://www.confoo.ca/foobaz
https://www.confoo.ca/bar
SmokeTest

Client
CI Server
Application

to test
Production Server
Source: http://www.ve7kfm.com/fcc-server.jpg
How do SmokeTests work?
BUT
BUT
There is a library for it
DjThossi/smoke-testing-php
How do SmokeTests work?
DataProvider
Single Test
Application
HTTP Requests
HTTP Responses
PHPUnit
Calls
Result[ ]
Result
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;

/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];

$options = new SmokeTestOptions(

UrlCollection::fromStrings($urls),

new RequestTimeout(2),

new FollowRedirects(true),

new Concurrency(3),

new BodyLength(500)

);

return $this->runSmokeTests($options);

}
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;

/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];
$options = new SmokeTestOptions(

UrlCollection::fromStrings($urls),
new RequestTimeout(2),

new FollowRedirects(true),

new Concurrency(3),

new BodyLength(500)

);

return $this->runSmokeTests($options);

}
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;

/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];

$options = new SmokeTestOptions(
UrlCollection::fromStrings($urls),
new RequestTimeout(2),
new FollowRedirects(true),
new Concurrency(3),
new BodyLength(500)
);
return $this->runSmokeTests($options);

}
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;
/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];

$options = new SmokeTestOptions(

UrlCollection::fromStrings($urls),

new RequestTimeout(2),

new FollowRedirects(true),

new Concurrency(3),

new BodyLength(500)

);

return $this->runSmokeTests($options);

}
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;

/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];

$options = new SmokeTestOptions(

UrlCollection::fromStrings($urls),

new RequestTimeout(2),

new FollowRedirects(true),

new Concurrency(3),

new BodyLength(500)

);

return $this->runSmokeTests($options);

}
class SmokeTest extends PHPUnit_Framework_TestCase

{

use SmokeTestTrait;
/**

* @dataProvider myDataProvider

*/

public function testExample(Result $result)

{

$this->assertSuccess($result);

$this->assertTimeToFirstByte(new TimeToFirstByte(100), $result);

$this->assertBodyNotEmpty($result);

$this->assertHeaderExists(Header::fromPrimitives(‘App-Server', ‘Fury’), $result);

}

public function myDataProvider()

{

$urls = ['http://www.kartenmacherei.de', …];

$options = new SmokeTestOptions(

UrlCollection::fromStrings($urls),

new RequestTimeout(2),

new FollowRedirects(true),

new Concurrency(3),

new BodyLength(500)

);

return $this->runSmokeTests($options);

}
How do SmokeTests work?
Recommended

Server Architecture
Webserver (Router)
Webserver
Application
Box A
K/V
Store
DB
Webserver
Application
Box B
K/V
Store
DB
Webserver (Router)
Webserver
Application
Box A
K/V
Store
DB
Webserver
Application
Box B
K/V
Store
DB
active = A
Webserver (Router)
Webserver
Application
Box A
K/V
Store
DB
Webserver
Application
Box B
K/V
Store
DB
active = B
Webserver (Router)
Webserver
Application
Box A
K/V
Store
DB
active = B
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
active = B
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
Configure
active = B
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
Configure
Smoke Tests
active = B
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
Configure
Smoke Tests
active = B
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
Configure
Smoke Tests
Switch to A
active = Bactive = A
Webserver
Application
Box A
K/V
Store
DB
Webserver (Router)
Build Server
Deploy Code
Configure
Smoke Tests
active = B
SmokeTesting

as a Service
Smest.it
smoke test it
Smest.it
Smest.it
Smest.it
Smest.it
Smest.it
Smest.it
Smest.it
Try it for free!
better ventures group
DjThossi/smoke-testing-php
better__groupbetter.ventures.group
better.ventures
better.group/jobs

SmokeTests - What, Why & How - ConFoo 2019