Testbox is a tool we all should be using to test our ColdFusion Applications which was created and is maintained by Ortus Solutions, the people that brought you ColdBox. We will have Gavin from Ortus in house on this day to go over some testbox examples, talk about its importance, and answer any questions you have.
SO --- if you have and high level questions for Gavin, reply to this post (or hit me up) so I can get the questions to Gavin a head of time just in case he needs to consult others at Ortus.
Gavin Pickin is a proud ColdFusion developer, starting with ColdFusion in the late 90s. Now working with Ortus Solutions, a leading force in CFML Development frameworks and tools, Gavin gets to work on a lot of great projects, for a big variety of clients. At Ortus Solutions, a big focus is on free and open source tools, on open source Fridays, Gavin spend most of his open source time working on ContentBox Content Management System.
2. Who am I
Gavin Pickin – developing Web Apps since late 90s
● New Addition to Ortus Solutions
● ContentBox Evangelist
What else do you need to know?
● Blog - http://www.gpickin.com
● Twitter – http://twitter.com/gpickin
● Github - https://github.com/gpickin
Let’s get on with the show.
4. Trick Question - You are all testing your apps.
● Most people look like this guy
Clicking around in the browser yourself
● Setup Selenium / Web Driver to click
around for you
● Structured Programmatic Tests
6. Bugs Hurt
•Bugs hurt – the later in the process, the harder to
fix.
•Test Early and Often
–Find them before they rot your foundation
•Testable Code is Maintainable Code
7. Types of Testing
● Black/White Box
● Unit Testing
● Integration Testing
● Functional Tests
● System Tests
● End to End Tests
● Sanity Testing
● Regression Test
● Acceptance Tests
● Load Testing
● Stress Test
● Performance Tests
● Usability Tests
● + More
8. Important Testing Types
•Unit Testing
–Test behavior of individual objects
•Integration Testing
–Test Entire Application from Top Down
•UI verification testing
Verification via HTML/Visual elements
9. Important Testing Tools
•TestBox (Run BDD and MXUnit style)
•IDE - CF Builder / Eclipse
•Mocking Framework
•ANT
•Jenkins, Bamboo, Teamcity, other Cis
•Selenium
•Jmeter or Webstress Tool, Apache AB
10. What is TestBox?
TestBox is a next generation testing framework for ColdFusion (CFML)
that is based on BDD (Behavior Driven Development) for providing a
clean obvious syntax for writing tests. It contains not only a testing
framework, runner, assertions and expectations library but also ships
with MockBox, A Mocking & Stubbing Framework. It also supports
xUnit style of testing and MXUnit compatibilities.
More info: https://www.ortussolutions.com/products/testbox
11. Integration Tests
● Integration Tests several of the pieces together
● Most of the types of tests are variations of an Integration Test
● Can include mocks but can full end to end tests including DB /
APIs
12. What is Unit Testing
“unit testing is a software verification and validation method
in which a programmer tests if individual units of source code
are fit for use. A unit is the smallest testable part of an
application”
- wikipedia
13. Unit Testing
•Can improve code quality -> quick error discovery
•Code confidence via immediate verification
•Can expose high coupling
•Will encourage refactoring to produce > testable code
•Remember: Testing is all about behavior and expectations
14. Style of Testing - TDD vs BDD
● TDD = Test Driven Development
● Write Tests
● Run them and they Fail
● Write Functions to Fulfill the Tests
● Tests should pass
● Refactor in confidence
● Test focus on Functionality
15. Style of Testing - TDD vs BDD
● BDD = Behavior Driven Development
● Actually similar to TDD except:
● Focuses on Behavior and Specifications
● Specs (tests) are fluent and readable
● Readability makes them great for all levels of testing in the
organization
● Hard to find TDD examples in JS that are not using BDD
describe and it blocks
16. TDD Example
Test( ‘Email address must not be blank’, function(){
notEqual(email, “”, "failed");
});
17. BDD Example
Describe( ‘Email Address’, function(){
It(‘should not be blank’, function(){
expect(email).not.toBe(“”);
});
});
21. BDD Example
describe("Hello world function", function() {
it(”contains the word world", function() {
expect(helloWorld()).toContain("world");
});
});
22. New BDD Example
feature( "Box Size", function(){
describe( "In order to know what size box I need
As a distribution manager
I want to know the volume of the box", function(){
scenario( "Get box volume", function(){
given( "I have entered a width of 20
And a height of 30
And a depth of 40", function(){
when( "I run the calculation", function(){
then( "the result should be 24000", function(){
// call the method with the arguments and test the outcome
expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
});
});
23. Installing Testbox
Install Testbox – Thanks to Commandbox - this is easy.
# box install testbox
Next, we need to decide how you want to run Testbox
24. Create a Runner.cfm File to Run your tests
<cfsetting showDebugOutput="false">
<!--- Executes all tests in the 'specs' folder with simple reporter by default --->
<cfparam name="url.reporter" default="simple">
<cfparam name="url.directory" default="tests.specs">
<cfparam name="url.recurse" default="true" type="boolean">
<cfparam name="url.bundles" default="">
<cfparam name="url.labels" default="">
<!--- Include the TestBox HTML Runner --->
<cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >
25. Create a test suite
// tests/specs/CFCTest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
it( "will error with incorrect login", function(){
var oTest = new cfcs.userServiceRemote();
expect( oTest.login( 'gavin@gavin.com', 'topsecret').result ).toBe('400');
});
}
}
26. Create 2nd Test Suite
// tests/specs/APITest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
describe("userService API Login", function(){
it( "will error with incorrect login", function(){
var email = "gavin@gavin.com";
var password = "topsecret”;
var result = "";
http
url="http://www.testableapi.local.com:8504/cfcs/userServiceRemote.cfc?method=login&e
mail=#email#&password=#password#" result="result”;
expect( DeserializeJSON(result.filecontent).result ).toBe('400');
});
});
}
35. TestBox JSON output
Testbox has several runners, you have seen the HTML one, this
Runner uses the JSON runner and then formats it.
http://www.testableapi.local.com:8504/tests/runner.cfm?report
er=JSON&directory=%2Ftests%2Fspecs&recurse=true
36. Running TestBox in Sublime 2
Install PackageControl into Sublime Text
Install Grunt from PackageControl
https://packagecontrol.io/packages/Grunt
Update Grunt Sublime Settings for paths
{
"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }
}
Then Command Shift P – grunt
38. Run TestBox with CommandBox
Run the tests in the box.json file.
# testbox run
Watch the folders and run tests on change
# testbox watch
More info on TestBox usage from CommandBox:
https://www.ortussolutions.com/blog/using-testbox-watch-to-a
utomate-your-testing-suite
39. More Testing - Mocking with MockBox
"A mock object is an object that takes the place of
a ‘real’ object in such a way that makes testing
easier and more meaningful, or in some cases,
possible at all"
by Scott Bain - Emergent Design
40. More Testing - Integrated
Start using an easy, fluent API for your integration tests!
More Info: https://github.com/elpete/integrated
Api - https://elpete.github.io/integrated/
41. Some real life examples
https://github.com/framework-one/fw1/tree/develop/tests maybe?
https://github.com/Ortus-Solutions/TestBox/tree/development/tests
https://github.com/foundeo/cfdocs/tree/master/tests
https://github.com/foundeo/cfmlparser/tree/master/tests/tests
https://github.com/foundeo/toscript/tree/master/tests
https://github.com/foundeo/bolthttp/tree/master/test
https://github.com/foundeo/cfml-security/tree/master/tests/tests/secureupload
42. Some real life examples cont.
https://github.com/ryanguill/emit/tree/master/tests
https://github.com/ryanguill/cfmlBase62/tree/master/tests
https://gitlab.com/ryanguill/FPcfc/tree/master/tests
https://gitlab.com/ryanguill/template-cfc/tree/master/tests
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic