This document provides tips and techniques for testing C++ code, including:
- Use a test framework and get to know it well
- Prioritize testable code over perfect code
- Be creative in testing legacy code, such as using golden master tests or approval tests
- Maintain tests by filtering slow tests from continuous integration and fixing flickering tests
Optimizing AI for immediate response in Smart CCTV
Cpp Testing Techniques Tips and Tricks - Cpp Europe
1. 1
C++ Testing: Techniques, Tips and Tricks
Clare Macrae (She/her)
clare@claremacrae.co.uk
25 February 2020
Cpp Europe
2. 2
About Me
• C++ and Qt developer since 1999
• My mission: Sustainable and efficient testing and refactoring of legacy code
– Co-author of “Approval Tests for C++”
• Consulting & training
– https://claremacrae.co.uk
• All links from this talk via:
– github.com/claremacrae/talks
3. 3
Why?
• Share things I wish I knew earlier
– Some well known
– Some less so…
• High-level overview, with links
– Avoiding detail
• Anything unclear?
– Please ask!
4. 4
Assumptions
• Value of testing – safety net!
• No worries about types of tests
– (unit, integration, regression)
9. 9
Rarely that simple!
• Long methods
• Interdependent code
• Tests hard to set up
• Not designed to be testable
10. 10
Tip: Take any reasonable measure!
• … to get code tested!
• Don’t get hung up on perfect code…
• What does that mean?
11. 11
Tip: Take any reasonable measure!
• When you are stuck, write the test you want
• Then safely do whatever you need to your implementation – temporarily!
– Make method static to call from outside?
– Add getters – and maybe even setters?
– Public data?
• Hold your nose!
• Then refactor
14. 14
Tip: Don’t test private code
• Automated tests give freedom to safely change implementation
• Testing private code breaks that
• Only test observable – public – behaviours
15. 15
What could possibly go wrong?
• It’s easy to think your test is passing…
• I’ve been bitten by all of these:
– Forget to add test source to build
– IDE calls a different test
– Test is doing the wrong thing
– Test is accidentally disabled
18. 18
Summary: Foundations
• Use a test framework – get to know it well
• Take any reasonable measure to add tests
– Then improve it
• Test public behaviour – not private implementation
• Testable code over perfect code
• Never trust a test until you have seen it fail
21. 21
But what if…
• Tiny real world example
• “GoToLineTool” widget
• User can type in to the spinner
22. 22
Testing goal
• Test that widget communicates changes correctly when user types in to widget
• lineNumberChanged()
• How to test that?
23. 23
Options – none are satisfactory
• Ideas
– Search the widget hierarchy for widgets by name
– Make the spinner widget data member public
– Provide a public accessor to the data member
– Make the test a friend of the class GoToLineTool class
• All if these require the test to be reworked if widget changes
24. 24
Goal: Make tests super easy to write
• And super easy to maintain…
• Test code accessing widgets directly makes that hard
25. 25
Technique: Separate concerns with Fixture
class GoToLineTool : public
QWidget
{
...
public/protected/private:
QSpinBox* spinBox();
QToolButton* goButton();
Implementation Tests – lots of them!Test Fixture
class GoToLineToolFixture
{
private:
GoToLineTool mGoToLineWidget;
public:
void typeCharacterIntoSpinner(
QChar character);
...
TEST_CASE_METHOD(
GoToLineToolFixture,
"GoToLineTool …")
{
// Ask Fixture to act
// on GoToLineTool
typeCharacterIntoSpinner('1');
}
28. 28
Typical Scenario
• I've inherited some
legacy code
• It's valuable
• I need to add feature
• Or fix bug
• How can I ever break
out of this loop?
Need to
change
the code
No tests
Not
designed
for testing
Needs
refactoring
to add
tests
Can’t
refactor
without
tests
37. 37
Summary: Legacy Code
• Approval Tests
• Be creative to make your tests work for you!
• After adding tests, start refactoring for maintainability
40. 40
Introducing ApprovalTests.cpp.Qt
•Goals
• Quickly start testing Qt code
– Useful even if you don’t use Approval Tests!
• Approval Tests support for Qt types
– Easy saving of state in Golden Master files
• https://github.com/approvals/ApprovalTests.cpp.Qt
• https://github.com/approvals/ApprovalTests.cpp.Qt.StarterProject
• v.0.0.1
41. 41
Example: Checking Table Contents
• Inherited complex code to set up a table
• Want to add at least a first test – of the text in the cells
42. 42
Verifying a QTableWidget
TEST_CASE("It approves a QTableWidget")
{
// A note on naming: QTableWidget is a concrete class that implements
// the more general QTableView. Here we create a QTableWidget,
// for convenience.
QTableWidget tableWidget;
populateTable(tableWidget);
ApprovalTestsQt::verifyQTableView(tableWidget);
}
44. 44
Summary: ApprovalTests.cpp.Qt
• More on this and on using Fixtures to write Expressive, Maintainable Tests
– slideshare.net/ClareMacrae/quickly-testing-qt-desktop-applications
46. 46
Scenario: Tests too slow for CI
• “Builds take 2 to 3 days!” => No hope of Continuous Integration
• Most of that was running tests
• No prospect of speeding up some very slow tests
47. 47
Solution: Don’t run slow tests during day
• Mark some tests as Slow
– Filtering built in to Google Test Framework
• Teach CI system to run sub-set of tests
– Only run the Slow tests at night
– Everything else run on every commit
• Thanks to Michael Platings for making this happen!
48. 48
Tip: Never tolerate Flickering tests
• Tests that fail randomly
• Rapidly devalue other tests
• Fix them, or delete them!
• Especially don’t mix performance tests with unit tests
50. 50
Summary
• It’s never too late to start testing!
– Pick a framework and practice it
– Explore different types of testing
• You can test legacy code too!
• Keep maintaining your tests
• Even GUIs can be tested
51. 51
C++ Testing: Techniques, Tips and Tricks
• All links from this talk, and more, via:
– bit.ly/CppTestingTips
– github.com/claremacrae/talks
• Sustainable and efficient testing and refactoring of legacy code
• Consulting & Training
– https://claremacrae.co.uk
– clare@claremacrae.co.uk
• Any Questions?
• Any More Tips?
Editor's Notes
Harder than testing non-Graphical User Interface code
But it’s still a learnable skill
Harder than testing non-Graphical User Interface code
But it’s still a learnable skill
Harder than testing non-Graphical User Interface code
But it’s still a learnable skill
Harder than testing non-Graphical User Interface code
But it’s still a learnable skill
Harder than testing non-Graphical User Interface code
But it’s still a learnable skill
Note the dependency order
Another phrase for the Golden Master output is “Known Good” output.
A lot of this stuff you can get going on your own. If you get stuck, I can help you!