SlideShare a Scribd company logo
1 of 73
1
Quickly Testing Qt Desktop Applications
Clare Macrae (She/her)
clare@claremacrae.co.uk
15 November 2019
Meeting C++ 2019, Berlin, Germany
2
Audience: Developers using Qt
Approval Tests: claremacrae.co.uk/conferences/presentations.html
3
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
4
About Me
• Scientific C++ and Qt developer since 1999
• My mission: Sustainable and efficient testing and refactoring of legacy code
– Co-author of “Approval Tests for C++”
• Consulting & training via “Clare Macrae Consulting Ltd”
– https://claremacrae.co.uk
• All links from this talk via:
– bit.ly/TestingQt
– github.com/claremacrae/talks
5
Typical Scenario
• I've inherited some Qt
GUI code
• It's valuable
• I need to add feature
• Or fix bug
• How can I ever break
out of this loop?
Need to
change
the code
No tests
Not
designed
for testing
Needs
refactoring
to add
tests
Can’t
refactor
without
tests
6
Typical Scenario
•
•
•
•
•
Need to
change
the code
No tests
Not
designed
for testing
Needs
refactoring
to add
tests
Can’t
refactor
without
tests
Topics of
this talk
7
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
8
9
Qt’s GUI powers make
Automated Testing Harder
10
Give you Confidence
to Start testing
your Qt application
Effectively
11
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
12
A: Ah – Good point – You don’t
Q: How do I add tests to existing app?
13
Introduce static lib for tests
Executable
Color Key
GUI.exe
main() and *.cpp
*.ui
14
Introduce static lib for tests
Static Library
Executable
Color Key
GUI.exe
main()
UI.lib
*.cpp
*.ui
15
Introduce static lib for tests
Static Library
Executable
Color Key
GUI.exe
main()
UITests.exe
main() and tests/*.cpp
UI.lib
*.cpp
*.ui
If too hard, could
move just the code
you are going to test
16
Doing the Separation
• For CMake, see Arne Mertz’s excellent tutorial:
• https://arne-mertz.de/2018/05/hello-cmake/
• Remember to make the library static!
17
Context: Type of testing
• Glass-box or transparent-box testing
• Have access to the code
• And can potentially even make changes to it
18
Creating the Test Executable
19
C++ Test Runners
• Lots to choose from:
• You can use any – I’m using Catch2, with a little bit of Qt Test
• I found there were tricks needed to set up Catch2 – you will probably need to do the same if
you use a different framework
20
Setting up testsuite main()
// Demonstrate how to create a Catch main() for testing Qt GUI code
#define CATCH_CONFIG_RUNNER
#include <Catch.hpp>
#include <QApplication>
int main(int argc, char* argv[])
{
QApplication app(argc, argv);
int result = Catch::Session().run(argc, argv);
return result;
}
21
Context: SuperCollider
• supercollider.github.io
• platform for audio
synthesis and algorithmic
composition
Picture credit: Chad Cassady, @beatboxchad
22
Context: ColorWidget color-picker
This set of widgets is
”our” code:
• we want to test it
This set of widgets is
”Qt” code:
• not our job to
test it
• hard to test
anyway
• only test our
stuff!
23
ColorWidget setup – QColor object
#include "catch.hpp"
#include "color_widget.hpp"
#include "catch_qt_string_makers.hpp"
using namespace ScIDE;
TEST_CASE("ColorWidget initial state") {
ColorWidget widget;
QColor expected_color(0, 0, 0, 255);
CHECK(widget.color() == expected_color);
}
24
ColorWidget setup – approval testing
#include "catch.hpp"
#include "color_widget.hpp"
#include "catch_qt_string_makers.hpp"
using namespace ScIDE;
TEST_CASE("ColorWidget initial state - approval testing") {
ColorWidget widget;
auto color = Catch::StringMaker<QColor>::convert(widget.color());
CHECK(color == "(0, 0, 0), alpha = 1");
}
25
ColorWidget changing state
TEST_CASE("ColorWidget changing color updates correctly") {
// Arrange
ColorWidget widget;
QColor red("red");
// Act
widget.setColor(red);
// Assert
CHECK(widget.color() == red);
}
26
“Quickly Testing?”
• Getting from 0 to 1 tests is always the hardest – that’s normal
• Getting from 1 to 2, and then 2 to 3, always much easier
• Step 1: Make the first test as easy as possible – it’s going to be hard!
• Step 2: Write at least 3 tests before deciding if it’s a good idea!
27
Context: “Go To Line” Panel
28
Goal: Demo the basic code
• Warning: Some detail coming up!
– Later I’ll show how to avoid it…
• Basic steps:
– Arrange
– Act
– Assert
29
High level view – version 1
class GoToLineTool : public QWidget
{
...
private:
QSpinBox* mSpinBox;
QToolButton* mGoBtn;
TEST_CASE("GoToLineTool …")
{
// Peek inside GoToLineTool to
// access widgets
...
}
Implementation Tests
30
Arrange: Set up Widget
TEST_CASE("GoToLineTool emits signal when Go button clicked")
{
// --------------------------------------------------------
// Arrange
GoToLineTool widget;
widget.raise();
widget.show();
widget.setMaximum(27);
// This is needed to make sure text in the spinner is selected, so that
// as we type in characters, the initial text ("1") is erased
widget.setFocus();
31
Arrange: Accessing private widgets – V1
// Allow us to interact with widgets inside GoToLineTool:
auto spinner = widget.findChild<QSpinBox*>();
REQUIRE(spinner != nullptr);
auto goButton = widget.findChild<QToolButton*>();
REQUIRE(goButton != nullptr);
✘ Beware: findChild() not recommended,
especially in production code!
32
Test actions – Signals and Slots
1 7
emit activated(17);
QSignalSpy(&mGoToLineWidget,
&GoToLineTool::activated)
33
Arrange: Listen to activate() signal
// Enable tracking of one signal
// New-style code
QSignalSpy activatedSpy(&widget, &GoToLineTool::activated);
REQUIRE(activatedSpy.isValid());
// Old-style code
//QSignalSpy activatedSpy(&widget, SIGNAL(activated(int)));
//REQUIRE(activatedSpy.isValid());
34
Act: Generate user events 1 7
// --------------------------------------------------------
// Act
// Type a number, one digit at a time
QTest::keyClicks(spinner, "1");
QTest::keyClicks(spinner, "7");
// Clicking the Go button:
goButton->click();
35
Assert: Check the changes
// --------------------------------------------------------
// Assert
// Check the activated() signal was emitted only once:
REQUIRE(activatedSpy.count() == 1);
// And check that the signal emitted the new value:
QList<QVariant> arguments = activatedSpy.takeFirst();
const QVariant& argument = arguments.at(0);
CHECK(argument.type() == QVariant::Int);
CHECK(argument.toInt() == 17);
}
emit activated(17);
36
High level view – version 1
Implementation Tests
class GoToLineTool : public QWidget
{
...
private:
QSpinBox* mSpinBox;
QToolButton* mGoBtn;
TEST_CASE("GoToLineTool …")
{
// Peek inside GoToLineTool to
// access widgets
auto spinner =
widget.findChild<QSpinBox*>();
...
}
37
Goal: Make tests super easy to write
• findChild() is really fragile
– prone to future breakage at run-time
– not checked at compile-time
38
Testing Custom Qt Widgets – part 1
Option Advantages Disadvantages
widget.findChild()/findChildren() Don’t need to change code
being tested
No compile time feedback.
Maintenance overhead
Add public accessor to widget
being tested
Testing easier Maybe encourages misuse of
library if internals exposed
39
High level view – version 2
Implementation Tests
class GoToLineTool : public QWidget
{
...
public:
QSpinBox* spinBox();
QToolButton* goButton();
TEST_CASE("GoToLineTool …")
{
// Ask GoToLineTool for
// access widgets
auto spinner = widget.spinBox();
...
}
40
Goal: Hide implementation details
• Want to avoid tests knowing insides of widgets being tested
– As much as possible
• Can we improve readability of test?
• Can we guard against changes to implementation?
41
High level view – version 3
class GoToLineTool : public
QWidget
{
...
public:
QSpinBox* spinBox();
QToolButton* goButton();
Implementation Tests – lots of them!Test Fixture
class GoToLineToolFixture
{
private:
GoToLineTool mGoToLineWidget;
public:
void typeCharacterIntoSpinner(
QChar character);
...
TEST_CASE_METHOD(
GoToLineToolFixture,
"GoToLineTool …")
{
// Ask Fixture to act
// on GoToLineTool
typeCharacterIntoSpinner('1');
}
42
Introduce a Fixture
class GoToLineToolFixture
{
private:
GoToLineTool mGoToLineWidget;
QSpinBox* mSpinner = nullptr;
QToolButton* mGoButton = nullptr;
std::unique_ptr<QSignalSpy> mActivatedSpy;
protected:
GoToLineToolFixture()
{
mGoToLineWidget.raise();
mGoToLineWidget.show();
// ... and so on ...
43
GoToLineTool – testing events, expressively
TEST_CASE_METHOD(GoToLineToolFixture,
"GoToLineTool emits signal when Go button clicked")
{
// Arbitrary upper limit in number of lines.
// When used in the application, this would be obtained from the open docunent
setMaximumLineCount(27);
// Type a number, one digit at a time
typeCharacterIntoSpinner('1');
typeCharacterIntoSpinner('7’);
clickGoButton();
checkActivatedSignalCount(1);
checkActivatedSignalValue(17);
}
44
Testing Custom Qt Widgets – part 1
Option Advantages Disadvantages
widget.findChild()/findChildren() Don’t need to change code
being tested
No compile time feedback.
Maintenance overhead
Add public accessor to widget
being tested
Testing easier Maybe encourages misuse of
library if internals exposed
45
Testing Custom Qt Widgets – part 2
Option Advantages Disadvantages
widget.findChild()/findChildren() Don’t need to change code
being tested
No compile time feedback.
Maintenance overhead
Add public accessor to widget
being tested
Testing easier Maybe encourages misuse of
library if internals exposed
Add protected accessor …
Make the test fixture inherit the
widget
Mostly hides the
implementation
Maybe encourages inheritance
to customize behavior.
Test fixture has to inherit class
being tested – maybe confusing
Add private accessor …
Make the test fixture a friend of
the widget
Hides the implementation
more
Friendship not inherited by test
implementation – so there’s
more work to create the test
fixture
46
Test Pyramid
martinfowler.com/bliki/TestPyramid.html
47
Bonus Points: Separate logic from UI code
Static Library
Executable
Color Key
GUI.exe
main()
UITests.exe
main() and tests/*.cpp
UI.lib
*.cpp
*.ui
Model.lib ModelTests.exe
*.cpp main() and tests/*.cpp
48
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
49
• Run-time error checking:
Pitfall: Qt4-style Signal/Slot connect()
connect(
ui->quitButton, SIGNAL(clicked()),
this, SLOT(quit()));
QObject::connect: No such slot SampleMainWindow::quit()
QObject::connect: (sender name: 'quitButton')
QObject::connect: (receiver name: 'SampleMainWindow')
50
Safer: Qt5-style Signal/Slot connect()
• Use Qt5 pointer-to-member-function connections
connect(
ui->quitButton, SIGNAL(clicked()),
this, SLOT(quit()));
// Solution: Qt5-style connections get checked at compile-time:
connect(
ui->quitButton, &QPushButton::clicked,
this, &SampleMainWindow::quit
);
51
Pitfall: Event processing in tests
• Normally, Qt’s event loop processes events automatically
– QCoreApplication::processEvents()
• Can’t rely on that in test programs
• Know your options:
– bool QTest::qWaitForWindowActive( window/widget, timeout )
– bool QTest::qWaitForWindowExposed( window/widget, timeout )
– bool QSignalSpy::wait( timeout )
• If your test runner is Qt Test:
– QTRY_COMPARE, QTRY_COMPARE_WITH_TIMEOUT
– QTRY_VERIFY, QTRY_VERIFY_WITH_TIMEOUT
– These macros include a return statement on failure
52
Running Qt tests in CI system
• Windows:
– Will probably just work
• Linux:
– Typically use virtual display, e.g. xvfb
– Could investigate -platform offscreen
• Mac:
– Found needed to keep a user logged in
• Make tests that need a GUI behave gracefully if no display found:
– IF_NO_GUI_PRINT_AND_RETURN()
53
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
54
Introducing ApprovalTests.cpp.Qt
55
Introducing ApprovalTests.cpp.Qt
•Goals
• Rapid start testing Qt code
– Useful even if you don’t use Approval Tests!
• Approval Tests support for Qt types
– Easy saving of state in Golden Master files
• https://github.com/approvals/ApprovalTests.cpp.Qt
• https://github.com/approvals/ApprovalTests.cpp.Qt.StarterProject
• v.0.0.1
56
Introducing ApprovalTests.cpp.Qt
• https://github.com/approvals/ApprovalTests.cpp.Qt
Catch2ApprovalTests.cpp Qt5 Test Qt5 Widgets
ApprovalTests.cpp.Qt
57
Setting up testsuite main()
// main.cpp:
#define APPROVALS_CATCH_QT
#include "ApprovalTestsQt.hpp"
58
Checking Table Contents
• Inherited complex code to set up a table
• Want to add at least a first test – of the text in the cells
59
Verifying a QTableWidget
TEST_CASE("It approves a QTableWidget")
{
// A note on naming: QTableWidget is a concrete class that implements
// the more general QTableView. Here we create a QTableWidget,
// for convenience.
QTableWidget tableWidget;
populateTable(tableWidget);
ApprovalTestsQt::verifyQTableView(tableWidget);
}
60
Approval file: .tsv
61
Checking Screenshots? No!
• Not recommended, in general
• Styles differ between Operating Systems
• Better to test behaviour, not appearance
62
Checking rendering
• But if you have to do it…
63
Verifying a QImage
TEST_CASE("It approves a QImage")
{
QImage image(10, 20, QImage::Format_RGB32);
image.fill(Qt::red);
ApprovalTestsQt::verifyQImage(image);
}
64
65
Caution!
• Avoid Qt Test macros in this project
– QCOMPARE, QVERIFY, QTRY_COMPARE and so on
– Any test failures will be silently swallowed
– Tests will spuriously pass.
– [November 2019: Fabian Kosmale at Qt has been really helpful on this – there is hope!]
66
What next for ApprovalTests.cpp.Qt?
• Fix Qt Test macros problem
• Seek feedback
• Could…
– Support more test frameworks
– Support approving more Qt types
– Add real-world examples to the docs
67
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone Things
– Approval Tests
• Extras
– Tools
– Summary
68
Navigation around your GUI
• Where is that widget in the source code?
– Find strings in UI and search
– Use KDAB GammaRay
kdab.com/development-resources/qt-tools/gammaray
69
clazy
• Qt related compiler warnings and fixits
• Lots of useful warnings
• github.com/KDE/clazy
• Level 2 check: old-style-connect
– Read the warnings on this check, though
70
Squish
• Automated GUI Testing – of your whole application
– Semi-opaque-box testing
• froglogic.com/squish – Commercial tool
• Basics
– Record actions in your application
– Refactor in to reusable tests, e.g. in Python
• Opinion
– Not a replacement for unit tests
– Requires commitment to training and maintenance
– If used well, can replace lots of manual testing
71
Contents
• Introduction
• Qt
– Setting Up Testing
– Error-prone things
– Approval Tests
• Extras
– Tools
– Summary
72
Summary
• It’s never too late to start testing!
• What to test
– Try to test smaller units of code, such as individual widgets
– Test behaviors, not appearance (mainly)
– Keep application logic separate from GUI
• Make tests …
– Easy to write
– Hide details of custom widgets behind helper functions, in tests
– Expressive and readable with fixtures
• ApprovalTests.Cpp.Qt feedback welcome!
73
Quickly Test Qt Desktop Applications
• All links from this talk, and more, via:
– bit.ly/TestingQt
– github.com/claremacrae/talks
• Sustainable and efficient testing and refactoring of legacy code
• Consulting & training via “Clare Macrae Consulting Ltd”
– https://claremacrae.co.uk
– clare@claremacrae.co.uk

More Related Content

What's hot

An introduction to Google test framework
An introduction to Google test frameworkAn introduction to Google test framework
An introduction to Google test frameworkAbner Chih Yi Huang
 
Best Practices in Qt Quick/QML - Part I
Best Practices in Qt Quick/QML - Part IBest Practices in Qt Quick/QML - Part I
Best Practices in Qt Quick/QML - Part IICS
 
Best Practices in Qt Quick/QML - Part II
Best Practices in Qt Quick/QML - Part IIBest Practices in Qt Quick/QML - Part II
Best Practices in Qt Quick/QML - Part IIICS
 
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing FrameworkC++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing FrameworkHumberto Marchezi
 
QThreads: Are You Using Them Wrong?
QThreads: Are You Using Them Wrong? QThreads: Are You Using Them Wrong?
QThreads: Are You Using Them Wrong? ICS
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming LanguageGiuseppe Arici
 
Mocking in Java with Mockito
Mocking in Java with MockitoMocking in Java with Mockito
Mocking in Java with MockitoRichard Paul
 
Unit Testing And Mocking
Unit Testing And MockingUnit Testing And Mocking
Unit Testing And MockingJoe Wilson
 
Best Practices in Qt Quick/QML - Part 3
Best Practices in Qt Quick/QML - Part 3Best Practices in Qt Quick/QML - Part 3
Best Practices in Qt Quick/QML - Part 3ICS
 
Introduction to Kotlin coroutines
Introduction to Kotlin coroutinesIntroduction to Kotlin coroutines
Introduction to Kotlin coroutinesRoman Elizarov
 

What's hot (20)

An introduction to Google test framework
An introduction to Google test frameworkAn introduction to Google test framework
An introduction to Google test framework
 
Best Practices in Qt Quick/QML - Part I
Best Practices in Qt Quick/QML - Part IBest Practices in Qt Quick/QML - Part I
Best Practices in Qt Quick/QML - Part I
 
Junit
JunitJunit
Junit
 
Best Practices in Qt Quick/QML - Part II
Best Practices in Qt Quick/QML - Part IIBest Practices in Qt Quick/QML - Part II
Best Practices in Qt Quick/QML - Part II
 
Introduction to JUnit
Introduction to JUnitIntroduction to JUnit
Introduction to JUnit
 
Test driving QML
Test driving QMLTest driving QML
Test driving QML
 
C++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing FrameworkC++ Unit Test with Google Testing Framework
C++ Unit Test with Google Testing Framework
 
QThreads: Are You Using Them Wrong?
QThreads: Are You Using Them Wrong? QThreads: Are You Using Them Wrong?
QThreads: Are You Using Them Wrong?
 
Mockito
MockitoMockito
Mockito
 
Qt Application Programming with C++ - Part 2
Qt Application Programming with C++ - Part 2Qt Application Programming with C++ - Part 2
Qt Application Programming with C++ - Part 2
 
Kotlin
KotlinKotlin
Kotlin
 
Swift Programming Language
Swift Programming LanguageSwift Programming Language
Swift Programming Language
 
Junit
JunitJunit
Junit
 
JUNit Presentation
JUNit PresentationJUNit Presentation
JUNit Presentation
 
Mocking in Java with Mockito
Mocking in Java with MockitoMocking in Java with Mockito
Mocking in Java with Mockito
 
Unit Testing And Mocking
Unit Testing And MockingUnit Testing And Mocking
Unit Testing And Mocking
 
Junit 4.0
Junit 4.0Junit 4.0
Junit 4.0
 
JUnit Presentation
JUnit PresentationJUnit Presentation
JUnit Presentation
 
Best Practices in Qt Quick/QML - Part 3
Best Practices in Qt Quick/QML - Part 3Best Practices in Qt Quick/QML - Part 3
Best Practices in Qt Quick/QML - Part 3
 
Introduction to Kotlin coroutines
Introduction to Kotlin coroutinesIntroduction to Kotlin coroutines
Introduction to Kotlin coroutines
 

Similar to Quickly Testing Qt Desktop Applications

Qt for beginners part 4 doing more
Qt for beginners part 4   doing moreQt for beginners part 4   doing more
Qt for beginners part 4 doing moreICS
 
Cpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp EuropeCpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp EuropeClare Macrae
 
The Ring programming language version 1.3 book - Part 53 of 88
The Ring programming language version 1.3 book - Part 53 of 88The Ring programming language version 1.3 book - Part 53 of 88
The Ring programming language version 1.3 book - Part 53 of 88Mahmoud Samir Fayed
 
Quality of life through Unit Testing
Quality of life through Unit TestingQuality of life through Unit Testing
Quality of life through Unit TestingSian Lerk Lau
 
Meet the Widgets: Another Way to Implement UI
Meet the Widgets: Another Way to Implement UIMeet the Widgets: Another Way to Implement UI
Meet the Widgets: Another Way to Implement UIICS
 
The Ring programming language version 1.2 book - Part 51 of 84
The Ring programming language version 1.2 book - Part 51 of 84The Ring programming language version 1.2 book - Part 51 of 84
The Ring programming language version 1.2 book - Part 51 of 84Mahmoud Samir Fayed
 
The Ring programming language version 1.4 book - Part 19 of 30
The Ring programming language version 1.4 book - Part 19 of 30The Ring programming language version 1.4 book - Part 19 of 30
The Ring programming language version 1.4 book - Part 19 of 30Mahmoud Samir Fayed
 
The Ring programming language version 1.10 book - Part 72 of 212
The Ring programming language version 1.10 book - Part 72 of 212The Ring programming language version 1.10 book - Part 72 of 212
The Ring programming language version 1.10 book - Part 72 of 212Mahmoud Samir Fayed
 
[COSCUP 2021] A trip about how I contribute to LLVM
[COSCUP 2021] A trip about how I contribute to LLVM[COSCUP 2021] A trip about how I contribute to LLVM
[COSCUP 2021] A trip about how I contribute to LLVMDouglas Chen
 
3 Ways to test your ColdFusion API - 2017 Adobe CF Summit
3 Ways to test your ColdFusion API - 2017 Adobe CF Summit3 Ways to test your ColdFusion API - 2017 Adobe CF Summit
3 Ways to test your ColdFusion API - 2017 Adobe CF SummitOrtus Solutions, Corp
 
3 WAYS TO TEST YOUR COLDFUSION API -
3 WAYS TO TEST YOUR COLDFUSION API - 3 WAYS TO TEST YOUR COLDFUSION API -
3 WAYS TO TEST YOUR COLDFUSION API - Ortus Solutions, Corp
 
3 WAYS TO TEST YOUR COLDFUSION API
3 WAYS TO TEST YOUR COLDFUSION API3 WAYS TO TEST YOUR COLDFUSION API
3 WAYS TO TEST YOUR COLDFUSION APIGavin Pickin
 
Breaking Dependencies Legacy Code - Cork Software Crafters - September 2019
Breaking Dependencies Legacy Code -  Cork Software Crafters - September 2019Breaking Dependencies Legacy Code -  Cork Software Crafters - September 2019
Breaking Dependencies Legacy Code - Cork Software Crafters - September 2019Paulo Clavijo
 
Integrazione QML / C++
Integrazione QML / C++Integrazione QML / C++
Integrazione QML / C++Paolo Sereno
 
Strategy Design Pattern
Strategy Design PatternStrategy Design Pattern
Strategy Design PatternGanesh Kolhe
 
Introduction to Griffon
Introduction to GriffonIntroduction to Griffon
Introduction to GriffonJames Williams
 
Building a DRYer Android App with Kotlin
Building a DRYer Android App with KotlinBuilding a DRYer Android App with Kotlin
Building a DRYer Android App with KotlinBoonya Kitpitak
 
The Ring programming language version 1.9 book - Part 81 of 210
The Ring programming language version 1.9 book - Part 81 of 210The Ring programming language version 1.9 book - Part 81 of 210
The Ring programming language version 1.9 book - Part 81 of 210Mahmoud Samir Fayed
 
Gitlab ci e kubernetes, build test and deploy your projects like a pro
Gitlab ci e kubernetes, build test and deploy your projects like a proGitlab ci e kubernetes, build test and deploy your projects like a pro
Gitlab ci e kubernetes, build test and deploy your projects like a prosparkfabrik
 

Similar to Quickly Testing Qt Desktop Applications (20)

Qt for beginners part 4 doing more
Qt for beginners part 4   doing moreQt for beginners part 4   doing more
Qt for beginners part 4 doing more
 
Cpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp EuropeCpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp Europe
 
The Ring programming language version 1.3 book - Part 53 of 88
The Ring programming language version 1.3 book - Part 53 of 88The Ring programming language version 1.3 book - Part 53 of 88
The Ring programming language version 1.3 book - Part 53 of 88
 
Quality of life through Unit Testing
Quality of life through Unit TestingQuality of life through Unit Testing
Quality of life through Unit Testing
 
Meet the Widgets: Another Way to Implement UI
Meet the Widgets: Another Way to Implement UIMeet the Widgets: Another Way to Implement UI
Meet the Widgets: Another Way to Implement UI
 
The Ring programming language version 1.2 book - Part 51 of 84
The Ring programming language version 1.2 book - Part 51 of 84The Ring programming language version 1.2 book - Part 51 of 84
The Ring programming language version 1.2 book - Part 51 of 84
 
The Ring programming language version 1.4 book - Part 19 of 30
The Ring programming language version 1.4 book - Part 19 of 30The Ring programming language version 1.4 book - Part 19 of 30
The Ring programming language version 1.4 book - Part 19 of 30
 
The Ring programming language version 1.10 book - Part 72 of 212
The Ring programming language version 1.10 book - Part 72 of 212The Ring programming language version 1.10 book - Part 72 of 212
The Ring programming language version 1.10 book - Part 72 of 212
 
[COSCUP 2021] A trip about how I contribute to LLVM
[COSCUP 2021] A trip about how I contribute to LLVM[COSCUP 2021] A trip about how I contribute to LLVM
[COSCUP 2021] A trip about how I contribute to LLVM
 
3 Ways to test your ColdFusion API - 2017 Adobe CF Summit
3 Ways to test your ColdFusion API - 2017 Adobe CF Summit3 Ways to test your ColdFusion API - 2017 Adobe CF Summit
3 Ways to test your ColdFusion API - 2017 Adobe CF Summit
 
Qt Workshop
Qt WorkshopQt Workshop
Qt Workshop
 
3 WAYS TO TEST YOUR COLDFUSION API -
3 WAYS TO TEST YOUR COLDFUSION API - 3 WAYS TO TEST YOUR COLDFUSION API -
3 WAYS TO TEST YOUR COLDFUSION API -
 
3 WAYS TO TEST YOUR COLDFUSION API
3 WAYS TO TEST YOUR COLDFUSION API3 WAYS TO TEST YOUR COLDFUSION API
3 WAYS TO TEST YOUR COLDFUSION API
 
Breaking Dependencies Legacy Code - Cork Software Crafters - September 2019
Breaking Dependencies Legacy Code -  Cork Software Crafters - September 2019Breaking Dependencies Legacy Code -  Cork Software Crafters - September 2019
Breaking Dependencies Legacy Code - Cork Software Crafters - September 2019
 
Integrazione QML / C++
Integrazione QML / C++Integrazione QML / C++
Integrazione QML / C++
 
Strategy Design Pattern
Strategy Design PatternStrategy Design Pattern
Strategy Design Pattern
 
Introduction to Griffon
Introduction to GriffonIntroduction to Griffon
Introduction to Griffon
 
Building a DRYer Android App with Kotlin
Building a DRYer Android App with KotlinBuilding a DRYer Android App with Kotlin
Building a DRYer Android App with Kotlin
 
The Ring programming language version 1.9 book - Part 81 of 210
The Ring programming language version 1.9 book - Part 81 of 210The Ring programming language version 1.9 book - Part 81 of 210
The Ring programming language version 1.9 book - Part 81 of 210
 
Gitlab ci e kubernetes, build test and deploy your projects like a pro
Gitlab ci e kubernetes, build test and deploy your projects like a proGitlab ci e kubernetes, build test and deploy your projects like a pro
Gitlab ci e kubernetes, build test and deploy your projects like a pro
 

More from Clare Macrae

Testing Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests EasilyTesting Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests EasilyClare Macrae
 
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cppQuickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cppClare Macrae
 
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval TestsQuickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval TestsClare Macrae
 
How to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ EffectivelyHow to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ EffectivelyClare Macrae
 
C++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ LondonC++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ LondonClare Macrae
 
Code samples that actually compile - Clare Macrae
Code samples that actually compile - Clare MacraeCode samples that actually compile - Clare Macrae
Code samples that actually compile - Clare MacraeClare Macrae
 
Quickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval TestsQuickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval TestsClare Macrae
 
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019Clare Macrae
 
Quickly testing legacy code cppp.fr 2019 - clare macrae
Quickly testing legacy code   cppp.fr 2019 - clare macraeQuickly testing legacy code   cppp.fr 2019 - clare macrae
Quickly testing legacy code cppp.fr 2019 - clare macraeClare Macrae
 
Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019Clare Macrae
 
Quickly testing legacy code
Quickly testing legacy codeQuickly testing legacy code
Quickly testing legacy codeClare Macrae
 
Escaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releasesEscaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releasesClare Macrae
 
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...Clare Macrae
 

More from Clare Macrae (13)

Testing Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests EasilyTesting Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests Easily
 
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cppQuickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
 
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval TestsQuickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
 
How to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ EffectivelyHow to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ Effectively
 
C++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ LondonC++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ London
 
Code samples that actually compile - Clare Macrae
Code samples that actually compile - Clare MacraeCode samples that actually compile - Clare Macrae
Code samples that actually compile - Clare Macrae
 
Quickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval TestsQuickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval Tests
 
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
 
Quickly testing legacy code cppp.fr 2019 - clare macrae
Quickly testing legacy code   cppp.fr 2019 - clare macraeQuickly testing legacy code   cppp.fr 2019 - clare macrae
Quickly testing legacy code cppp.fr 2019 - clare macrae
 
Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019
 
Quickly testing legacy code
Quickly testing legacy codeQuickly testing legacy code
Quickly testing legacy code
 
Escaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releasesEscaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releases
 
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...
CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery ...
 

Recently uploaded

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)jennyeacort
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesPhilip Schwarz
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesŁukasz Chruściel
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Developmentvyaparkranti
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commercemanigoyal112
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfStefano Stabellini
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf31events.com
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZABSYZ Inc
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based projectAnoyGreter
 

Recently uploaded (20)

SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
Call Us🔝>༒+91-9711147426⇛Call In girls karol bagh (Delhi)
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
Folding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a seriesFolding Cheat Sheet #4 - fourth in a series
Folding Cheat Sheet #4 - fourth in a series
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New Features
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. Salesforce
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
 
2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Development
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commerce
 
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZ
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based project
 

Quickly Testing Qt Desktop Applications

  • 1. 1 Quickly Testing Qt Desktop Applications Clare Macrae (She/her) clare@claremacrae.co.uk 15 November 2019 Meeting C++ 2019, Berlin, Germany
  • 2. 2 Audience: Developers using Qt Approval Tests: claremacrae.co.uk/conferences/presentations.html
  • 3. 3 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 4. 4 About Me • Scientific C++ and Qt developer since 1999 • My mission: Sustainable and efficient testing and refactoring of legacy code – Co-author of “Approval Tests for C++” • Consulting & training via “Clare Macrae Consulting Ltd” – https://claremacrae.co.uk • All links from this talk via: – bit.ly/TestingQt – github.com/claremacrae/talks
  • 5. 5 Typical Scenario • I've inherited some Qt GUI code • It's valuable • I need to add feature • Or fix bug • How can I ever break out of this loop? Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests
  • 6. 6 Typical Scenario • • • • • Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests Topics of this talk
  • 7. 7 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 8. 8
  • 9. 9 Qt’s GUI powers make Automated Testing Harder
  • 10. 10 Give you Confidence to Start testing your Qt application Effectively
  • 11. 11 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 12. 12 A: Ah – Good point – You don’t Q: How do I add tests to existing app?
  • 13. 13 Introduce static lib for tests Executable Color Key GUI.exe main() and *.cpp *.ui
  • 14. 14 Introduce static lib for tests Static Library Executable Color Key GUI.exe main() UI.lib *.cpp *.ui
  • 15. 15 Introduce static lib for tests Static Library Executable Color Key GUI.exe main() UITests.exe main() and tests/*.cpp UI.lib *.cpp *.ui If too hard, could move just the code you are going to test
  • 16. 16 Doing the Separation • For CMake, see Arne Mertz’s excellent tutorial: • https://arne-mertz.de/2018/05/hello-cmake/ • Remember to make the library static!
  • 17. 17 Context: Type of testing • Glass-box or transparent-box testing • Have access to the code • And can potentially even make changes to it
  • 18. 18 Creating the Test Executable
  • 19. 19 C++ Test Runners • Lots to choose from: • You can use any – I’m using Catch2, with a little bit of Qt Test • I found there were tricks needed to set up Catch2 – you will probably need to do the same if you use a different framework
  • 20. 20 Setting up testsuite main() // Demonstrate how to create a Catch main() for testing Qt GUI code #define CATCH_CONFIG_RUNNER #include <Catch.hpp> #include <QApplication> int main(int argc, char* argv[]) { QApplication app(argc, argv); int result = Catch::Session().run(argc, argv); return result; }
  • 21. 21 Context: SuperCollider • supercollider.github.io • platform for audio synthesis and algorithmic composition Picture credit: Chad Cassady, @beatboxchad
  • 22. 22 Context: ColorWidget color-picker This set of widgets is ”our” code: • we want to test it This set of widgets is ”Qt” code: • not our job to test it • hard to test anyway • only test our stuff!
  • 23. 23 ColorWidget setup – QColor object #include "catch.hpp" #include "color_widget.hpp" #include "catch_qt_string_makers.hpp" using namespace ScIDE; TEST_CASE("ColorWidget initial state") { ColorWidget widget; QColor expected_color(0, 0, 0, 255); CHECK(widget.color() == expected_color); }
  • 24. 24 ColorWidget setup – approval testing #include "catch.hpp" #include "color_widget.hpp" #include "catch_qt_string_makers.hpp" using namespace ScIDE; TEST_CASE("ColorWidget initial state - approval testing") { ColorWidget widget; auto color = Catch::StringMaker<QColor>::convert(widget.color()); CHECK(color == "(0, 0, 0), alpha = 1"); }
  • 25. 25 ColorWidget changing state TEST_CASE("ColorWidget changing color updates correctly") { // Arrange ColorWidget widget; QColor red("red"); // Act widget.setColor(red); // Assert CHECK(widget.color() == red); }
  • 26. 26 “Quickly Testing?” • Getting from 0 to 1 tests is always the hardest – that’s normal • Getting from 1 to 2, and then 2 to 3, always much easier • Step 1: Make the first test as easy as possible – it’s going to be hard! • Step 2: Write at least 3 tests before deciding if it’s a good idea!
  • 27. 27 Context: “Go To Line” Panel
  • 28. 28 Goal: Demo the basic code • Warning: Some detail coming up! – Later I’ll show how to avoid it… • Basic steps: – Arrange – Act – Assert
  • 29. 29 High level view – version 1 class GoToLineTool : public QWidget { ... private: QSpinBox* mSpinBox; QToolButton* mGoBtn; TEST_CASE("GoToLineTool …") { // Peek inside GoToLineTool to // access widgets ... } Implementation Tests
  • 30. 30 Arrange: Set up Widget TEST_CASE("GoToLineTool emits signal when Go button clicked") { // -------------------------------------------------------- // Arrange GoToLineTool widget; widget.raise(); widget.show(); widget.setMaximum(27); // This is needed to make sure text in the spinner is selected, so that // as we type in characters, the initial text ("1") is erased widget.setFocus();
  • 31. 31 Arrange: Accessing private widgets – V1 // Allow us to interact with widgets inside GoToLineTool: auto spinner = widget.findChild<QSpinBox*>(); REQUIRE(spinner != nullptr); auto goButton = widget.findChild<QToolButton*>(); REQUIRE(goButton != nullptr); ✘ Beware: findChild() not recommended, especially in production code!
  • 32. 32 Test actions – Signals and Slots 1 7 emit activated(17); QSignalSpy(&mGoToLineWidget, &GoToLineTool::activated)
  • 33. 33 Arrange: Listen to activate() signal // Enable tracking of one signal // New-style code QSignalSpy activatedSpy(&widget, &GoToLineTool::activated); REQUIRE(activatedSpy.isValid()); // Old-style code //QSignalSpy activatedSpy(&widget, SIGNAL(activated(int))); //REQUIRE(activatedSpy.isValid());
  • 34. 34 Act: Generate user events 1 7 // -------------------------------------------------------- // Act // Type a number, one digit at a time QTest::keyClicks(spinner, "1"); QTest::keyClicks(spinner, "7"); // Clicking the Go button: goButton->click();
  • 35. 35 Assert: Check the changes // -------------------------------------------------------- // Assert // Check the activated() signal was emitted only once: REQUIRE(activatedSpy.count() == 1); // And check that the signal emitted the new value: QList<QVariant> arguments = activatedSpy.takeFirst(); const QVariant& argument = arguments.at(0); CHECK(argument.type() == QVariant::Int); CHECK(argument.toInt() == 17); } emit activated(17);
  • 36. 36 High level view – version 1 Implementation Tests class GoToLineTool : public QWidget { ... private: QSpinBox* mSpinBox; QToolButton* mGoBtn; TEST_CASE("GoToLineTool …") { // Peek inside GoToLineTool to // access widgets auto spinner = widget.findChild<QSpinBox*>(); ... }
  • 37. 37 Goal: Make tests super easy to write • findChild() is really fragile – prone to future breakage at run-time – not checked at compile-time
  • 38. 38 Testing Custom Qt Widgets – part 1 Option Advantages Disadvantages widget.findChild()/findChildren() Don’t need to change code being tested No compile time feedback. Maintenance overhead Add public accessor to widget being tested Testing easier Maybe encourages misuse of library if internals exposed
  • 39. 39 High level view – version 2 Implementation Tests class GoToLineTool : public QWidget { ... public: QSpinBox* spinBox(); QToolButton* goButton(); TEST_CASE("GoToLineTool …") { // Ask GoToLineTool for // access widgets auto spinner = widget.spinBox(); ... }
  • 40. 40 Goal: Hide implementation details • Want to avoid tests knowing insides of widgets being tested – As much as possible • Can we improve readability of test? • Can we guard against changes to implementation?
  • 41. 41 High level view – version 3 class GoToLineTool : public QWidget { ... public: QSpinBox* spinBox(); QToolButton* goButton(); Implementation Tests – lots of them!Test Fixture class GoToLineToolFixture { private: GoToLineTool mGoToLineWidget; public: void typeCharacterIntoSpinner( QChar character); ... TEST_CASE_METHOD( GoToLineToolFixture, "GoToLineTool …") { // Ask Fixture to act // on GoToLineTool typeCharacterIntoSpinner('1'); }
  • 42. 42 Introduce a Fixture class GoToLineToolFixture { private: GoToLineTool mGoToLineWidget; QSpinBox* mSpinner = nullptr; QToolButton* mGoButton = nullptr; std::unique_ptr<QSignalSpy> mActivatedSpy; protected: GoToLineToolFixture() { mGoToLineWidget.raise(); mGoToLineWidget.show(); // ... and so on ...
  • 43. 43 GoToLineTool – testing events, expressively TEST_CASE_METHOD(GoToLineToolFixture, "GoToLineTool emits signal when Go button clicked") { // Arbitrary upper limit in number of lines. // When used in the application, this would be obtained from the open docunent setMaximumLineCount(27); // Type a number, one digit at a time typeCharacterIntoSpinner('1'); typeCharacterIntoSpinner('7’); clickGoButton(); checkActivatedSignalCount(1); checkActivatedSignalValue(17); }
  • 44. 44 Testing Custom Qt Widgets – part 1 Option Advantages Disadvantages widget.findChild()/findChildren() Don’t need to change code being tested No compile time feedback. Maintenance overhead Add public accessor to widget being tested Testing easier Maybe encourages misuse of library if internals exposed
  • 45. 45 Testing Custom Qt Widgets – part 2 Option Advantages Disadvantages widget.findChild()/findChildren() Don’t need to change code being tested No compile time feedback. Maintenance overhead Add public accessor to widget being tested Testing easier Maybe encourages misuse of library if internals exposed Add protected accessor … Make the test fixture inherit the widget Mostly hides the implementation Maybe encourages inheritance to customize behavior. Test fixture has to inherit class being tested – maybe confusing Add private accessor … Make the test fixture a friend of the widget Hides the implementation more Friendship not inherited by test implementation – so there’s more work to create the test fixture
  • 47. 47 Bonus Points: Separate logic from UI code Static Library Executable Color Key GUI.exe main() UITests.exe main() and tests/*.cpp UI.lib *.cpp *.ui Model.lib ModelTests.exe *.cpp main() and tests/*.cpp
  • 48. 48 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 49. 49 • Run-time error checking: Pitfall: Qt4-style Signal/Slot connect() connect( ui->quitButton, SIGNAL(clicked()), this, SLOT(quit())); QObject::connect: No such slot SampleMainWindow::quit() QObject::connect: (sender name: 'quitButton') QObject::connect: (receiver name: 'SampleMainWindow')
  • 50. 50 Safer: Qt5-style Signal/Slot connect() • Use Qt5 pointer-to-member-function connections connect( ui->quitButton, SIGNAL(clicked()), this, SLOT(quit())); // Solution: Qt5-style connections get checked at compile-time: connect( ui->quitButton, &QPushButton::clicked, this, &SampleMainWindow::quit );
  • 51. 51 Pitfall: Event processing in tests • Normally, Qt’s event loop processes events automatically – QCoreApplication::processEvents() • Can’t rely on that in test programs • Know your options: – bool QTest::qWaitForWindowActive( window/widget, timeout ) – bool QTest::qWaitForWindowExposed( window/widget, timeout ) – bool QSignalSpy::wait( timeout ) • If your test runner is Qt Test: – QTRY_COMPARE, QTRY_COMPARE_WITH_TIMEOUT – QTRY_VERIFY, QTRY_VERIFY_WITH_TIMEOUT – These macros include a return statement on failure
  • 52. 52 Running Qt tests in CI system • Windows: – Will probably just work • Linux: – Typically use virtual display, e.g. xvfb – Could investigate -platform offscreen • Mac: – Found needed to keep a user logged in • Make tests that need a GUI behave gracefully if no display found: – IF_NO_GUI_PRINT_AND_RETURN()
  • 53. 53 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 55. 55 Introducing ApprovalTests.cpp.Qt •Goals • Rapid start testing Qt code – Useful even if you don’t use Approval Tests! • Approval Tests support for Qt types – Easy saving of state in Golden Master files • https://github.com/approvals/ApprovalTests.cpp.Qt • https://github.com/approvals/ApprovalTests.cpp.Qt.StarterProject • v.0.0.1
  • 57. 57 Setting up testsuite main() // main.cpp: #define APPROVALS_CATCH_QT #include "ApprovalTestsQt.hpp"
  • 58. 58 Checking Table Contents • Inherited complex code to set up a table • Want to add at least a first test – of the text in the cells
  • 59. 59 Verifying a QTableWidget TEST_CASE("It approves a QTableWidget") { // A note on naming: QTableWidget is a concrete class that implements // the more general QTableView. Here we create a QTableWidget, // for convenience. QTableWidget tableWidget; populateTable(tableWidget); ApprovalTestsQt::verifyQTableView(tableWidget); }
  • 61. 61 Checking Screenshots? No! • Not recommended, in general • Styles differ between Operating Systems • Better to test behaviour, not appearance
  • 62. 62 Checking rendering • But if you have to do it…
  • 63. 63 Verifying a QImage TEST_CASE("It approves a QImage") { QImage image(10, 20, QImage::Format_RGB32); image.fill(Qt::red); ApprovalTestsQt::verifyQImage(image); }
  • 64. 64
  • 65. 65 Caution! • Avoid Qt Test macros in this project – QCOMPARE, QVERIFY, QTRY_COMPARE and so on – Any test failures will be silently swallowed – Tests will spuriously pass. – [November 2019: Fabian Kosmale at Qt has been really helpful on this – there is hope!]
  • 66. 66 What next for ApprovalTests.cpp.Qt? • Fix Qt Test macros problem • Seek feedback • Could… – Support more test frameworks – Support approving more Qt types – Add real-world examples to the docs
  • 67. 67 Contents • Introduction • Qt – Setting Up Testing – Error-prone Things – Approval Tests • Extras – Tools – Summary
  • 68. 68 Navigation around your GUI • Where is that widget in the source code? – Find strings in UI and search – Use KDAB GammaRay kdab.com/development-resources/qt-tools/gammaray
  • 69. 69 clazy • Qt related compiler warnings and fixits • Lots of useful warnings • github.com/KDE/clazy • Level 2 check: old-style-connect – Read the warnings on this check, though
  • 70. 70 Squish • Automated GUI Testing – of your whole application – Semi-opaque-box testing • froglogic.com/squish – Commercial tool • Basics – Record actions in your application – Refactor in to reusable tests, e.g. in Python • Opinion – Not a replacement for unit tests – Requires commitment to training and maintenance – If used well, can replace lots of manual testing
  • 71. 71 Contents • Introduction • Qt – Setting Up Testing – Error-prone things – Approval Tests • Extras – Tools – Summary
  • 72. 72 Summary • It’s never too late to start testing! • What to test – Try to test smaller units of code, such as individual widgets – Test behaviors, not appearance (mainly) – Keep application logic separate from GUI • Make tests … – Easy to write – Hide details of custom widgets behind helper functions, in tests – Expressive and readable with fixtures • ApprovalTests.Cpp.Qt feedback welcome!
  • 73. 73 Quickly Test Qt Desktop Applications • All links from this talk, and more, via: – bit.ly/TestingQt – github.com/claremacrae/talks • Sustainable and efficient testing and refactoring of legacy code • Consulting & training via “Clare Macrae Consulting Ltd” – https://claremacrae.co.uk – clare@claremacrae.co.uk

Editor's Notes

  1. Harder than testing non-Graphical User Interface code But it’s still a learnable skill
  2. main() + other source files linked directly to exe? Introduce static library Still uses dynamic link of Qt (essential if using “free” version) New test exe links new library OR – just move the bit you are going to test!!!
  3. What’s the point of pulling stuff in to static library or exe? Faster builds Consistent builds Distribution process unchanged
  4. https://github.com/approvals/ApprovalTests.cpp.Qt/blob/96e9f12c618fda43991463d4356a5bf8c5ca4f7a/examples/vanilla_catch_qt_tests/main.cpp
  5. https://github.com/claremacrae/supercollider/blob/57bd9c2ef9b4fb610c45fdc9ff4e6740300d7196/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/util/color_widget_tests.cpp#L9-L13
  6. https://github.com/supercollider/supercollider/blob/9378f406156a39024b58c0010acd65b38d966852/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/util/color_widget_tests.cpp#L15-L19
  7. https://github.com/claremacrae/supercollider/blob/57bd9c2ef9b4fb610c45fdc9ff4e6740300d7196/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/util/color_widget_tests.cpp#L15-L25 setColor() is public in the widget we are testing
  8. The UI depends on the Model, and not the other way round – logic is separated
  9. https://github.com/claremacrae/supercollider/blob/b253166c6c69d07ea992b1e6d50036f4cba583fa/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp Qt test framework handles lots of this. always a good idea to give your widget focus
  10. https://github.com/claremacrae/supercollider/blob/b253166c6c69d07ea992b1e6d50036f4cba583fa/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp White-box testing – just don’t use it in production code
  11. https://github.com/claremacrae/supercollider/blob/b253166c6c69d07ea992b1e6d50036f4cba583fa/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp
  12. https://github.com/claremacrae/supercollider/blob/b253166c6c69d07ea992b1e6d50036f4cba583fa/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp
  13. https://github.com/claremacrae/supercollider/blob/b253166c6c69d07ea992b1e6d50036f4cba583fa/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp
  14. The UI depends on the Model, and not the other way round – logic is separated
  15. The UI depends on the Model, and not the other way round – logic is separated
  16. The UI depends on the Model, and not the other way round – logic is separated
  17. https://github.com/claremacrae/supercollider/blob/d9d5693811980884a4bf362f0d7bf699dc03ac34/testsuite/editors/sc-ide/catch2_sc-ide_tests/widgets/goto_line_tool_tests.cpp#L60-L73
  18. The UI depends on the Model, and not the other way round – logic is separated
  19. Qt4-style signal/slot connections only get checked at run time They are string-based macros. If application doesn’t have a console, users (and developers) will never see the error
  20. Qt4-style signal/slot connections only get checked at run time They are string-based macros. If application doesn’t have a console, users (and developers) will never see the error If using Qt5, avoid the old syntax
  21. Qt Test does this for you
  22. https://github.com/approvals/ApprovalTests.cpp.Qt/blob/199bd998cefc4fa15389872a50af102711f898b1/tests/Catch2_Tests/main.cpp#L3-#L4
  23. A lot of this stuff you can get going on your own. If you get stuck, I can help you!