Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cpp Testing Techniques Tips and Tricks - Cpp Europe

78 views

Published on

Links from the talk are available at:
https://github.com/claremacrae/talks/blob/master/Cpp_Testing_Techniques_Tips_and_Tricks.md#top

An assortment of practical patterns and techniques to make it easier to write effective automated tests of C++ code, both old and new.

Clare will share some valuable techniques for easier handling of commonly troublesome testing scenarios. Whatever test framework you use, you will take away practical ideas to writer tests more easily and more effectively, to handle challenging automated test scenarios.

This talk is independent of test frameworks, and even covers a little for those creating Qt desktop applications.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Cpp Testing Techniques Tips and Tricks - Cpp Europe

  1. 1. 1 C++ Testing: Techniques, Tips and Tricks Clare Macrae (She/her) clare@claremacrae.co.uk 25 February 2020 Cpp Europe
  2. 2. 2 About Me • C++ and Qt developer since 1999 • My mission: Sustainable and efficient testing and refactoring of legacy code – Co-author of “Approval Tests for C++” • Consulting & training – https://claremacrae.co.uk • All links from this talk via: – github.com/claremacrae/talks
  3. 3. 3 Why? • Share things I wish I knew earlier – Some well known – Some less so… • High-level overview, with links – Avoiding detail • Anything unclear? – Please ask!
  4. 4. 4 Assumptions • Value of testing – safety net! • No worries about types of tests – (unit, integration, regression)
  5. 5. 5 Foundations
  6. 6. 6 Tip: Pick a Test Framework • Lots to choose from, including: • Get to know it well!
  7. 7. 7 Catch2 basic example // Let Catch provide main(): #define CATCH_CONFIG_MAIN #include <catch2/catch.hpp> int Factorial( int number ) { return number <= 1 ? 1 : Factorial( number - 1 ) * number; } TEST_CASE( "Factorial of 0 is 1" ) { REQUIRE( Factorial(0) == 1 ); } TEST_CASE( "Factorials of 1 and higher are computed" ) { REQUIRE( Factorial(1) == 1 ); REQUIRE( Factorial(2) == 2 ); REQUIRE( Factorial(3) == 6 ); REQUIRE( Factorial(10) == 3628800 ); }
  8. 8. 8 But what about Real World Code?
  9. 9. 9 Rarely that simple! • Long methods • Interdependent code • Tests hard to set up • Not designed to be testable
  10. 10. 10 Tip: Take any reasonable measure! • … to get code tested! • Don’t get hung up on perfect code… • What does that mean?
  11. 11. 11 Tip: Take any reasonable measure! • When you are stuck, write the test you want • Then safely do whatever you need to your implementation – temporarily! – Make method static to call from outside? – Add getters – and maybe even setters? – Public data? • Hold your nose! • Then refactor
  12. 12. 12 Prioritise Testable code Over Perfect code
  13. 13. 13 How do I test private code? Don’t!
  14. 14. 14 Tip: Don’t test private code • Automated tests give freedom to safely change implementation • Testing private code breaks that • Only test observable – public – behaviours
  15. 15. 15 What could possibly go wrong? • It’s easy to think your test is passing… • I’ve been bitten by all of these: – Forget to add test source to build – IDE calls a different test – Test is doing the wrong thing – Test is accidentally disabled
  16. 16. 16 Never trust a test until you have seen it fail!
  17. 17. 17 Book recommendations
  18. 18. 18 Summary: Foundations • Use a test framework – get to know it well • Take any reasonable measure to add tests – Then improve it • Test public behaviour – not private implementation • Testable code over perfect code • Never trust a test until you have seen it fail
  19. 19. 19 Private code revisited
  20. 20. 20 How do I test private code? Don’t!
  21. 21. 21 But what if… • Tiny real world example • “GoToLineTool” widget • User can type in to the spinner
  22. 22. 22 Testing goal • Test that widget communicates changes correctly when user types in to widget • lineNumberChanged() • How to test that?
  23. 23. 23 Options – none are satisfactory • Ideas – Search the widget hierarchy for widgets by name – Make the spinner widget data member public – Provide a public accessor to the data member – Make the test a friend of the class GoToLineTool class • All if these require the test to be reworked if widget changes
  24. 24. 24 Goal: Make tests super easy to write • And super easy to maintain… • Test code accessing widgets directly makes that hard
  25. 25. 25 Technique: Separate concerns with Fixture class GoToLineTool : public QWidget { ... public/protected/private: QSpinBox* spinBox(); QToolButton* goButton(); Implementation Tests – lots of them!Test Fixture class GoToLineToolFixture { private: GoToLineTool mGoToLineWidget; public: void typeCharacterIntoSpinner( QChar character); ... TEST_CASE_METHOD( GoToLineToolFixture, "GoToLineTool …") { // Ask Fixture to act // on GoToLineTool typeCharacterIntoSpinner('1'); }
  26. 26. 26 Summary: Fixtures • Test Fixtures can make tests of complex code expressive • And Maintainable
  27. 27. 27 Inheriting Legacy/Existing Code
  28. 28. 28 Typical Scenario • I've inherited some legacy code • It's valuable • I need to add feature • Or fix bug • How can I ever break out of this loop? Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests
  29. 29. 29 Typical Scenario • • • • • Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests Topics of this part
  30. 30. 30 Golden Master Test Setup Input Data Existing Code Save “Golden Master”
  31. 31. 31 Golden Master Tests In Use Input Data Updated Code Pass Fail Output same? Yes No
  32. 32. 32 Approval Tests TEST_CASE( "Factorials up to 10" ) { // Create a container with values 0 ... 10 inclusive std::vector<int> inputs(11); std::iota(inputs.begin(), inputs.end(), 0); // Act on all values in inputs container: Approvals::verifyAll("Factorial", inputs, [](auto i, auto& os) { os << i << "! => " << Factorial(i); }); }
  33. 33. 33 More Info: Quickly Testing Legacy C++ Code with Approval Tests
  34. 34. 34 Scenario: Golden Master is a log file • Dates and times? • Object addresses? 2019-01-29 15:27
  35. 35. 35 Options for unstable output • Introduce date-time abstraction? • Customised comparison function? • Or: strip dates from the log file – Credit: Llewellyn Falco
  36. 36. 36 Tip: Rewrite output file, before testing it 2019-01-29 15:27 [date-time-stamp]
  37. 37. 37 Summary: Legacy Code • Approval Tests • Be creative to make your tests work for you! • After adding tests, start refactoring for maintainability
  38. 38. 38 Testing Qt User Interfaces
  39. 39. 39
  40. 40. 40 Introducing ApprovalTests.cpp.Qt •Goals • Quickly start testing Qt code – Useful even if you don’t use Approval Tests! • Approval Tests support for Qt types – Easy saving of state in Golden Master files • https://github.com/approvals/ApprovalTests.cpp.Qt • https://github.com/approvals/ApprovalTests.cpp.Qt.StarterProject • v.0.0.1
  41. 41. 41 Example: Checking Table Contents • Inherited complex code to set up a table • Want to add at least a first test – of the text in the cells
  42. 42. 42 Verifying a QTableWidget TEST_CASE("It approves a QTableWidget") { // A note on naming: QTableWidget is a concrete class that implements // the more general QTableView. Here we create a QTableWidget, // for convenience. QTableWidget tableWidget; populateTable(tableWidget); ApprovalTestsQt::verifyQTableView(tableWidget); }
  43. 43. 43 Approval file: .tsv
  44. 44. 44 Summary: ApprovalTests.cpp.Qt • More on this and on using Fixtures to write Expressive, Maintainable Tests – slideshare.net/ClareMacrae/quickly-testing-qt-desktop-applications
  45. 45. 45 Maintaining Tests
  46. 46. 46 Scenario: Tests too slow for CI • “Builds take 2 to 3 days!” => No hope of Continuous Integration • Most of that was running tests • No prospect of speeding up some very slow tests
  47. 47. 47 Solution: Don’t run slow tests during day • Mark some tests as Slow – Filtering built in to Google Test Framework • Teach CI system to run sub-set of tests – Only run the Slow tests at night – Everything else run on every commit • Thanks to Michael Platings for making this happen!
  48. 48. 48 Tip: Never tolerate Flickering tests • Tests that fail randomly • Rapidly devalue other tests • Fix them, or delete them! • Especially don’t mix performance tests with unit tests
  49. 49. 49 Summary: Maintaining Tests • Fast feedback from Continuous Integration • Stomp on flickering tests
  50. 50. 50 Summary • It’s never too late to start testing! – Pick a framework and practice it – Explore different types of testing • You can test legacy code too! • Keep maintaining your tests • Even GUIs can be tested
  51. 51. 51 C++ Testing: Techniques, Tips and Tricks • All links from this talk, and more, via: – bit.ly/CppTestingTips – github.com/claremacrae/talks • Sustainable and efficient testing and refactoring of legacy code • Consulting & Training – https://claremacrae.co.uk – clare@claremacrae.co.uk • Any Questions? • Any More Tips?

×