Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp

Presented at MUC++, 6 August 2020

Links from this talk are at: https://github.com/claremacrae/talks/blob/main/Quickly_and_Effectively_Testing_Legacy_C++_Code_with_Approval_Tests.md#top

Abstract:

You've inherited some legacy code: it's valuable, but it doesn't have tests, and it wasn't designed to be testable, so you need to start refactoring. But you can't refactor safely until the code has tests, and you can't add tests without refactoring! How can you ever break out of this loop?

Whether Legacy code for you means "old code", "code without tests", or "code you wish to redesign for new features or unit-tests", this talk will enable you to become productive and work safely, quickly.

The simplicity, convenience, ease-of-use, power and flexibility of Llewellyn Falco's "Approval Tests" approach has long been proven in a dozen programming languages. And now all this is now available to C++ developers too!

Clare will present a small but surprisingly effective C++11 library for applying "Approval Tests" to cross-platform C++ code - for both legacy and green-field systems, and with a range of testing frameworks.

She will demonstrate its use in some real-world situations, including how to quickly lock down the behaviour of legacy code. She will show how to quickly achieve good test coverage, even for very large sets of inputs.

Attendees will discover some quick, practical techniques to use for common challenges that can be applied very easily using Approval Tests.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp

  1. 1. 1 Quickly and Effectively Testing Legacy C++ Code with Approval Tests Clare Macrae (She/her) clare@claremacrae.co.uk 6 August 2020 MUC++
  2. 2. 2 About Me • C++ and Qt developer since 1999 • My mission: Sustainable and efficient testing and refactoring of legacy code – Co-author of “Approval Tests for C++” • Consulting & training – https://claremacrae.co.uk • All links from this talk via: – github.com/claremacrae/talks
  3. 3. 3 Goal Share the Power of Approval Tests
  4. 4. 4 Contents •Intro • Getting Started • Legacy Example • Non-deterministic Output • Customisability • Visualising Differences • Summary
  5. 5. 5
  6. 6. 6 Quickly and Effectively Testing Legacy Code
  7. 7. 7 What does Legacy Code really mean? • Dictionary –“money or property that you receive from someone after they die ” • Michael Feathers –“code without unit tests” • J.B. Rainsberger –“profitable code that we feel afraid to change.” • Kate Gregory –“any code that you want to change, but are afraid to”
  8. 8. 8 Typical Scenario • I've inherited some legacy code • It's valuable • I need to add feature • Or fix bug • How can I ever break out of this loop? Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests
  9. 9. 9 Typical Scenario • • • • • Need to change the code No tests Not designed for testing Needs refactoring to add tests Can’t refactor without tests Topics of this talk
  10. 10. 10 Assumptions • Value of automated testing • Evaluate existing tests – Code coverage tools – Mutation testing • No worries about types of tests – (unit, integration, regression)
  11. 11. 11
  12. 12. 12
  13. 13. 13 “Approval Tests allow you to verify a chunk of output (such as a file) in one operation as opposed to writing test assertions for each element”
  14. 14. 14 Questions?
  15. 15. 15 Contents • Intro •Getting Started • Legacy Example • Non-deterministic Output • Customisability • Visualising Differences • Summary
  16. 16. 16 Demo 1: Hello Approvals
  17. 17. 17 Main Methods • Approvals::verify(std::string, Options) – 2 other overloads take template types • Approvals::verifyAll() – For use with containers – Writes one element at a time • All are in namespace ApprovalTests
  18. 18. 18 object verify() writes "received" reads "approved" Same? Pass Fail Reporter Stages of Approvals::verify()
  19. 19. 19 Easy to: • … Add tests quickly • … Visualise differences • … Update output • Convenience – Separates test data from test code – Ignore end-of-line differences
  20. 20. 20 Easy setup • Single header – https://github.com/approvals/ApprovalTests.cpp/releases • Finds diff tool automatically • Sensible filenames automatically – source_directory/test_filename.test_name.approved.txt • “It just works”
  21. 21. 21 Supported Test Frameworks • Lots to choose from: • Get to know chosen one well!
  22. 22. 22 Questions?
  23. 23. 23 Contents • Intro • Getting Started •Legacy Example • Non-deterministic Output • Customisability • Visualising Differences • Summary
  24. 24. 24 Demo 2: Legacy Code: Gilded Rose
  25. 25. 25 Approvals and Legacy Code • Before you start refactoring… • Achieve good coverage • verifyAllCombinations() – Not just for legacy code
  26. 26. 26 CombinationApprovals::verifyAllCombinations() • One line from GildedRose .approved.txt file: (Wibble, 0, 0) => name: Wibble, sellIn: -1, quality: 0 verifyAllCombinations() writes one set of input values and then the result of the lambda for that input set: here, an Item object
  27. 27. 27 Lots of String Conversion Options • String Conversions – Pass in a std::string – Write custom operator<<( std::ostream, … ) – Use Approvals::verify( object, lambda ) – Specialize ApprovalTests::StringMaker::toString( … ) – Use TApprovals<YourStringConvertingClass> • How to Use the Fmt Library To Print Objects – Use {fmt} library via FmtApprovals::verify()
  28. 28. 28 String Design Guidelines • Objects print their relevant data • The data is consistent between runs – (no times, pointers, random) • The data is easy to read, at a glance: • Advice: Tips for Designing Strings [0] = [x: 4 y: 50 width: 100 height: 61] [1] = [x: 50 y: 5200 width: 400 height: 62] [2] = [x: 60 y: 3 width: 7 height: 63] (x,y,width,height) = (4,50,100,61) (x,y,width,height) = (50,5200,400,62) (x,y,width,height) = (60,3,7,63)
  29. 29. 29 Questions?
  30. 30. 30 Contents • Intro • Getting Started • Legacy Example •Non-deterministic Output • Customisability • Visualising Differences • Summary
  31. 31. 31 Demo 3: Log Files
  32. 32. 32 Key Points • Approvals::verifyExistingFile() – For when the file is already written • Scrubbers for non-deterministic output – More scrubber options coming soon…
  33. 33. 33 Questions?
  34. 34. 34 Contents • Intro • Getting Started • Legacy Example • Non-deterministic Output •Customisability • Visualising Differences • Summary
  35. 35. 35 Customisation Points Overview
  36. 36. 36 Scrubber Namer Writer Comparator Pass Fail Reporter Stages of Approvals::verify()
  37. 37. 37 Everything is customisable • We take great pride in the docs • All examples generated from compiled, tested code github.com/approvals/ApprovalTests.cpp/blob/master/doc/ approvaltestscpp.readthedocs.io/en/latest/
  38. 38. 38 Questions?
  39. 39. 39 Contents • Intro • Getting Started • Legacy Example • Non-deterministic Output • Customisability •Visualising Differences • Summary
  40. 40. 40 Demo 4: SVG Files
  41. 41. 41 Key Points • Custom Reporters can help understand failures • Make it easy to visualise differences • No need to version control .png files
  42. 42. 42 See Also ImageApprovals • github.com/p-podsiadly/ImageApprovals • Built on ApprovalTests.cpp • Not yet released
  43. 43. 43 Questions?
  44. 44. 44 Contents • Intro • Getting Started • Legacy Example • Non-deterministic Output • Customisability • Visualising Differences •Summary
  45. 45. 45 Summary
  46. 46. 46 “Approval Tests allow you to verify a chunk of output (such as a file) in one operation as opposed to writing test assertions for each element”
  47. 47. 47 Approval Tests for C++ • Easy to use • Convenient • Powerful • Flexible • It just works!
  48. 48. 48
  49. 49. 49
  50. 50. 50
  51. 51. 51 Quickly & Effectively Test Legacy C++ Code with Approval Tests
  52. 52. 52 References • All links from this talk, and more, via: – github.com/claremacrae/talks • Sustainable and efficient testing and refactoring of legacy code • Consulting & Training – claremacrae.co.uk – clare@claremacrae.co.uk Any Questions?
  53. 53. 53 Contents • Intro • Getting Started • Legacy Example • Non-deterministic Output • Customisability • Visualising Differences • Summary •Appendix: Challenges
  54. 54. 54 Appendix: Common Challenges
  55. 55. 55 External Resources • I/O, Networking, System API calls… • Be creative! • Can you intercept and log calls? – And save them as “approved” results? – Then intercept again during later runs, and compare results? • You can even use this data as inputs to later tests! • Excellent example from Angie Jones: – https://angiejones.tech/verifying-entire-api-responses/
  56. 56. 56 Embedded Systems • Separate as much logic as possible • Test the logic on desktop machines and CI • Then if you get a failure on device, you’ve got less to test • It may become possible to run Approval Tests from cross-built code
  57. 57. 57 Output is OS-specific • Include the OS in the file-names • So verify() compares against output from same platform • For example – TestQtDialog.loginScreen.onMacOSX.approved.png – TestQtDialog.loginScreen.onWindows.approved.png – TestQtDialog.loginScreen.onLinux.approved.png • See Multiple output files per test
  58. 58. 58 Approving is fiddly or takes too long • Approving images? • Approving hundreds of files? • Try one of: – AutoApproveIfMissingReporter – AutoApproveReporter (and compare the diffs in version control!)

×