This document discusses strategies for maintaining a legacy Python codebase. It presents a three part "maintenance framework": 1) Understand the system through documentation, debugging, profiling and testing; 2) Establish a safety net using automated tests, continuous integration, monitoring and automation; 3) Improve the code through linting, complexity analysis and code reviews, and improve development practices through test-driven development. The document argues that software maintenance is a major cost and recommends these strategies to help understand an existing codebase and establish processes to prevent issues and improve quality over time.

1. 1
So you "want" to maintain a
Python legacy code base
PyConUK 2016
César Cardenas Desales

2. 2
About me
● Software Architect at Webrepublic AG
● Python user since v.2.0
● Co-organizer of the Swiss Python Summit (i.e.
PyCon Switzerland) and the Zurich Python User
Group

3. 3
Agenda
❖ Introduction
❖ Maintenance “framework”
1. Understand (your system)
2. Safety net
3. Improve

4. 4
Introduction

5. 5
Why discuss maintenance?
● 50% software cost - Barry
Boehm
● 40 to 80% software costs -
Robert L. Glass

6. 6
Computer programming

7. 7
Software maintenance

8. 8

9. 9
Software maintenance?

10. 10

11. 11
Automotive Industry
● Warranty? 5 years or 100,000 KM
● Life-time: 100,000 to 200,000 KM

12. 12
Rail Industry
● Short cycle maintenance: 80 years

13. 13
Rail Industry
● Short cycle maintenance: 80 years
● Long cycle maintenance: 150 years

14. 14
IT Industry?
● Linux kernel: 1991, 25 years
● LaTeX: 1985, 31 years
● U.S. Treasury Department - Individual Master File: 56
years

15. 15
IT Industry?
● Linux kernel: 1991, 25 years
● LaTeX: 1985, 31 years
● U.S. Treasury Department - Individual Master File: 56
years
● Probably many Fortran or Cobol systems
COBOL programmer wanted
- 5 years experience life expectancy

16. 16
IT Industry?
● Linux kernel: 1991, 25 years
● LaTeX: 1985, 31 years
● U.S. Treasury Department - Individual Master File: 56
years
● Probably many Fortran or Cobol systems
● Mine WAS 4 years
● Yours?

17. 17
Maintenance: Raffle of a Tiger

18. 18
The life of a maintainer (i.e. me)
1. “Our code is messy”
2. “Everyone’s code is messy” (3k LOC class)
3. “Somebody’s code is messy”

19. 19
1 Understand

20. 20
1 Understand - Context
● Code doesn’t live in a
void
● Ask users
● THE business case
● Interview power users
● Create personas
● Read use cases
● Write use cases

21. 21
1 Understand - Architecture
● Understand the
architecture
● Create diagrams

22. 22
1 Understand - Architecture
● Print critical code
● Read the code
● Do code walkthroughs
● Rubber duck debugging

23. 23
1 Understand - Docs
● Read the docs
● Write the docs
● Comment the code
● Docstrings
● Doxygen
● Sphingx

24. 24
1 Understand - Debug it
● Observe program behaviour
● For the sake of it
● pdb
● pudb
● ipdb
● Bugjar
● Winpdb
● Your IDE

25. 25
1 Understand - Profile it
● Does 80/20 hold?
● cProfile, profile
● timeit
● heapy
● memory_profiler
● Your IDE
● top, htop
● IPython SnakeViz

26. 26
1 Understand - Break it
● Scalability?
● What breaks?
● Why does it break?
● Does it matter?
● Recovery?
● nose, pytest
● Apache Benchmark
● Locust
● JMeter
● Adhoc scripts

27. 27
2 Safety net

28. 28
2 Safety net - Automated tests
● Regression tests
● Acceptance tests
● Data driven tests
● nose, pytest
● coverage
● selenium
● tox

29. 29
2 Safety net - Fail often and early
● Neutral/Nightly build
● Continuous Integration
● Broken build =>
Not safe to release

30. 30
2 Safety net - The build
● Github protected branches
● Jenkins
● Buildout
● Buildbot
● Travis CI

31. 31
2 Safety net - Monitoring
● What errors are current?
● Opbeat

32. 32
2 Safety net - Monitoring
● Visualize and filter log entries
● Logstash + Kibana

33. 33
2 Safety net - Automate it all
● Because humans make
mistakes
● Provisioning deployment
● Ansible
● Salt
● Puppet
● Chef

34. 34
3 Improve

35. 35
3 Improve - The code
● Leave the code cleaner than you found it
● pep8 -> pycodestyle
● Flake8 (PyFlakes + pycodestyle)
● pylint
● radon
● Your IDE

36. 36
3 Improve - The code
● Radon
○ LOC, SLOC
○ Cyclomatic Complexity
○ Maintainability Index
○ Halstead Metrics

37. 37

38. 38
3 Improve - The code
$ radon cc -o SCORE -s -a processing/tasks.py

39. 39
3 Improve - The code
$ pylint bot.py

40. 40
3 Improve - The people
● TDD if possible
● Do Code Reviews

41. 41
3 Improve - The people
● Code Reviews
○ Spread knowledge
○ The Ego Factor
○ Increase quality of software
“Your reputation at stake
with every commit”

42. 42
3 Improve - The people
● Code Reviews
○ Easily done with Github
○ Like washing your hands
after going to the toilet

43. 43
Ask me
● Ascend a Swiss
(Python) summit
● Why can you buy
cornish pasties in
Mexico

44. 44
Let’s make it happen.
Thank you.
@ccdesales