The Professional Programmer


Published on

Tutorial that I gave at the London Perl Workshop in December 2009.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Professional Programmer

  1. The Professional Programmer Dave Cross Magnum Solutions Ltd [email_address]
  2. The Professional Programmer <ul><li>Looking beyond programming skills
  3. What else do you need to be a successful programmer? </li></ul>
  4. The Pragmatic Programmer <ul><li>The Pragmatic Programmer </li><ul><li>Andrew Hunt
  5. David Thomas </li></ul></ul>
  6. What We Will Cover <ul><li>Unit testing
  7. Source code control
  8. Continuous integration
  9. Coding Standards
  10. Agile techniques </li></ul>
  11. Unit Testing
  12. Unit Testing <ul><li>Never program without a safety net
  13. Does your code do what it is supposed to do?
  14. Will your code continue to do what it is supposed to do?
  15. Write unit tests
  16. Run those tests all the time </li></ul>
  17. What is Unit Testing? <ul><li>Code that exercises an individual element (unit) of your code
  18. Provide known inputs
  19. Look for expected outputs
  20. “Bottom-up” testing </li></ul>
  21. Typical Unit Tests <ul><li>Create an object
  22. Call methods with known inputs
  23. Look for expected outputs </li></ul>
  24. Non-OO Unit Tests <ul><li>Possible to test non-OO code
  25. Call functions rather than methods
  26. Even call programs and examine their output </li><ul><li>Output files
  27. Effect on databases
  28. Etc </li></ul></ul>
  29. Unit Testing Advantages <ul><li>Confidence in the building blocks of your program
  30. Modular code is easier to test
  31. Well-tested code is good
  32. Easy to test code is better </li></ul>
  33. Unit Testing in Perl <ul><li>Perl has a culture of unit testing
  34. All Perl modules ship with test suites
  35. Lots of work in this area since 2001
  36. Perl makes it easy to write test suites </li></ul>
  37. Simple Test Program <ul><li>use Test::More tests => 4; BEGIN { use_ok('My::Object'); } ok(my $obj = My::Object->new); isa_ok($obj, 'My::Object'); $obj->set_foo('Foo'); is($obj->get_foo, 'Foo'); </li></ul>
  38. Simple Test Output <ul><li>$ prove -v test.t test.... 1..4 ok 1 - use My::Object; ok 2 ok 3 - The object isa My::Object ok 4 ok All tests successful. Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.05 cusr 0.00 csys = 0.07 CPU) Result: PASS </li></ul>
  39. How Good Are Your Tests? <ul><li>How much of your code is exercised by your tests?
  40. Devel::Cover can help you to find out
  41. Deep internal magic
  42. Draws pretty charts </li><ul><li>./Build testcover </li></ul></ul>
  43. Devel::Cover Output
  44. Devel::Cover Output
  45. Devel::Cover Output
  46. Other Advantages <ul><li>A test suite gives you confidence in your code
  47. But it's also a specification of your code
  48. Your code works if it passes the tests
  49. Once the tests pass you can stop writing code
  50. See Test-Driven Development later </li></ul>
  51. Further Information <ul><li>Perl Testing – A Developer's Notebook </li><ul><li>Ian Langworth
  52. chromatic </li></ul></ul>
  53. Source Code Control
  54. Source Code Control <ul><li>Source code is a programmer's most valuable asset
  55. If you lose your source code you are in trouble
  56. Ensure your source code is always safe </li></ul>
  57. Other Advantages <ul><li>Rewind to previous versions of code </li><ul><li>Encourages experimentation </li></ul><li>Mark particular versions as important </li><ul><li>Released versions </li></ul><li>Branching and merging </li></ul>
  58. Brief History <ul><li>SCCS
  59. RCS
  60. CVS
  61. Subversion
  62. Git </li></ul>
  63. Typical Workflow <ul><li>Network-available centralised repository of source code </li><ul><li>LAN or internet </li></ul><li>Developer checks out local copy of code
  64. Makes changes </li><ul><li>Ensures unit tests still pass </li></ul><li>Uploads new versions </li></ul>
  65. Locking <ul><li>Older systems locked code whilst a developer worked on it
  66. Only one developer working on code at a time
  67. No chance of conflicts
  68. Development bottlenecks </li></ul>
  69. Non-Locking <ul><li>“Optimistic concurrency”
  70. Anyone can work on any file at any time
  71. Conflicts detected at check-in
  72. Most of the time people don't work on the same files </li><ul><li>Or, at least, on the same parts of the same file </li></ul><li>More effective use of time </li></ul>
  73. Branching & Merging <ul><li>Multiple versions of code available simultaneously </li><ul><li>Released version
  74. Development version
  75. Experimental version </li></ul><li>Merge changes back into older branches
  76. Merging easier with more modern systems </li></ul>
  77. Distributed Version Control <ul><li>Network model allows many people to work on same codebase
  78. Single master repository on the internet
  79. Need internet access to check in
  80. Distributed model has no master repository
  81. Many equal peers
  82. Check into local repository
  83. Merge with peers when online </li></ul>
  84. Git <ul><li>Git is currently the most popular source code control system
  85. Written by Linus Torvalds for the Linux kernel
  86. Distributed model
  87. Strong support for branching and merging </li></ul>
  88. GitHub <ul><li>GitHub offers free Git hosting to Open Source projects
  89. Paid hosting also available
  90. Very popular choice
  91. No excuse for not having public git hosting for your project
  92. Also wiki and ticket tracker </li></ul>
  93. Hooks <ul><li>Most systems support some kind of hooks
  94. Code that is run when something is checked in </li><ul><li>Or at other times </li></ul><li>Send email for every check-in
  95. Ensure all checked-in code compiles
  96. Run perltidy
  97. Etc... </li></ul>
  98. Conclusions <ul><li>Source code control is a problem that all programmers have
  99. No excuse for not being familiar with source code control systems
  100. At some point in your career, source code control will save you from an embarrassing mistake
  101. The file you accidentally delete is never under source code control </li></ul>
  102. Continuous Integration
  103. Continuous Integration <ul><li>Continuously monitoring the health of your codebase
  104. Giving visibility to the changes that are being made </li></ul>
  105. What Does It Do? <ul><li>Monitors source code repository
  106. Every time it sees a check-in
  107. Run unit tests
  108. Produce a web site
  109. See if your build is broken </li></ul>
  110. Advantages <ul><li>Instant feedback on the quality of codebase
  111. Instantly know when tests are failing
  112. Instantly know which tests are failing
  113. Instantly know who broke the build </li></ul>
  114. Software <ul><li>Easy to build something to do this
  115. Perl would be an excellent choice
  116. Plenty of ready-built software </li><ul><li>Much of it Open Source </li></ul><li>Most popular choice is currently CruiseControl </li></ul>
  117. Enhancements <ul><li>Don't just run tests
  118. Measure test coverage
  119. Generate documentation
  120. Code complexity statistics
  121. Coding standards checks </li><ul><li>Perl Critic </li></ul><li>Etc... </li></ul>
  122. Build Radiator <ul><li>Standard practice is to build an internal web site
  123. But people have to choose to visit it
  124. Large monitor on the wall of the development office
  125. Rolling dashboard of important stats
  126. No excuse for not seeing the data </li></ul>
  127. Coding Standards
  128. Coding Standards <ul><li>Perl is a very loosely-structured language
  129. Anything goes
  130. Should probably tighten up on that
  131. “perldoc perlstyle” </li><ul><li>How Larry Wall writes Perl </li></ul></ul>
  132. Perl Best Practices <ul><li>Perl Best Practices </li><ul><li>Damian Conway </li></ul></ul>
  133. Perl Best Practices <ul><li>Damian Conway's ideas on what makes a good Perl program
  134. Naming conventions
  135. Code layout
  136. Disallowed constructs
  137. Recommended CPAN modules </li></ul>
  138. Disagreeing With PBP <ul><li>You don't need to agree with everything that Damian says
  139. But you should consider his ideas
  140. And have a good reason for any disagreements </li></ul>
  141. Code Layout <ul><li>Perl compiler doesn't care about your code layout
  142. But the maintenance programmer will
  143. Make your code as easy as possible to follow
  144. Consistent formatting is a big help </li></ul>
  145. Maintenance Programmers <ul><li>Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live </li></ul>
  146. Maintenance Programmers <ul><li>Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live </li></ul>
  147. Maintenance Programmers <ul><li>Bear that image in mind whilst writing code
  148. I find it helps me focus </li></ul>
  149. Enforcing Code Layout <ul><li>Different programmers have different ideas about good code layout
  150. No right answer
  151. Brace positioning
  152. Cuddled else
  153. Size of indent
  154. Maximum line length </li></ul>
  155. perltidy <ul><li>Perl tidy automatically reformats code according to various rules
  156. Decide on a project standard
  157. Create a perltidy config file with those rules
  158. Everyone can use their own local rules
  159. But run standard config before check-in </li><ul><li>Or even during check-in </li></ul></ul>
  160. Enforcing Best Practices <ul><li>Many suggestions in Perl Best Practices are well worth following
  161. Enforce them automatically
  162. Perl::Critic does this
  163. Command line program “perlcritic”
  164. $ perlcritic
  165. Very depressing output </li></ul>
  166. perlcritic <ul><li>Five levels of criticism available
  167. Gentle, stern, harsh, cruel, brutal
  168. Gentle is default </li><ul><li>Reports only most severe violations </li></ul><li>Change that on command line
  169. $ perlcritic –brutal </li></ul>
  170. Configuring perlcritic <ul><li>Not everyone agrees with all perlcritic rules
  171. Configure behaviour with .perlcriticrc
  172. Turn off certain policies
  173. Or invent your own </li><ul><li>Based on PPI </li></ul><li>Have a project-wide .perlcriticrc </li></ul>
  174. Criticism Statistics <ul><li>For continuous integration it's useful to get statistics from a file
  175. $ perlcritic –statistics
  176. $ perlcritic –statistics-only </li></ul>
  177. Criticism Statistics <ul><li>$ perlcritic --statistics-only lib/Calendar/ 1 files. 4 subroutines/methods. 128 statements. 269 lines. Average McCabe score of subroutines was 8.50. 1 violations. Violations per file was 1.000. Violations per statement was 0.008. Violations per line of code was 0.004. 1 severity 5 violations. 1 violations of BuiltinFunctions::ProhibitStringyEval. </li></ul>
  178. Coding Standards <ul><li>Many valid options in Perl
  179. Choose one set of options
  180. Enforce with perltidy and perlcritic </li></ul>
  181. Agile Techniques
  182. Systems Analysis and Design <ul><li>In the bad old days, Systems Analysis and Design was a huge task
  183. Analysts would spend months analysing a problem
  184. Huge specification documents would be written
  185. All before a line of code was written </li></ul>
  186. System Design Diagrams <ul><li>The specs were always full of diagrams
  187. How the system looks now
  188. How the system should look
  189. Data flow diagrams
  190. Entity-Relationship diagrams
  191. Jackson Structured Programming </li></ul>
  192. The Flaw <ul><li>This all took too long
  193. Months before spec was complete
  194. Then more time whilst system was coded, tested and deployed
  195. By the time the system was complete, the problem had changed
  196. Back to the drawing board </li></ul>
  197. Agile Programming <ul><li>Analysis and design needed to be more agile
  198. Able to respond to demands more quickly
  199. Turn around projects in less time
  200. Deliver value sooner </li></ul>
  201. Agile Methodologies <ul><li>Many “lightweight” methods arose in mid 1990s
  202. Extreme programming
  203. Scrum
  204. Meeting Utah Feb 2001
  205. Agile Manifesto </li></ul>
  206. Agile Manifesto <ul><li>We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: </li><ul><li>Individuals and interactions over processes and tools
  207. Working software over comprehensive documentation
  208. Customer collaboration over contract negotiation
  209. Responding to change over following a plan </li></ul><li>That is, while there is value in the items on the right, we value the items on the left more. </li></ul>
  210. Individual & Interactions <ul><li>People are more important than processes
  211. Development teams know how to organise their work
  212. Business says what they need
  213. Development team works out how to achieve that
  214. Giving people more control and responsibility </li></ul>
  215. Working Software <ul><li>Documentation is often out of date before it is finished
  216. Mismatch between business language and technical language
  217. Business often don't know what they want until they see the options
  218. Build prototypes and discuss alternatives </li></ul>
  219. Customer Collaboration <ul><li>Don't just involve users when the project is specified
  220. Have user representatives as a part of the development team
  221. Instant response to developer questions
  222. More involvement leads to more business ownership of project </li></ul>
  223. Responding to Change <ul><li>Requirements will change during the project lifetime
  224. Be flexible
  225. Plan for change </li></ul>
  226. Scrum <ul><li>Common agile process for software development
  227. In use at many companies </li></ul>
  228. Scrum Roles <ul><li>Scrum Master: manages the process (like project manager)
  229. Product Owner: represents the stakeholders
  230. Team: people responsible for analysis, design, implementation and testing
  231. Stakeholders: people who are requesting the work </li></ul>
  232. Scrum Process <ul><li>Team decides on the length of a sprint (between one and four weeks)
  233. Stakeholders build up a product backlog
  234. Stakeholders (with product owner) prioritise most important work
  235. Team work out how much they can do in one sprint
  236. Team carry out work
  237. Team demonstrate progress at end of sprint </li></ul>
  238. Scrum Meetings <ul><li>Sprint planning: start of sprint </li><ul><li>Converts product backlog to sprint backlog </li></ul><li>Daily scrum: short meeting where team members report on progress and plan day
  239. Sprint demo: end of sprint </li><ul><li>Demo work to stakeholders </li></ul><li>Sprint retrospective: end of sprint </li><ul><li>Team discusses what went well and what can be improved </li></ul></ul>
  240. Scrum Artifacts <ul><li>Product backlog </li><ul><li>Long list of things to do
  241. High level </li></ul><li>Sprint backlog </li><ul><li>More detailed descriptions of current projects
  242. Broken down to tasks of about a day </li></ul><li>Burn down chart </li><ul><li>Lists work planned for current sprint
  243. Current progress plotted </li></ul></ul>
  244. YAGNI <ul><li>“You Ain't Gonna Need It”
  245. Do not plan for the future
  246. Only implement what is required now
  247. The simplest thing that could possibly work
  248. Refactor and add functionality later </li></ul>
  249. Refactoring <ul><li>Changing internal structure without modifying external behaviour
  250. Changes made to make next steps easier
  251. Run test plan
  252. Refactor
  253. Run test plan again </li></ul>
  254. Refactoring <ul><li>Refactoring </li><ul><li>Martin Fowler </li></ul></ul>
  255. Test Driven Development <ul><li>Test plans give you confidence that you code does what it is supposed to do
  256. And that your refactoring hasn't broken anything
  257. They can also act as a specification for a change
  258. If the code passes this test then this feature has been implemented </li></ul>
  259. Test Driven Development <ul><li>Run existing test plan
  260. Write test for new feature
  261. Run new test plan
  262. Code/refactor until new tests pass
  263. Run complete test plan </li></ul>
  264. Test Driven Development <ul><li>Test-Driven Development </li><ul><li>Kent Beck </li></ul></ul>
  265. Pair Programming <ul><li>Code review is useful
  266. Constant code review is better
  267. Two people working together on a problem
  268. Two heads are better than one
  269. Better initial design
  270. Fewer logic and syntax errors
  271. Benefits outweigh costs </li></ul>
  272. Agile Development <ul><li>Agile methods are becoming more popular
  273. I see many clients using versions of Scrum or Extreme Programming
  274. More common in smaller companies
  275. Larger companies of still prefer their huge specifications
  276. This is changing slowly </li></ul>
  277. The Professional Programmer
  278. The Professional Programmer <ul><li>A few more things to consider
  279. Operating system
  280. Editor/IDE
  281. Language
  282. Business </li></ul>
  283. That's All Folks <ul><li>Any Questions? </li></ul>