When you decide to re-write a product from the ground up, there are a lot of decisions to make, and functionality that cannot be changed. We chose to rebuild in Pyramid. This talk will cover our experiences of building a large product in Pyramid. There will be a side by side comparison between the old and the new product, and a bit of background into the reasons for some of the choices we made. It will look into the process of the build, encompassing the following subjects: - Fast prototyping - Theming the app with Diazo - Test Driven Development - Translation - Minimising page reloads using JSON - Supporting per-client customisations both in theme and in functionality - Pitfalls - Problems found and solutions discovered - Does Pyramid support agile development?

1. Migrating a
legacy product
to Pyramid
Tom Blockley

2. Who am I?
• Tom
• Developer for Delib
• Plone for 5 years
• Pyramid for 2 years
• Scrum Master for 18 months
• Before that? Java.

3. Who are Delib?
• The other Plone company in Bristol
• Est. 2001: Consultation solutions for government
• Plone for 8 years
• First to use Dexterity in production sites
• 3 main products, 2 based on Plone
• >150 clients around the world
• >150,000 responses in 2013
• Peak of 42,000 responses in a single week in 2013
• Biggest ever: 47k Users and 110k Responses in 3 months

4. What is this?
• Migration of an old product to the Pyramid framework
• Tech choices
• Problems
• Agile
• Translation
• Testing
• Theming

5. The old
• Conceived 2005
• ZMI product with Kubes
• Completely customisable per-client
• >60 clients, large and small
• Ugly. Really Ugly...

6. The old

7. The new
• Conceived May 2013
• Chose Pyramid
• Must be:
• Pretty
• Responsive
• Accessible
• Translatable
• Customisable
• Tested
• ... and done by October 2013

8. Decisions we made
• Build it using agile methodology
• Use:
• Pyramid
• ZODB
• Buildout
• Diazo
• Colander
• Deform
• repoze.workflow
• Pyramid auth
• i18n

9. How did it go?
• Actually only started in July
• 10 week long sprints
• Fast prototyping to test concepts and UX
• Still a lot to do
• BUT the public side of the app nearly finished

10. Sprint 1 - Ada Lovelace
• 544 lines of python (95 in tests)
• 48%Test coverage

11. Sprint 2 - Bertrand Russell
• 1693 lines of python (272 in tests)
• 76%Test coverage

12. Sprint 3 - Charlie Chaplin
• 2299 lines of python (779 in tests)
• 74%Test coverage

13. Sprint 4 - Don McCullin
• 2419 lines of python (931 in tests)
• 84%Test coverage

14. Sprint 5 - Edwin Hubble
• 2683 lines of python (1079 in tests)
• 85%Test coverage

15. Sprint 6 - Francis Crick
• 2867 lines of python (1266 in tests)
• 86%Test coverage

16. Sprint 7 - Grace Hopper
• 3020 lines of python (1301 in tests)
• 85%Test coverage

17. Sprint 8 - Harry Houdini
• 3414 lines of python (1416 in tests)
• 86%Test coverage

18. Sprint 9 - Indiana Jones
• 3752 lines of python (1494 in tests)
• 86%Test coverage

19. Sprint 10 - Judi Dench
• 4602 lines of python (2027 in tests)
• 87%Test coverage

20. Technologies used with and
in Pyramid

21. Combined Traversal and Dispatch
• Gives you all the good stuff from traversal
• Without defining your URL structure for you

22. Combined Traversal and Dispatch
+
+

23. Combined Traversal and Dispatch
+

24. Auth framework
• Easy to configure
• Can set it up incrementally
• Hooking up with object classes & __acl__ is a piece of cake

25. Predicates
• Request Parameter predicates
• Route prefixing
• Stacking predicates
• Custom predicates
• Exception views
• default_views

27. Colander & Deform
• Does the forms and schema so you don’t have to
• Colander Schema is canonical source for layout

28. Diazo
• Build basically an HTML API
• Keep basic, semantically correct templates
• Apply anyone’s theme to the templates
• Good introduction to XSL
• Don’t pollute your app code with crazy skin compromises
• Designer & Front end developer could go and do their thing

29. Things we really liked about Pyramid
• Auth & group finder
• Super flexible predicates
• Combining traversal and dispatch
• Documentation
• Everything runs in one process
• Fast tests
• Minimal boiler plate
• pshell

30. How did we do against our
original goals?

31. Pretty
• 3 demos in different colour schemes
• Configured with a few lines of LESS

35. Responsive
• Javascript
• Mobile andTablet layouts

36. Accessible
• Well, we’ve not done it yet
• BUT
• We have a set of accessible colour schemes ready to go
• We can fix the accessibility of the markup in Diazo

37. Translatable
• Every single piece of text on the site has corresponding i18n
tag
• No translations yet
• Some of the current theme is too fixed width

38. Customisable
• Fork it
• Theming for clients is easy

39. Tested
• Jenkins integration
• No selenium / robot framework yet
• 87% Coverage

40. Agile
• Fast prototyping
• Populate script
• Make initial implementations as naïve as possible
• We’ve already decided to change the design again

41. What went wrong?

42. Pitfalls
• “Protoduction”
• Short term technical debt
• Leaving interns on their own
• Meeting fatigue
• Arbitrary deadlines

43. Problems
• Incomplete
• Contracted Front End developers
• One tree, many branches
• Rebase is a b*tch
• repoze.workflow

44. Thanks.