Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Green Screen ci at Travis Perkins
Similar to Green Screen ci at Travis Perkins (20)
Recently uploaded
Recently uploaded (20)
Green Screen ci at Travis Perkins
- 1. Using XP practices on 1960s green screen technology The system that time forgot Brian Leach Nik Silver
- 2. Context
- 3. How we went… from this… …to this
- 4. Wider Context • Senior management buy-in It’s not legacy, it’s a • More team-working heritage system. Knowledge silos and functional teams. Pairing, independent teams, breaking down of knowledge silos, closer involvement of stakeholders.
- 6. What is It? The What is UniVerse? Original No SQL Database
- 7. 1960s 1980s MySQL NOSQL 1970s SYBASE SQL SERVER UNIVERSE UNIX REALITY SQL ORACLE GIM-1 NELSON PICK A History Lesson 1990s
- 9. The UniVerse Business Language Embedded Procedural Runtime Loaded Data centric Concurrent Business focused No encapsulation No type-safety No standard tooling
- 10. Do you have global variables ? Hell, yes
- 11. Challenges from the UniVerse Environment The database is NOT a dependency Bad Code is platform independent
- 12. TDD
- 13. What engineers were seeing TDD + FitNesse training for Java/.NET Sometimes learning Eclipse, and Java, too There was a basic test “template” With if/then instead of assert Could not see how unit testing applied to them: Procedural code Not designed to be tested Not part of the culture
- 14. The Challenge – In Numbers •40,000+ concurrent connections Over 700 lines of code added every working day for 25 years • 4x HP Superdomes • 1 million+ reads/machine/second • 12,000+ programs • 4,644,481 lines of application code • 31% have cyc. complexity > 50 • 12 TB changes every day • 120 software engineers • ... And growing
- 19. Lest we Forget This application has powered a business for over 25 years to an annual turnover of 5 billion pounds
- 20. What we did • Wrote our own framework Created own refactoring catalogue Started UniVerse-specific TDD training • Rewrote the framework Simpler API Collaborative effort • Extended the framework File configuration Subroutine fakes
- 21. Some Lessons Learned .. and shared.. Units are different in UniVerse. Via refactoring catalogue The database is under test Via TDD champions There are limits to breaking down procedural code. Critically, they drive each other Starting to inherit tests Community site for discussion of techniques
- 22. Areas of resistance “It’s a waste of time” “It takes too long” “We’re going to throw it away anyway” “We can’t spend time on it if our product owner doesn’t prioritise it” Effective pairing is more difficult with procedural code.
- 23. We had to name the framework
- 24. We had to name the framework
- 26. Git
- 27. Experiences Introducing git for UniVerse • February 2013: TFS introduced for source control Very slow File-driven/fragmented in a central repository • April 2013: CloudShop moved to git (practical and strategic), trigger for similar move for UniVerse • Ambition: Switch over in June firebreak, but lots of anguish…
- 28. Git Anguish • Change of workflow Whole codebase snapshots File locking • Inclination to want all problems are solved. “Git is not suitable for UniVerse”
- 29. Git anguish resolutions… and disappointment • Regular discussions and steering • Some people still use old editors • Adjusting world view • Difficulties with three way merges • Git training • Adopting better tools • Finally went live in… September
- 30. CI Pipeline
- 32. Consistent reliable environments • Chaining to/calling fatal errors or stop • Calls which kill the CI process • Lots of assumptions about the environment, and continually challenging to ensure it’s clean. • Non-compiling programs
- 33. CI Surprises • Needed to clarify the development process • Engagement around non-compiling code • Reaction to global CI email Good results: Whole codebase compiled for the first time Real collective ownership Commits circulated to whole of Engineering as a matter of pride.
- 34. Wrap-up
- 35. Achievements • Human rewards… • Engineers feel part of the modern world Hackathon Coding katas • Feel invested-in • Have more transferable skills • TP held up as an exemplar • “Best Agile Team” award 2013
- 36. Where Next ? • Improve CI process • Extend packaging • BDD • And on and on…
- 37. Sep Jun Mar Dec Jun Sep 1996 1987 2013 2012 Timeline Switch-over to git + CI Start git training for UniVerse Start building CI pipeline Target for git switch-over (missed) Planning git and CI for UniVerse Started TDD for UniVerse Cloudshop moves to git Source control introduced (TFS) teams start working in step TDD training starts Scrum teams arranged Senior team assembled New CIO arrives Coding .... 1996: Migration to UniVerse 1987: Development starts 1987: Development starts Tech appendix follows…
Editor's Notes
- Synopsis: Travis Perkins has introduced some very modern technologies in the last few years, but half of the 120-strong software engineeringteam is actively developing in BASIC on its 1960s green screen technology platform. Brian and Nik show how it is possible to introduce XP practices to technology that predates the moon landings, and will discuss lessons learned. They will cover unit testing, source control, and continuous integration, and will touch briefly on future objectives.
- Wider ContextTravis Perkins has embarked on a transformation of its IT practices with support and leadership from senior management.Where before small teams chipped away at the code face in single-function isolated silos, now they have grown their teams and embraced Agile principles, pairing and close working with stakeholders.
- Does it have Global Variables?Yes.All variables have global scope within a program or subroutine.It also has ‘very global’ variables that can be shared between programs and subroutines.And ‘very, very global’ variables that exist for the duration of a login session (permanent).
- What Engineers were SeeingThe engineers were learning java or sitting next to teams using java.So they were seeing those engineers working with TDD, frameworks and standard build processes.And they thought – does this apply to us?
- The ChallengeStu had to change four lines of code in a small routine.Here is the context ..
- .. And a bit more ..
- .. And a bit more ..
- .. And a bit more ..(This is one of the shorter routines. And Remember: every variable is global)
- What We DidWe wrote our own unit test framework for UniVerse and taught engineers about unit testing and pair programming.Then we rewrote the framework to simplify and extend it as the engineers began to take ownership and to request features.The we extended it with configuration and faking.
- Some LessonsUnits in UniVerse are different. The concept of a unit covers a wider block – you can’t build in Single Responsibility Principle in the same way. So we developed patterns for splitting and refactoring code to make the sections addressable for testing.The database is part of the code: we can’t separate it out as a dependency (though faking to isolate tests is possible).UniVerse engineers like TDD! And have taken it on board enthusiastically.. Eventually.
- A framework has to have a name. All our unit test program names started “UNIT.” But UniVerse has a limit of 62 characters for a label, and the team was worried about wasting them. So they elected to drop one character, and thus the framework was named…
- A token of RegardFrom the team to Nik...
- The codebase did get source control in early 2013, but TFS had its problems. The Java teams’ move to git was the catalyst to move to the UniVerse teams, too.
- Git AnguishMoving to git meant a change in mindset.UniVerse uses file locking to protect access to code. Previous source code control (TFS) also implemented locking.Moving to a distributed version control system with no visibility of who was working on what raised fears.
- CI PipelineWe built a CI pipeline based around gitlab post-commit hooks. These are captured using HTML listeners in UniVerse and commit details logged to file. A cron job runs a CI process on commit changes, feeding a segregated release system and packager.Unusually, the decision was taken to email notifications of passes as well as failures (the opposite of many black box CI systems).The aim is to publicize and celebrate success as well as notify failures.
- Consistent Reliable EnvironmentsShifting to TDD and CI led to changes in expectation about what would/could be run in a clean environment.In addition to data files, the application sets up configuration information on login. This is unavailable to the CI process.Many assumptions are made in the code about the existence of this and of standard parameter records.And this is a distributed system, so code expects to call routines on other machines.Fatal errors stop that UniVerse process: a problem as the CI runs in the same process (and cannot report back).And there were many programs that could not compile.
- September 2013: Switch-over to git + CIAugust 2013: Start git training for UniVerseJuly 2013: Start building CI pipelineJune 2013: Target for git switch-over (missed)May 2013: Started discussion of git and CI for UniVerse Apr 2013: Started TDD for UniVerse, Cloudshop moves to gitFeb 2013: Source control introduced (TFS), teams start working in stepJan 2013: TDD training startsDec 2012: Scrum teams arrangedSep 2012: Senior team assembled