Downloaded 67 times
Uploaded byKerry Buckley
Software Development Trends
This document discusses trends in software development, including programming languages like lambda and open source software. It also covers architectural approaches like web and cloud computing as well as scaling techniques. Additional trends mentioned are document databases, agile development, lean software development, craftsmanship, and DevOps. A variety of images are included to illustrate different concepts.
More Related Content
Software Development Trends
- 1. Trends in SoftwareDevelopment Software Development Kerry Buckley
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
Editor's Notes
- #3 First a disclaimer: this talk is based on my personal impressions of software development trends, so don’t treat it as company or community policy. I certainly don’t claim to be an expert on any of this, and a lot of it is just gleaned from blogs and twitter, so will reflect the particular echo chamber I happen to inhabit (mostly the Ruby, Agile and Software Craftsmanship communities). I’m not in charge of anything either – I’m just a developer…
- #4 … and just to prove it, here’s my directory entry.
- #5 I’ll start with the most obvious thing people probably think of when they consider development trends – languages. The general consensus seems to be that there isn’t going to be a single Next Big Language in the same way as we had FORTRAN, C, Java etc. The buzzword these days is polyglotism, meaning that people are proficient in multiple languages and use the one most suitable for the job (maybe DSLs and dynamic languages at the top of the stack, and functional languages lower down for reliability and scaling).
- #6 One of the things that’s made mixing and matching multiple languages much easier has been the growth in the number of languages that can run on common platforms alongside the usual Java and C#. These all run on the Java platform, and the story’s similar on .NET.
- #7 Unless you’ve lived in a cave for the past few years, it’s hard to have missed the growth of dynamic languages like Ruby and Python. This graph shows that demand for these is still way below that for Java, but it gets more interesting if you look at the graphs for growth…
- #8 You can see here that Ruby in particular appears to be growing at a fast rate, presumably driven by the use of Rails for web development. But there’s another language that’s growing even faster…
- #9 I searched for the term ‘Ajax’ instead of ‘Javascript’ because that’s the specific use case that’s driving the resurgence of Javascript, but you can see from the graph that demand has rocketed. Ajax had a bad reputation for years, but libraries like JQuery have removed most of the pain from using it, and allow dynamic behaviour to be added to a page in a structured unobtrusive way.
- #10 As well as dynamic languages, another category that’s getting a lot of interest lately is functional languages, including for example Erlang, Haskell, OCaml, Clojure and F#. Because they avoid state and side-effects, functional languages are well-suited for massively parallel execution, which is becoming more important with the rise of multi-core processors and cloud computing. As well as these production applications, many people are learning functional languages to give them a different viewpoint, which can add another tool to your collection even when working on OO languages.
- #11 Outside the enterprise world, use of open source software is almost universal, but even in large companies adoption is starting to become much more widespread (or at least more officially-approved). These are just a sample of the open source tools and products already in use in BT. Consumption is only one side of the coin though, and businesses are starting to realise the benefits of contributing to open source projects, and opening up the source of their own software. This allows collaboration and sharing of effort on common tools, leaving more capacity to concentrate development on systems and products that are core to the company’s business.
- #12 Contributing to open source projects has become much easier thanks to distributed version control, and particularly sites like GitHub and BitBucket. Unlike traditional source control systems like Subversion, distributed tools like Git, Mercurial and Bazaar mean that each developer has a clone of the entire source repository, and can pull changes from anyone else. This means you can work on a fix or enhancement, control your changes locally, merge in concurrent upstream changes then either submit a patch or request that the project maintainer pulls the commits from your copy of the repository. DVCSs also allow you to easily maintain a fork of a project – for example to add support for some local requirements – without losing track of changes to the parent project. This screenshot shows part of the network of repositories for a single project – each coloured line represents one user’s copy, and the arrows show changes being merged from one to another. Obviously in BT we’re standardising on Subversion for various reasons – some of which are no doubt perfectly valid ;-) – but it’s worth being aware of tools like Git anyway, as they are rapidly becoming the norm in the open source world, and have lots of advantages.
- #13 A few words about software architecture. Don’t forget that as I said earlier I’m just a developer – if anyone tries to call me an architect there’ll be trouble!
- #14 From the “stating the bleedin’ obvious” department, the web (with whichever version number you choose to append to that word) continues to be central to an awful lot of software architectures, both within and outside the company. As well as the human web, this also includes web services, which are increasingly moving away from SOAP towards REST (or at least simple data over HTTP).
- #15 Another buzzword that no-one can have avoided is cloud, which again means different things to different people.
- #16 One thing that’s certainly already a reality is the growth of web-based services that can handle everything from hosting your application to source control, continuous integration and fault tracking. These allow small companies or independent developers to release staggeringly quickly with minimal startup cost, and to scale successful projects relatively easily.
- #17 Many modern web application frameworks are designed to scale out by adding more servers, rather than scale up by running a single instance on a massive machine. This is well-suited to deployment to commodity hardware or cloud compute instances, but distributing the database layer can be difficult with traditional relational databases…
- #18 The problem of distributing relational databases is one of the reasons why document-based or key-value databases are becoming more popular. Another is the tendency of frameworks like Rails to treat the database as simply somewhere to persist objects, with much of the logic that traditionally lived in the database moving up to the application layer. Document databases store relatively unstructured data in an ad hoc fashion as simple key-value pairs, a bit like a giant persistent hash. They don’t have schemas, and don’t use SQL for querying (for example CouchDB uses a Javascript view model, exposed using a RESTful JSON API.
- #19 Obviously agile development methodologies continue to spread, but as adoption grows, particularly in large organisations, there’s a concern that the original intention is being watered down, with too much concentration on process at the expense of technical excellence and delivery of value.
- #20 Most large organisations that adopt agile development start with Scrum, which provides a good framework to get started from, with its emphasis on iteration and lightweight project management. Unfortunately Scrum doesn’t tell you what to do within those iterations to actually develop quality software, and that’s where the trouble starts…
- #21 If you start trying to develop software in two-week iterations without also having the right technical practices, it’s going to end in tears. Scrum explicitly doesn’t cover these, and Scrum experts generally recommend following the engineering practices from Extreme Programming within the management framework of Scrum.
- #22 Some of the key XP practices are automated testing, pair programming, continuous integration and refactoring, and we’re starting to see many of these pushed more strongly inside BT with the work that DSO are doing on CIT. In certain parts of the wider world (such as much of the Ruby and Rails community) practices such as test-driven development have become the norm rather than the exception, but in general adoption still seems to be relatively low.
- #23 While continuous integration is becoming more of a baseline practice, some teams are taking it a step further to continuous deployment, where changes are deployed almost as soon as the CI tests have completed. Flickr is a well-known example, with changes deployed multiple times in a day.
- #24 Ideas from the Lean world are also starting to be applied more widely in software development, particularly in terms of reducing waste and limiting work in progress. Some teams are starting to apply a Kanban approach to user story flow, with continuous delivery instead of fixed iterations.
- #25 Many of the original authors of the Agile Manifesto are now trying to promote a shift back to those original values, away from a watered-down, corporate-friendly version of Agile. One example is Brian Marick’s AR⊗TA, which stands for ‘artisanal retro-futurism crossed with team-scale anarcho-syndicalism’. The aims are similar to those of the original agile movement, but the name was chosen deliberately to stop people being able to claim ‘oh, we already do that’ without at least stopping to read about what it actually means. There’s more detail on arxta.net.
- #26 Another related trend is the emerging Software Craftsmanship movement. This is a reaction to the attempted industrialisation and deskilling of software development, which many people think isn’t working out so well (I’m sure everyone has their own horror stories of outsourced development). Part of the problem with the engineering view of software development is that it sometimes makes too strong an analogy with production engineering, assuming that if we can just get the design right, then we can leave the mechanical task of ‘coding’ to lower-skilled people. In reality software development is a creative process, with a large part of the design only emerging as the code is written. The ‘production line’ part of the process is actually the steps performed by compilers and other tools, not by the people writing the code. Software Craftsmanship re-emphasises the role of the skilled developer. It encourages developers to take responsibility for their own journey from apprenticeship to mastery, by deliberate practice and learning from other craftsmen.
- #27 Software Craftsmanship also recognises that a critical aspect of software development is the software itself. A professional developer should be able to take pride in their work, to insist on high quality code and refuse to deliver ‘quick hacks’ that they know will compromise maintainability in the future. Dragging things back to the BT world again, my hope is that the creation of the SE community is an indication that the technical skills of software development are starting to be taken seriously again. It’s certainly an opportunity for us as a community to demonstrate what can be achieved by small teams of good developers, and to help improve the quality and agility of BT’s software delivery.