Download to read offline
Uploaded byDaniel Vaughan
Trends in Software Development
The document discusses techniques for developing world-class software quickly and deploying it cost-effectively. It notes that minimizing the time between an idea and delivering business value is important. Fast and sustainable development allows for frequent delivery of new features and value. Deploying software quickly can save significant costs and generate substantial additional revenue compared to traditional slow development processes. The key is having developers work directly with users to rapidly iterate based on feedback.
Trends in Software Development
- 1. or Developing world-class softwarewhilst sitting in Starbucks Trends in Software Development Daniel Vaughan www.ebi.ac.uk
- 2. “The goal whenbuilding software is to minimise the time between idea and business value.”
- 7. “We have thecapability to build code fast and sustainably deliver a constant stream of value.”
- 10. Potential Business Value Businessper advisor / year £200k Potential extra per hour saved £100 Advisors 1,500 Potential hours saved per year 7,500 (50 per advisor) Potential extra revenue / year £750k Estimated cost of development £30k
- 11. Deployment Costs Item CostsRunning Total Oracle Application Server license £20k £20k Oracle Database Server license £30k £50k Upgrade to multi-core license x2 £100k Server hardware £10k £110k Staff, backups, hosting costs £15k £125k Load balancing severs x2 £250k Redundant fall-over servers x2 £500k Frankfurt disaster recovery replica x2 £1m
- 12. “The business oftenhas difficulty understanding why getting software deployed is slow and expensive.”
- 18. “We have thecapability to deploy software fast and cost effectively.”
- 21. “We now havetechniques for building sustainable code fast and capability to deploy it fast and cost effectively.”
- 22. “We now havethe opportunity to reduce the time from idea to business value to hours instead of months. We can achieve this only by developers being part of the team that engages with users directly.”
- 23. Dan North • Tuesday20th September • 9:30am • Everyone welcome
- 24.
Editor's Notes
- #3 The goal when building software is to minimise the time between idea and business value. As a software developer I aim to explain why I believe this, how we have been moving towards it and what is still left to do.
- #4 When I was a child I loved to play with Lego. I loved the freedom to create. I had an idea and an hour later had built what I imagined. Having that short cycle between idea and reality was what made it fun.
- #5 When I got my first computer I treated it the same way as Lego. I had an idea and attempted to make it into reality on the computer. It was slow and frustrating at first but I stuck at it and as the cycle between idea and reality shortened I started to enjoy it as much as I had with my Lego.
- #6 Turning other people’s ideas into software became my job. I find being a software developer, able to take someone’s idea and create something they value rewarding. Yet again having a short cycle between taking an idea, creating software and getting feedback is when it feels best to me. However, this is all very fluffy and idealistic. The reality is that cycle becomes far too long. In my experience, non-software developers, especially those who have coded themselves and experienced the short cycle have difficulty understanding why software takes so long.
- #7 There is a big difference between creating software for yourself and creating for, and with, others. As software grows maintaining it gets a lot harder. Every new idea or feature adds to the complexity of what is already there. The more complex the software the more chance of something breaking. When you break something people aren’t happy. It is at this point that fear takes control and progress slows down dramatically. The focus switches from creating a feature of the highest value to doing a hack that will best avoid breaking what you already have. Deployments become something to avoid as they are painful. Features and bug fixes are batched up and deployed in releases weeks or months apart. Work becomes more about correcting bugs than delivering features. The business starts to be perceive software development as a necessary but unwelcome cost. No longer is money spent seen as an investment in the business but rather a drain on resources. It is not surprising then, that an organisation in this state will start seeing developers as commodities and seek to minimise their cost. Pay rises or bonuses - forget it. Would you willing pay more for your tap water or petrol for your car? When you are seen as a commodity people will seek the lowest price. One way software companies avoid this situation is with dedicated testers and quality assurance teams. The disadvantage of this however is that it creates another handover in the process and increases the time from idea to business value further. The common pain has led developers to focus on quality and sustainability in recent years resulting in many useful new techniques. By adding comprehensive testing as part of the development process and following sustainable software design principles we spend less time dealing with problems and can return to a focus on quality and business value. It is important though that when focusing on quality we don’t go too far, don’t get obsessed with the code and lose sight of its purpose. No one cares about the quality of software that is delivered too late to be useful. With testing in place and fast automated deployments we are able to return to fast, frequent, painless deployments that nobody notices; often several times per day. We have the ability to deliver a constant stream of value rather than a stream of problems.
- #8 We have the capability to build code fast and sustainably deliver a constant stream of value.
- #9 Another factor is when building software for yourself, or to an extent at somewhere like EBI, you have a vast amount of open source tools freely available to you. The only difficulties are picking the best ones and getting them to work together.
- #10 In my last job that wasn’t the case. The desire was to use Oracle for everything. As the saying goes “No one got fired for choosing Oracle”. It was the safe choice but a very limiting and expensive one. It is hard to understand just how much so, so here is an example.
- #11 At the company I worked for financial advisors made up the core of the business, bringing in on average £200k of business per year each. We had an idea for software to free up an hour per week of administration per advisor. This meant fitting in an extra appointment with a client. Assuming an advisor worked 50 weeks per year and 40 hours per week that worked out at potentially £100 extra revenue per hour saved. As there were 1,500 financial advisors this was potentially an extra £750,000 per year of value to the business. We estimated that the solution would take a team of three developers two months to deliver at a cost of £30k. Investing £30k to generate a potential extra £750k per year is a compelling business case.
- #12 The problem was when I took the software to the architecture committee to plan how to deploy it. The conversation went like this: Ok, we need an application server licence @ £20k and a database licence at @ £30k (£50k) The servers we host them on have 4 processors we will need the multicore licence they are twice the price. (x2) Then we have the server hardware itself @ £5k each (£10k) We also need to factor in hosting, staff, networking, backups @ £15k What if we have a rush to use the service we cannot have it slowing down. Let’s get two sets of machines and load balance them (x2) Then of course we need a redundant failover in case the first setup falls over. And of course we need a disaster recovery installation in Frankfurt incase the London data centre becomes unavailable.(x2) I referred to this as the million pound conversation. Every time we tried to get anything deployed the cost always came out around a million pounds. Of course spending a million pounds per year to save £750k per year no longer made business sense. Not only was the cost a problem, but time was a problem too. The time to get budgets signed off and then get everything scheduled and setup could easily take months. Software that took 2 months to write could take an extra 4 months to get deployed. In that time, what was required could have changed or no longer be needed at all.
- #13 The business often has difficulty understanding why getting software deployed is slow and expensive.
- #14 It is not surprising that organisations get frustrated when the cycle between idea and business value is long and expensive. When someone says, ‘But my 15-year-old nephew could write this in a couple of days and you need months’ it is a hard position to justify and it is about to get harder, as I now explain. In recent years Cloud Computing has come to the fore. Put simply: it allows organisations to rent computing infrastructure by the hour instead of buying and running their own. In the case I just described the price of infrastructure started to add up because of redundancy and “what-if” planning. 90% of the cost was for infrastructure that would sit completely idle 90% of the time. Renting infrastructure by the hour enables an organisation to only pay for what we use when they use it. It is easy to see why this is highly desirable in the case I described.
- #15 The market-leading cloud provider is Amazon Web Services. I, or anyone’s 15-year-old nephew or niece with a credit card, can rent servers by the hour. Not only that, I can rent a wide range of services such as databases, storage, message queues, email systems and networking at the cost of pennies per day. These services are configured on demand rather than have to spend days setting them up yourself or weeks getting systems to set them up for you. More importantly, as soon as I have finished with a service then it can be switched it off and I stop paying straight-away - making it great for experimentation.
- #16 When I log in to Amazon Web Services I see a control panel that shows all the tools available. As a developer, this excitement is hard to get across!
- #17 But to me it looks like a toolkit, like this: all the tools I need neatly laid out before me. Now I have an even bigger box of Lego to play with! I now have the tools and infrastructure I need to deploy software so that thousands of people can start getting value in hours rather than months. Most of all I could do it as a solo developer sitting in Starbucks, at a monthly cost of less than the cup of coffee I am drinking. I no longer need tens of thousands of pounds, a large organisation or a generous VC behind me.
- #18 A great example was a recent story about the Australian Census 2016 Website. The site, developed by IBM cost $10m to build and was tested with 1 million users and 260 submissions per second. It fell over when used for real due to the architecture used. A few weeks later two students at a hackathon weekend in Sydney build a replica of the site using AWS Lambda. This coped with 4 million users and 10,000 submissions per second. Total cost? $500.
- #19 We have the capability to deploy software fast and cost effectively. The length of the cycle of idea to business value can get really short again. I am starting to have the same feeling as when playing with Lego. This new opportunity is loaded with promise. A sole developer sitting in Starbucks can provide more business value for £1000 in days than a team of developers could in months for hundreds of thousands a few years ago. That is an exciting prospect.
- #20 As we have heard, we are moving to a world of short cycles, where a small teams can create high quality software for thousands of people at a low cost. So why are more organisations not using this to deliver new features in their software every day?
- #21 There is one more problem left slowing us down. Knowing what to build next, and most importantly what not to build remains the biggest problem. When building for yourself you know what is needed. When building for others you need to engage with the users and customers to fully understand. In this respect, we put up a barrier. It is common to use intermediaries such as business analysts to communicate between users and developers. This wall slows down the cycle of idea to business value reverting it back to weeks instead of days as feedback and questions are passed indirectly. Most of all, without having the user’s perspective developers will always have to assume and “approximate” what is needed. To achieve the short cycles we aim for, the people actually building the software need to be part of the team that directly engaged with the users and have them in mind with every feature they produce; A so-called “matrix” team.
- #22 We now have techniques for building sustainable code fast and capability to deploy it fast and cost effectively.
- #23 We now have the opportunity to reduce the time from idea to business value to hours instead of months. We can achieve this only by developers being part of the team that engages with users directly.
- #24 The trouble is that I don’t the best way to do this. What is the best way of having developers and users engaging with each other? What is a shared language that they can use to fully understand each other? For that reason we with support from Elixir have invited Dan North, someone who has made his name answering these questions to come to EBI next week and help us answer that. He will be speaking at 9:30am next Tuesday in the Kendrew. There will even be pastries before. Everyone is welcome and I encourage you to come along and learn more.