Distributed Development: The Dynamic Dance of Dispersed Data

Editor's Notes

1. The phrase “distributed development” describes a development project carried out across a number of geographically separated places of business. So just what does that look like in practice?
2. Think of a chessboard. On its own, it’s inert. Game mechanics given form. It’s a tool that enables play. But play requires players.
3. Switching to single-player mode. With a one person team all the elements available for the project are in one spot. Communication is seamless. But working alone has its limits. And large scale software development is not a lone wolf operation.
4. Now jump to two players. Instantly the game is much more complex.Communication is pretty straightforward and easy to manage. The outcome of the project is no longer in one person’s complete control.
5. Moving to three players the difficulty level starts to escalate exponentially. Whose turn is it? What strategies are being employed by each player? Who’s an ally or a rival? How do you plan your attack when the board constantly changes?
6. Now imagine 5, 10, 20, 30 people or many more, playing the same game. How does each member of the team coordinate their movements with the rest of the group? Just how fast can the whole project descend into disorganized chaos? (hint: fast)
7. And all those players may be spread out across the globe, in different countries and time-zones. They have different cultures and languages.
8. Imagine coordinating the programmers in Estonia…
9. An art team in Hong Kong…
10. And writers in Hollywood…
11. All reporting back to headquarters in London.
12. How does this all come together? The simple truth is, it’s not easy to pull off. Conventional wisdom has held that distributed development is risky and the quality of software developed with this methodology suffers in comparison to that developed in a collocated fashion – by teams with everyone working side-by-side.
13. But is that really true? A 2009 study by the University of California, Davis, the University of Zurich and Microsoft Research looked at the results of the development of Windows Vista and found negligible difference in the quality of results delivered by distributed vs. collocated development. [Does Distributed Development Affect Software Quality? An Empirical Case Study of Windows Vista / ACM August 2009]
14. Q/A: Has anyone here experienced with this type of development? Why did you do it? What were your experiences, good and bad?
15. So it may just be possible to pull off. But given the challenges, why would you want to? There has to be some discernable benefit to make distributed development worthwhile, even enticing.
16. If you’re thinking of going this route just to save money, you’re might want to reconsider. This doesn’t work like outsourcing to a cheaper labor market. You might save money upfront but there are additional costs that are going to come into play. Distributed development is not necessarily cheaper than its collocated cousin.
17. If you’re thinking of going this route just to go faster, you’re might want to reconsider. It’s quite often slower than a comparable effort with a colocated team. If cheap and fast is your primer motivator, this may not be the right track for you.
18. However, it does give you the leeway to hire the best talent in the world, wherever they may be. If you need a specific skillset, you don’t have to settle for what you have on hand or can source locally. You can reach around the world and pull in the top talent available. And they can sign on without having to relocate to another country. That flexibility lets you pick the best possible people for the project at hand. If you can establish and maintain a workflow that doesn’t suffer due to the geographic distribution of the teams, then you have a very powerful tool to wield.
19. And because you can source the best talent anywhere in the world, you can be bigger and stronger than are on your own. If you do it right
20. If you do this well, it can sometimes be cheaper and faster, but those are the side effects of a well planned and well enacted development cycle. If cheap and fast is your primary goal, you might want to look elsewhere. So let’s talk about how it works.
21. First off, distributed development is not the same thing as outsourcing. I hear it described that way a lot. They are not the same thing. Can anyone define outsourcing?I’ll define it as having an entire project, or a compartmentalized piece of a project, completed by an external team. Distributed development isn’t about external teams. It’s about redefining the meaning of the word internal. This is an internal team, potentially spread all over the world.
22. So how do you establish that workflow? First and foremost, communication. The importance of seamless, free-flowing and continuous communication in this arena cannot be overstated. If the lines of communication falter in a distributed development environment, the train can go off the tracks very quickly. The exact nature of the communication will depend upon the project itself.
23. Right off the bat, beware silos. When communication falters, silos begin to form. Silos create an Us vs. Them mentality that is anathema to this process.
24. You have one team, even though the members of that team may live in different countries and speak different languages. It’s one team. That’s the magic. It allows you to bring an incredibly diverse group of skills together as needed for your specific project. One team, one project, one goal.
25. How do you avoid silos? You talk. You communicate. Preferably face-to-face. There are numerous tools like Skype that can be leveraged to drive communication between far flung team members. Face-to-face conversation is a powerful tool, but if used obsessively or inconsiderately it can become a negative. You’ll need to fine tune the tools you use as you progress.
26. Regular team meetings are important. Videoconferencing can be a fantastic way to create and maintain the personal relationships necessary to drive the project forward. Meet daily or weekly as the project demands. Keep the meetings as short as possible. Longer meetings can be scheduled ahead of time. Don’t surprise people. If videoconferencing is overused it becomes a drag with many participants considering it a waste of their time. How do you determine how much cam-to-cam time is appropriate?
27. There is no hard and fast rule. But I recommend following the left brain/right brain split. The more right brain the project or team in question, the more regular face-to-face time is needed. Conversely, the more left brain the project or team, the less cam time is required or advisable. Game designers and artists need to communicate live to fire the cauldron that powers the ideas that drive the product. Techs may tend to prefer asynchronous communication via daily reports. Figure out what will work best for your teams. Then monitor the results and adjust as needed.
28. Once you’ve got your ideal communication patterns in place, lock them down. Make them a standard part of everyone’s daily workflow. Then every once in awhile, mix it up. When you feel a lull in energy or creativity, spice it up.
29. Debates continue about the value of getting everyone together in person. It costs money and it’s hard to pin an ROI to the cost of a night at a pub. There may well be teams for whom this type of familial bonding offers no discernable value. But this industry isn’t driven by data alone. It’s powered by people. And most people place a high value on human contact. It becomes “Us” instead of “Us and Them”. Once again, look at the Left/Right brain breakdown, and then determine which key people need to make the trip.
30. And don’t just bring far flung teams to your headquarters. Go to them. Meet them on their home turf. This isn’t outsourcing. Near or far, they are part of the same team. Treat everyone as equals no matter where they are located or you risk building walls that can impede development.
31. Your tools are the building blocks of everything you do, including communication. Make certain the tools you are using are optimized for a distributed environment. Everything needs to be globally accessible to all parties. Beware of legacy tools that worked well in a collocated environment, but might import ongoing problems to a distributed project.
32. And make certain everyone gets onboard and is using the same tools. Whether you’re talking source control, bug tracking, or an interoffice chat network, your tools lay the foundation for technical and creative communication. If one office is using different tools they will develop different habits and that can hurt you over time. Your employees may speak 5 different languages, but the tools are the universal translator.
33. Defining requirement up front in extensive detail is difficult, time consuming has great difficulty accounting for the iterative nature of software development. It’s also necessary, especially in a distributed workflow. The key is identifying the difference between a key requirement and a directional goal. This isn’t about busywork. This is about painting the path ahead in bright colors so the entire team can follow the map. Regular written global and team updates are a must. It’s easy to get out of step when you are thousands of miles away from your compatriots. Walls get thrown up quickly and you’re back to “Us and Them” before you know it.
34. Whatever can be measured, should be measured. Whatever your key metrics are, they should be tracked and visible to all. This creates a culture of accountability and can save a lot of heartache.
35. Keep your time zones in mind. Groups who need to communicate regularly will require a window of opportunity to talk during acceptable hours each day. Think about that at the outset of the project, before placing interdependent teams exactly 12 hours opposite each other. And if it’s 3am for someone on the call, don’t expect them to be on video.
36. Recognize this guy? That’s a problem. 6 months of 3am conference calls carry a cost. Don’t underestimate that. You need to monitor the team. Tired people make mistakes. Mistakes are amplified by distance and miscommunication. Be careful of this.
37. Don’t fall into a mentality of Home Team and Visitors. Keep a level playing field for all team members. Conference calls often feature one main group gathering in a conference room while their distributed comrades dial in separately. That leads to a lot of internal dialog that no one dialing in can hear. It limits and devalues their participation. Try making everyone dial into the call and skip the conference room. See what that does to the level of participation from the distributed callers.
38. Things to remember. Everyone is watching. All the time. Just because the team is offsite doesn’t mean they aren’t paying attention. They are likely far more attuned to every little piece of communication, both verbal and non, that pass between you. That’s all they have to go on so they pay attention.
39. Things will go wrong. Depend on it. Prepare for it. When the car goes off course, you’ll need to correct quickly to avoid wasting time and money.
40. Diving into Distributed Development unprepared is not a wise idea. But if you are determined to find proof that it doesn’t work, that’s a guaranteed way to predetermine your results. Be prepared.
41. No one solution set will work for every company using Distributed Development. Explore, test, reset. Find what works for your team.
42. Distributed development is not a half-measure. You can’t dip your toe in and test the waters. If you go this route, go all the way. Commit to it from day one and stay the course. No retreat. Or this little alien guy will shoot you. Seriously, he’s watching.
43. Questions, comments reach out and let me know.