How 5 people with 4 day jobs in 3 time zones enjoyed 2 years writing 1 book
How 5 people with 4 day jobs in 3 time zones enjoyed 2 years writing 1 book Ian Dees Open Source Bridge 2011This is the story of a big collaborative writing project. I’m telling it for therapeutic reasons—butif you’re a freelancer, a project manager, a telecommuter, or a member of a remote team, you mayﬁnd the arc of this project familiar. Perhaps some of the tools and techniques we used along theway will be helpful to you as well.
5 authors Ian Nick Charlie Ola Tom sketches by Lukas KetnerThe ﬁve authors are: Charles Nutter and Tom Enebo, the two JRuby project leads;Nick Sieger and Ola Bini, the other two members of the JRuby core team; andme (Ian Dees), a happy JRuby user and the only non-core team member in the group.
4 day jobs 1. Sun (Charlie, Nick, Tom) 2. Engine Yard (Charlie, Nick, Tom) 3. ThoughtWorks (Ola) 4. Tektronix (Ian)Collectively, we’ve worked at these four companies...
3 time zones 1. Paciﬁc (Ian) 2. Central (Charlie, Nick, Tom) 3. Central European (Ola)...in these three places.
2 years June 2008 - 2010We started in June of ’08 and ﬁnished writing in August of ’10.
1 bookThe book is Using JRuby, the deﬁnitive guide to the Java-based Ruby implementation.
IT’S NOT A BOOK IT’S A SOFTWARE PROJECTSince this is such a code-heavy book, I should have realized much earlier on that it’s as much asoftware project as a book. We could have used practices that have helped software teams, such asworking in iterations and having frequent standup meetings. (Later on, we ﬁgured this out.)
the startup curve http://adam.heroku.com/past/2008/4/23/the_startup_curveBefore I tell you our story, let me show you this famous graph drawn by Paul Graham and TrevorBlackwell at the Y Combinator headquarters.
the startup curve The Process Upside of Buyer Acquisition Wearing Off of Liquidity TechCrunch of Novelty of Initiation Wiggles of False Hope The Promised Trough of Land! Sorrow Releases Crash of of Improvement IneptitudeHere’s a cleaned-up version. It’s meant to represent the ebb and ﬂow of web traffic at a startup.
the book curve The Process Release! Conservation Harshness of Momentum Euphoria of Reality of Kickoff Wiggles of False Hope Establishment Trough of of Rhythm Sorrow Nagging of Stakeholders Abandonment?It also ﬁts the ebb and ﬂow of morale during the writing of our book. This curve should feel familiarto anyone who’s worked on a multi-year, multi-person effort.
euphoria of kickoff Harshness Euphoria of Reality of KickoffLet’s talk about how it all began.
dramatis personaeFirst, a word about my co-conspirators.
Ola The Prospector TECHNOLOGY SURVEYS, (RE-)STARTING PROGRESSOla is like a prospector. You can turn him loose in Testing Canyon, and he’ll suddenly turn up acouple of weeks later with a twelve-pound chunk of gold, saying, “Smelt that, baby.” Someone thisgood at working without a map is crucial for getting projects unstuck.
Tom The Analyst BREAKING DOWN HUGE QUESTIONSTom has an analyst’s mindset. He likes to consider every angle of a subject, write some code, studysome more, and then weave a compelling narrative. He doesn’t ﬂinch at huge topics that wouldoverwhelm most people, such as “Tell me everything about user interfaces in JRuby.”
Charlie The Professor BUILDING GREAT DEMONSTRATIONSCharlie has a library of examples available to ﬁt every occasion. His writing style is verydemonstrative: “In this example, you will encounter problem X. To solve it, use technique Y. Forinstance: ....” He also has a good sense of what things we should explain in the book, and whatthings we should just ﬁx in JRuby itself.
NickThe ConversationalistKEEPING THE READER ENGAGED, FINDING GAPS IN FLOWNick is a natural conversationalist. He connects material to previous ideas, then challenges thereader to absorb the new information. Someone like this is crucial for ﬁnding gaps in ﬂow.
Ian The Yak Shaver THE THOUSAND LITTLE THINGS THAT NEED DOINGAnd ﬁnally, there’s the yak shaver—the guy who sets out to write a chapter, creates some codeexamples to talk about, and eventually falls down a rabbit hole poring through the JRubyimplementation. There’s even a place in most projects for a scatterbrain like this: taking care of thethousand little things like maintaining tools, talking to the editor, and so on.
the old plan I. Introduction V. Working With Data 1. Introduction 8. JDBC II. JRuby and Java 9. XML 2. Ruby 101 10. Web Services/SOAP 3. Java from Ruby V. Rails 4. Ruby from Java 11. Overview/Tutorial III. Environment 12. JDBC 5. Standard Libraries/YAML 13. Deploying 6. Conﬁguring/Running VI. Testing 7. Inside JRuby 14. JUnit/Test::Unit/Mocking a. Architecture 15. RSpec b. Interpreter 16. Benchmarking/Debugging c. Compiler VII. Other Topics d. Java Integration 17. Security e. JRuby API 18. Application Servers/ f. Extending JRuby Frameworks 19. Swing 20. ToolsIn late 2007, the JRuby core team decided it was time for an official book. Here was an early outline.This would have made for a great book, but we didn’t have the time to write a work this ambitious.We needed to negotiate scope.
IT’S NOT A WRITING GIG IT’S PROJECT MANAGEMENTLesson #2: This project was as much about management—breaking down work, making harddecisions—as it was about writing. This was yet another thing I failed to realize early on. We spunour wheels for a while because no one was making project-management decisions. One thing wedid get right though: we narrowed the scope down to something we could ﬁnish together.
At RailsConf in June 2008, the ﬁve of us sat down at the awesome Doug Fir Lounge in Portland tosee if there was enough rapport for us to work together. This is a scan of the original notes Iexcitedly took.
notes, transcribed I. JRuby III. Appendices 1. Basics / Installation A. Ruby 101 2. Java from Ruby B. Contributing 3. Ruby from Java 4. Compiler II. Libraries 5. Rails 6. Hibernate 7. Rake 8. Swing 9. JMX 10.JMSThis is what those notes look like when arranged hierarchically. It’s about half the size of theoriginal plan.
the ﬁnal book I. JRuby III. Appendices 1. Basics / Installation A. Ruby 101 2. Java from Ruby B. Contributing 3. Ruby from Java C. Conﬁguration 4. Compiler D. C Code E. JMX + Sysadmins II. Libraries 5. Rails 6. Hibernate + Databases 7. Rake + Deployment 8. Testing 9. Swing 10.JMSAnd here’s the ﬁnal structure of the book. We were able to stay pretty faithful to our vision for twoyears, because it all came from our simple mantra: using Java from Ruby, and Ruby from Java.
FOCUS ON A SIMPLE RALLYING CRYLesson #3: avoid scope creep by asking of each proposed addition, “Does it help us do the thing weset out to do?”
work vs. “work”After the meeting, we set up our tools based on the way we expected to collaborate: IRC forconversation, Google Groups for ﬁles and wiki pages, and so on. Alas, none of these things arework.
DON’T CHOOSE YOUR TOOLS BEFORE YOU KNOW WHY YOU NEED THEMIt should have been a yellow ﬂag that we were choosing tools based on how we thought we mightwork, rather than on how we were actually working.
things we tried • IRC • Google Groups • Skype + Audio HijackIRC was great for helping real JRuby users, but for a small group like us, there was never a criticalmass in the channel for our book. Google Groups was also overkill for the kinds of informal thingswe should just have used e-mail or AWS for. Skype helped us get unstuck, but there was little pointin recording it.
things we didn’t try • actually writingOkay, so maybe we wrote after all—it just didn’t feel like it.
AN EXPERIMENT THAT PROVES A RESULT YOU DIDN’T EXPECT is still A SUCCESSIt’s okay that IRC and some of the other tools we tried didn’t work out for us. It was still a validexperiment to try, as long as we cut our losses and moved on.
harshness of realityIt’s at this point that we near the second phase of the project...
trough of sorrow Wiggles of False Hope Trough of Sorrow Nagging of Stakeholders Abandonment?...the Trough of Sorrow. This is where it looks like there’s no progress—either because there is noprogress, or because it’s all on things invisible to the reader.
July 2008 Jackie: What’s happening with JRuby? Ian: Nothing. I have no authors. Couldn’t track anyone down.... Jackie: Can you try again this weekend? Don’t get discouraged just yet.This phase is best told in conversation. Notice that at this point, I still haven’t learned the famousApple dictum: when you’re the one coordinating the project and it goes off kilter, “reasons don’tmatter.”
August 2008 Jackie: Anything happening with JRuby? Ian: Good news from Sweden. Ola’s starting the chapter on Hibernate....Aha! Forward progress!
September 2008 Jackie: BTW, if nothing is happening, we should talk about it soon. Ian: Ola just released a bunch of code... following up to see if he’s ready to write about it.... I’ve just proposed a “bookfest” where... we sit and do nothing but write....Infrastructure progress isn’t always visible to the reader, though. Note that this is the ﬁrst time weget the idea to write in person together. We’ll revisit that idea several months later.
October 2008 Ola: It seems that the book will take a long time to get finished..., and I really want to get going.... Ian: Its just been difficult [for the team] to justify writing about JRuby when they could either be hacking JRuby or helping someone.... On the other hand, a phone call gives us a solid block of time with each other, where we have (nearly) 100% of one anothers attention.Four months in, and even the ﬁve authors are getting antsy—but we don’t know what to do about ityet. We have, however, discovered that voice chat is much less distraction-prone than text chat.
February 2009 Jackie: Is [Ola’s testing chapter] ready for me to review? .... This looks really good!Not much progress expected or seen during the Thanksgiving/Christmas time period. And then, inlate January: a wiggle of hope!
April 2009 Jackie: How is it coming along? Ian: I feel a bit stalled..., and now things just seem... slow. Jackie: How can I help you get back on track? Are you getting discouraged?Two short months later, and we were close to abandoning the project.
http://www.ﬂickr.com/photos/danielygo/5242003647I’d love to be able to tell my past self: in any long-running project, there will be times when you feelalone. You’ll log into Skype, and no one will be there. You’ll send e-mail, and no one will answer.Don’t give up, don’t despair, and don’t give up.
http://www.harpercollins.comIn The Care of the Soul, Thomas Moore explains that there are a lot more facets to the Narcissusmyth than just some vain guy staring at his reﬂection.
SELF-DOUBT IS A FORM OF NARCISSISMIn particular, self-doubt is a form of narcissism. Telling oneself, “I’m terrible at this; might as wellgive up” is just looking for an excuse to give up. Who would blame a terrible writer for not writing?It’s important to acknowledge these feelings for what they are, and then push on.
establishment of rhythm Release! Conservation of Momentum Establishment of RhythmRight after near disaster, a few things happened that helped us establish a rhythm—both in thesense of playing well together, and in the sense of delivering at regular intervals.
a personal inﬂection pointFirst, JRuby helped me out of a jam at work. I started using it as my primary Ruby, instead of usingit as an interesting alternative. When you write about something you use all the time, you can writewith much more authority. See Charlie’s blog entry on writing vs. implementing: http://blog.headius.com/2007/09/are-authors-technological-poseurs.html
EAT YOUR OWN DOG FOODCompanies call this “dogfooding”—actually using the product you’re trying to persuade people toadopt.
three things to establish rhythmThere were three more things we did together as a team that were crucial to getting us movingagain.
HAVE A HEART(-BEAT)The ﬁrst was to establish a heartbeat for the project. (No wonder the patient almost died—we’dnever taken the pulse!)
Pivotal TrackerWe broke our work down into a piece at a time—no time estimates—and entered them as storiesinto Pivotal Tracker. For example, a story title might be “Section on Maven describes Rakeintegration.” After a few weeks, Tracker would tell us if we were going to make each deadline. Wegot better at making them.
MEET IN PERSON AT LEAST ONCEThe next thing we did was ﬁnally follow up on our plan to sit in person and write. I ﬂew toMinneapolis and wrote with Charlie and Nick around dining room tables, coffee shops, restaurants,and bars all over the twin cities.
Google DocsWe needed something more immediate than revision control so we could see each other’s changes.We turned to Google Docs. Notice we’re choosing our tools based on need now, not guesswork. Wewrote about JRuby as we wanted it to be; if a feature wasn’t implemented yet, we’d throw a TODO inthe text and keep writing.
FORM GOOD HABITS NOT EMPTY OBLIGATIONSThe third thing we did was to establish Skype calls every Monday, Wednesday, and Friday after thewritefest. We weren’t overly rigid about this. If we had nothing to talk about, we’d skip a call. If oneof us had to be somewhere loud, we’d use iChat instead. The point was to form good habits, notempty obligations.
August 2010 Ian: The final chapter is in our editors hands. The book is written. Tonight, Ill try this "sleep" thing Ive heard about.We kept these habits up for a year. During that year, we announced the book, released several betaversions, and made tons of improvements based on our readers’ feedback.
lessons and hopesIn the course of writing this book, we learned the importance of recognizing software managementwhen we see it. We learned the importance of in-person meetings, even for remote teams. Welearned how to establish and keep a rhythm, even when we’re geographically separate andextremely busy. We hope that there’s something in here that you can apply on your next freelance,collaborative, or remote project.