How to (Almost) Kill a Successful Project and BringIt Back to Life AgainRussell PavlicekXen Project EvangelistCitrix SystemsRussell.Pavlicek@XenProject.orgLessons Learned from the Xen Project
A guy with a lot of experience and areally big mouthSo Who is the Old, Fat Geek Up Front?
• Linux user since 1995; Linux desktop since 1997• Linux advocate before I ever saw the software• Early advocate in DEC, Compaq• Former columnist for Infoworld, Processor• Former panelist on The Linux Show• Wrote book, Embracing Insanity: Open Source SoftwareDevelopment• Speaker at 40+ conferences• Currently Xen Project Evangelist employed by CitrixAbout the Speaker...
• We will spend a couple minutes reviewing theproject• We will spend a few minutes considering itshistory• But we will spend the bulk of the timeconsidering lessons to take away• We are not here for the projects history; we arehere for your futureAbout This Talk
• The premier Open Source Hypervisor• Powering some of the biggest Clouds in theindustry (Amazon, Rackspace, Terremark)• Celebrating its 10th Anniversary• Now a Linux Foundation Collaborative Project– Sponsoring organizations include: Google, AMD,Intel, Samsung, Cisco, Oracle, Amazon, Verizon andmoreWhat is the Xen Project?
• The Xen Hypervisor, including ARM servers– Type 1 (Bare Metal)• Xen Cloud Platform & XAPI– Cloud readiness out of the box• Other Projects– Mirage– ARM Hypervisor for mobile devicesWhat Does the Xen Project Produce?
• It was the first industrial-strength Open SourceHypervisor• It enjoyed a very high rate of adoption• It employed excellent technology• It had a FOSS-friendly corporation behind it• And, yet, 2 years ago, it ran the risk of beingabandoned by the FOSS community before its10th birthdayThe Xen Project Story (30 secondversion)
• The project, though viable, developed an inwardfocus– Reach out to the rest of the Open Source communitywas limited– Reach out to its users was minimal– Code development continued, but the communitybecame insulated and stagnated– No one stepped up to contradict the rumor that Xenwas dying technology, overcome by competitorsHow Did This Happen?
• The project forgot the importance of workingwith its ecosystem– Upstream projects (Linux Kernel, QEMU) werebranched rather than engaged with patches– The project decided that others in the ecosystem(i.e., the distributions) would have to carry theburden of maintaining and supporting thosedifferences– This went on too long, and the ecosystem got fed upHow Did This Happen? (continued)
• The corporation backing it (XenSource) was soldto a company with a long closed sourcesoftware history (Citrix)• The new corporation was interested in thetechnology, but had no particular interest in theproject itselfHow Did This Happen? (continued)
• It was not about malice• It was not about fear• It was about disconnection– The project became disconnected from the FOSScommunity– The project became disconnected from the users– The new company became disconnected from theneeds of the project, because, in part, the projectnever really explained what it needed from thecompanyWhy Did This Happen?
• The prognosis was not good• Xen Hypervisor had been overtaken by acommercial offering in IT mindshare• Xen Project had been overshadowed by anotherOpen Source Hypervisor in the community• Distributions stopped facilitating Xen• The FOSS Community began to forget XenAbout Two Years Ago: Battling for aFuture
• About 2 years ago, a new direction was plotted• Citrix decided it wanted to understand FOSS andreinvigorate the Xen community• The company began to hire FOSS-savvy peopleto reconnect with the community and withusers• Brought about efforts to birth ApacheCloudStack, OpenDaylight, and to move the XenProject under the Linux FoundationA Conscious Reversal in Direction
• Common themes heard at FOSS events:– What is Xen?– Xen is dead, right?– Isnt Xen closed source?– No one uses Xen anymoreReality Two Years Ago: Xen Who?
• Linux kernel 3.0 contains all that Xen needs toexist by default• Most Linux distributions are Xen-enabled– CentOS has a project to give RHEL6 users a Xenoption• Xen Project now part of Linux Foundation• Launch of a new user-friendlier website(XenProject.org)Reality Today: Xen is Back!
• It is possible to die while you are winning– Being first is not enough– Great technology is not enough– Having a FOSS-friendly corporation behind you is notenough• A project must stay vibrant as an Open Sourceorganism or it will perishLesson 1
• Disconnection from users can make you a “DeadProject Walking”– You can be adding functionality, issuing newreleases, and still be dying– The connection between project and users isessential– Focusing on software alone is not enough• If you are not interacting with your users, youare at serious riskLesson 2
• Connecting with your developers != connectingwith your users– You need information sources for both users anddevelopers– If users have to dig through technical websites,wikis, etc. to answer simple questions, you are introuble– Even Linux kernel development – arguably one ofthe most insular projects – cannot thrive in avacuumLesson 2 (continued)
• Never ignore your projects Open Source rootstructure– Cut flowers are beautiful – until they die– Living plants need their roots– FOSS community is the root structure, and it mustspread wide– The project team cannot stand alone• Pay attention to your partners in FOSS: libraries,kernel, packagingLesson 3
• Never ignore your support structure– Xen needed cooperation from Distributions to beproperly supported– The relationship with the distributions was allowedto stagnate; it was not continuously cultivated– When one distribution invested in another OpenSource virtualization solution, other distributionswere swayed• Your distribution route can be critical to successLesson 4
• Having corporate backing is not enough– The corporation has its own set of goals, and theyrarely align exactly with the projects goals– When the project and the company dont mesh,friction can occur– This isn’t about good versus evil; business andprojects are just two separate animals with differentneedsLesson 5
• Having a FOSS company backing you is noguarantee– Even FOSS-centric companies can be sold– Sometimes they are sold to other FOSS companies(e.g., JBoss, Gluster)– Sometimes they are sold to closed sourcecompanies (e.g., MySQL, Xen, Cassatt)• If a project won’t survive without FOSS companybacking, consider options (e.g., LinuxFoundation)Lesson 6
• In FOSS, there is no such thing as autopilot– Intent is critical– If you are not planning to succeed, you are planningto fail– Great software is not enough; you can have the besttechnical solution, but if a “big dog” starts throwingits weight around, you need to be able to respond– If youre not looking at your whole ecosystem, youare inviting failureLesson 7
• If it aint growing, its dying– If your project team is seeing no new blood overtime, be concerned– Open Source organisms must move and grow– New folks are needed from time to time to add newideas and keep focus on what users needLesson 8
• Know where your project could fit in the worldand make a plan to get there– Competition means you probably wont be best fitfor every situation– It may not be possible to have every feature yourcompetitors have (especially if they have muchcorporate backing)– Figure out who your users are, what they need, andwhat you need for them to use your codeLesson 9
• Competition Increases Innovation– Lack of competition can cause stagnation (considerUnix CDE)– Competing technologies keep the ball movingforward continuously– Xens competition with KVM and VMware insuredthat new virtualization capabilities would keepflowing– A competing project has to stay on top of its gameor it wont make itLesson 10
• Major new features can keep your mindsharealive in the community– Large advances (e.g., ARM and Mirage) generateattention from the FOSS ecosystem and theuserbase– If you aren’t making headway, your root and supportstructures may stop working to give you what youneed– Periodic advances keeps the project growingLesson 11
• Sometimes, perception really is reality– You can have the best code in the world, but if noone cares, it’s useless– If the rumor arises that you are dying, outmoded, oroutdone by some other project, you must fight thatperception– The unchallenged lie will become fact for manypeopleLesson 12
• In contrast, KVM managed perceptions well– It could have looked like a purely Red Hat/IBMbusiness play when Red Hat purchased Qumranet– The relationship between Red Hat and the KVMproject was well-defined & appropriate; nodisconnect occurred– FOSS community embraced KVM project– Clearly, Red Hat and IBM are still focusing majorbusiness initiatives on KVM, but the communityaccepts that because it was done correctlyLesson 12 (continued)
• Go to local FOSS events– Submit talks• Get used to rejection and learn from it• Talk to the track chairman– “I don’t like speaking” – get over it; calculus wasway harder than this– Get an ORG booth for cheap• Stock it with flyers, CDs, business cards,stickers• Shoot your mouth off– Blogs– A usable website– Podcasts (TLLTS)– Social Media• Twitter, Facebook, Google+, LinkedIn• More mouthing off– YouTube demos and tutorials– Write for Linux.com LXer, LWN.net• Get a “kick-*aas” mascot!– But our buddy the Xen panda is taken!• Shout out and live, or shut up and die!– Passion is your ally– Let it leak over everyone– Don’t imitate the suits; do what fits youCrash Course in PerceptionManagement
• Theres a new reality for FOSS projects: thecorporate connection– Projects used to be primarily volunteers workingnights and weekends– Today, corporations play a big role in development• You need to have a good grip on what yourcorporate sponsors want from you, and whatyou need from them; disconnection can be fatalLesson 13
• Manage the relationship between business andproject– Prevent the loss of the project’s identity– If the project appears “owned” by a business, theFOSS community might become suspicious and backaway– In this case, perception is as dangerous as reality– If you forget what the project is, every else will too– Project ecosystem will wither away; only thebusiness remainsLesson 13 (continued)
• Establish a symbiotic relationship– Business provides user feedback, resources– Project provides overall focus, goal, direction, labor– Both sides need to color in the lines– Otherwise, you get “fake Open Source:” the code isopen, but there is no community, no support, noecosystemLesson 13 (continued)
• Make sure your project addresses its entireecosystem:– Is the code good?– Are you reaching out to your users?– Is your development community active, engaged,and growing?– Are reaching out to the FOSS community?– Have you insured you have proper support (libraries,distros, kernels, etc.)?Final Thoughts
• If you are in a relationship with a corporation, isthat relationship healthy?– Do you have freedom to do what you need to do?– Are you getting user feedback to seed new growth inthe project?– Is your projects identity getting lost in the corporateidentity? (if so, consider a foundation route)• Whatever else, dont give up!Final Thoughts (continued)