How to (Almost) Kill a Successful Project and Bring
It Back to Life Again
Russell Pavlicek
Xen Project Evangelist
Citrix Systems
Russell.Pavlicek@XenProject.org
Lessons Learned from the Xen Project
A guy with a lot of experience and a
really big mouth
So Who is the Old, Fat Geek Up Front?
• Linux user since 1995; Linux desktop since 1997
• Linux advocate before I ever saw the software
• Early advocate in DEC, Compaq
• Former columnist for Infoworld, Processor
• Former panelist on The Linux Show
• Wrote book, Embracing Insanity: Open Source Software
Development
• Speaker at 40+ conferences
• Currently Xen Project Evangelist employed by Citrix
About the Speaker...
• There is a Xen Project overview and status talk
later in the conference
• “Xen Project: A 10th Anniversary Status Report”
Sunday @2:45 PM
• If you want to learn about Xen Hypervisor
Architecture, details, and plans, please plan to
attend
About The Upcoming Talk
• We will spend a couple minutes reviewing the
project
• We will spend a few minutes considering its
history
• But we will spend the bulk of the time
considering lessons to take away
• We are not here for the project's history; we are
here for your future
About This Talk
• The premier Open Source Hypervisor
• Powering some of the biggest Clouds in the
industry (Amazon, Rackspace, Terremark)
• Celebrating its 10th Anniversary
• Now a Linux Foundation Collaborative Project
– Sponsoring organizations include: Google, AMD,
Intel, Samsung, Cisco, Oracle, Amazon, Verizon and
more
What is the Xen Project?
• The Xen Hypervisor, including ARM servers
– Type 1 (Bare Metal)
• Xen Cloud Platform & XAPI
– Cloud readiness out of the box
• Other Projects
– Mirage
– ARM Hypervisor for mobile devices
What Does the Xen Project Produce?
• It was the first industrial-strength Open Source
Hypervisor
• It enjoyed a very high rate of adoption
• It employed excellent technology
• It had a FOSS-friendly corporation behind it
• And, yet, 2 years ago, it ran the risk of being
abandoned by the FOSS community before its
10th birthday
The Xen Project Story (30 second
version)
• The project, though viable, developed an inward
focus
– Reach out to the rest of the Open Source community
was limited
– Reach out to its users was minimal
– Code development continued, but the community
became insulated and stagnated
– No one stepped up to contradict the rumor that Xen
was dying technology, overcome by competitors
How Did This Happen?
• The project forgot the importance of working
with its ecosystem
– Upstream projects (Linux Kernel, QEMU) were
branched rather than engaged with patches
– The project decided that others in the ecosystem
(i.e., the distributions) would have to carry the
burden of maintaining and supporting those
differences
– This went on too long, and the ecosystem got fed up
How Did This Happen? (continued)
• The corporation backing it (XenSource) was sold
to a company with a long closed source
software history (Citrix)
• The new corporation was interested in the
technology, but had no particular interest in the
project itself
How Did This Happen? (continued)
• It was not about malice
• It was not about fear
• It was about disconnection
– The project became disconnected from the FOSS
community
– The project became disconnected from the users
– The new company became disconnected from the
needs of the project, because, in part, the project
never really explained what it needed from the
company
Why Did This Happen?
• The prognosis was not good
• Xen Hypervisor had been overtaken by a
commercial offering in IT mindshare
• Xen Project had been overshadowed by another
Open Source Hypervisor in the community
• Distributions stopped facilitating Xen
• The FOSS Community began to forget Xen
About Two Years Ago: Battling for a
Future
• About 2 years ago, a new direction was plotted
• Citrix decided it wanted to understand FOSS and
reinvigorate the Xen community
• The company began to hire FOSS-savvy people
to reconnect with the community and with
users
• Brought about efforts to birth Apache
CloudStack, OpenDaylight, and to move the Xen
Project under the Linux Foundation
A Conscious Reversal in Direction
• Common themes heard at FOSS events:
– What is Xen?
– Xen is dead, right?
– Isn't Xen closed source?
– No one uses Xen anymore
Reality Two Years Ago: Xen Who?
• Linux kernel 3.0 contains all that Xen needs to
exist by default
• Most Linux distributions are Xen-enabled
– CentOS has a project to give RHEL6 users a Xen
option
• Xen Project now part of Linux Foundation
• Launch of a new user-friendlier website
(XenProject.org)
Reality Today: Xen is Back!
So What Did We Learn?
• It is possible to die while you are winning
– Being first is not enough
– Great technology is not enough
– Having a FOSS-friendly corporation behind you is not
enough
• A project must stay vibrant as an Open Source
organism or it will perish
Lesson 1
• Disconnection from users can make you a “Dead
Project Walking”
– You can be adding functionality, issuing new
releases, and still be dying
– The connection between project and users is
essential
– Focusing on software alone is not enough
• If you are not interacting with your users, you
are at serious risk
Lesson 2
• Connecting with your developers != connecting
with your users
– You need information sources for both users and
developers
– If users have to dig through technical websites,
wikis, etc. to answer simple questions, you are in
trouble
– Even Linux kernel development – arguably one of
the most insular projects – cannot thrive in a
vacuum
Lesson 2 (continued)
• Never ignore your project's Open Source root
structure
– Cut flowers are beautiful – until they die
– Living plants need their roots
– FOSS community is the root structure, and it must
spread wide
– The project team cannot stand alone
• Pay attention to your partners in FOSS: libraries,
kernel, packaging
Lesson 3
• Never ignore your support structure
– Xen needed cooperation from Distributions to be
properly supported
– The relationship with the distributions was allowed
to stagnate; it was not continuously cultivated
– When one distribution invested in another Open
Source virtualization solution, other distributions
were swayed
• Your distribution route can be critical to success
Lesson 4
• Having corporate backing is not enough
– The corporation has its own set of goals, and they
rarely align exactly with the project's goals
– When the project and the company don't mesh,
friction can occur
– This isn’t about good versus evil; business and
projects are just two separate animals with different
needs
Lesson 5
• Having a FOSS company backing you is no
guarantee
– Even FOSS-centric companies can be sold
– Sometimes they are sold to other FOSS companies
(e.g., JBoss, Gluster)
– Sometimes they are sold to closed source
companies (e.g., MySQL, Xen, Cassatt)
• If a project won’t survive without FOSS company
backing, consider options (e.g., Linux
Foundation)
Lesson 6
• In FOSS, there is no such thing as autopilot
– Intent is critical
– If you are not planning to succeed, you are planning
to fail
– Great software is not enough; you can have the best
technical solution, but if a “big dog” starts throwing
its weight around, you need to be able to respond
– If you're not looking at your whole ecosystem, you
are inviting failure
Lesson 7
• If it ain't growing, it's dying
– If your project team is seeing no new blood over
time, be concerned
– Open Source organisms must move and grow
– New folks are needed from time to time to add new
ideas and keep focus on what users need
Lesson 8
• Know where your project could fit in the world
and make a plan to get there
– Competition means you probably won't be best fit
for every situation
– It may not be possible to have every feature your
competitors have (especially if they have much
corporate backing)
– Figure out who your users are, what they need, and
what you need for them to use your code
Lesson 9
• Competition Increases Innovation
– Lack of competition can cause stagnation (consider
Unix CDE)
– Competing technologies keep the ball moving
forward continuously
– Xen's competition with KVM and VMware insured
that new virtualization capabilities would keep
flowing
– A competing project has to stay on top of its game
or it won't make it
Lesson 10
• Major new features can keep your mindshare
alive in the community
– Large advances (e.g., ARM and Mirage) generate
attention from the FOSS ecosystem and the
userbase
– If you aren’t making headway, your root and support
structures may stop working to give you what you
need
– Periodic advances keeps the project growing
Lesson 11
• Sometimes, perception really is reality
– You can have the best code in the world, but if no
one cares, it’s useless
– If the rumor arises that you are dying, outmoded, or
outdone by some other project, you must fight that
perception
– The unchallenged lie will become fact for many
people
Lesson 12
• In contrast, KVM managed perceptions well
– It could have looked like a purely Red Hat/IBM
business play when Red Hat purchased Qumranet
– The relationship between Red Hat and the KVM
project was well-defined & appropriate; no
disconnect occurred
– FOSS community embraced KVM project
– Clearly, Red Hat and IBM are still focusing major
business initiatives on KVM, but the community
accepts that because it was done correctly
Lesson 12 (continued)
• Go to local FOSS events
– Submit talks
• Get used to rejection and learn from it
• Talk to the track chairman
– “I don’t like speaking” – get over it; calculus was
way harder than this
– Get an ORG booth for cheap
• Stock it with flyers, CDs, business cards,
stickers
• Shoot your mouth off
– Blogs
– A usable website
– Podcasts (TLLTS)
– Social Media
• Twitter, Facebook, Google+, LinkedIn
• More mouthing off
– YouTube demos and tutorials
– Write for Linux.com LXer, LWN.net
• Get a “kick-*aas” mascot!
– But our buddy the Xen panda is taken!
• Shout out and live, or shut up and die!
– Passion is your ally
– Let it leak over everyone
– Don’t imitate the suits; do what fits you
Crash Course in Perception
Management
• There's a new reality for FOSS projects: the
corporate connection
– Projects used to be primarily volunteers working
nights and weekends
– Today, corporations play a big role in development
• You need to have a good grip on what your
corporate sponsors want from you, and what
you need from them; disconnection can be fatal
Lesson 13
• Manage the relationship between business and
project
– Prevent the loss of the project’s identity
– If the project appears “owned” by a business, the
FOSS community might become suspicious and back
away
– In this case, perception is as dangerous as reality
– If you forget what the project is, every else will too
– Project ecosystem will wither away; only the
business remains
Lesson 13 (continued)
• Establish a symbiotic relationship
– Business provides user feedback, resources
– Project provides overall focus, goal, direction, labor
– Both sides need to color in the lines
– Otherwise, you get “fake Open Source:” the code is
open, but there is no community, no support, no
ecosystem
Lesson 13 (continued)
• Make sure your project addresses its entire
ecosystem:
– Is the code good?
– Are you reaching out to your users?
– Is your development community active, engaged,
and growing?
– Are reaching out to the FOSS community?
– Have you insured you have proper support (libraries,
distros, kernels, etc.)?
Final Thoughts
• If you are in a relationship with a corporation, is
that relationship healthy?
– Do you have freedom to do what you need to do?
– Are you getting user feedback to seed new growth in
the project?
– Is your project's identity getting lost in the corporate
identity? (if so, consider a foundation route)
• Whatever else, don't give up!
Final Thoughts (continued)
Russell.Pavlicek@XenProject.org
XenProject.org
Questions?

Lessons Learned from Xen - SELF2013

  • 1.
    How to (Almost)Kill a Successful Project and Bring It Back to Life Again Russell Pavlicek Xen Project Evangelist Citrix Systems Russell.Pavlicek@XenProject.org Lessons Learned from the Xen Project
  • 2.
    A guy witha lot of experience and a really big mouth So Who is the Old, Fat Geek Up Front?
  • 3.
    • Linux usersince 1995; Linux desktop since 1997 • Linux advocate before I ever saw the software • Early advocate in DEC, Compaq • Former columnist for Infoworld, Processor • Former panelist on The Linux Show • Wrote book, Embracing Insanity: Open Source Software Development • Speaker at 40+ conferences • Currently Xen Project Evangelist employed by Citrix About the Speaker...
  • 4.
    • There isa Xen Project overview and status talk later in the conference • “Xen Project: A 10th Anniversary Status Report” Sunday @2:45 PM • If you want to learn about Xen Hypervisor Architecture, details, and plans, please plan to attend About The Upcoming Talk
  • 5.
    • We willspend a couple minutes reviewing the project • We will spend a few minutes considering its history • But we will spend the bulk of the time considering lessons to take away • We are not here for the project's history; we are here for your future About This Talk
  • 6.
    • The premierOpen Source Hypervisor • Powering some of the biggest Clouds in the industry (Amazon, Rackspace, Terremark) • Celebrating its 10th Anniversary • Now a Linux Foundation Collaborative Project – Sponsoring organizations include: Google, AMD, Intel, Samsung, Cisco, Oracle, Amazon, Verizon and more What is the Xen Project?
  • 7.
    • The XenHypervisor, including ARM servers – Type 1 (Bare Metal) • Xen Cloud Platform & XAPI – Cloud readiness out of the box • Other Projects – Mirage – ARM Hypervisor for mobile devices What Does the Xen Project Produce?
  • 8.
    • It wasthe first industrial-strength Open Source Hypervisor • It enjoyed a very high rate of adoption • It employed excellent technology • It had a FOSS-friendly corporation behind it • And, yet, 2 years ago, it ran the risk of being abandoned by the FOSS community before its 10th birthday The Xen Project Story (30 second version)
  • 9.
    • The project,though viable, developed an inward focus – Reach out to the rest of the Open Source community was limited – Reach out to its users was minimal – Code development continued, but the community became insulated and stagnated – No one stepped up to contradict the rumor that Xen was dying technology, overcome by competitors How Did This Happen?
  • 10.
    • The projectforgot the importance of working with its ecosystem – Upstream projects (Linux Kernel, QEMU) were branched rather than engaged with patches – The project decided that others in the ecosystem (i.e., the distributions) would have to carry the burden of maintaining and supporting those differences – This went on too long, and the ecosystem got fed up How Did This Happen? (continued)
  • 11.
    • The corporationbacking it (XenSource) was sold to a company with a long closed source software history (Citrix) • The new corporation was interested in the technology, but had no particular interest in the project itself How Did This Happen? (continued)
  • 12.
    • It wasnot about malice • It was not about fear • It was about disconnection – The project became disconnected from the FOSS community – The project became disconnected from the users – The new company became disconnected from the needs of the project, because, in part, the project never really explained what it needed from the company Why Did This Happen?
  • 13.
    • The prognosiswas not good • Xen Hypervisor had been overtaken by a commercial offering in IT mindshare • Xen Project had been overshadowed by another Open Source Hypervisor in the community • Distributions stopped facilitating Xen • The FOSS Community began to forget Xen About Two Years Ago: Battling for a Future
  • 14.
    • About 2years ago, a new direction was plotted • Citrix decided it wanted to understand FOSS and reinvigorate the Xen community • The company began to hire FOSS-savvy people to reconnect with the community and with users • Brought about efforts to birth Apache CloudStack, OpenDaylight, and to move the Xen Project under the Linux Foundation A Conscious Reversal in Direction
  • 15.
    • Common themesheard at FOSS events: – What is Xen? – Xen is dead, right? – Isn't Xen closed source? – No one uses Xen anymore Reality Two Years Ago: Xen Who?
  • 16.
    • Linux kernel3.0 contains all that Xen needs to exist by default • Most Linux distributions are Xen-enabled – CentOS has a project to give RHEL6 users a Xen option • Xen Project now part of Linux Foundation • Launch of a new user-friendlier website (XenProject.org) Reality Today: Xen is Back!
  • 17.
    So What DidWe Learn?
  • 18.
    • It ispossible to die while you are winning – Being first is not enough – Great technology is not enough – Having a FOSS-friendly corporation behind you is not enough • A project must stay vibrant as an Open Source organism or it will perish Lesson 1
  • 19.
    • Disconnection fromusers can make you a “Dead Project Walking” – You can be adding functionality, issuing new releases, and still be dying – The connection between project and users is essential – Focusing on software alone is not enough • If you are not interacting with your users, you are at serious risk Lesson 2
  • 20.
    • Connecting withyour developers != connecting with your users – You need information sources for both users and developers – If users have to dig through technical websites, wikis, etc. to answer simple questions, you are in trouble – Even Linux kernel development – arguably one of the most insular projects – cannot thrive in a vacuum Lesson 2 (continued)
  • 21.
    • Never ignoreyour project's Open Source root structure – Cut flowers are beautiful – until they die – Living plants need their roots – FOSS community is the root structure, and it must spread wide – The project team cannot stand alone • Pay attention to your partners in FOSS: libraries, kernel, packaging Lesson 3
  • 22.
    • Never ignoreyour support structure – Xen needed cooperation from Distributions to be properly supported – The relationship with the distributions was allowed to stagnate; it was not continuously cultivated – When one distribution invested in another Open Source virtualization solution, other distributions were swayed • Your distribution route can be critical to success Lesson 4
  • 23.
    • Having corporatebacking is not enough – The corporation has its own set of goals, and they rarely align exactly with the project's goals – When the project and the company don't mesh, friction can occur – This isn’t about good versus evil; business and projects are just two separate animals with different needs Lesson 5
  • 24.
    • Having aFOSS company backing you is no guarantee – Even FOSS-centric companies can be sold – Sometimes they are sold to other FOSS companies (e.g., JBoss, Gluster) – Sometimes they are sold to closed source companies (e.g., MySQL, Xen, Cassatt) • If a project won’t survive without FOSS company backing, consider options (e.g., Linux Foundation) Lesson 6
  • 25.
    • In FOSS,there is no such thing as autopilot – Intent is critical – If you are not planning to succeed, you are planning to fail – Great software is not enough; you can have the best technical solution, but if a “big dog” starts throwing its weight around, you need to be able to respond – If you're not looking at your whole ecosystem, you are inviting failure Lesson 7
  • 26.
    • If itain't growing, it's dying – If your project team is seeing no new blood over time, be concerned – Open Source organisms must move and grow – New folks are needed from time to time to add new ideas and keep focus on what users need Lesson 8
  • 27.
    • Know whereyour project could fit in the world and make a plan to get there – Competition means you probably won't be best fit for every situation – It may not be possible to have every feature your competitors have (especially if they have much corporate backing) – Figure out who your users are, what they need, and what you need for them to use your code Lesson 9
  • 28.
    • Competition IncreasesInnovation – Lack of competition can cause stagnation (consider Unix CDE) – Competing technologies keep the ball moving forward continuously – Xen's competition with KVM and VMware insured that new virtualization capabilities would keep flowing – A competing project has to stay on top of its game or it won't make it Lesson 10
  • 29.
    • Major newfeatures can keep your mindshare alive in the community – Large advances (e.g., ARM and Mirage) generate attention from the FOSS ecosystem and the userbase – If you aren’t making headway, your root and support structures may stop working to give you what you need – Periodic advances keeps the project growing Lesson 11
  • 30.
    • Sometimes, perceptionreally is reality – You can have the best code in the world, but if no one cares, it’s useless – If the rumor arises that you are dying, outmoded, or outdone by some other project, you must fight that perception – The unchallenged lie will become fact for many people Lesson 12
  • 31.
    • In contrast,KVM managed perceptions well – It could have looked like a purely Red Hat/IBM business play when Red Hat purchased Qumranet – The relationship between Red Hat and the KVM project was well-defined & appropriate; no disconnect occurred – FOSS community embraced KVM project – Clearly, Red Hat and IBM are still focusing major business initiatives on KVM, but the community accepts that because it was done correctly Lesson 12 (continued)
  • 32.
    • Go tolocal FOSS events – Submit talks • Get used to rejection and learn from it • Talk to the track chairman – “I don’t like speaking” – get over it; calculus was way harder than this – Get an ORG booth for cheap • Stock it with flyers, CDs, business cards, stickers • Shoot your mouth off – Blogs – A usable website – Podcasts (TLLTS) – Social Media • Twitter, Facebook, Google+, LinkedIn • More mouthing off – YouTube demos and tutorials – Write for Linux.com LXer, LWN.net • Get a “kick-*aas” mascot! – But our buddy the Xen panda is taken! • Shout out and live, or shut up and die! – Passion is your ally – Let it leak over everyone – Don’t imitate the suits; do what fits you Crash Course in Perception Management
  • 33.
    • There's anew reality for FOSS projects: the corporate connection – Projects used to be primarily volunteers working nights and weekends – Today, corporations play a big role in development • You need to have a good grip on what your corporate sponsors want from you, and what you need from them; disconnection can be fatal Lesson 13
  • 34.
    • Manage therelationship between business and project – Prevent the loss of the project’s identity – If the project appears “owned” by a business, the FOSS community might become suspicious and back away – In this case, perception is as dangerous as reality – If you forget what the project is, every else will too – Project ecosystem will wither away; only the business remains Lesson 13 (continued)
  • 35.
    • Establish asymbiotic relationship – Business provides user feedback, resources – Project provides overall focus, goal, direction, labor – Both sides need to color in the lines – Otherwise, you get “fake Open Source:” the code is open, but there is no community, no support, no ecosystem Lesson 13 (continued)
  • 36.
    • Make sureyour project addresses its entire ecosystem: – Is the code good? – Are you reaching out to your users? – Is your development community active, engaged, and growing? – Are reaching out to the FOSS community? – Have you insured you have proper support (libraries, distros, kernels, etc.)? Final Thoughts
  • 37.
    • If youare in a relationship with a corporation, is that relationship healthy? – Do you have freedom to do what you need to do? – Are you getting user feedback to seed new growth in the project? – Is your project's identity getting lost in the corporate identity? (if so, consider a foundation route) • Whatever else, don't give up! Final Thoughts (continued)
  • 38.