The document discusses complexity in Drupal and other software ecosystems using an analogy to biological ecosystems and the platypus. It argues that software like Drupal evolves in unpredictable ways due to contributions from many independent actors. This results in emergent complexity that is difficult to plan for but can be understood by viewing the software as a dynamic, interdependent ecosystem. The post encourages understanding complexity rather than ignoring it, and adapting to changes by studying the habitat and key contributors that software depends on.
Drupal's Complexity Explained Through Platypus Evolution
1. Hi. I’m eaton.
twitter.com/eaton
Saturday, October 16, 2010
Jeff Eaton.
Core Drupal developer.
Co-author of Using Drupal.
Consultant and architect at Lullabot.
2. Saturday, October 16, 2010
Primarily large clients
Helping them plan, get out of pickles, etc.
Lots of training, public and private.
Although I got involved with code, I spend most of my time planning and coordinating. I’m
that guy.
3. WARNIN
G
NO COD
E
Saturday, October 16, 2010
A warning: Today is a NO CODE ZONE. We’re thinking meta.
This is my cheesemonger. No money to think meta.
This is the White House. No time to think meta.
Both of them are affected - mom’s D5 site broke, and drupal is now different
At events like this we have the luxury of it.
We’re planning for the future.
4. Saturday, October 16, 2010
Let’s go to imagination land and think about some big questions.
5. 1: Platypodes
2: Carnivore, Bill
3: Mammal, Eggs
4: Electric Nose
5: Milk Sweat
6: Poison Feet
Saturday, October 16, 2010
Plural is NOT platypi
Chipmunk cheeks, too.
Two ovaries, but they only use one
6. Phase 2?
Nuclear Hooves
Saturday, October 16, 2010
7. 1: Three Queues
2: Profile Module
3: Two Actions
4: OOrrays
5: Block Regions
6: Yeah, Images
Saturday, October 16, 2010
Remind anyone of something?
Drupal - our complexity is platypus-style
8. Version 8?
Poisonous Feet
Saturday, October 16, 2010
11. Construction
http://www.flickr.com/photos/77422674@N00/693945631/
Saturday, October 16, 2010
“There’s a module for that” -- or 100 modules
How many tools do you have on your largest site? How many modules?
Did you use the RIGHT ONES? How do you know?
Are they released? are they changing?
Security update? Cross-module compatability?
12. Code
http://www.flickr.com/photos/27164277@N00/495517899/
Saturday, October 16, 2010
Solving bigger and bigger problems.
We have more contributors, we have more moving parts.
Dries doesn't get it all, webchick can't keep track, chx doesn't recognize it all
I don't even know what modules were released this week
Same problem in contrib: Views for Drupal 6 is larger than Drupal 5
Multiple solutions emerging every day
13. Community
http://www.flickr.com/photos/8897633@N02/4923221504/
Saturday, October 16, 2010
It's not a club anymore.
There are whole swaths of people who didn't get the 'chx' joke
Who should do X? Who should I ASK about Y?
Who’s responsible for Y? Pay vs Hobbyists? What is Karma FOR?
Who explains Karma to the newcomers?
14. Yes.
It’s a Problem.
Saturday, October 16, 2010
How do we cope with it?
How do we understand it?
The good news is that it’s not just our problem.
15. “I put a GridView onto my
CallBack control and now I get a
ViewState is corrupt error”. The
real answer to this question is
not the one supplied: “this is
fixed in the next version.”
http://whatupdave.tumblr.com/post/1170718843/leaving-net
Saturday, October 16, 2010
ASP.Net projects rely on an ecosystem of Web Controls
Upgrade compatabilities, support hassles, etc.
16. “It’s a pile of spaghetti with insane
indirection and module
dependencies and you’re lucky if you
manage to import anything at all.
Every other function tries to import
modules for you automatically and
circular imports are the norm.”
http://mockit.blogspot.com/2010/04/mess-djangos-in.html
Saturday, October 16, 2010
Django is a cluster of inter-dependent components
Build tools manage the complexity, but it’s there!
17. “From a system
administration standpoint,
Plone is just shy of being the
absolute devil.”
http://stackoverflow.com/questions/348044/what-could-justify-the-complexity-of-plone
Saturday, October 16, 2010
See, that’s just funny.
Wordpress plugin bloat is a growing problem
ExpressionEngine users are complaining that “really” building a site requires piles of addons
18. Saturday, October 16, 2010
Wordpress plugin bloat is a growing problem
Joomla! users have to manage their themes’ compatability with their plugins
ExpressionEngine users are complaining that “really” building a site requires piles of
addons...
Linux is and always will be Linux.
19. NOT US
http://www.flickr.com/photos/21165371@N00/3245301713
Saturday, October 16, 2010
We are not special snowflakes.
Drupal IS particularly susceptible to this
Even the core software itself is built that way
Culture of cross-module interaction
But it's a general problem.
20. But… But…
Architecture!
It’s great to plan
what you control.
http://www.flickr.com/photos/12392252@N03/1839810842/
Saturday, October 16, 2010
Architecture was supposed to protect us from this.
Architecture is about plans and execution.
Blueprints and timelines and tradeoffs
OH THE GANNNNNNNNT CHARTS
21. for what
YOU
Saturday, October 16, 2010
control
Can this be emphasized enough?
Cathedral and Bazaar has its dark side
Big danger is in ignoring it
House with sump pumps
22. Saturday, October 16, 2010
Back to the platypus... Think of Drupal like evolution.
We talk a lot about it, but rarely take it seriously.
Even use "Natural selection" to talk about module success
Dries said patches are mutations!
We call Drupal an Ecosystem, but we ignore it when we plan.
23. Drupal: An
Ecosystem?
Saturday, October 16, 2010
We call Drupal an Ecosystem, but we ignore it when we plan.
All the organisms, and habitats, etc that make up an area.
The whole that emerges from all the little parts.
They're independent, but related.
Interdependent.
24. Drupal: An
Ecosystem?
Saturday, October 16, 2010
Core! Everyone Depends on it!
Third-party modules -- jillions of them!
Participants come and go, submit patches
Teeny modules emerge to fill gaps, and vanish when the techniques fall out of favor
25. Drupal: An
Ecosystem?
Saturday, October 16, 2010
Modules live on sites
Some sites add resources back in
sponsoring development with money or developer-hours
26. Drupal: An
Ecosystem?
Saturday, October 16, 2010
No roadmap, no master plan.
The otter can’t force the turtle to help.
Wild and crazy. It works!
But seriously... what does it look like if we take it seriously?
27. Every module
is a species!
Core is a
critter, too.
http://www.flickr.com/photos/markscott/930501687
Saturday, October 16, 2010
They evolve over time, compete, swallow each other up, etc.
Sometimes they die because there isn't a place for them or they're superceded
Flexinode to CCK to FieldAPI -- and modules that used to do those focused things.
Views today isn't the Views of 2 years ago, and other modules have evolved to depend on it.
28. Mutations!
Patches are
where change
happens
http://www.flickr.com/photos/dullhunk/439737660
Saturday, October 16, 2010
They come constantly.
Sometimes they get folded back into the main 'tree' of code
sometimes they die off
but they're how drupal core and all the modules evolve.
29. Habitats:
The web sites
where modules
live and learn
Saturday, October 16, 2010
Modules live on sites.
Modules that live in lots of habitats tend to thrive better
Selection pressures = size or traffic demands or specific features
These filter out modules that are not well adapted
30. http://www.flickr.com/photos/casch/292720481
Profiles?
Biomes! Well-
known types of
habitats. Er, sites.
Saturday, October 16, 2010
Biomes are They are the commonalities across many habitats, patterns that we can recognize.
They let us learn and share the knowledge.
They let us spot what DOESN’T make sense, too
31. Developers?
NOM
NOM
NOM
http://www.flickr.com/photos/23382958@N02/2497966913/
Saturday, October 16, 2010
More accurately, their time is food.
High-protein food for cats is different than high-fiber bamboo for a panda
different developers have different areas of expertise and knowledge.
32. Drupal: An
Ecosystem.
Saturday, October 16, 2010
33. Does it
help us
understand?
http://www.flickr.com/photos/sylvain_masson/4195880838
Saturday, October 16, 2010
34. Don’t Hack
Core!
Saturday, October 16, 2010
Eventually, you inevitably diverge so much that you can't cross-breed. You're a mule! The
Onion faced that problem. Platypuses face that problem, too. All their living relatives are
extinct. "Hack core, and you're dead to me."
35. Why is it
so ugly?
Saturday, October 16, 2010
Key word -- design. The best work tends to emerge from the specific habitat rather than the
evolutionary process. Why isn't the platypus PRETTIER? Although great design has been
*dropped* on Drupal several times, it rarely emerges from the evolutionary process.
36. There’s a module
for that!
Saturday, October 16, 2010
Diversity is a good thing! Lots of little critters that fill lots of niches is good. It means that
there are lots of solutions being created, and lots of evolution happening.
37. …But don’t
duplicate modules!
Saturday, October 16, 2010
Different needs, different solutions. But resource starvation is dangerous. If you're
duplicating someone else's work, that's two of you who could be working on (feeding?) the
same module.
38. Complexity
is a disease!
(…but we need
wysiwyg so bad…)
Saturday, October 16, 2010
...But we need more stuff in core.
Small things need less food, and are more nimble.
It means more interdependencies, but it doesn't put all the chips in one spot.
It puts the complexity into the SYSTEM, not into the SPECIES.
It’s a tradeoff that makes us more agile in some ways but more vulnerable in others
39. Talk is silver
code is gold
Saturday, October 16, 2010
The 'natural selection' quality only kicks in when something is out there. Ideas are important
and need to be discussed and evaluated, but the process that the vast majority of the
community
41. Study your
habitat
Saturday, October 16, 2010
Look at your site using that habitat analogy
Think of it in terms of the high level functions that it needs: user feedback, product sales, etc
Consider what kinds of ‘creatures’ are well suited for it
Putting dev hours into contrib is like feeding the fauna
42. Know your
honeybees
Saturday, October 16, 2010
Anyone know about the honeybee crash?
Study the subtle interdependencies in your habitat/site.
Watch the vulnerable parts - does your site live and die by hook_cron?
Is there one contributor that helps in every issue?
43. Spot the
mammals
Saturday, October 16, 2010
Change comes, and the tools you built with will change.
Ber Kessels -- God bless him -- stuck with Flexinode.
Watch new modules, screencasts, blog posts, site writeups
Watch for trends that affect you.
44. Plan for
extinction
Saturday, October 16, 2010
Modules go away.
Contributors get busy, start dating.
Maintaining dead modules is like captive breeding
EXERCISE: pick five modules, pretend they vanished
45. Ar r r r r r.
Respect the
platypodes
Saturday, October 16, 2010
Inexplicable, emergent complexity can baffle us all.
The biggest danger is pretending it isn’t there
Architecture needs context.
Don’t punt on complexity - it’s like saying, “Use sump pumps.”