Drupal's Complexity Explained Through Platypus Evolution

Hi. I’m eaton.
twitter.com/eaton

Saturday, October 16, 2010

Jeff Eaton.
Core Drupal developer.
Co-author of Using Drupal.
Consultant and architect at Lullabot.


Primarily large clients
Helping them plan, get out of pickles, etc.
Lots of training, public and private.
Although I got involved with code, I spend most of my time planning and coordinating. I’m
that guy.

WARNIN
G

NO COD
E

A warning: Today is a NO CODE ZONE. We’re thinking meta.
This is my cheesemonger. No money to think meta.
This is the White House. No time to think meta.
Both of them are affected - mom’s D5 site broke, and drupal is now different
At events like this we have the luxury of it.
We’re planning for the future.


Let’s go to imagination land and think about some big questions.

1: Platypodes
2: Carnivore, Bill
3: Mammal, Eggs
4: Electric Nose
5: Milk Sweat
6: Poison Feet

Plural is NOT platypi
Chipmunk cheeks, too.
Two ovaries, but they only use one

Phase 2?

Nuclear Hooves

1: Three Queues
2: Proﬁle Module
3: Two Actions
4: OOrrays
5: Block Regions
6: Yeah, Images

Remind anyone of something?
Drupal - our complexity is platypus-style

Version 8?

Poisonous Feet

The Platypus Problem:

Inexplicable
Emergent
Complexity

http://www.ﬂickr.com/photos/striatic/2134277399

Is complexity truly a problem?
Isn’t it something that we can throw more people at?
More documentation?
Isn’t it just the eternal tug-of-war?

Construction

http://www.ﬂickr.com/photos/77422674@N00/693945631/

“There’s a module for that” -- or 100 modules
How many tools do you have on your largest site? How many modules?
Did you use the RIGHT ONES? How do you know?
Are they released? are they changing?
Security update? Cross-module compatability?

Code


Solving bigger and bigger problems.
We have more contributors, we have more moving parts.
Dries doesn't get it all, webchick can't keep track, chx doesn't recognize it all
I don't even know what modules were released this week
Same problem in contrib: Views for Drupal 6 is larger than Drupal 5
Multiple solutions emerging every day

Community


It's not a club anymore.
There are whole swaths of people who didn't get the 'chx' joke
Who should do X? Who should I ASK about Y?
Who’s responsible for Y? Pay vs Hobbyists? What is Karma FOR?
Who explains Karma to the newcomers?

Yes.
It’s a Problem.

How do we cope with it?
How do we understand it?
The good news is that it’s not just our problem.

“I put a GridView onto my
CallBack control and now I get a
ViewState is corrupt error”. The
real answer to this question is
not the one supplied: “this is
ﬁxed in the next version.”

http://whatupdave.tumblr.com/post/1170718843/leaving-net

ASP.Net projects rely on an ecosystem of Web Controls
Upgrade compatabilities, support hassles, etc.

“It’s a pile of spaghetti with insane
indirection and module
dependencies and you’re lucky if you
manage to import anything at all.
Every other function tries to import
modules for you automatically and
circular imports are the norm.”

http://mockit.blogspot.com/2010/04/mess-djangos-in.html

Django is a cluster of inter-dependent components
Build tools manage the complexity, but it’s there!

“From a system
administration standpoint,
Plone is just shy of being the
absolute devil.”

http://stackoverﬂow.com/questions/348044/what-could-justify-the-complexity-of-plone

See, that’s just funny.
Wordpress plugin bloat is a growing problem
ExpressionEngine users are complaining that “really” building a site requires piles of addons


Wordpress plugin bloat is a growing problem
Joomla! users have to manage their themes’ compatability with their plugins
ExpressionEngine users are complaining that “really” building a site requires piles of
addons...
Linux is and always will be Linux.

NOT US

http://www.ﬂickr.com/photos/21165371@N00/3245301713

We are not special snowﬂakes.
Drupal IS particularly susceptible to this
Even the core software itself is built that way
Culture of cross-module interaction
But it's a general problem.

But… But…
Architecture!

It’s great to plan
what you control.

Architecture was supposed to protect us from this.
Architecture is about plans and execution.
Blueprints and timelines and tradeoffs
OH THE GANNNNNNNNT CHARTS

for what

YOU
control
Can this be emphasized enough?
Cathedral and Bazaar has its dark side
Big danger is in ignoring it
House with sump pumps


Back to the platypus... Think of Drupal like evolution.
We talk a lot about it, but rarely take it seriously.
Even use "Natural selection" to talk about module success
Dries said patches are mutations!
We call Drupal an Ecosystem, but we ignore it when we plan.

Drupal: An
Ecosystem?


We call Drupal an Ecosystem, but we ignore it when we plan.
All the organisms, and habitats, etc that make up an area.
The whole that emerges from all the little parts.
They're independent, but related.
Interdependent.

Drupal: An
Ecosystem?


Core! Everyone Depends on it!
Third-party modules -- jillions of them!
Participants come and go, submit patches
Teeny modules emerge to ﬁll gaps, and vanish when the techniques fall out of favor

Drupal: An
Ecosystem?


Modules live on sites
Some sites add resources back in
sponsoring development with money or developer-hours

Drupal: An
Ecosystem?


No roadmap, no master plan.
The otter can’t force the turtle to help.
Wild and crazy. It works!
But seriously... what does it look like if we take it seriously?

Every module
is a species!

Core is a
critter, too.
http://www.ﬂickr.com/photos/markscott/930501687

They evolve over time, compete, swallow each other up, etc.
Sometimes they die because there isn't a place for them or they're superceded
Flexinode to CCK to FieldAPI -- and modules that used to do those focused things.
Views today isn't the Views of 2 years ago, and other modules have evolved to depend on it.

Mutations!

Patches are
where change
happens

http://www.ﬂickr.com/photos/dullhunk/439737660

They come constantly.
Sometimes they get folded back into the main 'tree' of code
sometimes they die off
but they're how drupal core and all the modules evolve.

Habitats:

The web sites
where modules
live and learn


Modules live on sites.
Modules that live in lots of habitats tend to thrive better
Selection pressures = size or traffic demands or speciﬁc features
These ﬁlter out modules that are not well adapted

http://www.ﬂickr.com/photos/casch/292720481

Proﬁles?

Biomes! Well-
known types of
habitats. Er, sites.

Biomes are They are the commonalities across many habitats, patterns that we can recognize.
They let us learn and share the knowledge.
They let us spot what DOESN’T make sense, too

Developers?

NOM
NOM
NOM

More accurately, their time is food.
High-protein food for cats is different than high-ﬁber bamboo for a panda
different developers have different areas of expertise and knowledge.

Drupal: An
Ecosystem.


Does it
help us
understand?

http://www.ﬂickr.com/photos/sylvain_masson/4195880838

Don’t Hack
Core!


Eventually, you inevitably diverge so much that you can't cross-breed. You're a mule! The
Onion faced that problem. Platypuses face that problem, too. All their living relatives are
extinct. "Hack core, and you're dead to me."

Why is it
so ugly?


Key word -- design. The best work tends to emerge from the speciﬁc habitat rather than the
evolutionary process. Why isn't the platypus PRETTIER? Although great design has been
*dropped* on Drupal several times, it rarely emerges from the evolutionary process.

There’s a module
for that!


Diversity is a good thing! Lots of little critters that ﬁll lots of niches is good. It means that
there are lots of solutions being created, and lots of evolution happening.

…But don’t
duplicate modules!


Different needs, different solutions. But resource starvation is dangerous. If you're
duplicating someone else's work, that's two of you who could be working on (feeding?) the
same module.

Complexity
is a disease!
(…but we need
wysiwyg so bad…)


...But we need more stuff in core.
Small things need less food, and are more nimble.
It means more interdependencies, but it doesn't put all the chips in one spot.
It puts the complexity into the SYSTEM, not into the SPECIES.
It’s a tradeoff that makes us more agile in some ways but more vulnerable in others

Talk is silver
code is gold


The 'natural selection' quality only kicks in when something is out there. Ideas are important
and need to be discussed and evaluated, but the process that the vast majority of the
community

The take-away


Study your
habitat


Look at your site using that habitat analogy
Think of it in terms of the high level functions that it needs: user feedback, product sales, etc
Consider what kinds of ‘creatures’ are well suited for it
Putting dev hours into contrib is like feeding the fauna

Know your
honeybees


Anyone know about the honeybee crash?
Study the subtle interdependencies in your habitat/site.
Watch the vulnerable parts - does your site live and die by hook_cron?
Is there one contributor that helps in every issue?

Spot the
mammals


Change comes, and the tools you built with will change.
Ber Kessels -- God bless him -- stuck with Flexinode.
Watch new modules, screencasts, blog posts, site writeups
Watch for trends that affect you.

Plan for
extinction


Modules go away.
Contributors get busy, start dating.
Maintaining dead modules is like captive breeding
EXERCISE: pick ﬁve modules, pretend they vanished

Ar r r r r r.

Respect the
platypodes

Inexplicable, emergent complexity can baffle us all.
The biggest danger is pretending it isn’t there
Architecture needs context.
Don’t punt on complexity - it’s like saying, “Use sump pumps.”

Drupal's Complexity Explained Through Platypus Evolution

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Drupal's Complexity Explained Through Platypus Evolution

Similar to Drupal's Complexity Explained Through Platypus Evolution (20)

More from Jeff Eaton

More from Jeff Eaton (19)

Recently uploaded

Recently uploaded (20)

Drupal's Complexity Explained Through Platypus Evolution