Taking Documentation to the Next Level
in Young Software Projects
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Do you contribute to documentation?



Have you created a web page, blog posting,
webinar, video, etc. about a software project?
Have you answered a question on a mailing list
or forum?
Have you asked a question on a mailing list or
forum?
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Structure of this talk
Problems of documentation in young software
projects

Some research into the use of mailing lists and
discussion forums



Comments about online Erlang documentation
Suggestions for developing more and better
documentation
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Documentation and the adoption curve
Innovators Early Adopters
Mainstream success
Documentation
needed
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media
Publishers (best case)
Decline
Publishers (worst case)

Documentation by the innovators
Software developers tend to write for other
people like themselves

Assume deep background knowledge that can
be transferred to their project

Elements of innovator documentation

Architectural justifications

Comparisons to other projects

Quick-starts

Reference material

Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Example: Riak documentation

Focuses on architecture

Useful to distinguish Riak from similar projects

But does not connect architecture to use cases
to make the information practical
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Early adopters arrive



They possess enough background to use
innovators' documentation
Consider documentation adequate
Promote project and generate enthusiasm that
draws other users
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Mainstream users arrive



Confused and alienated by the documentation
Desperately search for help, which may or may
not arrive
Leave project if not engaged and aided
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Research on mailing lists

Used on every project

Easy to quantify results

Every question reflects a failure at documentation
Perhaps the feature is undocumented

Perhaps the documentation could not be found

Perhaps the documentation didn't seem
relevant to the question

Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Questions asked about mailing lists

How many technical questions get answered?

How long does it take to get an answer?

How much noise is on the thread?
http://praxagora.com/andyo/professional/mailing_list/mailing_list.html
http://www.praxagora.com/andyo/professional/mailing_list_follow_up
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

How many questions were answered?
Unanswered
7
7
7
5
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media
Answered
Fedora Linux
7
Ubuntu Linux
7
Rails
7
Perl
9

So, are mailing lists effective?
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

How is Erlang documentation?
Erlang is in a lucky position

It is not a young project

It has been able to develop innumerable
training materials over the decades

Several publishers have released
professional books

Functional principles have spread
throughout the programming field

One must consider the whole information
ecosystem

Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Comments on erlang.org

Site is run by Ericsson

Potentially backed by lots of resources

But how much attention does the company
give it?
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Comments on erlang.org

Getting Started
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Assumptions in documentation

Reader understands the math

Reader understands the notion of recursion

Reader has seen factorials used as an example
of recursive programming
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Focusing on the code

Reader must be able to pick out “active ingredients”
in code
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Focusing on the executable code

Reader must determine when and why each
function runs
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Comments on erlang.org

An Erlang Course
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Questions left by course


Why does Erlang offer both tuples and lists?
Do tuples and lists work like they do in other
languages?
When do you use a tuple and when do you use
a list?

Other lapses in course: unexplained syntax, isolated
examples that don't build on each other, etc.

Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Comments on erlangcentral.org

Site run by the Industrial Erlang User Group

Points to wide range of documents and other sites

Solicits user contributions to wiki
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Criteria for evaluating documentation

Availability—does it exist at all?

Findability—can readers get answers?

Quality—accuracy, readability, relevance, etc.
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Availability—try crowdsourcing




Your developers don't have time to write
everything
Users have innate sympathy for other users
They'll generate content anyway, so you can
coordinate it intelligently
Give up on “One Ring to Rule Them All”
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Crowdsourcing has a history
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Models for crowdsourcing

Development and study of Talmud

Greek symposium
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Mobilizing users for documentation

Search for bloggers, contributors to forums

External motivations can encourage contributions

Access to developers is a motivation

Don't strive for unity—embrace multivocality

Let people use their own favorite media
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Availability—FLOSS Manuals
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Availability—FLOSS Manuals

Coordinate volunteers around the world

Create conventional books

Foster translations
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

FLOSS Manuals—sprints

Bring volunteers together in one space

Usually sponsored by an open source project

Google has hosted many documentation sprints
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Findability—more than a search engine

Add boxes for people to recommend documents

“Read before this document”

“Read after this document”

Long-term goal: spider creates recommended
paths through documents
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Quality—What's the ultimate goal?

After reading the document, can the reader
accomplish the task she set out to do?
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Quality—A/B testing



Attach a quiz to a document
As you rewrite the document, check how well
the readers can answer the quiz
Remember: a user experience encompasses more
than one document
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

Summary



Documentation and training materials are crucial
to mainstream adoption of young projects
A serious need exists for better documentation
Invest special efforts to create documentation,
make it findable, and improve its quality
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media

About me
@praxagora
andyo@oreilly.com
http://programming.oreilly.com/andyo
Erlang Factory, 23 October 2013
Andy Oram, O'Reilly Media