Tools -- things used to make other things -- are at the very core of our humanity. But despite a charter to make things, engineers are too often discouraged from making their own tools: time spent on tooling is viewed as regrettable distraction rather than essential endeavor. This is short-sighted: purpose-built tooling in fact allows for better systems to be delivered faster and with greater confidence -- especially when the systems themselves must be high-performing and/or highly reliable. In this talk, we will explore the primacy of toolmaking in engineering, drawing from both historical examples and personal experience. We will make the argument that when engineers encounter pain due to missing tools, they should be encouraged to develop the tooling that they desperately need. Finally, we will discuss the organizational and cultural traits necessary to foster an encouragement of pragmatic and effective toolmaking.

1. Brought to you by
Sharpening the Axe:
The Primacy of Toolmaking
Bryan Cantrill
CTO of Oxide Computer Company

2. OXIDE
Give me six hours to chop down a tree and I
will spend the first four sharpening the axe.
– Abraham Lincoln
Sharpening the axe

3. OXIDE
Give me six hours to chop down a tree and I
will spend the first four sharpening the axe.
– Abraham Lincoln?
Sharpening the axe?

4. OXIDE
Give me six eight hours to chop down a tree and
I will spend the first four six sharpening the axe.
– Abraham Lincoln?
Sharpening the axe?

5. OXIDE
Give me six eight four hours to chop down a
tree and I will spend the first four six two
sharpening the axe.
– Abraham Lincoln?
Sharpening the axe?

6. OXIDE
Give me six eight four an hours to chop down
a tree and I will spend the first four six two
forty-five minutes sharpening the axe.
– Abraham Lincoln?!
Sharpening the axe?

7. OXIDE
Sharpening the axe?!
The Home and Foreign Record of the Presbyterian Church in the United States of America, Volume 7, Number 1
(January 1856), Sermons for the Times: No. 2: The Dull Axe as cited in Quote Investigator, March 29, 2014

8. OXIDE
“Wisdom is profitable to direct”?
Translations for Ecclesiastes 10:10, from Biblehub.com

9. OXIDE
A woodsman was once asked, “What would
you do if you had just five minutes to chop
down a tree?” He answered, “I would spend the
first two and a half minutes sharpening my axe.”
- C. R. Jaccarde
Sharpening the axe!
Objectives and Philosophy of Public Affairs Education, Increasing Understanding of Public Problems and Policies
(1956), as cited in Quote Investigator, March 29, 2014

10. OXIDE
Toolmaking
• Better tools yield better artifacts
• But the tools themselves are entirely invisible to the user – and are often
not even present in the end product
• Even though we know that toolmaking is important, we do not explicitly
encourage it – it feels like a distraction
• Many consequential developments in software history were in fact
groups seeking to develop better tools for themselves…

11. OXIDE
Unix, ca. 1969
The Evolution of the Unix Time-sharing System,
AT&T Bell Laboratories Technical Journal, Vol. 63, No.8, (October 1984)

12. OXIDE
TeX, ca.1977
Preliminary preliminary description of TeX,
available as https://www.saildart.org/TEX1[1,DEK]1

13. OXIDE
NSELite, ca. 1991
NSELite README, available as http://mcvoy.com/lm/nselite/README

14. OXIDE
DTrace, ca. 2003
• System for dynamic instrumentation of production systems developed at
Sun Microsystems, ~2001-2005
• Informed by 5+ years of trying to understand performance of systems
• Put a bright light on the hidden innards of the system, exposing many
(many!) glaring performance problems
• Many of those problems were much further afield than we envisioned!
• DTrace not only helped debug current issues, it greatly accelerated the
development of software that came after it (e.g., ZFS)

15. OXIDE
Jenkins (née Hudson), ca. 2004
Hudson, Kohsuke Kawaguchi as presented at JavaOne, 2007

16. OXIDE
Rust, 2010
Project Servo: Technology from the past come to save the future from itself,
Graydon Hoare as presented at the Mozilla Annual Summit, July 2010

17. OXIDE
Toolmaking
• These are just high profile examples – many more are never heard of
• There are commonalities across all of these cases:
○ Toolmakers were building for themselves
○ The tools that they were building did not have immediate value on
their own – their value was in delivering better artifacts
○ Toolmakers retained the responsibility for the underlying artifacts
• But why would toolmaking be discouraged?

18. OXIDE
Discouraged toolmaking?
• The challenge of toolmaking is that the payoff is often at a distance
from the investment – both in terms of time and use
• But the cost of toolmaking is immediate and clear: time spent making
the tool is time not spent making the underlying artifact!
• Toolmaking feels like it’s stealing time – it lurks in the shadows
• But while it’s (in principle?) possible to spend too much time on tooling,
it is unlikely to be the wrong use of time – just more difficult to quantify

19. OXIDE
Toolmaking is an investment
• With its clear short term costs and potential long term gains, toolmaking
is best thought of as an investment
• As with any investment, aggressive time horizons (e.g., days/weeks or
small number of months) will result in overly conservative decisions
• And like the best kind of investments, toolmaking also tends to be
compounding: gains early in the lifetime result in more investment that
yields yet more dividends
• If toolmaking is always paying off, you may need to take more risk!

20. OXIDE
Encouraging toolmaking
• Yes, there is risk – but the downside is bounded to time spent
• Indeed, the most common failure mode in engineering is not building too
many tools, it is building something that no one wants
• But in building a tool that one needs, at least one person wants it!
• This is why toolmaking has in fact saved companies (e.g. Flickr, Slack,
Docker): the tooling proved to have more interest than the thing!
• When engineers wish to make tooling, they should be encouraged

21. OXIDE
Toolmaking under fire
• Encouraging toolmaking is hardest when deadlines are tight and
opportunity cost seems high…
• …but engineers themselves are cognizant of these constraints!
• So when these constraints are present and engineers see a need for
purpose-built tooling, that tooling should be prioritized
• This is especially true for tools that improve the artifact by improving
understanding: the best time to develop a debugger is when debugging!

22. OXIDE
Trusting toolmakers
• Toolmaking – and the organizational freedom that engineers need to
engage in it – ultimately requires an essential ingredient: trust
• In creative endeavors, trust must be the organizational binding force!
• To encourage engineers to engage in toolmaking is saying: you are
trusted to use your time wisely – and your future time is valued
• To build an organization that values toolmaking, start by building trust

23. Brought to you by
Bryan Cantrill
bryan@oxidecomputer.com
@bcantrill