A concrete talk about abstraction.
Theo Schlossnagle @postwait 1
Founder: Circonus, OmniTI, Sparkpost, Fontdeck
Distinguish ACM Member
ACM Member at Large
Avid Open Source Contributor
Theo Schlossnagle @postwait 2
Boundaries in the software/hardware and software/human interface.
Also in the software/software interface.
Theo Schlossnagle @postwait 3
Instruction Set Architecture
Theo Schlossnagle @postwait 4
Abstractions boundaries are at a
significantly lower dimension
than the space they separate
(typically greater than four).
Theo Schlossnagle @postwait 5
Time Series Data
Abstractions are divides,
and the world isn’t always clean.
Performance requirements are
often a witness.
“Leaky Abstractions” or
Theo Schlossnagle @postwait 6
Time Series Data
Time Series Data
(for rollup ETL)
Abstractions aim to be generically useful.
The fewer initial consumers, the less likely this will be satisfied.
Theo Schlossnagle @postwait 7
Abstractions interfere with holistic
Problems span layers.
Therefore analysis spans layers.
Abstractions are loans.
If abstraction boundaries bisect
common analysis tasks, those end
up being high-interest loans.
Theo Schlossnagle @postwait 8
Abstractions in real-life can win, can lose, but most often do both.
Theo Schlossnagle @postwait 9
Time Series Database
• Resource Pools
Theo Schlossnagle @postwait 10
A sordid love story of layer violations and compromise.
Theo Schlossnagle @postwait 11
libmtev is a multi-threaded, event-driven framework for
building high-performance service applications in C
It makes it “easy”… for C developers that understand event-driven
Lua is an elegant and accessible functional language
LuaJIT (is lua, but fast)
LuaJIT’s embedding API make violating boundaries
Theo Schlossnagle @postwait 12
Mistakes layered on mistakes layered on mistakes.
Theo Schlossnagle @postwait 13
updating, appending, deleting
reading randomly, sequentially, bulk
It’s basically impossible to excel at all without layering
Theo Schlossnagle @postwait 14
Concurrency limitations and fair scheduling.
Theo Schlossnagle @postwait 15
Resources (CPU, memory, disk, L2 cache, network, etc.) are
Concurrency can help until… it hurts.
For performance, we abstract on resource usage
Theo Schlossnagle @postwait 16
Concrete lessons about abstractions.
Theo Schlossnagle @postwait 17
No one would get anything done. Ever.
Theo Schlossnagle @postwait 18
Microservices (in particular) afford
isolation choices beyond code
complexity: security, compliance,
availability, ownership, etc.
The dimension on which the abstraction
boundary is drawn should be selected to
yield the most benefit.
Theo Schlossnagle @postwait 19
The more things (and ultimately people)
that use the abstraction, the better it will
be designed (or subsequently
Never build an API without a real
consumer (TDD doesn’t count). This
should apply to all simplifications.
Theo Schlossnagle @postwait 20
They are not a goal, but they are not
evil. It should be intentional.
Some things behind an abstraction
should intuitively be hidden. Intuition
isn’t always right. When these leaks
consistently benefit multiple
stakeholders, it simply means the line of
abstraction has been redrawn.
Theo Schlossnagle @postwait 21
They happen all the time.
I am unapologetic about this.
Violation carries cost burden.
Assess it, accept it, and move on.
Theo Schlossnagle @postwait 22
Theo Schlossnagle @postwait 23