How to Troubleshoot Apps for the Modern Connected Worker
Why we got to Docker
1. Why we got to Docker
by Jeff Nickoloff
Engineer, Manager, Entrepreneur, Consultant, and Docker Captain
Why do we Write Software and Use Tools?
2. –Terry Pratchett
“If you do not know where you come from, then
you don't know where you are, and if you don't
know where you are, then you don't know where
you're going. And if you don't know where
you're going, you're probably going wrong.”
3. Just Me
“If you don’t understand why you need to
change how can you understand what change
must be made?”
4. In the Beginning
(and for a while thereafter)
• Pre-Network or Purpose Built Networks
• Almost exclusively batch processing
• Course operational needs
(low/specific use means lower entropy)
• Mainframes / Workstations with direct link
• Hyper-localized data
• Bank / Card Networks
5. Dawn of the Age of Mass
Communication
• Networked Offices
• Centralized office services (comms)
• Mainframe emulation (3270 anyone?)
• Databases and back office software
• Operations require on-prem or on-call IT staff
(greater dependence on networked resources)
• Broader adoption of standard network application protocols like
SMTP, HTTP, Usenet, etc
• WWW - companies had web sites run by rouge geeks in IT
6. Mass Communication for the
Masses
• Internet and Consumer Broadband
• High school geeks had email addresses and Geocities accounts
• eCommerce - regular people started buying things online. Some
companies only exist on the WWW.
• A golden age for network protocols
• Hacks are common - software is weak and untested - execution
environments are laughably insecure
• Computing operations suddenly become critical to customer
experiences
7. Hyper-Growth of
eCommerce
• Mass Broadband - always-on connections
• All of a sudden there are “CTOs and CIOs”
• Streaming video, Web 2.0, “Service Oriented Architectures,” B2B
protocols with SOAP, XML, WSDL
• Developers detached from operations - building faster, standing on
taller giants, become detached from underlying systems - software
quality transformed into money problem by over consumption of
resources
• Testing become a popular concern for everyone (TDD, BDD, etc)
• Operations experiences extreme capacity planning pain, provisioning
pain, and software is still ridiculously buggy
8. Virtualization and
“The Cloud”
• Virtualized compute, network, and storage
• Explodes the quantity of logical machines in use
• Creates massive infrastructure level logistics burden
• Tools emerge for “configuration management”
• Operations are more important than ever and improving but now require a
whole new skill set
• “The Cloud”
• IaaS companies emerge
• Augment slow on-prem changes with “elastic” resources
• provides Lower lead times, geographically diverse
9. Problems
• Code bases large and tightly coupled. Business logic is written
at such a high level that most developers do not understand
how their software actually operates from a system perspective.
• Despite enhanced testing engineers cannot consistently
anticipate bugs and scale issues. Event triage is as painful as
ever.
• Operations are becoming more burdensome and struggle to
keep up with continued eCommerce growth.
• As monolithic applications grow “build engineering” teams
struggle to release with confidence - slowing release cycles.
10. Microservices
(2010 and later)
• Enforces coding to interface contracts
• Code reuse means resource reuse
• Independent release cycles
• Focused functional and integration testing feasible
• PaaS companies start emerging with proprietary
application level isolation
11. Containers
(2013 and later)
• Docker… and eventually others
• Abstracts process isolation techniques and creates a common
nomenclature
• Best practices become achievable practices and so become adopted
practices
• Encourages 12factor application patterns
• Well defined interface between application and infrastructure
• Infrastructure becomes application agnostic
• Applications become infrastructure agnostic
• Deployment times reduced from minutes to milli/microseconds
12. Containers Cont.
• Version control whole development environments
• Collocate stacks with library dependency conflicts
• New software packaging, distribution, and deployment
primitives, transports, and interfaces
• New opportunities for application level security
practices like payload signatures, and secure
transports
• Authoritative application configuration
13. Problem
• Explosion of new tools, distributed systems, infosec
practices as requisite knowledge
• Operations requires deep insight into applications,
systems, and their integration
• Increasing the number of services increases
operational burden at least linearly
14. What grew large and course
becomes small and fine again
• adopt simpler service stacks without “thick platforms”
like Application servers
• Movement to minimize contained material like
otherwise standard OS tools
• Multi-user doesn’t happen often for service software
• Minimal host operating systems - Kernel + Docker
• Unikernels - your application is your operating system
15. What was “tribal knowledge”
is becoming documented and automated
• Insight - Telemetry, monitors, and alarms
• Architecture Metadata - inter-service dependencies
• Secrets - Generation, storage, orchestration
• Code confidence - multi-dimensional testing
• Automation - Build, deployment, incident response
16. How to Solve Problems?
• Be engineers - think like engineers
• Choose success over perfection or trend
• Iterate toward your vision
(Choose Tools and Write Software)
