The document discusses the evolution of database administration from yesterday's gatekeeper role to today's database engineering role. A database engineer provides expertise to development teams to help them meet performance metrics and troubleshoot issues. The document outlines how database engineering aligns with devops goals and processes. It proposes that database engineers follow a craft approach focused on protecting data, eliminating waste, and data-driven decision making.

1. from antiquated to engineer
AllYourBase, November 2015
Laine Campbell, DB Architect/Engineer
lainevc@gmail.com @lainevcampbell
September, 2015

2. engineering, not administration
2

3. yesterday’s DBA
3
gatekeeper
master builder
superhero
siloed
specialized

4. engineering
4
quantitative
interdisciplinary
results focused
repeatable and code-driven

5. database engineering
5
shared services for:
cross-functional development teams
operations engineering
systems engineering

6. database engineering
6
provides expertise to other team’s processes to facilitate
all work (for example, SRE teams)
understands and teaches data access and storage
nuances to ensure all performance metrics can be met
anchors teams with expertise for troubleshooting,
recovery and other tasks requiring depth, not breadth

7. 7
paradigm
shifts

8. polyglot persistence
8
relational is not the end of the line
data exists as part of a pipeline
function dictates form
we cannot predict all uses

10. virtualization and cloud
10
forces horizontal scaling
forces designing for resilience
elasticity drives new data storage
management by API
enables continuous deployment

11. infrastructure as code
11
forces standardization
enables immutability
forces us to learn to code
we start building platforms
changes become deployments

12. continuous delivery
© 2014 Pythian 12
brings us to the source code control
paradigm
we must be teachers, not
gatekeepers
testing and compliance become
top priorities

13. devops cultures
13
lean manufacturing defines our workflows
theory of constraints (we are a constraint)
tighter feedback loops require organizational
shifts
experimentation and controlled failure shift
architecture and process design
integration drives empathy

14. database engineers:
O.G. devops
14
DEV OPS
DBE
shared goals, tools and processes

15. 15

16. db engineer’s manifesto
16
it’s all about the mission
protect the data
eliminate waste
data-driven decision making
databases are not special
eliminate the barriers between software and ops

17. the
craft
17

18. maslow’s hierarchy of needs
18
physiological
safety
love and belonging
esteem
self-actualization
with loving credit and glory to Charity F. Majors

19. physiological
19
physiological does it run and service your application?
is it stable?
can data be backed up and restored?
can basic levels of service be monitored?

20. safety
20
physiological
safety
do I have n+1 available copies?
is failover built, documented and tested?
is my data distributed to multiple physical
locations?
can I detect, and recover from corruption?
is access managed via roles, separation of duties
and appropriate authorization?
is appropriate data encrypted in transport and at
rest?

21. love and belonging
21
physiological are database changes part of integration and
deployment processes?
is continuous testing done with proper data sets?
is infrastructure managed via configuration
management, automation and orchestration?
is all database interaction done via scripted,
repeatable processes?
is consistency of configuration monitored and
managed?
safety
love and belonging

22. esteem
22
physiological
safety
love and belonging
esteem
are all relevant metrics being captured at
appropriate granularity?
can problems be detected preemptively?
are capacity correlations understand and
monitored?
are developers trained and knowledgeable on the
datastore?
can the system automatically remediate frequent
and simple faults?

23. self-actualization
23
physiological
safety
love and belonging
esteem
self-actualization
are you using the right datastore for the right job?
are your developers in tight feedback loops with
the datastore?
is experimentation and data driven decision
making supported by the datastore?
is the mission fully supported?

24. a day in the life...
24
selecting datastores to add to
production platform catalogs
dbms and feature education to
software, systems and
operations engineers
integrating with company-wide
services (reporting, monitoring,
config mgmt etc…)

25. a day in the life...
25
validating acceptable configurations
testing and benchmarking new
versions,
features and configurations
documenting and sharing standards

26. deploying infrastructure
26
configuration management definition
orchestration enablement
change and deployment test automation
building self-service tools for the team

27. deploying software
27
agile participation with engineers
using version control for schemas and
metadata
more flexible data model approaches
teach your engineering teams how to
assess risk, performance and impact
support test planning, building and
execution

28. a day in the life...
28
attending scrums, grooming
and planning
committing deployment scripts
to the codebase
building, running and
automating test cases
monitoring commits for key
areas that need your attention

29. a day in the life… (cont)
29
providing new or modified
recipes for CM and
automation
pairing with, and teaching
engineers, iterating on
the schema

30. the data must flow
30
data brokers
pub/sub systems
real-time analytics
batch processing
cache population, use, and
flushing
search stores

31. backup and recovery
31
this has not changed, we live and die by
the safety of our data
borrow ideas of continuous deployment,
for continuous recovery testing
build backup and recovery into every
possible process

32. a day in the life...
reviewing current workloads and
tuning
managing escalations on DB issues
continuous improvement
writing, testing and performing
change plans as part of the
deployment process
32

33. 33
how?

34. software engineering
34
bring DBEs into your agile
workflows
learn the software versioning
system
learn the testing frameworks
study the language, the
framework, the drivers and the
ORMs
teach, teach teach!

35. systems engineering
35
design your data pipelines
together
collaborate on data standards
together
learn the entire stack to
moderate proficiency
understand the entire flow
teach, teach, teach!

36. operations engineering
36
build your scripting chops
learn the build and deploy
frameworks
learn the configuration management,
automation and orchestration layers
and patterns
learn operational visibility layers and
patterns
plan, execute and learn from failures

37. further deep dives
37
understand the statistics and math
around distributions, anomaly
detection and correlation
write and push code!
answer the customer service
phones
dive into your network layers
teach everyone about the data