Aaron Volkmann, Senior Software/Research Engineer, Software Engineering Institute
All organizations face challenges in changing their culture and adopting DevOps philosophies. This is especially true in many federal government agencies. Through well-intentioned policies and procedures many agencies have created extremely silo’d environments where change is slow and difficult. Finishing the last leg of large scale software development project acquisitions can be particularly challenging and expensive. Barriers often impede getting hardware and software systems system fully tested, transitioned, and up and running in production on schedule.
Through our experience as a passionately DevOps focused software development group within Carnegie University’s Software Engineering Institute, a federally funded research and development center, creating, delivering and transitioning cutting edge software solutions to government organizations, we have struggled with and overcame challenges in helping government to adopt DevOps principles. Learn how we have conquered these challenges in shifting our government stakeholders’ thinking by coaching and initiating DevOps in their operational and development environments.
FFRDC
CERT CC – 1988 Morris Worm - We regularly partner with government, industry, law enforcement, and academia to develop advanced methods and technologies to counter large-scale, sophisticated cyber threats.
Develop and deliver prototype software solutions
Prototypes become production
Liason – Business Person – Users
No ops contact
Agile shop
CI / CD
No access to test systems on customer’s side
Environment was communicated to us through email
1 year of development
Testing in pseudo production
Production problems found late in the game before funding dried up
Right stakeholders not brought in at the right time (ops, security)
CYA
Blame
Friction with other projects
Gained access to proper test environments - Environment parity is key
Learned who the key people were who needed to be engaged
Customer used to monolithic releases
Get customer used to taking releases more often
Continuous Delivery into a staging environment on their side
Made documentation available via doc server
Approval process was big, did that once, then re-used authorization for iterative releases
Early Failures are good
No blame, encourage risk taking
Pilot release of new versions of software
PCSAM – Problem Cause Solution Action Measure
Cause – Why Why Why Why
Measure – Tie back to a business outcome to find a useful metric
Index cards
Went into backlog
Fix in order of how much pain they caused
Keeps mistakes from being repeated
Not having any problems in a certain amount of time should be considered a problem
Amplified feedback loops
Communication from users to developers direct
Specialized knowledge not in people’s heads, it’s documented
Brent effect
McGill University Empathy Study
A stronger partnership
Increased visibility of progress
Fewer defects
Faster lead time
More transparency