We will delve into the creation of the GSA's DevSecOps guide, progression towards componentized and lego-pieced ATO's (leveraging reusable Infrastructure and Configuration as-Code modules), Cloud.gov "Heroku for government", "how to" be Cloud agnostic, and more. Our DevSecOps meetup: https://www.meetup.com/DevSecOps-NoVA The Handbook: https://tech.gsa.gov/guides/dev_sec_ops_guide/ Our speakers group: https://handbook.tts.gsa.gov/tech-portfolio/ His team's areas of responsibility: https://digital.gov/services/

1. 1
DevSecOps at the
Technology Transformation Service (TTS)
@ the General Services Administration
(GSA)
John Jediny
Technology Transformation Service (TTS)

2. Powered By Technology Transformation Services 2
Challen
ge.gov
USWDS Bug
Bounty
Cloud.
gov
FedRAMP Login.
gov
Data.
gov
Search.
gov
USA.
gov
Digital.
gov
10x Federalist
Presidential
Innovation
Fellows
18F Centers of
Excellence
https://digital.gov/services/directory/
https://join.tts.gsa.gov/
Who we are

3. Powered By Technology Transformation Services
ACCELERATORS ARTIFICIAL
INTELLIGENCE
CLOUD &
INFRASTRUCTURE
DATA &
ANALYTICS
EXPERIENCE IDENTITY
Modern
organizations
that can build,
buy and
manage smart
scalable
tech
Leveraging
analytics and
machine
learning
to develop
deeper
insights
and automate
processes
Converged
infrastructure
that scales
and
reduce cost
securely
Applying
modern
data-driven
decision-maki
ng practices
Seamless
interactions
and
experiences
regardless of
channel
Shared
solutions for
increasing
trust
and avoiding
fraud
Who we are: How We Support Agency Partners

4. Powered By Technology Transformation Services
Software Authorizations
Who are we: Authority to Operate (ATOs)

5. Powered By Technology Transformation Services 5
● Have a single person (with a backup) in
charge of shepherding new
software/product requests
● Work with Vendors through FEDRAMP
tailored as a liaison
● Make the process more transparent
and efficient
● Make a list of available software that
can be used and licenses that can be
leveraged
● Ensure all TTS requests go through the
same process to allow staff to
understand where a product is at in the
process and when they can use it
Problems Actions
● Current software request process is
cumbersome and convoluted
● Tech Portfolio is sometimes left out of
communications around software and
is unable to check the status
● Purchase card holder is left out of the
loop on whether the product is
approved for purchase
● Wasted time for many involved
Who are we: Software Concierge

6. Powered By Technology Transformation Services
● Out of sync with GSA policies,
practices and procedures
● Incomplete picture of cyber posture at
an organizational level
● TTS gets a lot of vulnerability reports
for systems that aren’t ours
● Other agencies interested in
leveraging our Bug Bounty program
● Between Custom Developed Software
and Software as a Service, there are 36
ATOs that need managing from TTS
● Lack of visibility into the “ingredients”
that are in our software
6
● Serve as centralized point of contact to GSA
IT Security
● Standing up tooling to get a better picture
of cybersecurity at the portfolio level
● Informed Vulnerability Disclosure Binding
Operational Directive to ensure it’s easy to
report to agencies
● Considered offering Bug Bounty beyond TTS
● Lead DevSecOps Guild meetings, putting
action to identified problems
● Provide Subject Matter Expertise on
Software Integrity to TTS, GSA, and rest
Federal Gov
● Lead GSA-wide Software
Assurance-focused Supply Chain Risk
Management (SCRM) Working Group
Problems Actions
Who are we: Software-as-a-Service

7. Powered By Technology Transformation Services 7
● Auditing environment
○ Identification of unused
resources
● Single sign-on (IAM, etc.)
● Continuous monitoring/alerting
○ Re-using GRACE components
● Establish standards for better
cross-team management
○ Role-based management of
AWS
● Engagement with program teams to
better understand how centralized
solutions fit and extend their
existing workflows
Problems Actions
● Infrastructure managed ad-hoc by
program teams
● Disparity in how alerting is
implemented across program teams
● TTS has limited ability for oversight
at an organizational level
● Different processes create differing
levels of coverage and specificity
Who are we: Infrastructure Management

8. Powered By Technology Transformation Services
● Doubling down on cloud.gov
○ Expand cloud.gov’s service offerings
○ Focus infrastructure investments
from programs into TTS-wide
standardized services, such as
Kubernetes-as-a-Service
● Developing TTS-wide infrastructure
goals/roadmap through interviews with TTS
System Owners
● Standardization of AWS, Azure, and Google
Cloud management through code and
automation
○ Re-using GRACE components
● Unified account management,
synchronizing with GSA Active Directory
● Better secrets management
● Sharing technical components and their
compliance information (“Common Control
Platform”)
8
● TTS systems’ infrastructure varies
greatly
● Cloud.gov isn’t compatible with some
architectures, leading to greater
responsibility for programs to manage
their infrastructure
● Each program is configuring tools and
writing their SSPs independently,
duplicating effort
○ Larger issue: Inconsistent access
control and knowledge sharing
Problem Actions
Who are we: Infrastructure Improvement

9. Powered By Technology Transformation Services 9
● Minimize heavily siloed talent:
people work on a single system
● Lots of difference in architectures
and reinvention of wheels
● Teams have to make due with the
staff (skills) they have
○ May be missing critical skill
sets
● Improving knowledge
management for engineering
organizational structures
● Ensuring institutional knowledge
with heavy turn over
● Planning on interviews, surveys,
and/or Q&A sessions around
Solutions to better understand the
pain points and appetite for
change
● Working towards proposing a
different structure for engineering
Problem Actions
Who are we: Shared Engineering

10. https://hackerone.com/tts
Who are we: Bug Bounty Program Management

11. 11
https://tech.gsa.gov/guides/dev_sec_ops_guide/
Mission:
● Support the DevOps cultural transformation within GSA.
● “Assess the Gap” of hosting options between those systems manually provisioned
to those already cloud native systems w/ self-service deployment.
● Move security left and increase developers awareness of security “make the right
way the easy way”.
● “Don’t Repeat YOurselves” (DRY) by providing pre-hardened and reusable
Operating Systems and Common Components.
● Reduce the overall time of any GSA system’s “Authority to Operate”.
● “Plug the Gap” around the lack of hosting for systems “ready” for fully automated
systems using Infrastructure as Code, Configuration Management, and CI/CD.
DevSecOps Working Group at GSA
Cloud Native
Manual Provisioning /
Ticket Based Networking
DevSecOps
Infrastructure as Code
& Configuration
Management
Automation

12. 2
GSA IT Leadership (CIO/CTO/CISO)
❏ GSA IT Standards Pilot status for
testing new tools
❏ Authority to Operate (ATO) for Pilot
DevSecOps Sandbox
❏ Contract and In-house FTE support
❏ Organizational and Political
endorsement
GSA IT (IDI)
❏ Standardized Operating System
Repository
❏ Component Hardening
❏ Reusable Infrastructure as Code and
Configuration Management
❏ Centralized Pipeline Management
❏ User Onboarding & Account
Management
DevSecOps Roles and Responsibilities
GSA IT (SecOps)
❏ Hardening Guides
❏ Nessus/Twitlock Scanning
GSA IT (CTO Office)
❏ Platform Governance
❏ Financial Management
Technology Transformation Service (TTS)
❏ Engineering Support
All
❏ Collaboration
❏ Metrics

13. 2
Steps towards implementing DevSecOps...

14. 2
Step: Provide Software for Local Development

15. 2
Step 1: Use *-as-a-Service(s)
https://marketplace.fedramp.gov
https://digital.gov/services/directory/

16. 2
Step: Make Self-Servicing easy
https://github.com/openservicebrokerapi/servicebroker/
https://github.com/cloudfoundry-incubator/cloud-service-broker/blob/master/docs/brokerpak-intro.md
https://github.com/cloudfoundry-incubator/csb-brokerpak-aws
Application → Terraform → Cloud Service Provider → Database etc
Application ← Terraform ← Cloud Service Provider ← Database etc

17. 2
Step: Shared OS Hardening
Provide Ansible Roles for Hardening Operating System to existing
GSA Benchmarks:
https://github.com/GSA/security-benchmarks
Standard Hardened Images
● Ubuntu
● Red Hat
● CentOS
● Windows

18. Step: Create Reusable Security Components
https://github.com/gsa?q=odp
● SAML/Oauth Proxy server
● Encryption and Networking
○ VPN
○ SSL Cert (Lets Encrypt)
○ Storage at-Rest
● Host Intrusion Detection
● Logging Parsers/Aggregators
● Monitoring/Metrics (CPU/Network/Storage/Disk Monitoring)
● Search Indexing
● Visualization
● File Integrity
● Alerting

19. so they can...
Active
Scans
Passive
Scans
BDD &
ATDD
Testing
Compliance &
Configuration
Testing
Step: Focus on Continuous Integration, then Deployment
CI/CD

20. 2
Step: GitOps not Change Management Boards

21. 2
Step: Avoid Pets (Mutable Deployments)
STEP 1 - Test
Applications
are built and
tested
STEP 2
Stage Deploy
Jenkins executes
Ansible scripts on
each Dev/Staging
server to deploy
app via SSH
us-east-1c
Private
Subnet
EC2
us-east-1b
Private
Subnet
EC2
inventory file
STEP 3
Validation Testing
Automated and
manual validation,
integration,
acceptance testing
STEP 4
Prod Deploy
Jenkins executes
Ansible scripts on
each Production
server to deploy
app via SSH
us-east-1c
Private
Subnet
EC2
us-east-1b
Private
Subnet
EC2
inventory file

22. Step: Employ Cattle (Immutable Deployments)
Production/Staging Environment
Dev Environment
STEP 4 - Backup
Backup last 5 AMI(s) for
Rollback
Provision Production
(Auto-Scaling AMIs)
STEP 3 - Deploy
Deployment Platform
Test Deploy
STEP 2 - Provision
Applications on top
of Hardened Image
STEP 1 - Test
Applications
are built and
tested
Build
5x 5x 5x 5x 5x

23. 2
Step: Work Towards Compliance-as-Code
https://pages.nist.gov/OSCAL/
https://saf.mitre.org/#/

24. Demo / Questions?
https://tech.gsa.gov/guides/dev_sec_ops_guide/
https://github.com/18F/tts-tech-portfolio