DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012

DEV/OPS, CONTINUOUS DEPLOYMENT
& APIS, OH MY!

Matt Tesauro, Texas Linux Fest
– San Antonio, TX, August 2012

2

WHO AM I?
Matt Tesauro – Cloud Application Security Guy + OWASP

Racker since October 2011

Rackspace’s Cloud Product Group

Work with developers and QE

matt.tesauro@rackspace.com

OWASP International Foundation Board
Member and Treasurer

Project Leader of OWASP Live CD &
OWASP WTE projects

matt.tesauro@owasp.org

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

3

RACKSPACE® HOSTING
The Service Leader in Cloud Computing

4,000+
RACKERS WE SERVE
172,000+
CUSTOMERS
40% OF
THE

FORTUNE® 100
120 +
COUNTRIES

9 GLOBAL DATA
CENTERS
LEADER IN
GARTNER'S MAGIC
RAX QUADRANT FOR
MANAGED HOSTING

2008, 2010, 2011
& 2012


OUR VISION
To be recognized as one of the
World’s greatest service companies.

4

RAX CLOUD APPROACH
Open source orchestration, management & provisioning
cloud platform


6

THE FUTURE: FANATICAL SUPPORT
ANYWHERE
Rackspace Provides
The Fanatical Support

DEDICATED PUBLIC CLOUD PRIVATE CLOUD PRIVATE CLOUD PUBLIC CLOUD

RACKSPACE LOCATIONS CUSTOMER SITE PROVIDER DC

• One Control Panel across OpenStack connected clouds
• One Fanatical Support Team
• Our Cloud, Your Cloud, Partner Hosted OpenStack Cloud
• Global Reach
6

SECURING APPS IN A
DevOps WORLD

7

A quick Overview of DevOps
• The combination of traditional development activities with operations and
testing (QA/QE)

• Collaboration, communication and integration is key

• Agile development model (sprints, scrum, …)

• Release coordination and automation

"DevOps" is an emerging set of principles, methods and practices for
communication, collaboration and integration between software development
(application/software engineering) and IT operations (systems
administration/infrastructure) professionals.

8

CI, CD, CD, TDD and API
CI == Continuous Integration

CD == Continuous Deployment

CD == Continuous Delivery

TDD == Test Driven Development

API == Application Programming Interface

9

10

THE PROBLEM
• Cycle time for software is getting
shorter

• Continuous delivery is a goal

• Scanning windows are not viable

• First mover / first to market
advantage

10

THE PROBLEM – or at least more
• Traditional software development left little time to test

• DevOps, Agile and Continuous Delivery squeeze those windows
even more

• New languages and programming methods aren’t making
this better

• Growth of interpreted languages with loose typing
hurts static analysis efforts

• Few automated tools to test APIs especially
RESTful APIs

• Little time for any testing, manual testing is doomed

11

12

THE SOLUTION

• Automated software
testing
• Automated operational
infrastructure
• Automated security
testing

12

Think like a developer
Sprints break software into little pieces…
• Break your testing into little pieces
• Use your threat model to know the crucial bits to test

Long and short running tests

• Testing time drives testing frequency

• Code for tests needs to be optimized

Smoke test versus full regression test

• Smoke test early and often

• Full regression tests on regular intervals

13

Maximize what you’ve got
Make the most of your frameworks
• Embrace, understand and fill gaps where necessary

Make the best use of your time…
• Make tests easily repeatable
• Make tests easy to understand

• Make tests abstract and combine-able

• Ala carte tests for mixing and matching

• Think about the Unix pipe | and its power

14

Test Driven Development Security
Under the constraints of DevOps, Continuous Deployment

Your testing has to be nimble
Dare I say…Agile

In TDD, you know your code works
when the tests pass

In TD(S), you know your app has met
the baseline when the tests pass
15

A snail on fire!

16

17

AUTOMATING

• Declarative configuration language
• Plain-text configuration in source control
• Fully programmatic, no manual interactions

17

18

CHEF
Server / Hosted / Private

1. Solo
Racker
2. Server

3. Hosted

4. Private Hosted Node
Node
Node
Node
Node
Node
Node Node Node
Node Node Node
Node Node Node

18

19

COOKBOOKS

• Most major software
packages have cookbooks
• You will have to write your
own / customize
• Good place to spend
security cycles
- Merge patches upstream for
extra points.

19

20

GROUPING & TAGGING

Node • Tagging your
Node
Apache Node servers applies
Node
Web the required set of
recipes
Node • A base set of
Node
MySql Node
Node
recipes is
DB
common
• Each server will
Node
Node
have multiple tags
Memcache Node
Node set at bootstrap
Cache
time

Monitoring 20

21

LIMITATIONS

• Focus on single machines Cloud Load Balancer

• A multi-box configuration
is based on copying
Web Web Web Web
existing configurations
• No support for implicit
application or environment Memcached
Database as a
Service
configuration
• Applications include more
Cloud Files CDN
than just servers
• Images have security
issues
21

22

CHECKMATE

Inspector
• Verification
Contractor • Due Diligence
• Decomposition
Architect • Orchestration
• Templates
• Questions

A system to build generic application configurations
22

23

ARCHITECTURE • Components communicate
through a common queue
Architect • Each provisioning
component is independent

Checkmate Message Contractor Compute
Web Queue

Caching Storage

Message
Inspector
Queue

Load
Hadoop
Balancer

Database

23

24

base:

ARCHITECT name: wordpress large
environment-name: {tenantId}-
wordpress-large

Template
providers:
- rackspace:
- compute: &rax-cloud-servers
Generic Provider Definitions
endpoint: https://...
- loadbalancer: &rax-lbaas
Architecture Questions
endpoint: https://...
- database: &rax-dbaas
Scaling Factors endpoint: https://...
- common:
vendor: rackspace
credentials:
- token: {token}
24

25

ARCHITECT

Template • Requests per hour?
• Budget
Generic Provider Definitions • High availability
• Disaster resistant
Architecture Questions
• SSL
Scaling Factors • Backup
• CDN
…

25

26

ARCHITECT tiers:
- name: web
resource: &loadbalancer
min-occur: 1
Template type: loadbalancer
connection: public
port: [80, 443]
allow: all
Generic Provider Definitions isolation: none
resource: &webheads
min-occur: 2
Architecture Questions type: compute
os: Ubuntu 11.10
memory-min: 2Gb
memory-max: 4Gb
Scaling Factors
configs:
- wordpress-mp
attributes:
- role: web
connection: *database

26

27

28

29

30

31

32

33

34

35

37

CONTRACTOR
• Takes Architect’s plan and
builds it
• Task Decomposition
- Uses standard workflow
patterns
• Orchestration / Ordering
• Status Reporting
• Farms out tasks to sub-
Our current implementation uses an open source
contractors Python workflow engine, SpiffWorkflow.

37

38

INSPECTOR
• Takes Architect’s plan &
contractor’s output
• Focuses on checking for
code compliance
- Not perfection, bare minimums
• Can include multiple facets
- Security
- Scalability
Our current implementation includes WP Scan for
- Compliance WordPress and the Nikto vulnerability scanner.

38

39

INSPECTOR
+ Server: Apache/2.2.12 (Ubuntu)
+ No CGI Directories found (use '-C all' to force check all
possible dirs)
+ Apache/2.2.12 appears to be outdated (current is at least
Apache/2.2.17). Apache 1.3.42 (final release) and 2.0.64 are also
current.
+ ETag header found on server, inode: 12534048, size: 317, mtime:
0x4b9436dbea280
+ Allowed HTTP Methods: GET, HEAD, POST, OPTIONS
+ OSVDB-3268: /icons/: Directory indexing found.
+ OSVDB-3233: /icons/README: Apache default file found.
+ 6448 items checked: 0 error(s) and 5 item(s) reported

39

40

INSPECTOR
[!] The WordPress "http://---.com/readme.html" file exists.
[!] WordPress version 3.1 identified from meta generator.

[+] Enumerating installed plugins...Checking for 2394 total
plugins
[+] We found 2 plugins:
Name: disqus-comment-systemLocation:
Name: wordpress-popular-postsLocation:

[+] There were 1 vulnerabilities identified from the plugin
names:

[!] ["WordPress Plugin Disqus Comment System <= 2.68 Reflected
Cross-Site Scripting (XSS)"]*

40

41

FUTURE WORK

Monitor Architect
• Trending • Templates
• Thresholding • Questions

Inspector Contractor
• Verification • Decomposition
• Due Diligence • Orchestration

41

So I was talking with a friend…
He was bemoaning the pace of change and the speed at which software
was being pushed to production…
In essence, management has made the decision that
getting their app out the door with possible bugs is
more valuable to the business then having strong
assurance that the software has few or no significant
bugs.
You’ve got to up your game,
get automated, agile and
get on pace with your developers.

42

DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012

More Related Content

What's hot

Similar to DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012

More from Matt Tesauro

Recently uploaded

DevOps, CI, APIs, Oh My! - Texas Linux Fest 2012

Editor's Notes