Talk given at AppSec USA 2012. See the video here > https://vimeo.com/54250714

1. GAUNTLT:
RUGGED
BY EXAMPLE
JAMES WICKETT
MANI TADAYON
JEREMIAH SHIRK
SG: JASON CHAN

2. WE WANT YOU TO BE
SUCCESSFUL AND
MAKE A DIFFERENCE

3. James Wickett
CISSP, GWAPT,
CCSK, GSEC, GCFW
@wickett
@RuggedDevOps
@gauntlt

4. A BRIEF HISTORY OF
INFORMATION SECURITY

5. WE USED TO BE COOL

6. WE HAD
CINEMA

7. WE HAD
HEROES

8. WE MADE FREE
PHONE CALLS

9. WE WERE COOL

10. WE MADE IT INTO THE
ORGANIZATIONS
WE HAD PREVIOUSLY FOUGHT

11. WE COULDN’T STOP THE
VIRUSES AND WORMS

12. INSTEAD OF ENGINEERING
INFOSEC BECAME ACTUARIES

13. WE BECAME EXPERTS
IN BUYING INSURANCE POLICIES

14. “[RISK ASSESSMENT] INTRODUCES A
DANGEROUS FALLACY: THAT
STRUCTURED INADEQUACY IS
ALMOST AS GOOD AS ADEQUACY
AND THAT UNDERFUNDED
SECURITY EFFORTS PLUS RISK
MANAGEMENT ARE ABOUT AS
GOOD AS PROPERLY FUNDED
SECURITY WORK” - MICHAL ZALEWSKI

15. SOMETHING ELSE
HAPPENED GLOBALLY

16. DEVS BECAME COOL

17. ENTER DEVOPS

20. CODE BECAME SOCIAL

21. “I DON’T WANT YOU
TO SEND ME AN
INSTALLATION DVD”

22. WE SELL TIME NOW

23. WE SELL SOCIAL
AND FRIENDSHIPS

24. “IS THIS SECURE?”
-YOUR CUSTOMER

25. “ITS CERTIFIED”- YOU

26. WHY CAN’T YOU
GIVE A BETTER ANSWER?

27. THE INEQUITABLE
DISTRIBUTION OF
LABOR IN SECURITY
MIMICS THAT IN DEV/OPS

28. 2% OF AN ENGINEERING
DEV TEAM ARE WORKING
ON SECURITY
- BSIMM 2012 data, http://bsimm.com/

29. -LEARNING FROM
(PREFERABLY OTHER
PEOPLE’S) MISTAKES
-DEVELOPING TOOLS TO
CORRECT PROBLEMS
- PLANNING TO HAVE
EVERYTHING
COMPROMISED

30. ENTER RUGGED

33. Current Software

34. Rugged Software

35. ADVERSITY REQUIRES
RUGGED SOLUTIONS

36. ADVERSITY IS REAL OR
PERCEIVED NEGATIVE
ACTIONS AND EVENTS
THAT PROHIBIT NORMAL
FUNCTION AND OPERATION.

37. Building solutions to handle
adversity will cause
unintended, positive benefits
that will provide value that
would have been unrealized
otherwise.
RUGGEDIZATION
THEORY

38. NO PAIN, NO GAIN

39. "Secondly, our network
got a lot stronger as a
result of the LulzSec
attacks."
-Surviving Lulz: Behind the Scenes of LulzSec @SXSW 2012
by CloudFlare team

41. RUGGED BY DESIGN,
DEVOPS BY CULTURE

42. RUGGED DEVOPS

43. REPEATABLE – NO MANUAL STEPS, CI
RELIABLE - NO DOS HERE
REVIEWABLE – AKA AUDIT, INFRA AS CODE
RAPID – FAST TO BUILD, DEPLOY, RESTORE
RESILIENT – AUTOMATED RECONFIGURATION
REDUCED - LIMITED ATTACK SURFACE

44. ENTER GAUNTLT

45. Put your code through
the GAUNTLT

46. GAUNTLET, N.
AN ATTACK FROM
ALL SIDES

47. custom attacks dirbuster
metasploit sqlmap
fuzzers nessus
w3af nmap
Your web app You

48. gauntlt is built for doing security
testing in a DevOps world

49. GAUNTLT IS

50. AN ALWAYS-
ATTACKING
ENVIRONMENT FOR
DEVELOPERS

51. WITH ATTACKS
WRITTEN IN
EASY-TO-READ
LANGUAGE

52. ACCESSIBLE TO
EVERYONE INVOLVED
IN DEV, OPS,
TESTING, SECURITY, ...

53. WHY GAUNTLT?
SECURITY DOMAIN
KNOWLEDGE IS GENERALLY
A MYSTERY TO DEV TEAMS

54. GAUNTLT ALLOWS DEV AND
OPS AND SECURITY TO
COMMUNICATE

55. GAUNTLT JOINS
THE PHILOSOPHY OF
RUGGED SOFTWARE
& CONTINUOUS INTEGRATION

57. HTTPS://GITHUB.COM/THEGAUNTLET/GAUNTLT

58. $ gem install gauntlt
# download attacks
$ gauntlt

59. install gauntlt
$ gem install gauntlt
# download example attacks from github
# customize the example attacks
# now you can run gauntlt
$ gauntlt
Examples > https://github.com/thegauntlet/gauntlt/tree/master/examples

60. LETS LOOK INSIDE A
COUPLE OF THESE FILES

61. GAUNTLT ATTACKS

62. @slow
nmap.attack
Feature: nmap attacks for example.com
Background:
Given "nmap" is installed
And the target hostname is "www.example.com"
And the target tcp_ping_ports are "22,25,80,443"
Scenario: Verify server is open on expected set of ports
using the nmap fast flag
When I launch an "nmap" attack with:
"""
nmap -F <hostname>
"""
Then the output should contain:
"""
80/tcp open https
"""
Scenario: Verify that there are no unexpected ports open
When I launch an "nmap" attack with:
"""
nmap -F <hostname>
"""
Then the output should not contain:
"""
25/tcp
"""

63. running gauntlt with failing tests
wickett$ gauntlt
@slow
Feature: nmap attacks for example.com
Background:
Given "nmap" is installed
And the target hostname is "www.example.com"
And the target tcp_ping_ports are "22,25,80,443"
Scenario: Verify server is open on expected set of ports using the
nmap fast flag
When I launch an "nmap" attack with:
"""
nmap -F www-stage.cloudsourcery.com
"""
Then the output should contain:
"""
443/tcp open https
"""
1 scenario (1 failed)
5 steps (1 failed, 4 passed)
0m18.341s

64. running gauntlt with passing tests
wickett$ gauntlt
@slow
Feature: nmap attacks for example.com
Background:
Given "nmap" is installed
And the target hostname is "www.example.com"
And the target tcp_ping_ports are "22,25,80,443"
Scenario: Verify server is open on expected set of ports using the
nmap fast flag
When I launch an "nmap" attack with:
"""
nmap -F www-stage.cloudsourcery.com
"""
Then the output should contain:
"""
443/tcp open https
"""
1 scenario (1 passed)
5 steps (5 passed)
0m18.341s

65. gauntlt:
Netflix Use Case

66. Problem Statement
• Netflix is a heavy AWS user, and we provide self-
service deployment for dev teams
• AWS’ Elastic Load Balancer (ELB) provides cross-
datacenter traffic balancing, but no security
controls (if your cluster is attached to an ELB, it is
available to the Internet)
• Engineers may misunderstand use cases for ELBs,
security features, and/or other measures that can
be used to protect ELB-fronted clusters

67. How do we ensure the
100s of clusters associated
with ELBs are configured
and protected as intended?

68. Solution: Use gauntlt to
organize and perform
ELB testing

69. gauntlt test:
What response will an ELB provide
to an arbitrary Internet node, and is
it expected?

70. Process
1. Launch gauntlt test runner instance, loaded with
“master list” of ELBs and expected state
2. Determine “target list” of current ELBs to evaluate
3. Generate per-ELB listener gauntlt attack files
4. Execute attacks
5. Alert on failures and new ELBs
6. Triage findings and update ELB master list

71. gauntlt Attack Template
• Uses gauntlt curl feature
• Sub in protocol, port, hostname, and response
code from ELB master and target list

72. GAUNTLT
A VERY SHORT INTRODUCTION

73. ABOUT MANI
• Mani Tadayon
• Senior Software Engineer, ZestFinance
• Lots of experience in web development,
ruby and test automation
• Learning Clojure

74. CONWAY’S LAW
Any organization that designs a system ... will
inevitably produce a design whose structure is a
copy of the organization's communication
structure.
Melvin E. Conway, 1968

75. BEHAVIOR-DRIVEN
DEVELOPMENT
BDD is a second-generation, outside–in, pull-
based, multiple-stakeholder, multiple-scale, high-
automation, agile methodology. It describes a
cycle of interactions with well-defined outputs,
resulting in the delivery of working, tested
software that matters.
Dan North , 2009

76. CUCUMBER

77. ATTACK FILE
• Plain text file
• Gherkin syntax:
• Given
• When
• Then

78. Feature: Run sqlmap against a target
Scenario: Identify SQL injection vulnerabilities
Given "sqlmap" is installed
And the target URL is "http://localhost?id=1"
When I launch a "sqlmap" attack with:
"""
python <sqlmap_path> -u <target_url>
"""
Then the output should contain:
"""
sqlmap identified the following injection
points
"""

80. Feature: Run sqlmap against a target
verify
Scenario: Identify SQL injection vulnerabilities
tool
Given "sqlmap" is installed
setup steps
And the target URL is "http://localhost?id=1"
When I launch a "sqlmap" attack with: set
""" config
python <sqlmap_path> -u <target_url>
"""
Then the output should contain:
"""
sqlmap identified the following injection points
"""

81. Feature: Run sqlmap against a target
Scenario: Identify SQL injection vulnerabilities
Given "sqlmap" is installed
And the target URL is "http://localhost?id=1"
When I launch a "sqlmap" attack with:
"""
attack!
python <sqlmap_path> -u <target_url>
"""
env
Then the output should contain: get
param config
"""
sqlmap identified the following injection points
"""

82. Feature: Run sqlmap against a target
Scenario: Identify SQL injection vulnerabilities
Given "sqlmap" is installed
And the target URL is "http://localhost?id=1"
When I launch a "sqlmap" attack with:
"""
python <sqlmap_path> -u <target_url>
""" haystack
Then the output should contain:
"""
assert
sqlmap identified the following injection points
"""
needle

84. ATTACK ADAPTER
• Step definition for attack file
• Support code in ruby or java
• Support shell script

85. Given /^"sqlmap" is installed$/ do
step definition ensure_python_script_installed('sqlmap') ruby
end
When /^I launch an? "sqlmap" attack with:$/ do |command|
sqlmap_path = path_to_python_script("sqlmap")
command.gsub!('<target_url>', target_url)
command.gsub!('<sqlmap_path>', sqlmap_path)
run command
end

86. Given /^"sqlmap" is installed$/ do
ensure_python_script_installed('sqlmap')
end
When /^I launch an? "sqlmap" attack with:$/ do |command|
sqlmap_path = path_to_python_script("sqlmap")
step definition command.gsub!('<target_url>', target_url)
command.gsub!('<sqlmap_path>', sqlmap_path)
run command
end
execute

87. GAUNTLT DESIGN
• Simple
• Extensible
• UNIX™ : stdin, stdout, exit status
• Minimum features yield maximum utility

88. UPCOMING
FEATURES
• More output parsers
• More attack adapters
• More goats
• Better support for JRuby & Java
• Anything you want:
https://github.com/thegauntlet/gauntlt/issues

89. Gauntlt
Using the Gauntlt Starter Kit

90. About me
• Jeremiah Shirk
• Application & Infrastructure Manager,
Kansas State University
• 18 years doing unix admin, security, and
some open source contributions
• Keeper of tiny flocks

91. KSU 55 - WVU 14

92. Gauntlt Starter Kit

93. Dependencies
VirtualBox Vagrant

94. Download
• https://www.virtualbox.org/
• http://vagrantup.com/

95. Starter Kit on GitHub
• The starter kit is on GitHub at https://
github.com/thegauntlet/gauntlt-starter-kit
• Or, download a copy from:
www.gauntlt.org/...

96. Base box
$ vagrant box add precise32 http://files.vagrantup.com/precise32.box
[vagrant] Downloading with Vagrant::Downloaders::HTTP...
[vagrant] Downloading box: http://files.vagrantup.com/precise32.box
[vagrant] Extracting box...
[vagrant] Verifying box...
[vagrant] Cleaning up downloaded box...
$

97. Start the VM
$ cd gauntlt-starter-kit/vagrant/gauntlt
$ vagrant up
[default] Importing base box 'precise32'...
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
...

99. Vagrantfile
Vagrant::Config.run do |config|
config.ssh.private_key_path = "~/.ssh/id_rsa"
config.vm.box = "precise32"
config.vm.box_url = "http://files.vagrantup.com/precise32.box"
# config.vm.network :hostonly, "33.33.33.10"
# config.vm.network :bridged
# config.vm.forward_port 80, 8080
# config.vm.share_folder "v-data", "/vagrant_data", "../data"
config.vm.provision :chef_solo do |chef|
chef.cookbooks_path = ["cookbooks", "site-cookbooks"]
chef.add_recipe "vagrant_main"
end
end

100. SSH to the VM
$ vagrant ssh

101. Secure SSH Keys
$ vagrant ssh-config | grep Port
Port 2222
$ scp -i ~/.vagrant.d/insecure_private_key -P 2222
~/.ssh/ id_rsa.pub vagrant@localhost:~/.ssh/authorized_keys

102. vagrant@precise32:~$ gauntlt attacks/nmap
Feature: simple nmap attack (sanity check)
Background:
Given "nmap" is installed
And the target hostname is "google.com"
Scenario: Verify server is available on standard web ports
When I launch an "nmap" attack with:
"""
nmap -p 80,443 google.com
"""
Then the output should contain:
"""
80/tcp open http
443/tcp open https
"""
1 scenario (1 passed)
4 steps (4 passed)
0m0.112s
vagrant@precise32:~$

103. vagrant@precise32:~$ gauntlt attacks/sslyze
Feature: Run sslyze against a target
Background: # attacks/sslyze:3
Given "sslyze" is installed # gauntlt-0.0.8/lib/gauntlt/
attack_adapters/sslyze.rb:1
And the target hostname is "google.com" # gauntlt-0.0.8/lib/gauntlt/
attack_adapters/nmap.rb:7
Scenario: Ensure no anonymous certificates # attacks/sslyze:7
When I launch an "sslyze" attack with: # gauntlt-0.0.8/lib/gauntlt/
attack_adapters/sslyze.rb:5
"""
python /home/vagrant/sslyze/sslyze.py google.com:443
"""
Then the output should not contain: # aruba-0.5.0/lib/aruba/
cucumber.rb:111
"""
Anon
"""
1 scenario (1 passed)
4 steps (4 passed)
0m0.736s
vagrant@precise32:~$

104. Try it yourself
http://gauntlt.org/

105. Office hours
Hotel bar
Tonight, 10 p.m.

106. Questions?