My Little Webapp – DevOpsSec is Magic 
Apollo Clark 
@apolloclark 
apolloclark.com 
slideshare.net/ApolloClark/my-little-webap-devopssec-is-magic
About Me 
• Originally from Maine 
• Lived in Milwaukee, Chicago, Atlanta 
• Web developers since 2001 
• PHP, Python, Java, Perl, Visual Basic 
• Kali Linux, Burpsuite, SQLMap, XSSer, etc. 
• Got badly hacked in 2010, been learning since 
• I like making good software
What if we could fix anything 
in 10 minutes?
With DevOpsSec, you can!
How does it feel?
Prepare for a meme filled ride.
How do we do things today?
We need to build QA and security in.
What can we do?
Dev vs. Ops
Dev vs. Ops 
• Devs are paid to change code, high entropy
Dev vs. Ops 
• Devs are paid to change code, high entropy 
• Ops are paid to have stability, low entropy
Dev vs. Ops 
• Devs are paid to change code, high entropy 
• Ops are paid to have stability, low entropy 
• Change != Stability
Dev vs. Ops 
• Devs are paid to change code, high entropy 
• Ops are paid to have stability, low entropy 
• Change != Stability 
• IE8 only supports loading 31 CSS files
"One line of code can break everything."
What do we do?
Climbing the Pyramid
Performance 
• stress testing: "how many concurrent users?"
Performance 
• stress testing: "how many concurrent users?" 
• server latency: "how long is the response wait?"
Performance 
• stress testing: "how many concurrent users?" 
• server latency: "how long is the response wait?" 
• initial client-side load latency: "time to first tweet"
Performance 
• stress testing: "how many concurrent users?" 
• server latency: "how long is the response wait?" 
• initial client-side load latency: "time to first tweet" 
• client latency: "how long does action take?"
Performance 
• stress testing: "how many concurrent users?" 
• server latency: "how long is the response wait?" 
• initial client-side load latency: "time to first tweet" 
• client latency: "how long does action take?"
Don’t forget to DDoS yourself.
I like to DDoS myself on the weekends.
What we got:
What we want:
Code quality testing IS security testing.
Code Quality 
• linting, correct formatting
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target
2^6 possible code pathways
64 possible outcomes from 1 function.
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target 
• unsafe calls, change implementation
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target 
• unsafe calls, change implementation 
• e2e tests, detect regressions
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target 
• unsafe calls, change implementation 
• e2e tests, detect regressions 
• unit tests, detect integration issues
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target 
• unsafe calls, change implementation 
• e2e tests, detect regressions 
• unit tests, detect integration issues 
• coverage, testing thoroughness
Code Quality 
• linting, correct formatting 
• copy + paste, easily refactor 
• complexity, refactoring target 
• unsafe calls, change implementation 
• e2e tests, detect regressions 
• unit tests, detect integration issues 
• coverage, testing thoroughness 
• mocks, speed up testing
Unit Testing
Ready to try some Unit Testing?
Unit Testing 
GET /users/<account_name> 
• happy path: "aclark" 
• missing entry: "aclark2" 
• lower bounds: "a" 
• upper bounds: "aaaaaaaaa" 
• empty: "account_name" : "" 
• null: (null) 
• fuzzing: "a2$@o9(@1"
"a2$@o9(@1" eventually becomes "a or 1=1; --"
Supported
Supported 
• define supported devices, resolutions, 
browsers, and versions
You can’t support everything:
Supported 
• define supported devices, resolutions, 
browsers, and versions 
• use Selenium WebDriver
Supported 
• define supported devices, resolutions, 
browsers, and versions 
• use Selenium WebDriver 
• test locally in VM images
Supported 
• define supported devices, resolutions, 
browsers, and versions 
• use Selenium WebDriver 
• test locally in VM images 
• test on the cloud
Supported 
• define supported devices, resolutions, 
browsers, and versions 
• use Selenium WebDriver 
• test locally in VM images 
• test on the cloud
Try using unsupported systems. Hopefully fail 
gracefully. Might even find something…
Hint: Try setting your browser User-Agent to iPhone 3.0 
when visiting news websites :P
Deployable 
• atomic base box VM
Deployable 
• atomic base box VM 
• provisioning scripts
Deployable 
• atomic base box VM 
• provisioning scripts 
• deploy to local, AWS, Rackspace, etc.
Deployable 
• atomic base box VM 
• provisioning scripts 
• deploy to local, AWS, Rackspace, etc. 
• scan dependency list
Deployable 
• atomic base box VM 
• provisioning scripts 
• deploy to local, AWS, Rackspace, etc. 
• scan dependency list 
• scan server setup
Deployable 
• atomic base box VM 
• provisioning scripts 
• deploy to local, AWS, Rackspace, etc. 
• scan dependency list 
• scan server setup
My personal websites:
Monitoring 
• request origin
If you’re a ‘Murican only company, 
why are you letting your server talk 
to Russia?
Monitoring 
• request origin 
• request scans
Monitoring 
• request origin 
• request scans 
• invalid requests
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood 
• response flood
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood 
• response flood 
• server uptime
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood 
• response flood 
• server uptime 
• latency
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood 
• response flood 
• server uptime 
• latency 
• cpu load
Monitoring 
• request origin 
• request scans 
• invalid requests 
• request flood 
• response flood 
• server uptime 
• latency 
• cpu load
My startup has < 100 users. It gets 
scanned and attacked every day.
Your live servers are getting 
hammered all the time.
Security 
• what to test?
This is your attack surface:
Security 
• what to test? 
• how to test?
Security 
• what to test? 
• how to test? 
• monitor issues
Security 
• what to test? 
• how to test? 
• monitor issues 
• aggregate reports
Security 
• what to test? 
• how to test? 
• monitor issues 
• aggregate reports 
• prioritize issues
Security 
• what to test? 
• how to test? 
• monitor issues 
• aggregate reports 
• prioritize issues 
• automate tests
Security 
• what to test? 
• how to test? 
• monitor issues 
• aggregate reports 
• prioritize issues 
• automate tests
Give and request automated tests, 
not PDF docs.
Write "Malicious User Stories"
IF YOU SEE 
SOMETHING, 
SAY SOMETHING.
... but, at least write a test.
DevOpsSec is free, you can do it today.
Automation does not replace people.
Automation is people.
Repeat after me:
"I am DevOpsSec ..."
"... and so can you!"
Apollo Clark 
@apolloclark 
apolloclark.com 
slideshare.net/ApolloClark/my-little-webap-devopssec-is-magic 
github.com/apolloclark/py-jenkins-ci
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic
My Little Webap - DevOpsSec is Magic

My Little Webap - DevOpsSec is Magic

  • 1.
    My Little Webapp– DevOpsSec is Magic Apollo Clark @apolloclark apolloclark.com slideshare.net/ApolloClark/my-little-webap-devopssec-is-magic
  • 3.
    About Me •Originally from Maine • Lived in Milwaukee, Chicago, Atlanta • Web developers since 2001 • PHP, Python, Java, Perl, Visual Basic • Kali Linux, Burpsuite, SQLMap, XSSer, etc. • Got badly hacked in 2010, been learning since • I like making good software
  • 6.
    What if wecould fix anything in 10 minutes?
  • 7.
  • 8.
  • 10.
    Prepare for ameme filled ride.
  • 13.
    How do wedo things today?
  • 17.
    We need tobuild QA and security in.
  • 18.
  • 23.
  • 25.
    Dev vs. Ops • Devs are paid to change code, high entropy
  • 26.
    Dev vs. Ops • Devs are paid to change code, high entropy • Ops are paid to have stability, low entropy
  • 27.
    Dev vs. Ops • Devs are paid to change code, high entropy • Ops are paid to have stability, low entropy • Change != Stability
  • 28.
    Dev vs. Ops • Devs are paid to change code, high entropy • Ops are paid to have stability, low entropy • Change != Stability • IE8 only supports loading 31 CSS files
  • 29.
    "One line ofcode can break everything."
  • 30.
  • 33.
  • 36.
    Performance • stresstesting: "how many concurrent users?"
  • 38.
    Performance • stresstesting: "how many concurrent users?" • server latency: "how long is the response wait?"
  • 39.
    Performance • stresstesting: "how many concurrent users?" • server latency: "how long is the response wait?" • initial client-side load latency: "time to first tweet"
  • 41.
    Performance • stresstesting: "how many concurrent users?" • server latency: "how long is the response wait?" • initial client-side load latency: "time to first tweet" • client latency: "how long does action take?"
  • 42.
    Performance • stresstesting: "how many concurrent users?" • server latency: "how long is the response wait?" • initial client-side load latency: "time to first tweet" • client latency: "how long does action take?"
  • 43.
    Don’t forget toDDoS yourself.
  • 44.
    I like toDDoS myself on the weekends.
  • 47.
  • 49.
  • 51.
    Code quality testingIS security testing.
  • 52.
    Code Quality •linting, correct formatting
  • 54.
    Code Quality •linting, correct formatting • copy + paste, easily refactor
  • 55.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target
  • 57.
  • 58.
    64 possible outcomesfrom 1 function.
  • 59.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target • unsafe calls, change implementation
  • 61.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target • unsafe calls, change implementation • e2e tests, detect regressions
  • 62.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target • unsafe calls, change implementation • e2e tests, detect regressions • unit tests, detect integration issues
  • 63.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target • unsafe calls, change implementation • e2e tests, detect regressions • unit tests, detect integration issues • coverage, testing thoroughness
  • 64.
    Code Quality •linting, correct formatting • copy + paste, easily refactor • complexity, refactoring target • unsafe calls, change implementation • e2e tests, detect regressions • unit tests, detect integration issues • coverage, testing thoroughness • mocks, speed up testing
  • 66.
  • 68.
    Ready to trysome Unit Testing?
  • 70.
    Unit Testing GET/users/<account_name> • happy path: "aclark" • missing entry: "aclark2" • lower bounds: "a" • upper bounds: "aaaaaaaaa" • empty: "account_name" : "" • null: (null) • fuzzing: "a2$@o9(@1"
  • 72.
  • 77.
  • 78.
    Supported • definesupported devices, resolutions, browsers, and versions
  • 79.
  • 81.
    Supported • definesupported devices, resolutions, browsers, and versions • use Selenium WebDriver
  • 82.
    Supported • definesupported devices, resolutions, browsers, and versions • use Selenium WebDriver • test locally in VM images
  • 84.
    Supported • definesupported devices, resolutions, browsers, and versions • use Selenium WebDriver • test locally in VM images • test on the cloud
  • 85.
    Supported • definesupported devices, resolutions, browsers, and versions • use Selenium WebDriver • test locally in VM images • test on the cloud
  • 86.
    Try using unsupportedsystems. Hopefully fail gracefully. Might even find something…
  • 87.
    Hint: Try settingyour browser User-Agent to iPhone 3.0 when visiting news websites :P
  • 91.
  • 93.
    Deployable • atomicbase box VM • provisioning scripts
  • 95.
    Deployable • atomicbase box VM • provisioning scripts • deploy to local, AWS, Rackspace, etc.
  • 96.
    Deployable • atomicbase box VM • provisioning scripts • deploy to local, AWS, Rackspace, etc. • scan dependency list
  • 98.
    Deployable • atomicbase box VM • provisioning scripts • deploy to local, AWS, Rackspace, etc. • scan dependency list • scan server setup
  • 100.
    Deployable • atomicbase box VM • provisioning scripts • deploy to local, AWS, Rackspace, etc. • scan dependency list • scan server setup
  • 104.
  • 107.
  • 108.
    If you’re a‘Murican only company, why are you letting your server talk to Russia?
  • 109.
    Monitoring • requestorigin • request scans
  • 112.
    Monitoring • requestorigin • request scans • invalid requests
  • 114.
    Monitoring • requestorigin • request scans • invalid requests • request flood
  • 116.
    Monitoring • requestorigin • request scans • invalid requests • request flood • response flood
  • 118.
    Monitoring • requestorigin • request scans • invalid requests • request flood • response flood • server uptime
  • 119.
    Monitoring • requestorigin • request scans • invalid requests • request flood • response flood • server uptime • latency
  • 120.
    Monitoring • requestorigin • request scans • invalid requests • request flood • response flood • server uptime • latency • cpu load
  • 121.
    Monitoring • requestorigin • request scans • invalid requests • request flood • response flood • server uptime • latency • cpu load
  • 122.
    My startup has< 100 users. It gets scanned and attacked every day.
  • 123.
    Your live serversare getting hammered all the time.
  • 127.
  • 128.
    This is yourattack surface:
  • 132.
    Security • whatto test? • how to test?
  • 134.
    Security • whatto test? • how to test? • monitor issues
  • 136.
    Security • whatto test? • how to test? • monitor issues • aggregate reports
  • 137.
    Security • whatto test? • how to test? • monitor issues • aggregate reports • prioritize issues
  • 138.
    Security • whatto test? • how to test? • monitor issues • aggregate reports • prioritize issues • automate tests
  • 139.
    Security • whatto test? • how to test? • monitor issues • aggregate reports • prioritize issues • automate tests
  • 140.
    Give and requestautomated tests, not PDF docs.
  • 141.
  • 143.
    IF YOU SEE SOMETHING, SAY SOMETHING.
  • 144.
    ... but, atleast write a test.
  • 146.
    DevOpsSec is free,you can do it today.
  • 147.
    Automation does notreplace people.
  • 149.
  • 151.
  • 152.
  • 153.
    "... and socan you!"
  • 155.
    Apollo Clark @apolloclark apolloclark.com slideshare.net/ApolloClark/my-little-webap-devopssec-is-magic github.com/apolloclark/py-jenkins-ci