Successfully reported this slideshow.

Shell We Play A Game? CTF-as-a-Service for Security Education

1

Share

1 of 65
1 of 65

Shell We Play A Game? CTF-as-a-Service for Security Education

1

Share

Download to read offline

Although we are facing a shortage of cybersecurity professionals, the shortage can be reduced by using technology to empower all security educators to efficiently and effectively educate the professionals of tomorrow. One powerful tool in some educators' toolboxes are Capture the Flag (CTF) competitions. Although participants in all the different types of CTF competitions learn and grow their security skills, Attack/Defense CTF competitions offer a more engaging and interactive environment where participants learn both offensive and defensive skills, and, as a result, they develop their skills even faster. However, the substantial time and skills required to host a CTF, especially an Attack/Defense CTF, is a huge barrier for anyone wanting to organize one. Therefore, we created an on-demand Attack/Defense tool via an easy-to-use website that makes the creation of an Attack/Defense CTF as simple as clicking a few buttons. In this paper, we describe the design and implementation of our system, along with lessons learned from using the system to host a 24-hour 317 team Attack/Defense CTF.

Although we are facing a shortage of cybersecurity professionals, the shortage can be reduced by using technology to empower all security educators to efficiently and effectively educate the professionals of tomorrow. One powerful tool in some educators' toolboxes are Capture the Flag (CTF) competitions. Although participants in all the different types of CTF competitions learn and grow their security skills, Attack/Defense CTF competitions offer a more engaging and interactive environment where participants learn both offensive and defensive skills, and, as a result, they develop their skills even faster. However, the substantial time and skills required to host a CTF, especially an Attack/Defense CTF, is a huge barrier for anyone wanting to organize one. Therefore, we created an on-demand Attack/Defense tool via an easy-to-use website that makes the creation of an Attack/Defense CTF as simple as clicking a few buttons. In this paper, we describe the design and implementation of our system, along with lessons learned from using the system to host a 24-hour 317 team Attack/Defense CTF.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Shell We Play A Game? CTF-as-a-Service for Security Education

  1. 1. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing (SEFCOM) ● URL : sefcom.asu.edu ● BYENG 486 ASU Shell We Play A Game? CTF-as-a-service for Security Education Erik Trickel, Francesco Disperati, Eric Gustafson, Faezeh Kalantari, Mike Mabey, Naveen Tiwari, Yeganeh Safaei, Adam Doupé, and Giovanni Vigna
  2. 2. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 2
  3. 3. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 3
  4. 4. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 4
  5. 5. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 5 Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million
  6. 6. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 6 Current Cybersecurity Workforce 1.5 Million
  7. 7. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 7
  8. 8. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 8 Cost of Cybercrime Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million GlobalCostofCybercrime Years
  9. 9. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 9 Cybersecurity Workforce Needed by 2019 1.5 Million
  10. 10. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 10 Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million
  11. 11. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 11 Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million
  12. 12. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 12 Security Professionals Open Security Positions 1.5 Million By 2019
  13. 13. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 13
  14. 14. ARIZONA STATE UNIVERSITY Becoming a Security Samurai  Networking  Operating Systems (Linux, Windows, etc.)  C & Assembly  Vulnerability & Exploitation Patterns  Defensive Theories  Security Tools  … The Laboratory of Security Engineering for Future Computing Slide 14
  15. 15. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 15
  16. 16. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 16
  17. 17. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 17 Theory Practice Execution
  18. 18. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 19 Theory Practice Execution
  19. 19. ARIZONA STATE UNIVERSITY Capture the Flag (CTF) Competitions The Laboratory of Security Engineering for Future Computing Slide 20
  20. 20. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 21 {dev}
  21. 21. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 22
  22. 22. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 24 Theory Practice Execution
  23. 23. ARIZONA STATE UNIVERSITY Benefits of Capture the Flag Competitions  Hands on experience  Active learning  Small groups  Creates strong intrinsic motivation – Practice and research – Post competition analysis The Laboratory of Security Engineering for Future Computing Slide 25
  24. 24. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 26
  25. 25. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 27
  26. 26. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 28 Jeopardy Server Team1: 1600 Team2: 1100 Team 1 Team 2 Binary L33tness 300 Binary L33tness $100 $200 $300 $400 $500 Team1: 1900 Team2: 1100 Binary L33tness $100 $200 $300 $400 $500
  27. 27. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 29
  28. 28. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 30 Team 1 Service A Service B Service C Team 2 Service A Service B Service C Gamebot Scoring Team1: 10 Team2: 25 Team1: 10 Team2: 30 Service B
  29. 29. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 31 Team 1 Service A Service B Service C Team 2 Service A Service B Service C Gamebot Scoring Team1: 10 Team2: 30 Team1: 10 Team2: 35
  30. 30. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 32 Team 1 Service A Service B Service C Team 2 Service A Service B Service C Gamebot Scoring Team1: 10 Team2: 35
  31. 31. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 33 Team 1 Service A Service B Service C Team 2 Service A Service B Service C Gamebot Scoring Team1: 10 Team2: 30 Team1: 00 Team2: 30 Team1: 10 Team2: 35
  32. 32. ARIZONA STATE UNIVERSITY Create Your Own CTF  Accessibility – Adjust difficulty – Tailor to content of class – Control access – Less intimidating  Practice – Build/Test tools for competition The Laboratory of Security Engineering for Future Computing Slide 34
  33. 33. ARIZONA STATE UNIVERSITY Creating an Attack Defense CTF  Base Skills  Server Configuration and Setup  Create Vulnerable Services  Scoring & Tracking Application  Secure Everything The Laboratory of Security Engineering for Future Computing Slide 35 {dev}
  34. 34. ARIZONA STATE UNIVERSITY Creating an Attack Defense CTF  2014 UCSB Released iCTF Framework  2015 UCSB Created Pre-configured VMs The Laboratory of Security Engineering for Future Computing Slide 36
  35. 35. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 37 ARIZONA STATE UNIVERSITY
  36. 36. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 38 https://ShellWePlayAGame.org Current Cybersecurity Workforce
  37. 37. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 39 Current Cybersecurity Workforce
  38. 38. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 40 Theory Practice Execution
  39. 39. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 41
  40. 40. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 42 AWS On-Demand CTF 1 On-Demand CTF 2 On-Demand CTF 3 Games Controller James's Halliday’s AWS Acct Vigna’s AWS Acct Your-name-here AWS Acct
  41. 41. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 43 Current Cybersecurity Workforce
  42. 42. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 44 Current Cybersecurity Workforce
  43. 43. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 45 Current Cybersecurity Workforce
  44. 44. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 46 Current Cybersecurity Workforce
  45. 45. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 47 Current Cybersecurity Workforce
  46. 46. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 48 Current Cybersecurity Workforce
  47. 47. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 49 Current Cybersecurity Workforce
  48. 48. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 50 Current Cybersecurity Workforce
  49. 49. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 51 Current Cybersecurity Workforce
  50. 50. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 52 Current Cybersecurity Workforce
  51. 51. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 53 Current Cybersecurity Workforce
  52. 52. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 54 Current Cybersecurity Workforce
  53. 53. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 55 Current Cybersecurity Workforce
  54. 54. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 56 Current Cybersecurity Workforce
  55. 55. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 57 Current Cybersecurity Workforce
  56. 56. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 58 Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million
  57. 57. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 59 Current Cybersecurity Workforce Cybersecurity Workforce Needed by 2019 1.5 Million iCTF
  58. 58. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 60 Current Cybersecurity Workforce 1.5 Million March 2017 iCTF • 24 Hours • 317 Teams • 12 Services
  59. 59. ARIZONA STATE UNIVERSITY Incidence Report  18 Hours with few issues – Infrastructure handled load – Team VMs responsive – Service checking ran smoothly  Switchover – 650 VMs running concurrently  4 AM – DDos • Ouch The Laboratory of Security Engineering for Future Computing Slide 61
  60. 60. ARIZONA STATE UNIVERSITY Cost  Only pay for AWS costs – 6 Hour Game with 20 teams costs < $25  ShellWePlayAGame.org is free The Laboratory of Security Engineering for Future Computing Slide 62
  61. 61. ARIZONA STATE UNIVERSITY TODO:  Increase robustness of VM tests and automated restart  Custom services  Expand to more cloud platforms  Open source the framework The Laboratory of Security Engineering for Future Computing Slide 63
  62. 62. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 64 https://ShellWePlayAGame.org Current Cybersecurity Workforce
  63. 63. ARIZONA STATE UNIVERSITY The Laboratory of Security Engineering for Future Computing Slide 65 Shell We Play A Game? CTF-as-a-service for Security Education https://ShellWePlayAGame.org Erik Trickel Arizona State University Erik.Trickel@asu.edu @ErikTrickel https://www.trickel.com
  64. 64. ARIZONA STATE UNIVERSITY Game Overview The Laboratory of Security Engineering for Future Computing Slide 66 War Range Subnet Game Components Subnet Game Master Database Score Board Game Bot Team Interface RouterTeam 1 Scriptbot Team 2
  65. 65. ARIZONA STATE UNIVERSITY External F/W External F/W External F/W Team’s Network The Laboratory of Security Engineering for Future Computing Slide 67 Scriptbot Team 1 Team 2 Team 3 SSH Port 1338 SSH Port 22 Port 20000 Port 20000 Router Port20000 OriginTeam3

Editor's Notes

  • Thank you for the intro Mark,
    Good morning everyone, very happy to be here and excited for the workshop
    I’m Erik Trickel, I’m a PhD Student at Arizona State University,
    I’m here to talk about a very exciting CTF-as-a-service that we here at ASU created with my adviser Adam Doupe and Giovanni Vigna’s group at UCSB.
    Our tool makes it easy for anyone to run their own attack/defense CTF
    Set this up a bit, in the Internet stone age:
  • Ram was quite a bit larger
  • Network comm’n was quite a bit slower
  • Not only was UCSB one of the first nodes, but they were also the first to connect up Xbox 360
    Ok, not an Xbox, but is the predecessor
  • Originally, designers and developers were more focused on creating connections and developing basic applications
  • Security researchers, were the pioneers of the electron and the switch, exploring systems and trying to understand and boldly go where no electron has gone before.
  • The internet has become highly commercialized with trillions of dollars flowing over it daily and billions of nodes
    Making it a much more attractive target for criminal activities

  • The global cost of cyber crime was nearly 500 billion last year
  • Estimated to reach 2 trillion by 2019

  • The beautiful world of the electron and switch has transcended into a battlefield where organized crime and nation states all battle.
  • Constant threat
  • In 2019, 1.5m gap between the number of open cybersecurity positions and qualified cybersecurity professionals
  • It’s not that we need just more, we need more that are highly skilled
  • Objective problems solving skills
    Attacker & defender
  • Unless
  • Just like if you want to be good a football, or anything, you must have both
  • The highly skilled security professionals must go deeper than just lectures
  • Theory often comes from lectures
    Practice from HW

    But, how to get the hands on experience?
  • Fun and safe environment for participants to compete and practice their skills and deepen their understanding


  • Fun and safe environment for participants to compete and practice their skills and deepen their understanding

    Teams work to solve computer security puzzles, allowing them to uncover a hidden flag



  • Once the problem is solved, the flag is left and is evidence that you solved it.

    The problems range from crypto, binary exploitation, network detection, and programming puzzles

    We call them CTF’s but they are really security exercises testing and developing those skills necessary to become a security samurai

  • CTFs incorporate creative thinking, problem solving, OS, network, development, and security theory


  • Hands on experience with realistic scenarios
  • After these competitions, there’s often many blog posts about the problems
    improving the blogger’s learning while also contributing to the community
  • Talk about different areas and points

  • Find vulns
    Craft exploits

    Central Server
    No direct interaction
    No defending
  • Each team get’s their own server to defend and launch attacks from
  • Every so often, new flags get sent out to each of the teams
    Each team looks at their own services (instead of pulling from a central server), craft an exploit, run against opponent’s machines

    Similar services on each VM

  • Not only do you have to steal flags like in jeopardy but you have to automate exploitation and patch your own services

  • Disable, even though most secure, not the point

    SIMILAR & DIFFERENT
    Similar to the jeopardy style with additional moving parts

    Not perfect, b/c somewhat limited in types of problems
  • There’s a CTF every weekend, WHY?
  • I’m sure some of you out there have thought about creating a CTF but haven’t
    Even if you have the skills
  • Completely open sourced our framework for hosting ctfs (no body used it)
    Released a pre-configured setup and maybe 200 downloads over roughly a year

    WHY? This stuff is complicated!

    Even Adam,
    Hopefully, it’s ok to use you as an example,
    had issues while creating an 18 team game for a class, and he helped design and develop the platform

    That doesn’t even include the time it took to create the vulnerable services

  • ASU and UCSB partnered to create
  • As simple as pressing a button

  • One great way to give students hands on experience with security theories is to have them participate in capture the flag competitions

    Exercise those skills in a realistic scenario
  • When a game is created on SWPAG, the VMs are hosted on AWS

    While currently require AWS we plan to extend it to other platforms in the future.
  • Help community & security professionals of tomorrow we created an easy-to-use A/D that requires limited knowledge and skills
  • Add your own teams or incorporate teams adding by others
  • We will expand the number of vulnerable services in the future
  • After each VM is spun up, it’s tested
  • What happens if a component fails
    E.g., what if a team breaks their box and cannot fix it?
  • What happens if a component fails
    E.g., what if a team breaks their box and cannot fix it?
  • Does it work?
  • Does it work?
  • First time that open to all teams
    First time 24 hours
    First time been a DEFCON qualifying event

  • Due to a technical glitch with one of the components, we needed a fresh restart of all the servers, so for a period of time we had 2 games running concurrently on AWS

    Ruined, super successful, use of this tool
  • A/D CTFs are a fun way for participants to improve their security skills and now its easy and inexpensive to setup yourself!


  • I’m ET
  • Game Master – oversees game creation and comm’n with GC
    Database – central component of game’s operation
    Gamebot – moves the game forward and calculates the score
    Team Interface – Team’s interact with system
    Scoreboard – View scores
    Router – Traffic b/t teams and game components
    Teams -
    Scriptbot – tests services on team VMs and updates flags
  • ×