The document discusses research into scaling software security education online to help retrain professionals in the workforce. It describes flipping a university class into online video lectures and using the videos to create an online course. The research questions examine why software engineers enrolled, how their performance compared to university students, and feedback on the online format. Challenges discussed include technical issues, varying participant backgrounds, and underestimating time commitments. Improving future courses involves addressing these challenges, iterating based on lessons learned, and exploring additional online platforms.

1. Software Security Education At Scale
Chris Theisen, Laurie Williams, Emerson Murphy-Hill, Kevin Oliver
{crtheise, lawilli3, emurph3, kevin_oliver}@ncsu.edu
North Carolina State University
National Science Foundation Grant Number 4900-1318428.

2. Introduction
• Cisco 2014 Annual Security Report: Worldwide shortage
of 1 million security professionals
• Educating students is no longer enough!
• How do we help retrain people who are currently in the
workforce?
2Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

3. Solution: Online Coursework
3Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

4. “Flipping” A University Class
• Students watch video lectures, listen to Silver Bullet
Podcast before the class takes place, take a quiz
• Class time devoted to exercises, discussion, etc.
• Videos can then be reused for online course
4Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

5. 5

6. 6

7. Research Questions
• RQ1: Why did software engineers sign up for the online
course?
• RQ2: How do software engineers in the online course
perform on quiz and test questions relative to university
students being taught in an on-campus setting?
• RQ3: How well does the online course format work for
software engineering professionals? What could be
improved on for future courses?
7Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

8. Research Questions
• RQ1: Why did software engineers sign up for the online
course?
• RQ2: How do software engineers in the online course
perform on quiz and test questions relative to university
students being taught in an on-campus setting?
• RQ3: How well does the online course format work for
software engineering professionals? What could be
improved on for future courses?
8Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

9. Time Commitment
9Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
• For students: even asking 2-3 hours a week is a lot.
• Specific assignment deadlines should be relaxed.
• For instructors: Take your first guess, double it.
• Video editing, message boards, technical problems,
email, language barriers, etc…

10. Technical Issues
10Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
• Issues with Course Builder
– Quizzes stopped working night before we launched
– Slow response times of the site itself
– Fixed in latest version
• Peer review project had to be scrapped
– Should have required it be complete before course
launched

11. Consider Your Audience
11Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
• Wider spread of participants means…
– Can’t assume background knowledge
• Participants included:
– Administrative assistant working with sec.
professionals
– High school teacher teaching a CS class with minimal
background

12. Discussion Video
12Introduction | Methodology | Lessons Learned | Next Steps | Conclusion

13. Iterate and Improve
13Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
• Rerun the online course (tentatively this fall)
• OpenEDx, new Google Course Builder…?
• Better idea of what works/what doesn’t for videos
• Professionally shot videos for lectures/discussion

14. 14

15. Conclusion
15Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
@crtheisen
crtheise@ncsu.edu
theisencr.github.io