My talk for USENIX ASE

1. Mirror, Mirror, On the Wall:
What are we teaching them all?
Characterising the Focus of Cybersecurity Curricular Frameworks
Joseph Hallett

2. There are lots of Cyber Security
Curricular Frameworks

3. A couple of
questions…

4. 1. Are they all the same?
2. What are they all teaching?
3. Which one is best?

5. Are they the same?

6. Nope.

7. What are they
all teaching?

8. Risk management and
Security ops mostly…

9. But it depends on
the framework…
We’re going to describe them,
and present a method for describing others.

10. Which one is best?

11. Nope.

12. (depends)

13. They all serve different
purposes and we’re not going
to be making judgments here.

14. But we do want to clarify
what their content is…

15. 4 Curricular
Frameworks

16. Association for
Computing Machinery
ASSOCIATION FOR
INFORMATION SYSTEMS
ifip
CYBERSECURITY
CURRICULA 2017
Curriculum Guidelines for Post-Secondary
Degree Programs in Cybersecurity
A Report in the Computing Curricula Series Joint
Task Force on Cybersecurity Education
• Association for Computing Machinery (ACM)
• IEEE Computer Society (IEEE-CS)
• Association for Information Systems Special Interest Group on
• Information Security and Privacy (AIS SIGSEC)
• International Federation for Information Processing Technical
Version 1.0 Report

17. IISP Knowledge Framework
UK-based non-profit Cybersecurity professional organisation
What knowledge is required to work in information security?
Aims “to provide a foundation for curriculum development,
course accreditation and for individual
professional certification”

18. NCSC Certified Masters
Programmes in Cybersecurity
UK framework for cybersecurity degrees
Loosely based on IISP Knowledge Framework
Multiple pathways based around a common set of topics:
A: 4 year Computer science with cybersecurity
B: 4 year Cybersecurity
C: 4 year Digital forensics
CNIS: 4 year Computer networks and internet security
Masters: 1 year broad cybersecurity Masters programme

19. NICE Cybersecurity
Workforce Framework
NIST Special Publication 800-181
Aims to describe all cybersecurity work and act as a
reference guide for people implementing education
programmes.
Big lists (~600) of Knowledge, Skills and Abilities
Each tied and cross-referenced to jobs and roles within
cybersecurity

20. Joint Task Force
Cybersecurity Curricula
Collaboration between ACM, IEEE-CS, AIS SIGSEC and IFIP WG 11.8
Aims to “leading resource of comprehensive cybersecurity
curricular content for global academic institutions seeking to
develop a broad range of cybersecurity offerings at the post-
secondary level”

21. NCSC Certifed Masters NICE Cybersecurity Workforce Framework JTF Curriculum Guidelines
IISP Knowledge Framework
Security Discipline
Skills Group
Indicative Topic
Speciality Area
Work Role
K S A
Topic
Knowledge Units
Knowledge Area Learning Outcome
Level
Skill Area
Security Discipline

22. NCSC Certifed Masters NICE Cybersecurity Workforce Framework JTF Curriculum Guidelines
IISP Knowledge Framework
Security Discipline
Skills Group
Indicative Topic
Speciality Area
Work Role
K S A
Topic
Knowledge Units
Knowledge Area Learning Outcome
Level
Skill Area
Security Discipline

23. NCSC Certifed Masters NICE Cybersecurity Workforce Framework JTF Curriculum Guidelines
IISP Knowledge Framework
CyBOK
Security Discipline
Skills Group
Indicative Topic
Speciality Area
Work Role
K S A
Topic
Knowledge Units
Knowledge Area Learning Outcome
Level
Skill Area
Security Discipline

24. NCSC Certifed Masters NICE Cybersecurity Workforce Framework JTF Curriculum Guidelines
IISP Knowledge Framework
CyBOK
Security Discipline
Skills Group
Indicative Topic
Knowledge Area
Speciality Area
Work Role
K S A
Topic
Knowledge Units
Knowledge Area Learning Outcome
Level
Skill Area
Security Discipline
Category

26. Cybersecurity
Body
of
Knowledge

28. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability

29. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability
Attacks and Defences

30. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability
Human Organisational
and Regulatory Aspects

31. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability
Infrastructure Security

32. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability
Systems Security

33. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability
Software Platform Security

34. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability

35. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability

36. Adversarial
Behaviours
Forensics
Malware and
Attack Technologies
Security
Operations
and Incident
Management
Human Factors Law and Regulation
Privacy and
Online Rights
Risk Management
and Governance
Network Security Hardware Security
Cyber-Physical
Systems Security
Physical Layer
Security
Software
Security
Web and
Mobile
Security
Secure
Software
Lifecycle
Cryptography
Operating Systems
and Virtualisation
Security
Distributed
Systems
Security
Authentication,
Authorisation
and Accountability

37. Map the topics from the
curricular frameworks onto
CyBOK Knowledge Areas

38. Map the topics from the
curricular frameworks onto
CyBOK Knowledge Areas
Scope Document

39. —a learning outcome for the IISP Knowledge Framework Skill Area A6.1
“They shall be able to list the major applicable
legislation and regulations affecting an example
organization and describe their overall purpose.”

40. —a learning outcome for the IISP Knowledge Framework Skill Area A6.1
“They shall be able to list the major applicable
legislation and regulations affecting an example
organization and describe their overall purpose.”
—CyBOK Scope document for the Law and Regulation Knowledge Area
“International and national statutory and regulatory
requirements, compliance obligations including data
protection...”

41. —a learning outcome for the IISP Knowledge Framework Skill Area A6.1
“They shall be able to list the major applicable
legislation and regulations affecting an example
organization and describe their overall purpose.”

42. “They shall be able to list the major applicable
legislation and regulations affecting an example
organization and describe their overall purpose.”
—a learning outcome for the IISP Knowledge Framework Skill Area A6.1
Law and
Regulation

43. Curricular
Framework
Mapped /
Total
Mapped
Percentage
IISP 215/252 85%
JTF 286/287 100%
NICE 206/630 33%
NCSC 114/118 97%

44. Curricular
Framework
Mapped /
Total
Mapped
Percentage
IISP 215/252 85%
JTF 286/287 100%
NICE 206/630 33%
NCSC 114/118 97%

45. —NICE K0015
“Knowledge of computer algorithms.”

46. —NICE K0015
“Knowledge of computer algorithms.”
Too General
For CyBOK
:-(

47. Curricular
Framework
Mapped /
Total
Mapped
Percentage
IISP 215/252 85%
JTF 286/287 100%
NICE 206/630 33%
NCSC 114/118 97%

48. :-(
Not Cybersecurity
Research skills
Physical security
Intra-personal skills
General Computer Science

49. 34%
35%
13%
6%
11%
Attacks & Defences
HumanOrganisational&RegulatoryAspects
InfrastructureSecurity
Software&Platform
Security
SystemsSecurity
NICE
27%
27%
11%
19%
16%
Attacks & Defences
HumanOrganisational&RegulatoryAspects
InfrastructureSecurity
Software&Platform
Security
SystemsSecurity
NCSC
20%
33%
15%17%
15%
Attacks & Defences
HumanOrganisational&RegulatoryAspects
InfrastructureSecurity
Software&Platform
Security
SystemsSecurity
JTF
42%
34%
2%
19%
3%
Attacks & Defences
HumanOrganisational&RegulatoryAspects
InfrastructureSecurity
Software&Platform
Security
SystemsSecurity
IISP

50. IISP
18 (8%)
11 (5%)
10 (5%)
50 (23%)
6 (3%)
11 (5%)
1 (0%)
54 (25%)
1 (0%)
1 (0%)
2 (1%)
1 (0%)
36 (17%)
4 (2%)
1 (0%)
4 (2%)
1 (0%)
1 (0%)
1 (0%)
Adversarial Behaviours
Forensics
Malware & Attack Technology
Security Operations & Incident Management
Human Factors
Law & Regulation
Privacy & Online Rights
Risk Management & Governance
Cyber-Physical Systems Security
Hardware Security
Network Security
Physical Layer Security
Secure Software Design & Development
Software Security
Web & Mobile Security
Authentication, Authorisation & Accountability
Cryptography
Distributed Systems Security
Operating Systems & Virtualisation Security

51. 87 (9%)
20 (2%)
87 (9%)
146 (14%)
1 (0%)
86 (9%)
113 (11%)
158 (16%)
2 (0%)
130 (13%)
3 (0%)
22 (2%)
41 (4%)
2 (0%)
41 (4%)
40 (4%)
10 (1%)
21 (2%)
Adversarial Behaviours
Forensics
Malware & Attack Technology
Security Operations & Incident Management
Human Factors
Law & Regulation
Privacy & Online Rights
Risk Management & Governance
Cyber-Physical Systems Security
Hardware Security
Network Security
Physical Layer Security
Secure Software Design & Development
Software Security
Web & Mobile Security
Authentication, Authorisation & Accountability
Cryptography
Distributed Systems Security
Operating Systems & Virtualisation Security
NICE

52. 5 (2%)
10 (3%)
12 (4%)
29 (10%)
31 (11%)
23 (8%)
13 (5%)
26 (9%)
6 (2%)
10 (3%)
25 (9%)
2 (1%)
30 (10%)
18 (6%)
2 (1%)
15 (5%)
14 (5%)
5 (2%)
10 (3%)
Adversarial Behaviours
Forensics
Malware & Attack Technology
Security Operations & Incident Management
Human Factors
Law & Regulation
Privacy & Online Rights
Risk Management & Governance
Cyber-Physical Systems Security
Hardware Security
Network Security
Physical Layer Security
Secure Software Design & Development
Software Security
Web & Mobile Security
Authentication, Authorisation & Accountability
Cryptography
Distributed Systems Security
Operating Systems & Virtualisation Security
JTF

53. 2 (2%)
6 (5%)
9 (8%)
14 (12%)
9 (8%)
6 (5%)
4 (4%)
12 (11%)
4 (4%)
1 (1%)
7 (6%)
8 (7%)
4 (4%)
10 (9%)
7 (6%)
7 (6%)
1 (1%)
3 (3%)
Adversarial Behaviours
Forensics
Malware & Attack Technology
Security Operations & Incident Management
Human Factors
Law & Regulation
Privacy & Online Rights
Risk Management & Governance
Cyber-Physical Systems Security
Hardware Security
Network Security
Physical Layer Security
Secure Software Design & Development
Software Security
Web & Mobile Security
Authentication, Authorisation & Accountability
Cryptography
Distributed Systems Security
Operating Systems & Virtualisation Security
NCSC

54. Median

55. Median

56. Median

57. This doesn’t account
for time spent on
any topic…
This doesn’t look
at how knowledge
in these topics is
evaluated…

58. 1. Are they all the same?
2. What are they all teaching?
3. Which one is best?

59. Are they the same?

60. Nope.

61. What are they
all teaching?

62. Risk management and
Security ops mostly…

63. But it depends on
the framework…
Using CyBOK we can characterise what is in them

64. Which one is best?

65. It depends what you
want out of it…

66. But using CyBOK we can
see what the emphasis is

67. …

68. Any
Questions?

69. 24%
26%
9%
17%
24%
Attacks & Defences
HumanOrganisational&RegulatoryAspectsInfrastructureSecurity
Software&Platform
Security
SystemsSecurity
CISSP
3(1%)
5(2%)
2(1%)
41(19%)
9(4%)
9(4%)
10(5%)
29(13%)
2(1%)
14(6%)
3(1%)
22(10%)
4(2%)
11(5%)
20(9%)
16(7%)
6(3%)
11(5%)
AdversarialBehaviours
Forensics
Malware&AttackTechnology
SecurityOperations&IncidentManagement
HumanFactors
Law&Regulation
Privacy&OnlineRights
RiskManagement&Governance
Cyber-PhysicalSystemsSecurity
HardwareSecurity
NetworkSecurity
PhysicalLayerSecurity
SecureSoftwareDesign&Development
SoftwareSecurity
Web&MobileSecurity
Authentication,Authorisation&Accountability
Cryptography
DistributedSystemsSecurity
OperatingSystems&VirtualisationSecurity