Cyber Technologist Webinar - What assessment looks like

1. Cyber Technologist – Level 4
What Assessment Looks Like
Alison Pearce
Senior Quality Manager
alison.pearce@bcs.uk

2. Webinars
2
bcs.org
 Continuing across new standards
 Historical on Sharepoint ITST 8 February 2022
 Historical Digital Support Technician 15 February 2022
 AI Data Specialist 1 March 2022
 Data Technician 29 March 2022
 Data Analyst 19 April 2022
 Business Analyst 28 April 2022
 Digital Community Manager 5 May 2022
 Work in progress
 Stored on sharepoint
 Let us know additional support needs

3. Agenda
3
bcs.org
Process Reminders
The Assessment Plan – Read It!
DevOps Engineer Assessment Plan
Assessment Methods (AMs)
Delivery
Hints and Tips
The apprentice experience
Questions
Clarification required

4. Reminders
4
bcs.org
 Registrations – ESFA changes and audits
 Review Duties of standard with employer PRIOR to enrolment
 NEVER assume the next level is suitable progression
 Reasonable adjustments MUST BE submitted in advance
 BCS Apprenticeships - Access Arrangements Reasonable Adjustments and Special Considerations Policy V9.6.pdf - All Documents
(sharepoint.com)
 Ensure you are registered on e-Pro to deliver tests
 Email EPA team when Gateway submitted
 Allow time for Gateway and the EPA window
 Do NOT book Test within or close to the Gateway 2 weeks in case the application is rejected or delayed
 BCS assess against the criteria in the standard
 Sharepoint site BCS Apprenticeships - DevOps Engineer - All Documents (sharepoint.com)
 Contact the Channel Partner Quality Team cpqt@bcs.uk for access
 Put alerts on sharepoint folders for updates etc
 EPA window can not be stopped without medical evidence. Please send to BCS as soon as possible.
 Ask the apprentice NOT to book annual leave during the EPA window.

5. Assessment Methods for this Standard
5
bcs.org
 Assessment method 1: Professional
discussion underpinned by portfolio
 Fail, Pass or Distinction
 Assessment method 2: Scenario
demonstrations with questioning
 Fail, Pass or Distinction
 Assessment method 3: Project report
 Fail, Pass or Distinction
 Assessment Method 4: Knowledge
Test
 Fail or Pass
 EPA Window – 4 months
 NO CAP on grade
 a re-sit is typically taken within
two months
 a re-take is dependent on how
much re-training is required and
is typically taken within four
months

6. 6
bcs.org
Assessment Method 2
Assessment Method 1
Assessment Method 3
Assessment Method 4

7. AM1 – Professional discussion underpinned by
portfolio
7
bcs.org
 portfolio submitted at Gateway
 digital format containing evidence and mapping related to the KSBs
 should be presented under the following section headings:
 Section 1: Cyber security concepts and its importance to business and society (K3)
 Section 2: Rationale for security objectives (S6)
 Section 3: Ethical principles, codes of practice, law & regulation (K8, K9)
 Section 4: Preventing security breaches & continuous improvement (S9, S15)
 Section 5: Following organisations policies & processes (K6, S7)
 Section 6: Operation of security management systems & incident response (K7, K15)
 evidence should be presented holistically

8. AM1 – Portfolio (cont)
8
bcs.org
 evidence sources may include:
 workplace documentation/records, for example workplace policies/procedures, records
 witness statements
 annotated photographs
 video clips (maximum total duration 5 minutes); the apprentice must always be in view
and identifiable
 other evidence sources are allowed
 the portfolio should not include any methods of self-assessment
 any employer contributions should focus on direct observation of performance (for
example witness statements) rather than opinions
 the evidence provided should be valid and attributable to the apprentice
 the portfolio of evidence must contain a statement from the employer and apprentice
confirming this

9. AM1 Hints and Tips – Portfolio
9
bcs.org
 Use the STAR technique to present holistic narrative of activities
 One work based activity may meet multiple KSBs
 Do not evidence by KSB as it restricts the narrative
 Start early on programme creating the narrative and evidence
 Use “I” not “we”
 Challenge the apprentice on the WHAT, HOW and WITH WHOM and am
importantly the WHY for each project
 By month 6 the evidence should start to be robust and detailed
 Work-based application NOT knowledge
 Top Tips available

10. AM1 – The Professional Discussion
10
bcs.org
 The professional discussion will last for 90 minutes plus 10% to allow the
apprentice to complete their last answer
 a minimum of 12 open questions that include at least 2 questions focused on ‘law
& regulation’ (K8) and 1 question on ‘ethics’ (K9)
 follow up questions are permitted where clarity is required
 a one-to-one discussion
 carried out remotely
 apprentice has the opportunity to evidence all the KSB’s for the assessment
method
 portfolio supports the professional discussion but is not marked
 apprentice can draw on the contents of the portfolio to underpin the discussion,
selecting items to inform and enhance their answers

11. Hints and Tips
11
bcs.org
 review the submitted evidence for reference
 provide the assessor with as much wide detail (and relevant) detail
 take notes with them in addition to the project and portfolio
 their time to shine – build confidence
 if lacking confidence underpin in the portfolio to give the assessor a lead in
 practice discussing and knowing their material several times in advance
 make notes of new or additional activities since gateway and be prepared
to discuss

12. The apprentice experience
12
bcs.org
 The apprentice will
 be in a quiet room on their own
 have a computer with web cam and good internet connection
 logon with a link sent to them by BCS
 be asked for their original government approved ID
 be asked if they are ready to proceed
 be given an explanation of what will happen
 be prepared to discuss all aspects of their work
 be confident in sharing and explaining
 The assessor:
 MAY interrupt to refocus on areas they need additional information
 is not looking to catch them out
 wants them to Pass and meet the criteria
 works in the sector

13. AM2 – Scenario Demonstrations
13
bcs.org
 Apprentices must complete 4 scenario demonstrations
 the scenarios will be simulated and provided remotely online by BCS
 the assessment is invigilated
 the products of each scenario will be submitted to the assessor and these will be assessed.
 the scenario outputs will be supplemented by questioning.
 the scenario demonstrations as well the questioning component must be completed within
10 days, starting from when the apprentice undertakes their first scenario demonstration.
 the total time permitted for the scenario demonstrations is 7 hours 45 minutes typically
(working day = 7.5 hours long)
 over a minimum of 2 consecutive working days.
 in advance of the scenario demonstrations BCS provide the apprentice and employer with a
guidance document, with information on the format of the test, including timescales.
 can be completed in any order
 NO information on the context of the individual scenarios will be included within the
guidance document
 ALL information must be uploaded at the end of each day to ACE360

14. AM2 – Scenario Demonstrations
14
bcs.org
 On the day apprentices must be provided with clear instructions on the tasks
 the simulated environment, background material, and guidance provided by BCS
 the scenario demonstrations will each take the allotted amount of time as specified below:
 Attack and Threat Research 1 hour 45 minutes
 Risk Assessment 2 hours
 Set up and configure a system with security features 3 hours
 Computer programme/script writing 1 hour
 Each of the 4 scenario demonstrations may not be split, other than to allow comfort breaks as necessary.
 The following activities MUST be demonstrated during the practical demonstration
 Scenario 1 Attack and Threat Research
 research current threat and attack techniques
 discover vulnerabilities in a provided computer system
 describe the significance of threat research and vulnerability discovery in a given context in an electronic document within the scenario
 Scenario 2 – Risk Assessment
 conduct a risk assessment
 produce an electronic document that proposes mitigations with a supporting a rationale appropriate to the context of the employer within the scenario
 Scenario 3 - Set up and configure a system with security features
 set up a system that incorporates a computer, a network, and a cyber-security function (components to be provided and may be virtual, design to be
provided) and demonstrate that it functions as intended.
 configure all the main parts of the system (computer, network, and cyber security function) to implement the controls identified in a supplied security case.
 demonstrate that security controls are effective against the intended threat.
 Scenario 4 Computer programme/script writing
 write a program or script to meet a given requirement
 demonstrate that the programme or script functions as intended and has been written to a coding standard that the apprentice is familiar with from their
apprenticeship

15. Hints and Tips
15
bcs.org
 Read the pack in advance and refresh knowledge on areas identified in the
assessment plan
 Have pen and paper to take and make notes – real work scenario
 Know the KSBs and criteria and review – key to confidence
 Practice scenarios based on the topics
 Allocate the entire period to this assessment with no other plans
 Have an early night before and make sure they have snacks/drinks
 This is a marathon assessment – be prepared!

16. AM2 The apprentice experience
16
bcs.org
 The apprentice will
 be in a quiet room on their own for the entire period
 have a computer with web cam and good internet connection
 logon with a link sent to them by BCS
 be asked for their original government approved ID
 be asked if they are ready to proceed and informed the entire assessment is recorded
 be given an explanation of what will happen
 share their screen for the assessment period
 agree comfort breaks between each scenario
 lead the assessment guiding the assessor through their activities
 be confident in sharing and explaining
 The assessor:
 MAY interrupt to refocus on areas they need additional information
 is not looking to catch them out
 wants them to Pass and meet the criteria
 works in the sector

17. AM 2 - Questioning
17
bcs.org
 will take place following the BCS assessors review of all 4 scenario demonstration outputs
 questioning will not be used to extend the scope of the assessment and will focus on the outputs of
each scenario demonstration
 KSBs not demonstrated during the scenario demonstration outputs can instead be covered by
questioning, although these should be kept to a minimum.
 questioning must last 45 minutes plus up to 10%
 time is allocated across the scenarios according to the judgement of the assessor where questions will
add most value in increasing their understanding of the competence of the apprentice.
 the BCS assessor will use the full time available for questioning to allow the apprentice the opportunity
to evidence occupational competence at the highest level
 the BCS assessor will ask a minimum of 9 questions typically focussed on scenarios 1 to 3 at their
discretion
 follow up questions may be asked
 apprentices will be provided with a copy of the outputs from their scenario demonstrations to refer to
and to aid recall via a screen share facility.

18. AM3 – Project Report
18
bcs.org
 At Gateway
 a project brief must be submitted to BCS at the gateway.
 BCS will approve and sign-off the project’s subject, title and scope to confirm its suitability prior to the project commencing
 The project brief must scope out the project and should include:
 a summary of the stages covered by the project
 an overview of the tasks involved in the project
 the specific responsibilities and duties assigned to be undertaken by the apprentice
 The project brief is not assessed and should typically be no more than 500 words
 Each project will focus on the specialism undertaken by the apprentice within the Cyber Security Technologist standard and may be based
on any of the following:
 a specific problem
 a recurring issue
 an idea/opportunity
 Suggested Project scopes include
 Cyber Security Engineer Option
 design and configure a network to meet a requirement and troubleshoot to optimise performance
 analyse requirements and build a security system to provide effective defence against cyber threats.
 Risk Analyst Option
 undertake a cyber risk assessment and produce a report
 participate in a cyber-security audit and produce a report
 undertake a cyber-security culture assessment and design and implement a security awareness campaign
 undertake a security policy review and produce a report
 Cyber Defender & Responder Option
 develop an incident response plan for approval within an organisations' governance arrangements for incident response.
 manage local response to non-major incidents in accordance with a defined procedure.
 detect and analyse a security incident with action plan responses.
 implement security tool configuration in response to threat intelligence

19. AM3 – Proposed Project Scopes
19
bcs.org
 Suggested Project scopes include
 Cyber Security Engineer Option
 design and configure a network to meet a requirement and troubleshoot to optimise performance
 analyse requirements and build a security system to provide effective defence against cyber threats.
 Risk Analyst Option
 undertake a cyber risk assessment and produce a report
 participate in a cyber-security audit and produce a report
 undertake a cyber-security culture assessment and design and implement a security awareness
campaign
 undertake a security policy review and produce a report
 Cyber Defender & Responder Option
 develop an incident response plan for approval within an organisations' governance arrangements for
incident response.
 manage local response to non-major incidents in accordance with a defined procedure.
 detect and analyse a security incident with action plan responses.
 implement security tool configuration in response to threat intelligence

20. AM3 – Project Report Creation
20
bcs.org
 conduct their project and submit a project report after a maximum of 6 weeks
 the employer will ensure the apprentice has sufficient time and the necessary resources, within this
period, to plan and undertake the project.
 normal workplace supervision
 may work as part of a team which could include technical internal or external support however the report
will be the apprentice’s own work and will be reflective of their own role and contribution
 need to consider the availability of company and external resources required to complete their project.
 project report has a maximum word limit of 2,000 words plus or minus 10%
 Appendices, references, diagrams etc will not be included in this total.
 Where organisational documents are required, screenshots or extracts should be provided.
 Hyperlinks to external sources will not be permitted.
 All project reports should include:
 an introductory section (text only, i.e. no diagrams, screen shots or figures) that explains:
 description of the project
 approach
 project outcomes
 mapping how the KSB are evidenced through the project

21. AM3 - Project Report scope must cover
21
bcs.org
 For Cyber Security Engineer Option
 design of the network
 evidence that the network works to meet the requirement
 network optimisation metrics against performance requirements
 requirements analysis and its link to the eventual system, including security features
 schematics to show the build of a system to the design from provided components
 configuration metrics to show how the system to meet the security requirements
 demonstration of how the security features are effective
 For Cyber Risk Analyst Option
 description of the role taken in a cyber security risk assessment and audit
 a report explaining the conduct of the risk assessment & audit
 a report considering the cyber policies and cyber awareness campaign
 For Cyber Security Defender and Responder Option
 incident manager report of an incident response
 incident response plan submitted for approval
 detection of a security incident and action taken
 analysis of a security incident and action taken
 evidence of the implementation of tool configuration in response to threat intelligence

22. AM3 - Assessment
22
bcs.org
 Will be marked by the BCS assessor
 Will be marked against assessment criteria
 Outcome will be sent to Training Provider

23. AM3 - Hints and Tips – Project Report
23
bcs.org
 No interaction with the assessor
 Practice report writing and effective appendices
 2000 words there is no room for detailed narrative
 Assessors can not assess outside of the word count
 Ensure the mapping document is in an appendix
 Remember no external links

24. AM4 – Knowledge Test
24
bcs.org
 40 multiple-choice questions
 On line test
 Invigilated
 a maximum of 60 minutes
 auto marked – results immediate
 can be remote proctored (at additional cost)

25. AM4 - Hints and Tips
25
bcs.org
 Prior to Gateway trial the sample paper
 Create and test the criteria across different sector examples – transferable
skills
 Create spot check tests on programme to test knowledge of syllabus
content
 Read the questions fully
 Candidate should know to move on if unsure and go back to it if possible

26. AM4 - The Apprentice Experience
26
bcs.org
 Will require ID to sit
 Will be in an exam situation with an invigilator
 Can navigate back and forward through the test
 Can not ask questions or receive any support
 Need to submit at the end when finished
 Countdown clock showing duration of test will time out

27. Resits and Retakes
27
bcs.org
 Apprentices who fail one or more AM can re-sit or a re-take.
 A re-sit does not require further learning, whereas a re-take does.
 An apprentice who fails an assessment method, will be required to re-sit or
re-take any failed assessment methods only.
 Re-sits/re-takes must not be offered to apprentices wishing to achieve a
higher grade than pass.
 NO CAP on grade
 All assessment methods must be taken within a six-month period,
otherwise the entire EPA will need to be re-sat/re-taken.

28. Clarification
 Training Provider Query
 On the Cyber Security Technologist (2021) Level 4,
Assessment Method 2:
 What is the scope of assessment for Scenario 4
Computer Programme/Script writing
 Writes program code or scripts to meet a given design
requirement in accordance with employers' coding
standards. K17 S13
 We have been informed that this is not a large part of
the role and for all apprentices are they allowed access
during the scenarios to external data sources where
they can source code?
 The assessment plan states "The EPAO is responsible
for ensuring the security of scenario demonstrations
they administer to ensure the assessment remains valid
and reliable This includes any arrangements for access
to any data source locations which may be available
locally or online"
Ofqual Response
 Considering K17, S13 – the employers who participated in the TBG were clear that it was considered a core requirement
and sufficiently important to be explicitly assessed (SD4). Recognising the breadth of potential roles covered by the 3
options, employers thought it essential that an apprentice cover ‘programming’ in its broadest sense and the way it is
worded intends to give considerable latitude in terms of interpreting it in a given employers context.
 At its heart this is the ability to achieve an outcome (‘design requirement’) by devising a list of instructions for a machine
(‘programme or script’) to an acceptable level of quality as judged by the employer for the job (‘iaw an employer’s coding
standards’).
 The level of complexity should be appropriate for a Level 4 apprentice to be assessed in 1 hour, with access to the range of
resources that the apprentice would have in their employer’s environment. If an apprentice is given access to ‘open source
libraries for example then the complexity of the overall programming task should reflect that so that within the assessed
hour there is an equivalent level of skills required to create a list of machine instructions to achieve a defined outcome.
 We anticipated an EPAO would develop a bank of representative tests of comparable difficulty that cover a typical range of
employers scenarios (and pick one appropriate for the employer in question). These could include actual programming
(probably most relevant option 1); or developing scripts (more likely to be relevant for option 3) e.g. to automate security
testing (e.g. port scanning), e.g. create reports (e.g. combining computer log reports generated by SIEM tools); or (possibly
most relevant for option 2) writing scripts to combine cyber risk scores (e.g. a macro in Excel). ‘Coding standard’ means a
quality assurance standard appropriate to the example. I would expect any ‘coding standard’ to provide for a sufficient level
of documentation to explain what it does, how to use it, how it works and some reference to how it has been/would be
tested.
 In response to the key question – yes if appropriate for the employers context, but the functionality demonstrated should be
commensurately greater. (The apprentice can combine pre-existing components, at issue is the contribution the apprentice
makes during the assessed hour to heart of the ability.)
 Training in the technology of the moment (e.g. a programming language) is very quickly out of date but the underlying
logical reasoning, problem solving, creative skills, conceptual understanding required to create instructions to automate
tasks to meet a purpose and to a quality standard is enduring and very necessary for all the roles we envisage this
apprenticeship supporting.

29. OPEN DISCUSSION
bcs.org 29