Navigating Career Paths in the Age of AI
•Download as PPTX, PDF•
0 likes•54 views
This presentation from the OECD Disrupted Futures 2023: International lessons on how schools can best equip students for their working lives conference looks at Enhancing guidance through digital technologies “Navigating career paths in the age of AI”. Presented by Deirdre Hughes and Chris Percy. Discover the videos and other sessions from the OECD Disrupted Futures 2023 conference at https://www.oecd.org/education/career-readiness/conferences-webinars/disrupted-futures-2023.htm Find out more about our work on Career Readiness https://www.oecd.org/education/career-readiness/
Recommended
Recommended
More Related Content
Similar to Navigating Career Paths in the Age of AI
Similar to Navigating Career Paths in the Age of AI (20)
More from EduSkills OECD
More from EduSkills OECD (20)
Recently uploaded
Recently uploaded (20)
Navigating Career Paths in the Age of AI
- 1. Navigating Career Paths in the Age of AI OECD Disrupted Futures Conference Associate Professor Deirdre Hughes & Dr Chris Percy (UK) 2nd June 2023 ©CareerChat UK Ltd.
- 2. What is CareerChat’s purpose? Through CiCi the careers chatbot Enabling people to find fulfilment through their career choices ©CareerChat UK Ltd
- 3. Introduction
- 4. Focus • Present emerging findings from an applied digital and careers research programme underway in a school for children and young people aged 3 to 19 and a multi-academy trust (MAT) in England. • Share lessons learned, ideas and experiences on the use of chatbots, AI and large language models (LLMs) in education. • Hear about recent LLMs, their progress, and usage in careers IAG ©CareerChat UK Ltd.
- 5. How can a chatbot, powered by AI and machine learning, be used most effectively to improve career learning outcomes in a secondary school setting?
- 6. Recognition and Coverage • Shortlisted by UK Career Development Institute’s National Award for Best Practice Research and Innovation in the Use of Technology 2021 • CiCi features in a new International Labour Organisation (ILO) Digital Inventory of Career Guidance Tools • CiCi in Forbes, FE News, the Institute for Employability Practitioners (IEP), the UK Employment Related Services Association (ERSA), the IEP journal and Careers Matters journal • Book chapters published by Cedefop x 2 – Digital Transformations • SuperUser Groups x 12+ in differing regions in England and a fully customised version in a national/international private university CiCi so Far Working with Partners, Practitioners and Volunteers to get the idea just right
- 7. LLMs: A potted recent history Source: LinkedIn Source: OpenAI White Paper ©CareerChat UK Ltd
- 8. Two examples of deployed CIAG models Public-facing (France) • Chatbot-style interface for searching publicly- available data • Convenient integration of multiple databases in one place • Nudge tactics to promote users to engage and be proactive in their job search / course search • Main focus: help unemployed adults find work e.g. bob-emploi.fr Practitioner-facing (UK) Level 1: Repository of curated, QA-ed info for guidance practitioners e.g. LMI, trends, courses, vacancies, skills etc. Level 2: Professional supports access for client/class e.g. introduces tool, empowers for independent use – continues to help with reflection/action Level 3: Integrated public/professional usage, e.g. public front-end for simple queries + localised referrals e.g. careerchat.uk ©CareerChat UK Ltd
- 9. What about careers IAG? “Information” “Advice” “Guidance” • What qualifications to become a doctor? • Entry requirements for this course? • Average starting salary / hours for an engineer? • Help choosing a career & building an action plan • How to balance work and personal life? • How to stay positive during job search? • How to lay out my CV for this job? • How to contact local employers for work exp.? • What long-list career options suit my interests? ©CareerChat UK Ltd
- 10. A brief round-up What is CiCi? • CiCi is a distinctive AI chatbot with careers information and advice all in one place, quick and easy to use. • CiCi provides a personalised career journey experience for adults and young people, that can be integrated within careers and employability services • CiCi provides access to: • Searching and applying for jobs • Apprenticeships, T Levels, HE and training courses • Career discovery including ‘SkillsOmeter’ & CV support • Speaking to a Human Adviser • Self-employment and Volunteering • Volunteering
- 11. 26 000+ 40 000+ 25 000+ Jobs & skills information profiles 1 500+ Course information Full-time & Part-time jobs in England, Scotland and Wales Short inspirational career journey videos CiCi the chatbot gives you access to:
- 12. CiCi the chatbot is highly personalised with an inbuilt dashboard facility
- 13. Demo Screenshot CiCi is able to provide a record of the user journey, giving advisers a head start with interviews
- 14. THE BEMROSE SCHOOL • Inner-city school • Initially piloted with students, teachers and parents • Lower-secondary (ages 12 to 15) & Upper-secondary (ages 16-18) • Positive parental engagement • Now being embedded into school’s careers strategy • Teaching resources & training workshop with teachers to support school wide buy-in
- 19. BUDMOUTH ACADEMY & ASPIRATIONS ACADEMIES TRUST • School wide pilot led by Careers Leader • Links to access CiCi shared via Google Classrooms • Good initial take-up • Positive feedback received • Now being set tasks to complete using CiCi on Google Classrooms as part of careers curriculum
- 24. The floodgates of poor practice are already opening… ©CareerChat UK Ltd
- 25. Advice? “I’ve decided I want to change careers and become an accountant. How should I go about this?” Guidance? “I’m feeling under pressure to make a decision about what jobs to apply for, as I come up to finishing my degree in French and German from Leeds University.” As we go: please use the chat to share views on risks/opportunities to CIAG from LLM bots ©CareerChat UK Ltd
- 26. Reflections “What are the biggest risks and opportunities you see for LLMs in careers IAG?”
- 27. Reflections and feedback A few early thoughts on risks • Makes assumptions about user (no clarifying questions) • Over-confidence (generic caveats are shared but might be easily glanced over) • Risk of closed thinking or stereotyping in some cases • May invent information (if pushed too far, especially for local data, course/LMI etc.) • Does not know when to stop and strongly urge engaging with a person (red flags) • Corporate mis-use (no transparency) ©CareerChat UK Ltd
- 28. Benefits to organisations and individuals • Leading edge digital technology positioning at the forefront of AI and LLM innovation • Ability to save and to export profiles and key dialogue within CiCi – to be shared with and directed to the most well-suited adviser and make better use of everyone’s limited time and resource • Free access 24/7 to impartial and confidential careers information and advice • The result: better-prepared individuals can help to plan and manage career dialogue more effectively
- 29. Find Out More Visit: www.careerchat.uk Book a Demo - https://careerchat.uk/schedule-demo/ Or send us an Email If you would like to find out more, we’d love to hear from you. Here’s how we can chat.
Editor's Notes
- I’m Deirdre Hughes (Co-Founder of CareerChat, UK) joined by Dr Chris Percy (Co-Founder). We partner with organisations to add value to their careers and employability services by harnessing accessible and intelligent technology solutions with measurable outcomes. Since 2020, we have been researching the effective use of AI in a careers context and have co-developed CiCi the careers Chatbot, powered by AI and machine learning. In the short time we have with you, we will focus on our work in two contrasting schools (1) an inner-city school for children and young people aged 3 to 19, situated about 1 mile West of Derby City centre in the Midlands of England. This has a Nursery, approximately 300 pupils in the Primary phase, 780 students in secondary provision (11 – 16) and a successful and growing sixth form (17 -18-year-olds) and (ii) a multi-academy trust covering Dorset, Oxford and London. Academies are state schools that are not controlled by the local authority. Academies receive funding directly from the Government and are run by an academy trust. Schools controlled and funded by the local authority are called maintained schools. Academies have more control than maintained schools over some aspects of delivering education. For example, they do not have to follow the national curriculum and can set their own term dates. However, academies have to follow the same rules on admissions, special educational needs and exclusions as other state schools, and their students sit the same exams.
- Using the chat facility tell us which large language models, if any, you’ve had recent experience of. Don’t worry if this is something completely new to you, we’re all learning from each other. From our ongoing work, practitioners tell us they face a knowledge gap and report the growing need for professionals who understand AI-based technologies and ways of leveraging them for the advantage of students, teachers and parents.
- Before we share findings from the application of CiCi in the two contrasting school settings, followed by a brief overview from our AI literature review, it is worth noting that: People have been working on chatbot type models since the early years of computers, such as MIT's ELIZA in 1966 - a chatbot with scripts designed to imitate a therapist or psychologist. In the last few years, there has been an explosion of capability. OpenAi's first GPT model, GPT1 came out in 2016, but the impressive results really began in 2020 with GPT-3, which had 175 billion parameters. GPT-3 is basically a very good next word predictor. And if you think about it, to reliably predict the next word and then the next and the next, you need to encode certain reasoning and world modelling abilities to do it better. This is what we see with the LLMs - to various levels of ability - albeit with plenty of mistakes, given that it isn't actually grounded in the world at this stage and doesn't have a way of checking its reasoning separately from its corpus of text. GPT-3 sucked up about 500 billion tokens, approximately equivalent to words, mostly from online material. You then provide it with a prompt, which might be a few words or whole pages, and it then continues the text with what seems plausible given the vast corpus it has been trained. This isn't really a chatbot. If you show it an example conversation, it will continue the conversation, but keep producing text for both sides until told to stop. The usefulness of the result depends heavily on both the fine-tuning of the model and the quality of the prompt written immediately prior to the next word prediction, giving rise to the first of many grandiose careers predictions from the technology: that a new job called prompt engineer would seen be a major employer. At the end of Nov 2022, some six months ago, the world of LLMs changed. The same core model in GPT-3 was wrapped up with various behind-the-scenes fine-tuning and prompting to create something that worked intuitively as a fully natural language chatbot, rather than needing the user to provide the necessary prompts and illustrative examples to elicit the desired response: ChatGPT. The success of this, reaching 1 million users within 5 days of launch, has sent the tech world into meltdown, with Microsoft, Facebook, Google, and others quickly launching their own branded versions (Microsoft’s Bing is based on OpenAI). We’ll do some demos together later of GPT-4 and Google’s Bard. GPT-4 in particular seems to be performing at human-level, albeit on highly canned and structured tests.
- Two cases of deployed models focus strictly on Information and the simplest areas of generic advice. Both have worked together with careers advisers and sector organisations, with each having a different philosophy on how to help – likely to be complementary, support for different groups (e.g. the former more tech savvy, the latter needing a more curated/human-integrated space). There are also separate tools, powered by AI to various extents, that provide more tailored support to job seekers on very specific questions, such as Mockmate where users can record themselves doing very short interview questions and get AI-powered feedback or Rezi.Ai which has a suite of tools, some GPT powered, to help write or improve a CV. Here we will remain focused on questions/discussions within a core IAG setting rather than career management skills, although it is possible to imagine a future where many of these tools come together in an integrated way.
- While the IAG boundaries are blurred, particularly in career guidance sessions, there is a general spectrum with different levels of expertise needed at different levels. The question for careers chatbots is what sorts of services might usefully be provided by chatbots to what types of users? personalised advice, e.g. questions about a specific job, opportunity, or person, where the intention is to provide a fairly straight answer to a fairly straight question. As opposed to guidance, which typically requires more empathetic, targeted and iterative questioning from a human professional, considering the difference between presenting and underlying issues, and appears to be well beyond current off-the-shelf models (more on this later).
- Impressions on guidance: General statements are approximately tailored to the context and have reasonable caveats, including recommendations to reach out to people Words convey some empathy but not sensitive to the direct feedback (e.g. keeps producing fairly large itemised lists) Not actually delivering guidance, more of a single response to a single question with suggested actions for the user No asking questions to gather more context, no breaking up the conversation into smaller units, no trust building/contracting, no probing for underlying issues (e.g. why are feeling under pressure) etc. In practice, treating guidance questions more like advice. If more proactive as a user, can get some more value out of it It may be possible to fine-tune models to act more like an empathetic conversation, e.g. with InstructGPT and successors, but this is not available off the shelf, and if we want to RLHF it wouldn’t be a quick process either Impressions on advice: Doesn’t ask for context – without context it is hard to offer good advice. The generic statements are fine, and perhaps sharing those is fine, but it should say that a good answer requires context and ask if user willing to answer questions. No challenge, probing, testing – but perhaps this is okay (as in accept this by design) If ask for specific opportunities, it is likely to either refuse to answer (fine) or hallucinate details Acknowledges it’s an AI language model On the face of it, the simpler question turns out to be strangely harder for the LLM, perhaps because generic answers are less appropriate in this case. You would need the LLM plugged into local LMI and trained to probe for context in these kinds of contexts.
- Group discussions given what have seen and put in the chat
- https://www.bemrose.derby.sch.uk/ https://www.aspirationsacademies.org/our-academies/ http://www.budmouth-aspirations.org/