Keppel Ltd. 1Q 2024 Business Update Presentation Slides
M nolin pkerfoot spbt13_m_learning knowledge retention
1. 11
STICKY MLEARNING: HOW TO
ENSURE KNOWLEDGE RETENTION
IN MLEARNING PROGRAMS
Michelle Nolin, CPLP
Sr. Director, Product Strategy & Solutions
Informa Training Partners
Dr. B. Price Kerfoot
Associate Professor
Harvard Medical School
2. 2STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
LEARNING OBJECTIVES
Identify pedagogically sound strategies for mLearning
design
Appreciate findings from the latest instructional design
research for mLearning
Identify techniques for mLearning that ensure knowledge
and skills are gained, retained, and transferred
Apply 10 Tips for Knowledge Retention with mLearning
3. 3STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
What your learners really want
on their tablets and phones…
4. 4STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
What your learners don’t
want…
TAP
PINCH-
ZOOM
And then…
READ,
SCROLL,
READ, SCROLL,
READ, SCROLL, READ, SCR
OLL, READ, SCROLL, READ
SCROLL
5. 5STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Are you (and your learners) a…
Digital native OR digital immigrant?
6.
7. 7STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
What is mlearning?
M = Mobile
DEVICE
(Hardware)
Smartphones, iPads, othe
r tablets
TECHNOLOGY
(Software)
OS: Apple and Droid
Dozens of authoring tools
CONNECTIVITY
Cellular networks
Internet/Intranet
8. 8STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
eLearnng OR mLearning?
eLearning
• Real-time or self-paced
• Synchronous or
asynchronous
• Tethered (via
Web, Internet)
• Structured, formal
(GUI, architecture, etc.)
mLearning
• Self-paced
• Un-tethered
• Informal
• Unstructured, just-in-
time, field-based
• Anywhere, anytime
9. 9STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Polling Question
Are you using mLearning in your
organization today?
YES
Not yet, but actively working on mLearning
initiative
Maybe next year
No plans to use mLearning
10.
11. 11STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Challenges of mLearning
Device variability
Connectivity
Screen size and readability
Text entry
Limited memory
It’s MOBILE (and we work in a regulated
industry)
12. 12STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Poll
What are some of the challenges you face with
mLearning in your organization?
13.
14. 14STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Unique opportunities
Mobile devices are inexpensive and pervasive
Handle media well
Continuous, just-in-time
• In-the-field training and support
• ―Situational dependent knowledge‖
Rapid and urgent
15. 15STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Best uses of mlearning
Designed correctly, mLearning IS a knowledge
retention tool.
• LCOs: think “topics” NOT “courses”
• Job aids/support tools (quick “look ups”)
• Just-in-time training
• Rapid deployment of timely content
• Continuous touch points with your learners
• Evaluations, assessments, and remediation
16. 16STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Drive Retention with the
mLearning NINE
Start (so you can end
with) your desired
outcome
Bring it close to task
Consider Bloom’s
taxonomy
Set clear objectives—
and meet them
Match the objective to
the device
Engage learners
Establish logical
architecture and
navigation—and stick
to it
Establish and follow
good design standards
Meet the learner
where they live
17. 17STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Start with the desired outcome.
Instructional technology is great, BUT… it’s not
about the tool, or the platform. What outcome do
you need to achieve?
Content Calendar
Culture Cost
4Cs of
Training
18. 18STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Bring it close to task.
Examples
19. 19STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Consider Bloom’s taxonomy.
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
Lower order thinking skills
Higher order thinking skills
20. 20STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Set clear objectives
(and meet them).
Outcome
PerformanceLearning
21. 21STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Match the objective to
the platform.
Not all objectives can be achieved on a mobile
device
Some learning/performance objectives are better
suited for a tablet, others on a smartphone
22. 22STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Engage learners.
Draw the learner in… limit passive ―push down‖
of content
Use media where appropriate
• Audio
• Video
• Animations
Link mLearning with live interaction and
collaboration
Rethink assessments: gamify it
23. 23STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Establish logical architecture
and navigation (and stick to it).
Design an intuitive GUI with multiple navigation
paths through the content
• Let the learner drive and control access
Design for cross-platform, cross-device use
• Use responsive design
• Optimize for touch screen environments
24. 24STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Establish and follow good
design standards.
Micro chunk the content
More images and media, less text
Use large, easy to read fonts
Avoid ―porting‖ existing text/elearning content—
design and develop from the ground up for
mobile devices
Design according to the aspect ratio of the
device (not your computer monitor)
25. 25STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Meet the learner where they
live.
mLearning allows us to meet learners where
they live – on their PHONE and TABLETS.
• Cell phones = global population
• ½ the US population has smartphones
• Survey: 4 out of 5 people 18-44 check their
phone within 15 minutes of waking
• 84% time spent on email, texting, social
media, only 16% on voice calls
26. Sticky mLearning:
How to Ensure Knowledge Retention
in mLearning Programs
B. Price Kerfoot, MD EdM
VA Boston Healthcare System
Harvard Medical School
27. Conflict of Interest Disclosure
• Harvard submitted patent on spaced education
methodology.
• Harvard has launched a start-up company
Qstream which hosts this methodology outside of
the Harvard firewalls.
• As the ―inventor‖, I am an equity owner and
director of this company.
33. Spaced Education: the basics
• Spacing Effect
– spaced
– repeated
– repeated at increasing intervals
- Increased efficiency of learning
- Reduction in the slope of the forgetting curve
44. It Works!
• Increases knowledge & retention
- Medical Education 2007: 41:23-31 --- UGME
- Journal of General Internal Medicine 2008; 23(7):973-8 --- UGME
- Academic Medicine 2011 (in press) --- UGME
- Journal of Urology 2007; 177, 1481-1487 --- GME
– Journal of Urology 2009; 181, 2671-2673 --- GME
– Journal of the American College of Surgeons 2010; 211: 331-72673 --- GME
– Annals of Surgery 2009; 249: 744–749 --- CME
• Improves self-assessment of knowledge
– American Journal of Surgery 2009; 197(1):89-95
• Changes behavior
– American Journal of Surgery 2009: 197(2), 252-257
– American Journal of Preventive Medicine 2010; 39: 472-8
• Is well-accepted by learners
- Demonstrated in all trials to date
45. Interactive Spaced Education Versus
Web-based Modules for Teaching
Histopathology Diagnostic Skills:
a Randomized Controlled Trial
B. Price Kerfoot MD EdM, Yineng Fu MD, Michael Ritchey
MD, Donna Connelly MS & Elizabeth Genega MD
VA Boston Healthcare System
Harvard Medical School
Beth Israel Deaconess Medical Center
American Urological Association
J Am Coll Surg 2010;211:331-7
49. Spaced Education on GU Pathology *
Weeks
0 10 20 30 40 50
PercentageScores
50
55
60
65
70
75
80
Interactive
Spaced Education
Cycle 1
Cycle 2
Cycle 3
OC-1
OC-4
OC-3
OC-2
78% preferred SE over WBT
580/583 requested more SE
J Am Coll Surg 2010; 211: 331-7
50. Spaced Education Progress Testing
(SEPT) of Medical Students:
a Randomized Controlled Trial
B. Price Kerfoot, MD EdM, Kitt Shaffer MD PhD,
Graham T. McMahon MD MMSc, Harley Baker EdD,
Edward Krupat PhD, Jamil Kirdar, MBChB MRCP,
& Elizabeth G. Armstrong EdD
Harvard Medical School
Acad Med 2011, 86(3): 300-306
51. SEPT: Study Structure
• University of Virginia School of Medicine
• University of Pittsburgh School of Medicine
• Harvard Medical School
• University of Alabama School of Medicine
Acad Med 2011, 86(3): 300-306
52. SEPT: 120-item assessment tool
Year of Training
0 1 2 3 4 5
PercentageofItemsCorrect
35
40
45
50
55
60
65
All 120 Items
Acad Med 2011, 86(3): 300-306
56. 74% of participants (181/246) completed the SE program.
Of these, 97% (176/181) submitted the behavior change survey.
This randomized controlled trial was conducted from March 2009 to April
2010, immediately following the PriMed live CME conference in
Houston, Texas.
J Cont Educ Health Prof, 2011; 31(2):103–8
57. 86% agreed or strongly agreed that the SE program
enhanced the impact of the live CME conference.
97% requested to participate in future SE supplements to live CME courses.
J Cont Educ Health Prof, 2011; 31(2):103–8
58. Interactive Spaced Education
to Improve Prostate-Specific
Antigen Screening:
a Randomized Controlled Trial
B. Price Kerfoot MD EdM, Galina Sokolovskaya MS
Elizabeth Lawler MPH ScD, David Gagnon MD PhD,
& Paul R. Conlin, MD
VA Boston Healthcare System
Am J Prev Med 2010; 39: 472-8
59. VA PSA Screening Spaced Education Trial
Am J Prev Med 2010; 39: 472-8
60. VA PSA Screening Spaced Education Trial
93% requested to participate in future spaced education programs
67. Qstream Online Game
3N 2NSICU
Rank Team Ave. Score
1 3N 4083.7
2 SICU 3669.4
3 2N 3581.4
4 A1-MSDU 3425.8
5 CCU 3210.9
6 MICU 3206.0
7 AG 2906.2
8 A2 2493.1
9 PACU-PCU 2153.9
10 2S 1954.3
2S AGA2
Qstream Online Game
GO Team GO!!!
Nursescheck your email every,
TuesdayandThursday
got Qstream?
Lessons from Harvard:
Using Gamification to Juice
Your Sales Training
68. 68STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
CHOPPED ACTIVITY
Break into small groups
Ingredients you need are in your ―inbox‖
Cook up an mLearning intervention
Present Your DISH to the larger group
69. 69STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
10 Tips for Knowledge
Retention with mLearning
1. Select the right content at the right time for the right reason
in ALL mLearning.
2. Don’t let technology be the tail that wags the dog… desired
outcome should drive your mLearning initiatives.
3. Couple engagement through mobile devices with the
principles of spaced education to boost knowledge retention.
70. 70STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
10 Tips for Knowledge
Retention with mLearning
4. Investigate DIY, easy-to-learn, and cost-effective mLearning
tools to create continuous touch points with your learners.
5. Think beyond “just-in-time” training and performance
support for mobile learning… designed well, mLearning boosts
knowledge retention.
6. Use mLearning to meet learners “where they live,” providing
them with opportunities to access content when they need
it, confirm what they know (and don’t know), and retain their
knowledge.
71. 71STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
10 Tips for Knowledge
Retention with mLearning
7. Integrate mLearning tools with virtual instructor-led training
(VILT) to make distance learning intimate.
8. Integrate mLearning onsite at live meetings for learner
engagement.
9. mLearning is not reading on your phone… deliver learning
interactions on mobile devices to engage learners, foster
learning transfer, and sustain desired outcomes.
10. Recognize how digital natives and digital immigrants, like to
learn, and deliver training on mobile devices that balances the
needs of these two learner types.
72. 72STICKY MLEARNING: HOW TO ENSURE KNOWLEDGE RETENTION IN MLEARNING PROGRAMS
Questions?
SPEAKER EMAILS:
Michelle Nolin: mnolin@informatp.com
Dr. Kerfoot: price.kerfoot@gmail.com
Editor's Notes
Last year at my mLearning presentation I showed Angry Birds, this year, CANDY CRUSH
Poll, then ask, why does it matter to us as trainers? To our learners?
\r\nPoll Title: Are you a digital native, or a digital immigrant?\r\nhttp://www.polleverywhere.com/multiple_choice_polls/1ZCe6IO2AkIeroM
M = MobileDEVICE (Hardware)Smartphones, iPads, other tabletsTECHNOLOGY (Software)OS: Apple and DroidDozens of authoring toolsCONNECTIVITYCellular networksInternet/Intranet
Differentiating e-learning from mobile learning E-learning can be real-time or self-paced, also known as "synchronous" or "asynchronous" learning. Additionally, e-learning is considered to be “tethered” (connected to something) and presented in a formal and structured manner.In contrast, mobile learning is often self-paced, un-tethered and informal in its presentation. e-learning m-learninglecture in classroom or internet labs learning anywhere, anytimee-mail-to-e-mail instantaneous messagingprivate location no geographic boundariestravel time to reach to internet site no travel time with wireless internet connectivityBecause mobile devices have the power to make learning even more widely available and accessible, mobile devices are considered by many to be a natural extension of e-learning.
RUN POLLING VIA PHONES… then solicit people who said yes to share what they are doing.
\r\nAre you using mLearning in your organization today?\r\nhttp://www.polleverywhere.com/multiple_choice_polls/NAigQxELOsPj71P
M-learning design presents a unique set of challenges. All of these continue to improve, but represent challenges for us as trainers. Device variability. 3 TYPES of mobile phone devices: feature phones with tiny screens and numeric keypads; smartphones that include an A-Z keypad and a mid-sized screen; and touch phones featuring a device-sized screen activated by touch. And we can add tablets, namely,iPads to this mix. Not seeing a ton of uptake of droid and other operating systems in corporate training settings. Apple seems to be continuing to win in the market, and so for most of us, we are designing and pushing training to iPhones and iPads. ASK THE ROOM? MOBILE DEVICES IN USE? ASK: ALLOW CONTENT ON SOMEONE’S PERSONAL DEVICE OR ONLY COMPANY ISSUED? There is no single solution to push richly interactive mobile content onto every possible phone. Studies have shown that learners want to learn on their own mobile devices, but that’s a challenge for us in a regulated industry. Slow download speed and limited Internet access. This gets better all the time, but it’s an issue with mobile learning, we are pushing content out, and that requires connectivity at a decent speed. Small screen sizes present unique challenges, although resolution, color, and contrast has dramatically improved with each generation of smart phone and handheld device. Awkward text input. Regardless of the device being used, inputting text data into small devices also presents challenges for the user. Inputting information into a device using a numeric (0–9) keypad on a feature phone continues to be tedious and time-consuming. Again, the more sophisticated the device, the better its input capabilities. Limited memory. Handheld phones have limited internal information storage capacity or memory. ASK: ARE THERE OTHER CHALLENGES? POLL THE GROUP.
\r\nWhat are some of the challenges you face with mLearning in your organization?\r\nhttp://www.polleverywhere.com/free_text_polls/L85AsRaipq64suT
But m-learning design also presents a distinctive set of opportunities: Relatively inexpensive m-learning opportunities, and it can allow you to take your training global. In many parts of the world, mobile phone handsets are quickly becoming the Internet platform and multimedia device of choice. Multimedia content delivery and creation options… audio, text, pictures, video, and cameras are all standard features now, so we can’t forget about it in our designContinuous and situated learning support. Mobile devices allow ongoing learning to occur in multiple locations, including the potential to offer “scaffolded” support (Saye & Brush, 2002) to learners undertaking authentic tasks. Using these devices in a way that maximizes these learning benefits has the potential to offer educational opportunities that are both more inclusive and of higher quality. As Nyíri (2002) has explained:Mobile communication is enhanced everyday communication; and just as our everyday conversation is indifferent towards disciplinary boundaries, so, too, is m-learning. Situation dependent knowledge, the knowledge at which m-learning aims, by its nature transcends disciplines; its organising principles arise from practical tasks; its contents are multisensorial; its elements are linked to each other not just by texts, but also by diagrams, pictures, and maps. (p. 124) ASK: POLL THE GROUP… WHAT OTHER UNIQUE OPPORTUNITIES DOES MLEARNING OFFER OTHER DELIVERY METHODOLOGIES?
ASK: OTHERS?
Just because you can design a cool game-based app that is as addictive as Candy Crush or Angry Birds, doesn’t mean that a game for the iPad, iPhone, or Droid device makes instructional sense. In beginning, and all throughout, it’s about the desired business outcome, and then applying the filter I like to call the 4 cs of training to it.
Particularly true for mLearning! It’s all about situational dependent knowledge… providing learners with the knowledge, skills, and information they need, at the precise moment when they need it. That’s what mLearning can be particularly good at. Examples, two built in Jquery and one using Adobe (PDF).
Knowledge: Recall data or information.Examples: Recite a policy. Quote prices from memory to a customer. Knows the safety rules.Key Words: defines, describes, identifies, knows, labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words.Examples: Rewrites the principles of test writing. Explain in one's own words the steps for performing a complex task. Translates an equation into a computer spreadsheet.Key Words: comprehends, converts, defends, distinguishes, estimates, explains, extends, generalizes, gives an example, infers, interprets, paraphrases, predicts, rewrites, summarizes, translates.Application: Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the work place.Examples: Use a manual to calculate an employee's vacation time. Apply laws of statistics to evaluate the reliability of a written test.Key Words: applies, changes, computes, constructs, demonstrates, discovers, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses.Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.Examples: Troubleshoot a piece of equipment by using logical deduction. Recognize logical fallacies in reasoning. Gathers information from a department and selects the required tasks for training.Key Words: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.Examples: Write a company operations or process manual. Design a machine to perform a specific task. Integrates training from several sources to solve a problem. Revises and process to improve the outcome.Key Words: categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies, organizes, plans, rearranges, reconstructs, relates, reorganizes, revises, rewrites, summarizes, tells, writes.Evaluation: Make judgments about the value of ideas or materials.Examples: Select the most effective solution. Hire the most qualified candidate. Explain and justify a new budget.Key Words: appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.
Outcome = what are you looking to achieve with an mLearning training component, or program? I call this the Outcome Objective. Examples: My sales force needs to be able to follow certain steps when they deliver an inservice to nurses in the rheumatology clinic, and they need to be able to quickly access that information ondemand in the field, right before they deliver that inservice. Dual outcome: training and support. We often lump all objectives under learning objectives, but I also like to split these out… knowledge-based learning objectives and skills-based performance objectives. In the example of the inservice, the learner needs to know the steps in demonstrating a product during an inservice and be able to perform them. mLearning works best when we can isolate one or two clear objectives, design the training to meet those objectives, and then serve it up ondemand.
ASK: What would be an example of a type of mLearning intervention that would be better on a tablet? High density of content, graphics you want learners to be able zoom in and out, knowledge-based programs, etc. What is an example of a type of mLearning intervention that would be better on a smartphone?
Use media where appropriateAudioVideoAnimationsBut don’t forget to draw the learner in… limit passive “push down” of contentLink mLearning with live interaction : example: coaching guide on the iPad with checklists; worksheet-based activities that the learner completes and brings to an LILT or VILT event. mLearning does not have to be an island that is only visited by each individual learner.
Many of the rules of good eLearning design apply to mLearning.
Chunk contentMore images and media, less textLess than 100 words per screen is a good rule of thumbGo full-screen with graphicsUse large, easy to read fontsAvoid “porting” existing text/elearning content—design and develop from the ground up for mobile devicesDesign according to the aspect ratio of the device (not your computer monitor)Aspect ratio: ratio of the longer dimension to the shorter dimension of the screen. The iPhone screen is 9 cm (3.5 in) across. The pixel size is 480 x 320 which gives it an aspect ratio of 3:2. If you want to take advantage of this whole screen then you need to create content sizes of this ratio when using PowerPoint, Word, mp4 video or any other format. When using PowerPoint or Word a page size of 11” x 7.3” will provide this aspect ratio and will ensure that when the content is viewed on the iPhone it fits snugly to the whole screen.
Cell phones = global population ½ the US population has smartphonesSurvey: 4 out of 5 people 18-44 check their phone within 15 minutes of waking84% time spent on email, texting, social media, only 16% on voice calls
