Presentation material from ATD 2016 ICE W316 session.

1. LEARNING
EXPERIENCE
DESIGN
ATD 2016 ICE | Session W316

2. SPEAKERS
JAY CHON is business development and alliance director for IndustryMedia,
focused on developing cross-industry alliances utilizing learning solutions.
Jay is responsible for learning strategy, technology deployment and outsourcing at
major telecommunications, recruiting, and enterprise resource planning alliance
partner companies in Korea.
Jay was previously IBM Interactive Lead for IBM Korea and Senior Managing
Consultant for IBM GBS Korea.
CHAN LEE is professor of vocational education and workforce development at
Seoul National University in Korea.
He is also a current program chair of vocational education and workforce
development and department chair of agricultural and vocational education at
Seoul National University.
Chan worked at LG Electronics in the United States as the HRD team leader. Chan
holds a PhD and master’s degree in human resources development from the Ohio
State University.
©industrymedia 1

3. • Welcome and thank you!
• Understand learner expectations
• Understand experience design
• Examine actual learner behaviors
AGENDA
©industrymedia 2

4. Learner
Expectations
What do learners really
expect from us?
©industrymedia 3

5. • Learner-centered, self-directed …
• On-the-job, workplace learning …
• Just-in-time, any time, any topic, any path …
• Immersive, virtual, collaborative …
We focus too little on the experience! And, we’re already losing to competition.
©industrymedia 4
Our learners expect the Matrix because it is so many things our learning
experience is lacking.
Credit: The Matrix, Warner Bros, 1999

6. The closest medium that we have comparable to the Matrix is nearly always
in our pocket – our smartphone!
©industrymedia 5
We are assuming that you leave your Oculus at home.

7. Digital media usage time is exploding and it’s predominately being driven by
mobile apps.
©industrymedia 6
Source: The 2015 U.S. MobileApp Report, comScore
Growth in Digital Media Time Spent
Total digital media usage has grown 49% with mobile apps having grown 90%; contributing to 77% of
the total increase in time spent.

8. Mobile now represents almost 2 out of 3 media minutes, and mobile apps
alone now constitute a majority.
©industrymedia 7
Share of Digital Media Time Spent by Platform
Source: The 2015 U.S. MobileApp Report, comScore
Mobile now accounts for 62% of digital media time spent and mobile apps on their own now drive the
majority of digital media time spent at 54%.

9. “We’re good. Our learning is available through the mobile web.” Really?
©industrymedia 8
Mobile Unique Visitors (1M) Average Monthly Minutes per Visitor
Source: The 2015 U.S. MobileApp Report, comScore
Establishing app audiences is difficult, but their real value is in their loyalty. Mobile visitors spend 18-
times more time on apps than the mobile web.

10. The list of top 25 mobile apps is dominated by the leading digital media
companies and tends to concentrate within a few categories.
©industrymedia 9
Source: The 2015 U.S. MobileApp Report, comScore
The top 25 apps can be broadly classified as either social, entertainment, utilities, or retail.
Unfortunately, learning is not as popular.
Top 25 Mobile Apps by Unique Visitors (1K) with Y/Y Growth

11. Social media and entertainment account for the six top app categories and
drives two-thirds of total time spent on apps.
©industrymedia 10
Share of Mobile App Time Spent
Source: The 2015 U.S. MobileApp Report, comScore
The strength of the top categories highlights that mobile devices are more heavily used for
entertainment and communication. Again, learning does not account for any significant time.

12. The vast majority of apps skewing most heavily towards Millennials are social
in nature.
©industrymedia 11
Top 20 Apps with Highest Concentration of Millennials (Ages 18-34)
Source: The 2015 U.S. MobileApp Report, comScore
That vast majority of Millennial-heavy apps are related to social, chat and music.

13. YouTube mobile app’s explosion in engagement on smartphones and tablets
highlights the rapid shift to mobile video.
©industrymedia 12
Source: The 2015 U.S. MobileApp Report, comScore
YouTube is installed on 52% of all mobile devices; where smartphone users watch nearly 5-hours and
tablet users watch nearly 9-hours each month.
Average Hours Per Month Per Visitor YouTube Audience Penetration

14. More than 4 out of 10 Millennials listen to podcasts on smartphone apps at
least once a month.
©industrymedia 13
How often do you listen to podcasts using your smartphone?
Source: The 2015 U.S. MobileApp Report, comScore
While podcasting has been around for more than a decade, the popularity of new content combined
with the power of smartphone apps has helped it gain popularity – particularly among Millennials.

15. App installation does not guarantee continued use because apps, on
average, lose 77% of Daily Active Users in 3-days and lose 90% after 30-days.
©industrymedia 14
Average App Retention Rate (DAU)
Source: MobileIntelligence, Quettra, 2015 (Androidonly)
Defining app retention tactics during the app design process will increase the chance of continued
usage after installation.
App Retention Tactics
Progressive Onboarding
Notifications (Push, eMail)
Personalization
Incentives
App Updates

16. Learning Delivery
• Is your learning accessible via mobile?
• Is there an app for that?
Learning Experience Design
• Do you understand how your learners behave in mobile web and apps?
• Do you leverage prior experiences learners have gained using popular apps into
learning experience design?
Millennial Fever Prevention
• Do you leverage peer-to-peer learning with social and/or chat apps in mind?
• How much time do your children spend on YouTube? Do they need your help?
• How many of you have listened to a podcast in the last 30-days?
©industrymedia 15
SHORT DISCUSSION

17. • We're delighted to be here!
• The Matrix is yet to be. But, in the meantime, we must focus on mobile!
(Unless your department is obsessed with Oculus – which is a good thing.)
• Mobile web is okay, mobile apps are better!
• Videos and podcasts are here to stay. So, get ready!
• Understand function, interaction and emotion designs within popular apps; and,
assimilate implications to learning app design.
©industrymedia 16
QUICK SUMMARY

18. Experience
Design
What is the process of
experience design?
©industrymedia 17

19. Customer experience (CX) is the interaction between an organization and a
customer over the duration of their relationship.
©industrymedia 18
Consumer Decision Journey (CDJ)
Source: The Consumer Decision Journey, McKinsey & Company, 2009
The interaction includes a customer's consideration, evaluation, purchase and advocacy of a service –
commonly referenced together with McKinsey & Company’s Consumer Decision Journey (CDJ).

20. The learning experience is not different in the sense that learning involves
identical discovery, evaluation, selection, and advocacy phases.
©industrymedia 19
Learning Experience Virtuous Cycle (Point-of-View)
Learning
Discovery
Learning
Selection
Learning Evaluation
Post Selection
Loyalty Loop
Trigger
When in need of learning, learners will try to discover relevant assets (e.g., courses, Google, Wikipedia,
YouTube), evaluate the assets, select an asset, experience an asset; and, assuming the experience was
favorable, will continue to select the assets from the same provider.

21. So, how are learner experiences designed?
©industrymedia 20
Understand
the learner
Design the
optimal
experience
Define
implementation
requirements
Define
performance
indices
The end product is referred to as a ‘journey map’ and serves as a
continually evolving communication tool; rather than a definitive program specification.

22. The most important step is the first step. Utilize actual data and interviews
to define the personas accurately.
©industrymedia 21
ILLUSTRATIVE
1. Define demographics
2. Define key characteristics
e.g., computer literacy,
wants and needs, etc.
3. Define attitude towards
intended service
4. Define goals and decision
criteria for persona

23. The second step, designing the learner journey (or interactions), requires
thinking in terms of the persona – and a little imagination and creativity.
©industrymedia 22
ILLUSTRATIVE
1. Define learner decision
phases
2. Define phase goals
3. Define anticipated and
intended learner
behaviors
- What’s available today?
- What more can we do?
- What’s available aside
from learning?
Fundamental learning experiences should be designed first; but, additional experiences must be
incorporated to increase engagement and retention.

24. Defining the required functions and content are self-explanatory; but,
careful thought must be made when defining performance requirements.
©industrymedia 23
ILLUSTRATIVE
1. Define required functions
2. Define required content
3. Define performance
metrics; for continued
improvements
Performance metrics and their results will determine the need for change and the magnitude of
change after initial deployment.

25. Learning Experience Design Process
• How are your courses designed?
We’re excellent at creating courses; and, we apply methodology and discipline
similar to journey mapping when creating courseware.
• In comparison, how are your learning experiences being designed today?
Learning Experience Design Capabilities
• How many personnel in your department are knowledgeable in user experience
(UX) design, or human-computer-interaction (HCI), or digital marketing?
©industrymedia 24
SHORT DISCUSSION

26. Learner
Behaviors
How well do we know
our learners?
©industrymedia 25

27. • Examine data from over 400+ companies
• Types of data being examined
 Audience
 Behavior
 Enrolment / Completion
• Share learner survey results
©industrymedia 26
We wish to share our learning data; and, collectively identify implications to
and improvement opportunities for YOUR company.

28. AUDIENCE: Users (1-Year)
©industrymedia 27
A total of 103K ‘web’ users have joined during
the last 13-months.
During the same period, a total of 37K ‘app’
users have joined. (Approx. 35% of web)
AppWeb

29. AUDIENCE: Users (1-Month)
©industrymedia 28
Last month, 22K learners accessed the web. During the same period, 11K learners
accessed the app. (Approx. 53% of web)
AppWeb

30. AUDIENCE: New Users (1-Month)
©industrymedia 29
Out of the 22K web learners 16K (72%) are
new learners.
Out of 12K app learners, only 4K (34%) are
new learners; which is significantly lower.
AppWeb

31. AUDIENCE: Location (1-Month)
©industrymedia 30
Nearly all of our learners are from South
Korea. (We’re a Korean company.) We do see
minimal learners from other regions.
Because we do not know where the learners
will access our learning app, we heavily rely
on Content Delivery Networks (CDNs)

32. AUDIENCE: Technology (1-Month)
©industrymedia 31
Our web services are accessed primarily
through desktop browsers; with some from
mobile browsers.
Most users use our app on a mobile device.
(We don’t have a tablet app yet.)
AppWeb

33. AUDIENCE: Technology (Mobile-only)
©industrymedia 32
All iOS learners are on the most current
version (9.3.1) while Android learners are
fragmented across versions (4.4.2 – 6.0.1).
Although there is a predominant screen
resolution, other resolutions make up the
majority.

34. BEHAVIORS: Sessions (By Time of Day, By Channel, 1-Month)
©industrymedia 33
120K sessions (59%) were
created on the App
while 84K sessions (41%)
were created on the
Web.
Web usage is
concentrated during
business hours.
App usage is
concentrated before and
after work; and
particularly from Sunday
to Thursday.
AppWeb

35. BEHAVIORS: Sessions (By Time of Day, By Channel, 1-Year)
©industrymedia 34
App
The results are the
same, if not more
concentrated, when
observing app lifetime
data.
Web

36. BEHAVIOR: Session Duration (1-Year)
©industrymedia 35
App
App users spend nearly 11-minutes per
session
Web users spend 6-minutes per session
Web

37. BEHAVIOR: Session Duration (1-Month, Mobile-Only)
©industrymedia 36
On average, a learner accesses 14 screens per
session; and, is not different across mobile
operating systems.
On average, a learner spends 10-minutes per
session; 10.5 hours/year for 100+, 15.1
hours/year for 1-99 sized companies.

38. Learning category preferences vary by day and time, but the top 5 preferred
categories do not change.
©industrymedia 37
67.82%
30.74%
54.84%
26.21%
9.64%
23.31%
2.74%
25.56%
7.26%
17.56%
39.16%
18.48%
10.04%
14.86%
1.89%
15.54%
5.23%
13.53% 14.21%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
To Work Biz Hours Weekday Nights Weekends
Management Personal Development Leadership English MBA
Learning Category Preference

39. BEHAVIOR: Screen Views (By Menu)
©industrymedia 38
Other than the obvious Login and Home, the
Category browsing and Popular Courses
screens are accessed the most.

40. Our survey results from last year indicate that learners are satisfied with
mobile learning and are interested in using the service again.
©industrymedia 39
Q. What is the major reason for accessing this service?
Q. Are you interested in using the service again?
Q. Were you satisfied with the learning experience?
Q. Are you more satisfied with mobile vs. other options?
Very
Satisfied
Not
at All
Very
Satisfied
Not
at All
Very
Interested
Not at All
Interested
- Company policy
- Personal development
- Job requirements
- Promotion preparation
- Refreshment
- Other

41. Learners are positive about on-the-job performance improvements.
©industrymedia 40
Q. How likely are you to apply your learning to work?
Q. Has your job performance increased with mobile
learning?
Q. Have you applied any of your learning to work?
Q. Have you received feedback on better performance
from your peer/manager after learning?
Very
Much
Not
at All
Very
Likely
Not
at All
Almost
Always
Not
at All
Almost
Always
Not
at All

42. Learning Strategy
• Provide option to learn via mobile; especially for Millennials
• Maintain web and provide seamless experience with mobile
Learning Content
• Ensure content is effective on multiple screen resolutions
• Review length of current content; maintain length between 5-15 minutes
• Consider multiple content formats (i.e., video, podcasts, etc.)
Learning Operations
• Repeat experience design and behavior analysis throughout the lifecycle of the service
• Incorporate web/app analytics from the beginning and use as basis for continuous
improvements
• Extend engagement and retention tactics beyond the app realm (e.g., social sharing, push
notifications, SMS/MMS, incentives, etc.)
Learning Infrastructure (Systems)
• Be prepared to deliver learning services from any location, 24x7!
©industrymedia 41
QUICK SUMMARY

43. THANK YOU!
What change can we
start TODAY?
©industrymedia 42

44. ©industrymedia 43
Chan Lee (Professor)
Seoul National University
chanlee@snu.ac.kr
http://facebook.com/hrdream
http://twitter.com/hrdream
http://kr.linkedin.com/in/hrdream
http://hrdream.snu.ac.kr/
Jay Chon (Alliance Director)
IndustryMedia
Q&A
jwchon@ubob.com