Large Scale, Blended LearningDevelopment: BenchmarkA Chapman Alliance, Research StudyMarch 2013By Bryan Chapman, Chief Lea...
Research MethodologyHow did we collect the dataThe data contained in this research was collected bysurveying 188 organizat...
Demographics DashboardInformation about survey respondents and the blended courses represented in the resultsBasic188 – co...
How Long Does It Take to Create a Blended Learning Course“development time” per “finished hour” (seat time)Across all 188 ...
How Long Does It Take to Create a Blended Learning CourseBoil it down to “development time” per “finished hour” (seat time...
How does “Length of Course” change development ratios?Course length does appear to have an effect on development times and...
How does “Length of Course” change development ratios?Course length does appear to have an effect on development times and...
Using Benchmark Data for your ProjectsSimple scenario, using core metrics to create high-level estimatesSetup: you’ve been...
Adjusted for100K or moreemployeesHow Long Does It Take to Create a Blended Learning Course“development time” per “finished...
How Long Does It Take to Create a Blended Learning Course“development time” per “finished hour” (seat time)Filtered for re...
Budget AllocationHow budget amounts change based on internal development, outsourcing, or using a hybrid approachInternalD...
Lots of Different Kinds of BlendsObservation: The study picked up lots of different blends. No two blends are the same.See...
Blended Learning “Deliverables”Most common deliverables in blended learning development (sorted by most frequently used)En...
Delivery FormatsWhat modalities are used most often to deliver blended learning coursesIn Class (face-to-face) 87%eLearnin...
Itemized Development MetricsThe next several slides provide detailed summaries of time spentacross 30 developmental tasks ...
Analysis Phase – Development Tasks“Person Hours” spent in each task, organized by course lengthBlended Learning CourseDura...
Blended Learning CourseDurationDraftingLearningObjectivesCourseOutlineContentAnalysis &Data MappingCreateDetailedDesignDoc...
Develop Phase – Development Tasks“Person Hours” spent in each task during the develop phase, organized by course lengthTot...
Implementation & Evaluation Phases – Development Tasks“Person Hours” spent in each task during implementation and Evaluati...
Summary – Total Hours of DevelopmentThis chart includes totals for the entire ADDIE process (sorted by course duration)Ble...
Development Time by Role (“person hours”)Use this information to benchmark how teams are structured for blended learning d...
Summary of FindingsKey takeaways from this research• On average, the development ratio for blended learning is 49:1, meani...
About the AuthorBryan Chapman is Chief Learning Strategist at Chapman Alliance; a provider of research-centric consulting ...
Developing Large Scale Blended Learning: Research 2013
Upcoming SlideShare
Loading in...5
×

Developing Large Scale Blended Learning: Research 2013

10,808

Published on

This study measures time spent in 30 development tasks when creating large-scale, blended learning projects (courses than can span days, weeks, or even months) using multiple modalities of delivery. Great information for planning similar development projects.

Published in: Education, Technology
1 Comment
9 Likes
Statistics
Notes
No Downloads
Views
Total Views
10,808
On Slideshare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
669
Comments
1
Likes
9
Embeds 0
No embeds

No notes for slide

Developing Large Scale Blended Learning: Research 2013

  1. 1. Large Scale, Blended LearningDevelopment: BenchmarkA Chapman Alliance, Research StudyMarch 2013By Bryan Chapman, Chief Learning Strategistbryan@chapmanalliance.comwww.chapmanalliance.comAbstract: In previous studies, we tackled development times and costs for isolated, learning-development tasks (ILT and eLearning). The purpose of this research is to analyzedevelopment of large-scale, often complex, blended learning courses with several differentdelivery formats bundled into a cohesive and unified course. We had 188 respondents whograciously shared information about their development practices including developmenttimes, costs, deliverables produced during the process, team structure and itemizedinformation about each task in their work flow. Using these data we isolated a developmentration of 49:1 (meaning that it takes 49 hours of development to create 1 finished hour ofblended leaning (“seat time”). We also discovered that the per-finished-hour standard costis $3,938 ($US Dollars). There are many variables that change the development ratio andcost that can be gleaned from the study with data you can use to benchmark your ownpractice.Now the good news! You are welcome to share this blended learning development researchwith anyone you’d like. Because we feel that this information is so important to theindustry, we have released it under Creative Commons, meaning that you can share thewhole deck or pull-out slides/charts and add them to your presentations, whitepapers, conference presentations, etc. The only restrictions are (1) you must ALWAYS listthe source of information as “Chapman Alliance” and (2) you are NOT allowed to sell theresults without written permission from Chapman Alliance. That’s it. Enjoy!Source Citation: Chapman, B. (2013). Large Scale, Blended Learning Development:Benchmark [Research Study]. Published by Chapman Alliance LLC.www.chapmanalliance.com
  2. 2. Research MethodologyHow did we collect the dataThe data contained in this research was collected bysurveying 188 organizations involved in blendedlearning development. It was a daunting survey andwe would like to thank those of you who participated,often sharing sensitive information to develop thesemetrics. The chart, above, shows the flow of thesurvey.
  3. 3. Demographics DashboardInformation about survey respondents and the blended courses represented in the resultsBasic188 – companies /organizations represented in the survey (courses)7,025,269 – total # of employees across all companies/organizations2,174,663 - # of learners reached by represented, blended coursesLocated83% - in the United States17% - outside the USBy IndustryInformation Technology/High Tech 12%Government/Military 10%Healthcare 9%Banking/Financial Services 9%Transportation/Automotive/Aviation 7%Professional Services 6%Energy 5%Industrial/Manufacturing 5%Higher Education 5%Pharmaceutical 5%Telecommunications 5%Educational Provider (sells learning content) 4%Other 3%Insurance 3%Learning Vendor 3%Association 3%Retail 2%General Business 2%NGO 2%K-12 1%Learning Content TypesJob Specific/Technical Skills 39%Softskills (Leadership, Customer Service, etc.) 23%Computer Application Training 14%New Hire Orientation 10%Other 6%Product Knowledge & Sales 6%Compliance 2%Other includes: Professional Licensure or Certificate,Mixture of Multiple Types, Academic CoursesDeveloped…Internally 72%Outsource (External) 13%Hybrid (some of both) 15%Duration of Blended Courses53 hours (6.6 days) Average duration, blendedcourses in survey24 hours (3 days) Median duration, blendedcourses in survey45 minutes to 800 hours Range of blended coursesrepresented in the surveySMEs per course4.3 Average number ofSMEs used on blendedlearning projects0 - 50 Range of the number ofSMEs listed on projects7.3 Interesting fact: average# of SMEs used onSoftware/Applicationblended Learning devprojects
  4. 4. How Long Does It Take to Create a Blended Learning Course“development time” per “finished hour” (seat time)Across all 188 courses evaluated, the overall averagedevelopment ratio came out at 49:1; in short, 49hours of development for every 1 finished hour offinal, blended course (“seat time”)Next, we’ll show you how this ratio changes based onspecific variables…per finished hourSo how long does it take to create a blended learning course? Because of thevast differences in course durations (ranging from 45 minute courses to 800hours in seat time) we synthesized the information to oneratio….development time per “finished hour” (or seat time). Here’s what wefound…* $80.37 (US Dollars)estimated hourly rate * NOTE: survey respondents listed “personhours” and overall cost. Displayed hourlyrate is derived through calculation. Inshort, respondents did NOT provide thisnumber. This is true for all “rates” shownthroughout the summary.
  5. 5. How Long Does It Take to Create a Blended Learning CourseBoil it down to “development time” per “finished hour” (seat time)We cross-tabbed the data on outsource vs.internal vs. hybrid development. We alsolooked at how the development ratio andcost changes when developing “new” contentvs. repurposing existing content.per finished hourper finished hourFactor: Outsourced(Externally Developed)- 18+ $1,158per finished hourFactor: Hybrid(mix of internal and outsource)- 12- $1,130per finished hourFactor: “New” Content(60% or more of course is “new” content)+ 20+ 2,397per finished hourFactor: Repurposing Content(60% or more existing content)- 27- 2,217
  6. 6. How does “Length of Course” change development ratios?Course length does appear to have an effect on development times and costAnother observation is that blended learning “courselength” also changed the development ratio and cost.Advance to the next slide to see the results.per finished hour
  7. 7. How does “Length of Course” change development ratios?Course length does appear to have an effect on development times and costNotice how significantly the development ratios and cost changes whencreating smaller blended learning course (less than one day in length). Weattribute this to economies of scale during development..per finished hourper finished hourBlended courses 2 to 3 days in length- 7- $743per finished hourBlended courses less the 1 day in length+ 32+ $1,629per finished hourBlend courses 4 to 5 days in length- 14- $1,273NOTE: the development ratio came out thesame as 2 to 3 day courses in the study,with a slightly higher cost. This is not atypoper finished hourBlend courses 5 to 10 days in length- 15- $1,335per finished hourBlended courses longer than 10 days- 16- $1,631
  8. 8. Using Benchmark Data for your ProjectsSimple scenario, using core metrics to create high-level estimatesSetup: you’ve been tasked to create a 5-day, 40 hour, blended learning course1. Multiply 40 (the number of hours in the course (“seat time”) by 49 (benchmark average, according tothis study for blended learning course development)2. The total is 1,960 “person hours” of overall development time, whether you are producing the courseinternally, outsourcing development, or using a hybrid approach with both internal and outsourcedevelopment.3. Next, apply cost. For default purposes, you are welcome to use the benchmark cost of $3,938(average cost of development for 188 organizations per finished hour of course time). Multiply 40(number of hours in course) by $3,938 (or any price adjustment that best matches your scenario).4. Using benchmark data only, the estimated cost of development is $157,520Remember, this is a benchmark average, meaning there are many factors that can change pricingincluding:• Your choice to produce internally or outsource• New content vs. repurposed content• Development staff pay rate (NOTE: the US dollar amounts listed in this study is an average ofthe 188 participants of this study. You’re rate of pay may have a significant impact on overallprice)• Deliverables included with the course (covered later in this study)• Development tasks that may be omitted depending on the overall blend (i.e. your blendedcourse doesn’t use eLearning)• etc.
  9. 9. Adjusted for100K or moreemployeesHow Long Does It Take to Create a Blended Learning Course“development time” per “finished hour” (seat time)Across all 188 courses evaluated, the overall averagedevelopment ratio came out at 49:1; in short, 49hours of development for every 1 finished hour offinal, blended course (“seat time”)Next, we’ll show you how this ratio changes based onspecific variables…per finished hourSo how long does it take to create a blended learning course? Because of thevast differences in course durations (ranging from 45 minute courses to 800hours in seat time) we synthesized the information to oneratio….development time per “finished hour” (or seat time). Here’s what wefound…* $76.20 (US Dollars)estimated hourly rate* NOTE: survey respondents listed “personhours” and overall cost. Displayed hourlyrate is derived through calculation. In short,respondents did NOT provide this number.This is true for all “rates” shown throughoutthe summary.
  10. 10. How Long Does It Take to Create a Blended Learning Course“development time” per “finished hour” (seat time)Filtered for respondents with 100K or more employeesAll Respondents 100K or more employeesTime Cost cost per hour Time Cost cost per hourOverall 49 $ 3,938 $ 80.37 54 $ 4,115 $ 76.20Outsourced 31 $ 5,096 $ 164.39 50 $ 5,825 $ 116.50Hybrid 37 $ 2,808 $ 75.89 41 $ 3,116 $ 76.00New Content 69 $ 4,992 $ 72.35 70 $ 5,291 $ 75.59Repurposing Content 22 $ 1,721 $ 78.23 23 $ 1,979 $ 86.041 day or less 81 $ 5,567 $ 68.73 85 $ 5,811 $ 68.362 to 3 days 42 $ 3,246 $ 77.29 43 $ 3,505 $ 81.514 to 5 days 35 $ 2,685 $ 76.71 39 $ 2,841 $ 72.856 to 10 days 34 $ 2,603 $ 76.56 34 $ 2,748 $ 80.8210 days + 33 $ 2,307 $ 69.91 31 $ 2,459 $ 79.32Adjusted for100K or moreemployees
  11. 11. Budget AllocationHow budget amounts change based on internal development, outsourcing, or using a hybrid approachInternalDevelopmentOutsourcedDevelopmentHybridDevelopmentAverage across allrespondentsInternal Labor Cost = 87%Materials = 7%Outsourced Services = 4%Other = 2%Internal Labor Cost = 9%Materials = 3%Outsourced Services = 88%Other = 0%Internal Labor Cost = 48%Materials = 6%Outsourced Services = 46%Other = 0%Internal Labor Cost = 71%Materials = 6%Outsourced Services = 21%Other = 1%
  12. 12. Lots of Different Kinds of BlendsObservation: The study picked up lots of different blends. No two blends are the same.See random sample of blends from the actual survey below (actual listings by some respondents):[add text here]ILT(face to face)VirtualClassroomeLearning Hands-on Lab Mobile onSmartphoneMobile onTabletSocialCollaborationInformalLearningOn-the-Job(OJT)Coaching/MentoringOtherRespondent 1 80% 20%Respondent 2 10% 5% 10% 5% 5% 60% 5%Respondent 3 70% 10% 10% 5% 5%Respondent 4 60% 25% 5% 5%Respondent 5 15% 15% 60% 10%Respondent 6 5% 85% 10%Respondent 7 40% 20% 20% 20%Respondent 8 60% 15% 5% 15% 5%Respondent 9 75% 25%Respondent 10 15% 65% 20%Respondent 11 30% 5% 5% 5% 10% 15% 10% 10% 10%Respondent 12 40% 30% 10% 10% 10%Respondent 13 30% 10% 30% 15% 15%Respondent 14 20% 60% 20%Respondent 15 33% 67%Respondent 16 45% 20% 20% 5% 5% 5%Respondent 17 70% 20% 10%Respondent 18 40% 40% 20%Respondent 19 50% 20% 30%Respondent 20 80% 10% 10%Respondent 21 90% 5% 5%Respondent 22 25% 5% 35% 5% 5% 20% 5%Respondent 23 50% 50%Respondent 24 40% 20% 20% 10% 10%
  13. 13. Blended Learning “Deliverables”Most common deliverables in blended learning development (sorted by most frequently used)End-of-Course Evaluations 92%Instructor Guide (s) 84%Knowledge Test(s) (online or paper-based) 83%PowerPoint Deck(s )for Visuals 79%Student/Participant Guide(s) 78%Self-Paced eLearning (s) 68%Handouts or in class Activity Workbook(s) 62%Video (s) 54%Job Aide(s) 50%Virtual Classroom Material(s) 44%Reference Manual/Text Book (s) 41%Group Participation e-learning(s) 40%Pre-Reading (s) 32%Other 24%Poster(s) 18%Wiki(s) 13%Podcast(s) 12%Blog(s) 8%Common Answers for “Other” includes: Simulations, Skill Assessments (more than just a knowledge test), Games, VirtualLabs and Lab Guides, Participatory Exercises, Certification Tests, Webinar, SharePoint Discussion Board, Social Learning Site,and Procedural Checklists)
  14. 14. Delivery FormatsWhat modalities are used most often to deliver blended learning coursesIn Class (face-to-face) 87%eLearning (self paced) 87%Virtual Classroom 58%Social Collaborative Learning 49%Coaching/Mentoring 49%Assigned On-the-Job Training (OJT) 42%Informal Learning & Knowledgebases 37%Mobile on Tablet 31%Mobile on Smartphone 28%Other 23%Common Answers for “Other” includes: Self-Paced VideoTraining, Web Portal Resources, Teleconferences, Skype
  15. 15. Itemized Development MetricsThe next several slides provide detailed summaries of time spentacross 30 developmental tasks through the ADDIE process, whencreating large-scale blended learning courses. You can use thisinformation to further adjust the variables when calculatingexpected development times and costs (i.e. if eLearning is not partof your blend, remove the total hours for eLearning developmentfrom your projections).
  16. 16. Analysis Phase – Development Tasks“Person Hours” spent in each task, organized by course lengthBlended Learning CourseDurationScoping andProjectInitiationKick OffMeetingsFront EndAnalysis andDiscoveryOtherOne day or less 23 11 37 122 - 3 days 39 12 67 124 - 5 days 48 17 107 146 - 10 days 52 27 134 20Longer than 10 days 72 41 153 24Total83 person hours130 person hours186 person hours233 person hours290 person hours* All times shown above are “person hours” spent in each task
  17. 17. Blended Learning CourseDurationDraftingLearningObjectivesCourseOutlineContentAnalysis &Data MappingCreateDetailedDesignDocumentCreateDesignProtoype(s)OtherOne day or less 16 10 22 69 34 52 - 3 days 17 19 35 80 53 74 - 5 days 22 23 61 109 117 76 - 10 days 36 38 72 129 139 10Longer than 10 days 57 59 96 176 177 54Design Phase – Development Tasks“Person Hours” spent in each task during the design phase, organized by course lengthTotal156 person hours211 person hours339 person hours424 person hours619 person hours* All times shown above are “person hours” spent in each task
  18. 18. Develop Phase – Development Tasks“Person Hours” spent in each task during the develop phase, organized by course lengthTotal524 person hours699 person hours1228 person hours2151 person hours2956 person hoursBlended Learning CourseDurationCreateInstructorGuide(s)CreateStudentGuide(s)CreatePowerPoint(classroomVisuals)CreateHandout(s)or In-ClassActivityWorkbookCreate jobAide(s)Create Pre-Reading(s)Create Self-PacedeLearningeLearningModulesfor in-classexercisesCreateVideoCreateKnowledgeTest(s)CreateCourseEvaluationCreateOtherCourseMaterialsOne day or less 41 38 39 16 23 16 165 76 50 33 15 122 - 3 days 63 53 57 18 28 17 204 96 100 35 14 144 - 5 days 172 103 79 37 57 18 409 102 140 70 16 256 - 10 days 295 109 181 51 75 30 1021 112 147 78 15 37Longer than 10 days 482 170 232 86 101 36 1312 137 167 139 23 71* All times shown above are “person hours” spent in each task
  19. 19. Implementation & Evaluation Phases – Development Tasks“Person Hours” spent in each task during implementation and Evaluation, organized by course lengthTotal221 person hours257 person hours404 person hours538 person hours894 person hoursBlended LearningCourse DurationSet UpVirtualDelivey ofLearningPublishCoursesto theLMSSMEReviewsTeamMeetingsCourseWalkthroughPilotTestingRevisionof CourseMaterialsFinalReleaseDebrief/AfterActionReportOne day or less 23 11 27 30 18 72 20 10 102 - 3 days 28 17 31 40 19 74 25 11 124 - 5 days 40 18 66 81 24 88 56 11 206 - 10 days 41 34 100 84 32 109 100 14 24Longer than 10 days 58 26 151 130 76 267 116 39 31* All times shown above are “person hours” spent in each task
  20. 20. Summary – Total Hours of DevelopmentThis chart includes totals for the entire ADDIE process (sorted by course duration)Blended Learning Course Duration Analysis Phase Design Phase Develop PhaseImplement &Evaluate PhaseOne day or less 83 156 524 2212 - 3 days 130 211 699 2574 - 5 days 186 339 1228 4046 - 10 days 233 424 2151 538Longer than 10 days 290 619 2956 894TotalDevelopmentTime984 person hours1297 person hours2157 person hours3346 person hours4759 person hoursBlended Learning CourseDuration%% Analysis % Design % Develop% Implement &EvaluateOne day or less 8% 16% 53% 22%2 - 3 days 10% 16% 54% 20%4 - 5 days 9% 16% 57% 19%6-10 days 7% 13% 64% 16%Longer than 10 days 6% 13% 62% 19%* All times shown above are “person hours” spent in each task
  21. 21. Development Time by Role (“person hours”)Use this information to benchmark how teams are structured for blended learning developmentProject Manager(s) 13%Lead Instructional Designer(s) 15%Instructional Designer(s) 19%Writer(s) 6%Graphic Artist(s) 6%Programming/Authoring Specialist(s) 14%Audio/Video 3%Quality Assurance 6%Subject Matter Expert(s) 11%Other 7%
  22. 22. Summary of FindingsKey takeaways from this research• On average, the development ratio for blended learning is 49:1, meaning ittakes 49 hours to create one finished hour of blended learning (on average)• Using the data from 188 survey respondents, the average price per finishedhour of blended learning is $3,938• Creating “new “content can increase your development ratio to 69:1• Outsourcing content takes less time (31:1), but the cost per finished hour goesup by $1,158 per finished hour• Shorter blended courses (less than 1 day in duration) increase thedevelopment ratio average (81:1)• The median duration of a blended learning course is 3-days in length (24hours), although they range in the survey from 45 minutes to 800 hours (100days)• There is NO STANDARD BLEND. There were literally 188 different blends, usinga variety of delivery formats and integrating new, emerging technologies (suchas Social, Mobile, and Informal learning)
  23. 23. About the AuthorBryan Chapman is Chief Learning Strategist at Chapman Alliance; a provider of research-centric consulting solutions that assist organizations in defining, operating and optimizingtheir strategic learning initiatives. As a veteran in the industry, he has over 20 yearsexperience and has worked with such organizations as American Express, Shell, Kodak,Sprint, Sharp Electronics, Honda, IBM, Microsoft, Avon, UNICEF, American Red Cross, TheFood and Drug Administration, U.S. State Department, and many others; to help themoptimize learning efficiency through the use of innovative learning techniques andtechnologies. Click here for a full bio.Please feel free to contact me if you need some assistance…• working through learning management system (LMS) or any other learningtechnology selection project• finding and applying learning industry research; custom research projects• optimizing or establishing your learning governance approach• exploring best practices for rapid learning development• learning strategy “jam sessions” (available as a 2-hour block, full-day onsite, or as aworkshop)• Click here for more available servicesContact information:Bryan ChapmanChief Learning StrategistChapman Alliance(801) 568-7011bryan@chapmanalliance.comwww.chapmanalliance.comTo participate in future research studies, Click here (add you name to our mailing list)
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×