Measuring the Difficulty ofRevising/Remixing OER
First Rule of Delphi Study
Slides Availablehttp://www.slideshare.net/OpenEdBlogger/alms-analysis-presentation
4R’s1.Reuse2.Revise3.Remix4.Redistribute
OER Revise/Remix                   Photo by Crisitano Betta
Why?Photo by Greg Westfall.
Categories of Difficulty1. Content2.Pedagogical3. Technical
ALMS Analysis1.Access to editing tools?2.Level of expertise required to revise  or remix?3.Meaningfully editable?4.Source-...
Rubric1. Break down each part of the ALMS Analysis into parts2. Requires some technical knowledge and training3. Apply to ...
Access                                1            2            3          4Is appropriate software         No            ...
Access Part 2                          1              2            3          4How much does the   Program cost      Progr...
Access Part 3                           1                2               3              4Does the program        Program  ...
Access Part 4                           1                2               3              4Does the program        Program  ...
Access Part 5                            1               2               3             4How much does the     Service cost...
Access Part 6                       1    2   3   4Are you able to cut-   No   -   -   Yes   N/Aand-paste the textinto a wo...
Expertise                             1                  2.5                 4What degree of text    Extensive text    Som...
Expertise Part 2                              1                     2.5                   4What degree of            Exten...
Expertise Part 2                               1                    2.5                  4What degree of              Exte...
Editable                            1             2             3            4What portion of the     No text can   Some t...
Editable continued• What portion of the audio in the OER is  editable?(Example: Audio stored independently, not embedded i...
Source Files                               1               2            3            4What portion of the       Text sourc...
Source Files continued• What portion of the source files for the video in the OER  is available?(Example: Video project fi...
CalculationsEach category subtotal is the Total score dividedby the number of Non-N/A responses      Subtotal             ...
Delphi Study
Criteria1. Reliability2. Desirability3. FeasibilityParticipants asked to rater each cell, some gavean entire row the same ...
1st Round Results             Reliability   Desirability   Feasibility   MeanAccess            2.53         2.99          ...
Comments• Concerns about cost of technical reusability.• Concerns about technical reusability being only  one of many valu...
Future Directions1. Possible automated scoring of Access   component2. Revising other components for future Delphi   study...
Photo by walknboston
Thank YouSeth Gurellsethgurell@gmail.comOn Google+ or Facebook
Alms analysis presentation
Upcoming SlideShare
Loading in …5
×

Alms analysis presentation

830 views
792 views

Published on

Slides for ALMS Framework presentation.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
830
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Welcome everyone!
  • The first rule of the Delphi study is that you do not talk about the Delphi study.
  • Although OER has been produced
  • All of these issues are known, but there has been no way to systematic measurement of magnitude.
  • Merging of 2 and 3 to 2.5.
  • Group of experts surveyed through anonymous means. Results are presented through multiple rounds with the hope of reaching some kind of predetermined point of convergence.
  • Access the lowest part of ALMS analysis, while Meaningful was highest. Not surprised by the former, but was by the latter. Reliability is low, but no distinction between Desirability and Feasibility
  • Technical reusability is not synonymous with quality
  • Alms analysis presentation

    1. 1. Measuring the Difficulty ofRevising/Remixing OER
    2. 2. First Rule of Delphi Study
    3. 3. Slides Availablehttp://www.slideshare.net/OpenEdBlogger/alms-analysis-presentation
    4. 4. 4R’s1.Reuse2.Revise3.Remix4.Redistribute
    5. 5. OER Revise/Remix Photo by Crisitano Betta
    6. 6. Why?Photo by Greg Westfall.
    7. 7. Categories of Difficulty1. Content2.Pedagogical3. Technical
    8. 8. ALMS Analysis1.Access to editing tools?2.Level of expertise required to revise or remix?3.Meaningfully editable?4.Source-file access?Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The Four R’s of Openness and ALMS Analysis:Frameworks for Open Educational Resources. Open Learning: The Journal of Open and DistanceLearning, 25(1), 37-44.
    9. 9. Rubric1. Break down each part of the ALMS Analysis into parts2. Requires some technical knowledge and training3. Apply to a variety OER
    10. 10. Access 1 2 3 4Is appropriate software No - - Yes N/Aapplication pre-installedwith the operatingsystem?Is an appropriate software Not found Found Found Foundapplication downloadable through through through throughfrom the Internet? Internet significant some Internet search research research search easily
    11. 11. Access Part 2 1 2 3 4How much does the Program cost Program Some No N/Aapplication cost? is expensive cost is program program significant cost costHow large is the File to be File to be Found Foundprogram to be downloaded is downloaded through throughdownloaded? prohibitively a significant some Internet large size research search easily
    12. 12. Access Part 3 1 2 3 4Does the program Program Program Program Program N/Adepend on other depends on a depends on depends on does notsoftware or large amount a significant some depend onprogramming of libraries or amount of libraries or anylibraries? other libraries or other libraries or programs other programs files programsIs appropriate No Access with Access with Accessfunctionality significant some with noavailable from a restrictions restrictions restrictionsweb-based service? or sign-up only
    13. 13. Access Part 4 1 2 3 4Does the program Program Program Program Program N/Adepend on other depends on a depends on depends on does notsoftware or large amount a significant some depend onprogramming of libraries or amount of libraries or anylibraries? other libraries or other libraries or programs other programs files programsIs appropriate No Access with Access with Accessfunctionality significant some with noavailable from a restrictions restrictions restrictionsweb-based service? or sign-up only
    14. 14. Access Part 5 1 2 3 4How much does the Service cost is Service cost Some No service N/Aservice cost? expensive is significant service cost costDoes the service Service Service Service Servicerequire the user to requires requires an requires an does nothave an account? extensive account account require an account with with a with some account major privacy significant or no concerns amount of information information
    15. 15. Access Part 6 1 2 3 4Are you able to cut- No - - Yes N/Aand-paste the textinto a wordprocessor or texteditor?
    16. 16. Expertise 1 2.5 4What degree of text Extensive text Some text editing Minimal text N/Aediting expertise is editing expertise required editingrequired expertise (Example: footnotes, expertise required headers) required (Example: (Example: multi-column bolding, layout) headings)What degree of Extensive Some image editing Minimalimage editing image editing expertise required imageexpertise is expertise (Example: multiple editingrequired? required layers, filters) expertise (Example: required masking, clone (Example: stamp) cropping, brightness/ contrast)
    17. 17. Expertise Part 2 1 2.5 4What degree of Extensive Some audio editing Minimal N/Aaudio editing audio editing expertise required audio editingexpertise is required expertise (Example: audio filters, expertise required transitions) required (Example: (Example: remixing/revisi trimming and ng multiple saving in sources) appropriate format)What degree of Extensive Some video editing Minimalvideo editing video editing expertise required video editingexpertise is expertise (Example: transitions, expertiserequired? required filters) required (Example: (Example: remixing/revisi trimming and ng multiple saving in sources) appropriate format)
    18. 18. Expertise Part 2 1 2.5 4What degree of Extensive Some video editing Minimal video N/Aprogramming video editing expertise required editingexpertise is expertise (Example: functions, expertiserequired? required if/then statements) required (Example: data (Example: structures such variable as Object- manipulation, Oriented basic programming) arithmetic)What degree of Extensive Some video editing Minimal videospecialty expertise is specialty expertise required editingrequired? expertise (Example: intermediate expertise required animation or 3D required (Example: modeling) (Example: Advanced basic animation or animation or 3D modeling) 3D modeling)
    19. 19. Editable 1 2 3 4What portion of the No text can Some text A significant All of the N/Atext in the OER is be edited can be amount of text can beeditable? edited text can be edited(Example: Text stored editedindependently, notembedded in an image)What portion of the No images Some A significant All of theimages in the OER is can be images can amount of images caneditable? edited be edited the images be edited(Example: Images can bestored independently, editednot embedded in aPDF)
    20. 20. Editable continued• What portion of the audio in the OER is editable?(Example: Audio stored independently, not embedded in a video)• What portion of the video in the OER is editable?(Example: Video stored independently, not embedded in a Flash object)• What portion of the other components in the OER is editable?(Example: 3D Model)
    21. 21. Source Files 1 2 3 4What portion of the Text source Some text A significant All text N/Asource files for the files are not source files amount of source filestext in the OER is available are available the text areavailable? source files available(Example: a Word are availabledocument available fora PDF)What portion of the Image Some image A significant All imagesource files for the source files source files amount of source filesimages in the OER is are not are available the image areavailable? (Example: a available source files availablePhotoshop file with all are availablelayers available)
    22. 22. Source Files continued• What portion of the source files for the video in the OER is available?(Example: Video project files)• What portion of the source files for the programming code in the OER is available?(Example: Source code downloadable from a website)• What portion of the source files for the other components of the OER is available?(Example: FLA files for SWF Flash objects)
    23. 23. CalculationsEach category subtotal is the Total score dividedby the number of Non-N/A responses Subtotal Score Access to Editing Tools 3.75 Level of Expertise 3 Meaningfully Editable 4 Source Files Available 4 Average 3.69
    24. 24. Delphi Study
    25. 25. Criteria1. Reliability2. Desirability3. FeasibilityParticipants asked to rater each cell, some gavean entire row the same score.
    26. 26. 1st Round Results Reliability Desirability Feasibility MeanAccess 2.53 2.99 3.08 2.87Level 2.97 3.09 2.93 3Meaningful 3.17 3.41 3.35 3.31Source 3.07 3.18 3.31 3.19Total 2.93 3.17 3.17 3.09
    27. 27. Comments• Concerns about cost of technical reusability.• Concerns about technical reusability being only one of many values.• Concerns about properties of the OER itself versus the context in which reuse is occurring.
    28. 28. Future Directions1. Possible automated scoring of Access component2. Revising other components for future Delphi study rounds3. Rating a sample of OER for inter-rater reliability
    29. 29. Photo by walknboston
    30. 30. Thank YouSeth Gurellsethgurell@gmail.comOn Google+ or Facebook

    ×