Alternatives for the institutional implementation of e-learning: Lessons from 12 years of Webfuse

3,070 views

Published on

The practice of e-learning in universities suffers from a number of unquestioned perspectives that limit outcomes. This presentation describes a framework for understanding the full diversity of alternate perspectives and examines one successful set of perspectives arising out of 12+ years of designing, supporting and competing with the Webfuse system.

More information here - http://tr.im/E9JE

Published in: Education, Technology, Sports
  • Be the first to comment

  • Be the first to like this

Alternatives for the institutional implementation of e-learning: Lessons from 12 years of Webfuse

  1. 1. Alternatives for the institutional implementation of e-learning: Lessons from 12 years of Webfuse<br />David Joneshttp://tr.im/E9JE<br />
  2. 2. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  3. 3. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  4. 4. Culture always pays a price<br />Always winners and losers<br />Every tech. has a philosophy<br />It’s ecological, not additive<br />Media tends to become mythic<br />5 things to know about technological change<br />http://tr.im/E9KW<br />http://www.flickr.com/photos/tom1231/261029677/<br />
  5. 5. its capacity for good or evilrests entirely on human awarenessof what it does for us and to us<br />(Postman, 1998)<br />http://www.flickr.com/photos/lenore-m/2599969114/<br />
  6. 6. (Postman, 1998)<br />When a technology becomesmythic, it is always dangerous<br />because<br />it is then accepted as it is<br />and is there not easily susceptible to modification or control<br />http://www.flickr.com/photos/atoach/3543601334/<br />
  7. 7. almost every university is <br />planning to make use of an LMS<br />(Salmon, 2005)<br />(Feldstein, 2006)<br />the mindset in recent times has focused on the adoption of the one-size-fits-all LMS<br />http://www.flickr.com/photos/riverap1/3875718119/<br />
  8. 8. Awareness raising<br />http://www.flickr.com/photos/purpleslog/2881603057/<br />
  9. 9. http://www.civil.usyd.edu.au/current/undergraduate/learning.shtml<br />
  10. 10. Common sense is nothing more than <br />a deposit of prejudices laid down by <br />the mind before you reach eighteen. <br />Albert Einstein<br />http://flickr.com/photos/skoop/207145876/<br />
  11. 11. (Papert, 1995)<br />The structure of School is so deeply rooted that one reacts to deviations from it as one would to a grammatically deviant utterance:<br />Both feel wrong on a level deeperthan one’s ability to formulate reasons<br />http://www.flickr.com/photos/mccloud/145800121/<br />
  12. 12. There is no silver bullet<br />http://www.flickr.com/photos/gregoryjameswalsh/408773662/<br />
  13. 13. Awareness raising<br />http://www.flickr.com/photos/purpleslog/2881603057/<br />
  14. 14. As you solve the problemyou learn more about it<br />Wicked problems<br />(Rittel & Webber, 1973)<br />Solutions are neitherright nor wrong<br />Every one isessentiallyunique<br />There is no ultimate test of a solution<br />The problem can be described in many ways<br />You can’t identifyall potential solns.<br />There is no stopping rule<br />http://www.flickr.com/photos/wxmom/2052715752/<br />
  15. 15. http://www.flickr.com/photos/cleopold73/3677296594/<br />
  16. 16. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  17. 17. Nothing is so practical as a good theory<br />(Lewin, 1945)<br />http://www.flickr.com/photos/themadlolscientist/2543882652/<br />
  18. 18. Theories are practical because they allow knowledgeto be accumulated in a systematic manner and thisaccumulated knowledge enlightens professional practice<br />(Gregor & Jones, 2007)<br />http://flickr.com/photos/dalem/296189944/<br />
  19. 19. Raising awareness<br />http://www.flickr.com/photos/purpleslog/2881603057/<br />
  20. 20. The Ps Framework<br />Past experience<br />People<br />Frameworks offer new ways of looking at phenomena and provide information on which to base sound, pragmatic decisions <br />Place<br />Purpose<br />(Mishra & Koehler, 2006)<br />Product<br />Process<br />Pedagogy<br />http://flickr.com/photos/joits/225824796/<br />
  21. 21. The Ps Framework<br />PastExperience<br />People<br />Product<br />Purpose<br />Process<br />Place<br />Pedagogy<br />(Jones, Vallack, et al, 2008)<br />http://flickr.com/photos/joits/225824796/<br />
  22. 22. Raising awareness<br />http://www.flickr.com/photos/purpleslog/2881603057/<br />
  23. 23. The Mythic Technology<br />Product<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  24. 24. Context<br />PastExperience<br />People<br />Purpose<br />Place<br />Pedagogy<br />http://flickr.com/photos/joits/225824796/<br />
  25. 25. The alternative<br />Product<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  26. 26. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  27. 27. The Mythic Technology<br />Product<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  28. 28. The Mythic Technology<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  29. 29. Big up front design (BUFD)<br />(Truex, Baskerville and Klein, 1999)<br />Analysis & Design<br />Replacement<br />Cost<br />Maintenance<br />Time<br />http://www.flickr.com/photos/mauricedb/2706292588/<br />
  30. 30. Herding cats<br />Identify the goal<br />Make sure we have the right herders and herding equipment<br />Get the cats to the goal (and keep them there)<br />http://www.flickr.com/photos/australianshepherds/3593274951/<br />
  31. 31. Alternatives<br />http://www.flickr.com/photos/australianshepherds/3593274951/<br />
  32. 32. Limitations<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />Designer is able to manipulate system behaviour<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  33. 33. The Mythic Technology<br />Product<br />http://flickr.com/photos/joits/225824796/<br />
  34. 34. LMS is academic equivalent to ERP<br />(Morgan, 2003)<br />Reflects a …. move to standardisation<br />Will impose its own logic on a company’s strategy, structureand culture…push towards..generic structures<br />(Davenport, 1998)<br />Best practice..implement vanilla<br />(Robey, Ross et al, 2002; Pozzebon, Titah et al, 2006; Wagner, Scott et al, 2006)<br />http://flickr.com/photos/remedix/260741805/<br />
  35. 35. Identifying efficient procurement strategies<br />(Saarinen & Vepsalainen, 1994)<br />Internal<br />Development<br />SoftwareContracting<br />PackageAcquisition<br />http://www.flickr.com/photos/yourdon/3030229649/<br />
  36. 36. (Saarinen & Vepsalainen, 1994)<br />Type of Information System<br />Routine<br />Standard<br />Speculative<br />HighCost<br />Internal<br />Development<br />Innovator<br />Type of Developer<br />SoftwareContracting<br />Analyst<br />PackageAcquisition<br />Implementor<br />HighRisk<br />http://www.flickr.com/photos/yourdon/3030229649/<br />
  37. 37. (Saarinen & Vepsalainen, 1994)<br />Specificity of Design<br />Specific/Contingent<br />Common<br />Risky/Complex<br />HighCost<br />SpeculativeInvestment<br />Uncertainty of requirements<br />StandardApplication<br />RoutineSystem<br />Certain/Simple<br />HighRisk<br />http://www.flickr.com/photos/yourdon/3030229649/<br />
  38. 38. http://www.flickr.com/photos/tom1231/275402356/<br />
  39. 39. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  40. 40. Context<br />PastExperience<br />People<br />Purpose<br />Place<br />Pedagogy<br />http://flickr.com/photos/joits/225824796/<br />
  41. 41. Limitations<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />Designer is able to manipulate system behaviour<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  42. 42. Limitations<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />Designer is able to manipulate system behaviour<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  43. 43. (Saarinen & Vepsalainen, 1994)<br />Specificity of Design<br />Specific/Contingent<br />Common<br />Risky/Complex<br />HighCost<br />E-learningContext<br />SpeculativeInvestment<br />Uncertainty of requirements<br />StandardApplication<br />LMSProduct<br />RoutineSystem<br />Certain/Simple<br />HighRisk<br />http://www.flickr.com/photos/yourdon/3030229649/<br />
  44. 44. Context<br />PastExperience<br />http://flickr.com/photos/joits/225824796/<br />
  45. 45. Quality and Quantity e-learning<br />Fairly unsophisticated use of the tools<br />Substitutes for existingmedia and retainsexisting pedagogy<br />(Benson and Palaskas, 2006)<br />(Salmon, 2005)<br />Formal use of technology…bestdescribed as sporadic, unevenand often low-level<br />(Selwyn, 2007)<br />http://www.flickr.com/photos/dok1/3921218578/<br />
  46. 46. Level 1: Most used<br />Level 2: Moderately used<br />Level 3: Rarely used <br />More than 50% of courses<br />More than 20% of courses<br />(much) less than 20% of courses<br />CreatingClassInteractions<br />EvaluatingStudents<br />TransmittingContent<br /><ul><li> Chat
  47. 47. Discussion forum
  48. 48. Email
  49. 49. Quizzes
  50. 50. Drop box
  51. 51. Content/files
  52. 52. Announcements
  53. 53. Gradebook</li></ul>ComputerBasedInstruction<br />Evaluatingcourse and instructors<br /><ul><li> Some quizzes
  54. 54. Adaptive release
  55. 55. Surveys</li></ul>Move to features in unexplored categories, until each category is considered for different learning needs<br />(Malikowski, 2007)<br />http://www.flickr.com/photos/dok1/3921218578/<br />
  56. 56. Technology-mediated learning<br />(Pressy, 1926)<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1958)<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  57. 57. Technology-mediated learning<br />(Pressy, 1926)<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  58. 58. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  59. 59. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  60. 60. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  61. 61. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  62. 62. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Multimedia<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  63. 63. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Multimedia<br />Multimedia personal computers<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  64. 64. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Internet<br />Multimedia<br />Multimedia personal computers<br />Lone-ranger<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  65. 65. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Internet<br />Multimedia<br />Multimedia personal computers<br />Lone-ranger<br />WWW<br />Cottage industry<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  66. 66. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Internet<br />Multimedia<br />Multimedia personal computers<br />Lone-ranger<br />WWW<br />Cottage industry<br />ERPs<br />Industrial<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  67. 67. Technology spark<br />Motion pictures<br />Radio<br />Stereo-graphs, photos, motionpictures, radio, television<br />(Pressy, 1926)<br />Industrial revolution,mechanical machines<br />Audio-visual instruction<br />Teaching machines<br />(Skinner, 1956)<br />Computers<br />Networks<br />CBL/CML<br />CMC<br />Personal computers<br />Internet<br />Multimedia<br />Multimedia personal computers<br />Lone-ranger<br />WWW<br />Cottage industry<br />ERPs<br />Industrial<br />Web 2.0<br />Post-industrial<br />http://www.flickr.com/photos/extraketchup/459020985/<br />
  68. 68. Limitations<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />Designer is able to manipulate system behaviour<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  69. 69. Fad cycle<br />1. Technological spark<br />4. Resolution of dissonance<br />2. Growing revolution<br />3. Minimial impact<br />http://www.flickr.com/photos/moriza/308483890/<br />
  70. 70. Context<br />Place<br />http://flickr.com/photos/joits/225824796/<br />
  71. 71. Institutions that cannot continually change to keep up with the needs of the transforming economy will become irrelevant <br />(Klor de Alva, 2000)<br />(Agee, Yang et al, 2009)<br />The trend toward augmenting instruction with technology creates opportunities and substantial challenges for those who must respond to increasingly diverse and fluid instructional environments<br />http://www.flickr.com/photos/joshua/1154118/<br />
  72. 72. Limitations<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />Designer is able to manipulate system behaviour<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  73. 73. Assumption of Newtonian physics<br />Order<br />the great narrative of science shares with the great religious narratives the idea that there is order to the universe <br />(Postman, 1995)<br />Dominant ideology in organizationsand management science<br />(Snowden, 2005)<br />http://www.flickr.com/photos/r-z/1573332373/<br />
  74. 74. Cynefin Framework<br />(Snowden and Boone, 2007)<br />System and agents co-evolve<br />Knowable<br /><ul><li>Cause and effect separated over time and space
  75. 75. Analytical/Reductionist
  76. 76. Scenarios planning
  77. 77. Systems thinking
  78. 78. Sense-Analyse-Respond</li></ul>Complex<br />- Cause and effect are only coherent in retrospect and do not repeat<br /><ul><li> Pattern management
  79. 79. Perspective filters
  80. 80. Complex adaptive systems
  81. 81. Probe-Sense-Respond</li></ul>System constrains agents<br />- No cause and effect relationships perceivable<br /><ul><li> Stability-focused intervention
  82. 82. Enactment tools
  83. 83. Crisis management
  84. 84. Act-Sense-Respond
  85. 85. Cause and effect relationships repeatable, perceivable and predictable
  86. 86. Legitimate best practice
  87. 87. Standard operating procedures
  88. 88. Process reengineering
  89. 89. Sense-Categorise-Respond</li></ul>Chaos<br />Known<br />No constraints<br />http://www.flickr.com/photos/nov03/3663469686/<br />
  90. 90. Limitations<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />Designer is able to manipulate system behaviour<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  91. 91. Context<br />People<br />http://flickr.com/photos/joits/225824796/<br />
  92. 92. (Moore, 2002)<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  93. 93. Barriers to bridging the chasm<br />(Geoghegan, 1994)<br />arguments<br />1. Ignorance of the gap<br />modes of support<br />Homogeneity<br />Incentives<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  94. 94. Barriers to bridging the chasm<br />(Geoghegan, 1994)<br />arguments<br />1. Ignorance of the gap<br />modes of support<br />Homogeneity<br />Incentives<br />2. The technologists alliance<br />Early adopters<br />IT Staff<br />Vendors<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  95. 95. Barriers to bridging the chasm<br />(Geoghegan, 1994)<br />arguments<br />1. Ignorance of the gap<br />modes of support<br />Homogeneity<br />Incentives<br />2. The technologists alliance<br />Early adopters<br />IT Staff<br />Vendors<br />3. Alienation of the mainstream<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  96. 96. Barriers to bridging the chasm<br />(Geoghegan, 1994)<br />arguments<br />1. Ignorance of the gap<br />modes of support<br />Homogeneity<br />Incentives<br />2. The technologists alliance<br />Early adopters<br />IT Staff<br />Vendors<br />3. Alienation of the mainstream<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  97. 97. Barriers to bridging the chasm<br />(Geoghegan, 1994)<br />arguments<br />1. Ignorance of the gap<br />modes of support<br />Homogeneity<br />Incentives<br />2. The technologists alliance<br />Early adopters<br />IT Staff<br />Vendors<br />An application…that offers value substantiallyin excess of the costs of adoption<br />3. Alienation of the mainstream<br />4. Lack of a compelling reason to adopt<br />http://en.wikipedia.org/wiki/File:Technology-Adoption-Lifecycle.png<br />http://www.flickr.com/photos/nicmcphee/345850764/<br />
  98. 98. More time teaching is a negative influence on academic pay<br />The trend is worsening most rapidlyin institutions whose central missionfocuses on teaching and learning<br />(Fairweather, 2005)<br />http://www.flickr.com/photos/sylvar/1138341328/<br />
  99. 99. experience a loss of functionality and a degradation of performance as employees grapple to come to terms with the new technologies and processes that these systems force<br />(Katz, 2003)<br />
  100. 100. http://www.flickr.com/photos/knezovjb/2822498715/<br />
  101. 101. http://www.flickr.com/photos/kennymiller/796971032/<br />
  102. 102. Limitations<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  103. 103. People aren’t rational<br />Bounded rationality<br />(Simon, 1991)<br />Reliance on intuition, instinctsand simply heuristics<br />37 cognitive biases<br />(Arnott, 2006)<br />(Jamieson & Hyland, 2006)<br />Systematic biases influence judgment<br />(Tversky and Kahneman, 1974<br />Inherent limits in organisational substantive rationality<br />Innovation and change within universitiescan never be mere rational processes<br />(Cecez-Kecmanovic, et al, 2002)<br />(Jones and O’Shea, 2004)<br />http://www.flickr.com/photos/joelogon/3132985694/<br />
  104. 104. Limitations<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  105. 105. Context<br />Pedagogy<br />http://flickr.com/photos/joits/225824796/<br />
  106. 106. Application of webto teaching andlearning is stillevolving<br />(Bates, 2004)<br />Research into how people learnonline is in its infancy<br />(Herrington et al, 2005)<br />
  107. 107. Limitations<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />?<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  108. 108. http://www.tpack.org/<br />there is no single technological solution that applies for every teacher, every course, or every view of teaching <br />(Mischra & Koehler, 2006)<br />Productive technology integration in teaching needs to consider all three issues not in isolation, but rather within the complex relationships in thesystem defined by the three key elements <br />
  109. 109. Limitations<br />?<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />?<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  110. 110. http://www.gse.harvard.edu/~dedech/<br />
  111. 111. Limitations<br />?<br />?<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />?<br />?<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  112. 112. Context<br />Purpose<br />http://flickr.com/photos/joits/225824796/<br />
  113. 113. Purpose of universities<br />(Martin and Etzkowitz, 2000)<br />Immaculate<br />Utilitarian<br />(Shaul, 1972)<br />Education either functions as<br />becomes the ‘practice of freedom’, the means by which men and women deal critically with reality and discover how to participate in the transformation of their world. <br />an instrument which isused to facilitate the integration of generations into the logic of the present system and bring about conformity<br />http://www.flickr.com/photos/kylemackenzie/3576131493/<br />
  114. 114. Drivers for e-learning<br />(Coates et al, 2005)<br />Efficiency<br />Enriched student learning<br />New student expectations<br />Control<br />Competitive pressure<br />Responding to massification<br />http://www.flickr.com/photos/kylemackenzie/3576131493/<br />
  115. 115. Limitations<br />?<br />?<br />?<br />?<br />?<br />(Introna, 1996)<br />System behaviour is relatively stable and predictable<br />?<br />?<br />?<br />Designer is able to manipulate system behaviour<br />?<br />?<br />?<br />?<br />?<br />Designer able to accurately determine goals or criteria for success<br />http://www.flickr.com/photos/mwichary/2188958154/<br />
  116. 116. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  117. 117. The alternative<br />Product<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  118. 118. CQUniversity<br />1997-2010<br />2900+ Course sites<br />47000+ students<br />2500+ staff<br />90+% staff adoption<br />Web<br />http://www.flickr.com/photos/tonythemisfit/2718279207/<br />
  119. 119. The alternative<br />Principles<br />Product<br />Practice<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  120. 120. The alternative<br />Principles<br />Product<br />Practice<br />http://flickr.com/photos/joits/225824796/<br />
  121. 121. Best of breed<br />Adoption of different packages from different vendors or sources<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  122. 122. Skeleton + data = course site<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  123. 123. Different skeletons, different structure, different look<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  124. 124. Each page can be edited<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  125. 125. Each page can be edited<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  126. 126. PageType<br />Edit<br />Content<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  127. 127. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  128. 128. PageType<br />OtherServices<br />Content<br />Style<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  129. 129. Other services<br />External<br />Internal<br />Discussion forums<br />Student records system<br />Chat rooms<br />Online assignment submission<br />Social media<br />…..<br />Timetable system<br />…..<br />750+ classes<br />Web apps<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  130. 130. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  131. 131. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  132. 132. PortalWeb Interface<br />Ancillary services<br />e.g. virtual union& library<br />LMS(Blackboard)<br />LMS(Webfuse)<br />Simulations &other teaching environments<br />Shared education services<br />Monitoring performance & usage<br />Class & CourseManagement<br />Content management<br />Student profile& history<br />Assessment<br />Calendar<br />Timetable<br />Assignment tracking<br />Plagiarism detection<br />University wide services<br />Security authorisations<br />Student administration<br />axapta<br />Course administration<br />Single sign-on<br />Library management<br />Digital rights management<br />Digital repositories<br />Student profiles<br />Staff database<br />Finance database<br />Course profiles<br />Digital library &eReserve<br />Student Database<br />Learning content database<br />Library catalogue<br />Communication networks &distribution infrastructure<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  133. 133. PortalWeb Interface<br />Ancillary services<br />e.g. virtual union& library<br />LMS(Blackboard)<br />LMS(Webfuse)<br />Simulations &other teaching environments<br />Shared education services<br />Monitoring performance & usage<br />Class & CourseManagement<br />Content management<br />Student profile& history<br />Assessment<br />Calendar<br />Timetable<br />Assignment tracking<br />Plagiarism detection<br />University wide services<br />Security authorisations<br />Student administration<br />axapta<br />Course administration<br />Single sign-on<br />Library management<br />Digital rights management<br />Digital repositories<br />Student profiles<br />Staff database<br />Finance database<br />Course profiles<br />Digital library &eReserve<br />Student Database<br />Learning content database<br />Library catalogue<br />Review of grade<br />Gradebook<br />Plagiarism Tracking System<br />Faculty websites<br />ESOS compliance<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  134. 134. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  135. 135. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  136. 136. E-learning<br />Academics<br />Current practice<br />Current practice<br />Current practice<br />http://www.flickr.com/photos/cthe/2402577463/<br />
  137. 137. Academics<br />E-learning<br />Webfuse<br />Current practice<br />Current practice<br />Current practice<br />http://www.flickr.com/photos/cthe/2402577463/<br />
  138. 138. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  139. 139. we should change people&apos;s behaviourbecause information technology systems are difficult to change..<br />(Sturgess and Nouwens, 2004)<br />most commercial systems require the institution to modify its existing practices to accommodate the system<br />(Dodds, 2007)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  140. 140. Some folk unhappy with mailing lists<br />Some happy<br />Both/and NOT either/or<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  141. 141. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  142. 142. Staff choose which one they use<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  143. 143. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  144. 144. What’s the biggest discussion forum market?<br />A) Blackboard<br />B)Moodle<br />C) Sakai<br />D) Web/open source<br />http://www.flickr.com/photos/mckaysavage/3059483658/<br />
  145. 145. 27 March, 2009<br />http://www.flickr.com/photos/mckaysavage/3059483658/<br />
  146. 146. 30 October 2009<br />http://www.flickr.com/photos/mckaysavage/3059483658/<br />
  147. 147. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  148. 148. Web 2.0 course site<br />http://webfuse.cqu.edu.au/Courses/EDED11448/<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  149. 149. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  150. 150. Wordpress blog<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  151. 151. Wetpaint wiki<br />
  152. 152. http://redbubble.com.au<br />
  153. 153. http://redbubble.com.au<br />
  154. 154. http://del.icio.us<br />http://del.icio.us +http://feedburner.com<br />
  155. 155.
  156. 156. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  157. 157. 45 groups<br />6 discussion forums per group<br />270 forums<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  158. 158. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  159. 159. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  160. 160. (Light, Holland and Wills, 2001)<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  161. 161. (Light, Holland and Wills, 2001)<br />Apache<br />Perl<br />MySQL<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  162. 162. People<br />http://www.flickr.com/photos/sreejithk2000/2385193167/<br />
  163. 163. Learning and teaching<br />Not a technology problem<br />http://www.flickr.com/photos/wwworks/2970929574/<br />
  164. 164. The alternative<br />Principles<br />Practice<br />Process<br />http://flickr.com/photos/joits/225824796/<br />
  165. 165. Alternatives<br />
  166. 166.
  167. 167. (Introna, 1996)<br />(Jones, Danaher et al, 2005; Jones & Muldoon, 2007)<br />http://www.flickr.com/photos/spursfan_ace/2328879637/<br />
  168. 168. http://www.flickr.com/photos/spursfan_ace/2328879637/<br />
  169. 169. Big up front design (BUFD)<br />(Truex, Baskerville and Klein, 1999)<br />Analysis & Design<br />Replacement<br />Cost<br />Maintenance<br />Time<br />http://www.flickr.com/photos/mauricedb/2706292588/<br />
  170. 170. experience a loss of functionality and a degradation of performance as employees grapple to come to terms with the new technologies and processes that these systems force<br />(Katz, 2003)<br />
  171. 171. Cost<br />Time<br />http://www.flickr.com/photos/mauricedb/2706292588/<br />
  172. 172. Both/and NOT either/or<br />Mailing lists<br />Discussion forums<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  173. 173. http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  174. 174. Staff choose which one they use<br />http://www.flickr.com/photos/lyon_photography/3886352249/<br />
  175. 175. http://www.flickr.com/photos/spursfan_ace/2328879637/<br />
  176. 176. Committee<br />Developers<br />IT Director<br />ManagerDevelopers<br />ManagerHelpdesk<br />Dean<br />HoS<br />Helpdesk<br />Academics<br />http://www.flickr.com/photos/wetsun/53648963/<br />
  177. 177. Committee<br />Developers<br />IT Director<br />ManagerDevelopers<br />Business Analyst<br />ManagerHelpdesk<br />Dean<br />HoS<br />Helpdesk<br />Academics<br />http://www.flickr.com/photos/wetsun/53648963/<br />
  178. 178. Committee<br />Developers<br />Academics<br />“Helpdesk”<br />http://www.flickr.com/photos/bribri/126039806/<br />
  179. 179. A university is defined by thequality of its academic conversations,not by the technologies that service them.<br />(Laurillard, 2002)<br />http://www.flickr.com/photos/ktylerconk/3045268759/<br />
  180. 180. PEBKAC<br />C1234567x<br />Q12345678<br />S0123456<br />http://www.flickr.com/photos/tea610/3995308486/<br />
  181. 181. it is not the provision of features but<br />their uptake and use that really determines their educational value.<br />(Coates, 2005)<br />http://www.flickr.com/photos/spursfan_ace/2328879637/<br />
  182. 182. 3 levels of improving L&T<br />What management does.<br />What the teacher is.<br />What the teacher does.<br />http://www.flickr.com/photos/http2007/2788610963/<br />
  183. 183. Change agent effort<br />InnovationDecision<br />Rate ofAdoption<br />PerceivedAttributes<br />SocialSystem<br />CommsChannels<br />(Jones, Jamieson & Clark, 2003)<br />http://www.flickr.com/photos/airosan/2318527358/<br />
  184. 184. If we want to create passionate users, we have to help them get better.  <br />Nobody’s passionate about things they suck at. <br />Kathy Sierra<br />http://www.youtube.com/watch?v=Str2K98JnMc<br />http://www.flickr.com/photos/pressthebuttononthetop/277470840/<br />
  185. 185. http://www.thefuntheory.com<br />http://www.youtube.com/watch?v=2lXh2n0aPyw<br />
  186. 186. http://www.thefuntheory.com<br />http://www.youtube.com/watch?v=2lXh2n0aPyw<br />
  187. 187. Overview<br />The argument<br />Ps Framework<br />Mythic e-learning<br />The context<br />An alternative<br />Reflections and conclusions<br />http://www.flickr.com/photos/sookie/100608625/<br />
  188. 188. http://www.flickr.com/photos/14696088@N04/1606128341/<br />47000+ students<br />2500+ staff<br />Differentiation<br />Dead next year<br />
  189. 189. http://www.flickr.com/photos/neubie/2273635564/<br />
  190. 190. ..the precedent of other IT systems…suggests that it would be extremelyuser friendly for people with very limited competence/confidence<br />
  191. 191. The key to easy use of OASIS isthat it is not a off the shelf productthat is soooo generic that it has lostits way as a course delivery tool<br />
  192. 192. Level 1: Most used<br />Level 2: Moderately used<br />Level 3: Rarely used <br />More than 50% of courses<br />More than 20% of courses<br />(much) less than 20% of courses<br />CreatingClassInteractions<br />EvaluatingStudents<br />TransmittingContent<br /><ul><li> Chat
  193. 193. Discussion forum
  194. 194. Email
  195. 195. Quizzes
  196. 196. Drop box
  197. 197. Content/files
  198. 198. Announcements
  199. 199. Gradebook</li></ul>ComputerBasedInstruction<br />Evaluatingcourse and instructors<br /><ul><li> Some quizzes
  200. 200. Adaptive release
  201. 201. Surveys</li></ul>Move to features in unexplored categories, until each category is considered for different learning needs<br />(Malikowski, et al, 2007)<br />http://www.flickr.com/photos/dok1/3921218578/<br />
  202. 202. http://www.flickr.com/photos/dok1/3921218578/<br />
  203. 203. http://www.flickr.com/photos/dok1/3921218578/<br />
  204. 204. http://www.flickr.com/photos/dok1/3921218578/<br />
  205. 205. http://www.flickr.com/photos/dok1/3921218578/<br />
  206. 206. http://indicatorsproject.wordpress.com<br />Enabling comparisons of LMS usageacross institutions, platforms and time<br />(Beer, Jones & Clark, 2009)<br />
  207. 207. As you solve the problemyou learn more about it<br />Wicked problems<br />(Rittel & Webber, 1973)<br />Solutions are neitherright nor wrong<br />Every one isessentiallyunique<br />There is no ultimate test of a solution<br />The problem can be described in many ways<br />You can’t identifyall potential solns.<br />There is no stopping rule<br />http://www.flickr.com/photos/wxmom/2052715752/<br />
  208. 208. The Ps Framework<br />PastExperience<br />People<br />Product<br />Purpose<br />Process<br />Place<br />Pedagogy<br />http://flickr.com/photos/joits/225824796/<br />
  209. 209. http://www.flickr.com/photos/cleopold73/3677296594/<br />
  210. 210. There is nothing so useless as doing efficiently that which should not be done at all.<br />Peter Drucker<br />http://www.flickr.com/photos/joeshlabotnik/483769089/<br />
  211. 211. http://tr.im/E9JE<br />http://www.flickr.com/photos/eleaf/2536358399/<br />
  212. 212. Agee, A., C. Yang, et al. (2009). &quot;Top-Ten IT Issues, 2009.&quot; EDUCAUSE Review44(4): 44-59.<br />Arnott, D. (2006). &quot;Cognitive biases and decision support systems development: a design science approach.&quot; Information Systems Journal16: 55-78.<br />Bates, T. (2004). The promise and myths of e-learning in post-secondary education. The Network Society: A Cross-cultural Perspective. M. Castells. Cheltenham, UK, Edward Elgar: 271-292.<br />Beer, C., D. Jones, et al. (2009). The indicators project identifying effective learning, adoption, activity, grades and external factors. ASCILITE&apos;2009. Auckland, NZ.<br />Benson, R. and T. Palaskas (2006). &quot;Introducing a new learning management system: An institutional case study.&quot; Australasian Journal of Educational Technology22(4): 548-567.<br />Boehm, B. and R. Turner (2003). &quot;Using risk to balance agile and plan-driven methods.&quot; Computer36(6): 57-66.<br />Brews, P. and M. Hunt (1999). &quot;Learning to plan and planning to learn: Resolving the planning school/learning school debate.&quot; Strategic Management20(10): 889-913.<br />Cecez-Kecmanovic, D., M. Janson, et al. (2002). &quot;The rationality framework for a critical study of information systems.&quot; Journal of Information Technology 17: 215-227.<br />Coates, H., R. James, et al. (2005). &quot;A Critical Examination of the Effects of Learning Management Systems on University Teaching and Learning.&quot; Tertiary Education and Management11(1): 19-36.<br />Davenport, T. (1998). &quot;Putting the Enterprise into the Enterprise System.&quot; Harvard Business Review76(4): 121-131.<br />Dodds, T. (2007). &quot;Information Technology: A Contributor to Innovation in Higher Education.&quot; New Directions for Higher Education2007(137): 85-95.<br />Fairweather, J. (2005). &quot;Beyond the rhetoric: Trends in the relative value of teaching and research in faculty salaries.&quot; Journal of Higher Education76(4): 401-422.<br />Feldstein, M. (2006). Unbolting the chairs: Making learning management systems more flexible. eLearn Magazine. 2006.<br />Geoghegan, W. (1994). Whatever happened to instructional technology? 22nd Annual Conferences of the International Business Schools Computing Association, Baltimore, MD, IBM.<br />Gregor, S. and D. Jones (2007). &quot;The anatomy of a design theory.&quot; Journal of the Association for Information Systems8(5): 312-335.<br />Herrington, J., T. Reeves, et al. (2005). &quot;Online Learning as Information Delivery: Digital Myopia.&quot; Journal of Interactive Learning Research16(4): 353-367.<br />Hutchins, E. (1991). &quot;Organizing work by adaptation.&quot; Organization Science2(1): 14-39.<br />Introna, L. (1996). &quot;Notes on ateleological information systems development.&quot; Information Technology & People9(4): 20-39.<br />Jamieson, K. and P. Hyland (2006). Factors that influence Information Systems decisions and outcomes: A summary of key themes from four case studies. 17th Australasian Conference on Information Systems, Adelaide, Australia.<br />
  213. 213. Jones, D., K. Jamieson, et al. (2003). A model for evaluating potential Web-based education innovations. 36th Annual Hawaii International Conference on System Sciences, Hawaii, IEEE.<br />Jones, D., J. Luck, et al. (2005). The teleological brake on ICTs in open and distance learning. Conference of the Open and Distance Learning Association of Australia&apos;2005, Adelaide.<br />Jones, D. and N. Muldoon (2007). The teleological reason why ICTs limit choice for university learners and learning. ICT: Providing choices for learners and learning. Proceedings ASCILITE Singapore 2007, Singapore.<br />Jones, D., J. Vallack, et al. (2008). The Ps Framework: Mapping the landscape for the PLEs@CQUni project. Hello! Where are you in the landscape of educational technology? ASCILITE&apos;2008, Melbourne.<br />Jones, N. and J. O&apos;Shea (2004). &quot;Challenging hierarchies: The impact of e-learning.&quot; Higher Education48(3): 379-395.<br />Katz, R. (2003). &quot;Balancing Technology and Tradition: The Example of Course Management Systems.&quot; EDUCAUSE Review38(4): 48-59.<br />Klor de Alva, J. (2000). &quot;Remaking the Academy - 21st Century Challenges to Higher Education in the Age of Information.&quot; EDUCAUSE Review: 32-40. <br />Kurtz, C. and D.Snowden (2007). Bramble Bushes in a Thicket: Narrative and the intangiables of learning networks. Strategic Networks: Learning to Compete. Gibbert, Michel, Durand and Thomas, Blackwell.<br />Laurillard, D. (2002). Rethinking university teaching: a framework for the effective use of educational technology. London, Routledge.<br />Lewin, K. (1943-44). Problems of research in social psychology. Field Theory in Social Science. D. Cartwright. London, Social Science Paperbacks.<br />Light, B., C. Holland, et al. (2001). &quot;ERP and best of breed: a comparative analysis.&quot; Business Process Management Journal7(3): 216-224.<br />Malikowski, S., M. Thompson, et al. (2007). &quot;A model for research into course management systems: bridging technology and learning theory.&quot; Journal of Educational Computing Research36(2): 149-173.<br />March, J. (1991). &quot;Exploration and exploitation in organizational learning.&quot; Organization Science2(1): 71-87.<br />Martin, B. and H. Etzkowitz (2000). &quot;The origin and evolution of the university species.&quot; Journal for Science and Technology Studies13(3-4): 9-34.<br />Mintzberg, H. (1989). Mintzberg on Management, Inside our Strange World of Organisations. New York, Free Press.<br />Mishra, P. and M. Koehler (2006). &quot;Technological pedagogical content knowledge: A framework for teacher knowledge.&quot; Teachers College Record108(6): 1017-1054.<br />Morgan, G. (2003). Faculty use of course management systems, Educause Centre for Applied Research: 97.<br />Papert, S. (1995). &quot;Why School Reform is Impossible.&quot; The Journal of the Learning Sciences6(4): 417-427.<br />Postman, N. (1998). &quot;Five things we need to know about technological change.&quot; NewTech, from http://www.mat.upm.es/~jcm/neil-postman--five-things.html.<br />
  214. 214. Pozzebon, M., R. Titah, et al. (2006). &quot;Combining social shaping of technology and communicative action theory for understanding rhetorical closuer in IT.&quot; Information Technology & People19(3): 244-271.<br />Pressey, S. (1926). &quot;A simple apparatus which gives tests and scores - and teaches.&quot; School and Society23(586): 373-376.<br />Rittel, H. W. J. and M. M. Webber (1973). &quot;Dilemmas in a general theory of planning.&quot; Policy Sciences4(2): 155-169.<br />Robey, D., W. Ross, et al. (2002). &quot;Learning to implement enterprise systems: An exploratory study of the dialectics of change.&quot; Journal of Management Information Systems19(1): 17-46.<br />Saarinen, T. and A. Vepsalainen (1994). &quot;Procurement strategies for information systems.&quot; Journal of Management Information Systems11(2): 187-208.<br />Salmon, G. (2005). &quot;Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions.&quot; ALT-J, Research in Learning Technology13(3): 201-218.<br />Seely-Brown, J. and J. Hagel (2005) &quot;From push to pull: The next frontier of innovation.&quot; The McKinsey Quarterly<br />Selwyn, N. (2007). &quot;The use of computer technology in university teaching and learning: a critical perspective.&quot; Journal of Computer Assisted Learning23(2): 83-94.<br />Simon, H. (1991). &quot;Bounded rationality and organizational learning.&quot; Organization Science2(1): 125-134.<br />Skinner, B. F. (1958). &quot;Teaching Machines.&quot; Science128: 969-977.<br />Snowden, D. (2005). &quot;Strategy in the context of uncertainty.&quot; Handbook of Business Strategy6(1): 47-54.<br />Snowden, D. and M. Boone (2007). &quot;A leader&apos;s framework for decision making.&quot; Harvard Business Review85(11): 68-76.<br />Sturgess, P. and F. Nouwens (2004). &quot;Evaluation of online learning management systems.&quot; Turkish Online Journal of Distance Education5(3).<br />Truex, D., R. Baskerville, et al. (1999). &quot;Growing systems in emergent organizations.&quot; Communications of the ACM42(8): 117-123.<br />Truex, D., R. Baskerville, et al. (2000). &quot;Amethodical systems development: the deferred meaning of systems development methods.&quot; Accounting Management and Information Technologies10: 53-79.<br />Tversky, A. and D. Kahneman (1974). &quot;Judgment under uncertainty: Heuristics and biases.&quot; Science185(4157): 1124-1131.<br />Wagner, E., S. Scott, et al. (2006). &quot;The creation of &apos;best practice&apos; software: Myth, reality and ethics.&quot; Information and Organization16(3): 251-275.<br />Weick, K. and R. Quinn (1999). &quot;Organizational change and development.&quot; Annual Review of Psychology50: 361-386.<br />

×