ICT Role in 21st Century Education & its Challenges.pptx
Ashbaugh dissertation defense presentation
1. A Dissertation Study
ONLINE PEDAGOGICAL QUALITY QUESTIONED:
PROBING INSTRUCTIONAL DESIGNERS’
PERCEPTIONS OF LEADERSHIP COMPETENCIES
CRITICAL TO PRACTICE
By
Marcia L. Ashbaugh
Capella University
Doctoral Researcher
Mentor: Dr. Rod Sims
2. Naidu (2003)
Inferior found “a
Online schedule, brief
outline of the
Course course
Quality content, PowerPoi
nt slides of the
lecturer’s
notes, and
sometimes, sampl
e examination
papers” (p. 349)
and not
much more.
Context of Study- Observations
3. Common
Issues
• Economic
shortages
• Team
motivation
• Staying
current
Context of Study - Environment
4. Quality of Online
Learning Course
Design
Pedagogies Explored
for…
…Influence of
Leadership
Competencies on
Quality
Research Focus
5. Quality
Phipps & Merisotis (1999); Russell (2001); Cavanaugh (1999); Warren
and Holloman (2005) > No significant difference
Clark (1994); Merrill (2005) > Computers only tools
Dede et al. (2007) > Positive effect in online communities
Der-thanq, Hung, and Wang (2007) > Traditional pedagogies = low
course effectiveness
NSSE (2008) > Learning sought on deeper level; increased
collaboration
Allen & Seaman (2006); Seaman (2009) > Faculty consider online
inferior
Bernard et al. (2009) > Aligned strategies/learning context
Key Research - Quality
6. ID Competency
IBSTPI (2000); Spector et al. (2006) ~ 23 ID
competencies
Leadership Competency
Dooley and Lindner (2007) ~ Increased
leadership competencies
Scott, Coates, and Anderson (2008) ~ Poorly
understood
Fullan and Scott (2009) ~ Leadership lacking
Key Research - Competency
7. Leadership in Practice
Campbell, Schwier, and Kenny (2009) ~ Perceptions of
change agent, leader
Kim, Baylor, and Group (2006); Velletsianos and Miller
(2008) ~ Leadership exhibited in contemporary online
learning tools
Gessick and Derry (2010) ~ Leadership competencies
revealed during asynchronous activities meaning
DeBlois (2005) ~ Leadership competencies linked to
success for advancing ID field
Key Research - Leadership
8. Central Research
Phenomena:
Instructional Design Competency
Quality of online
academic pedagogies.
Theoretical
Framework
Leadership Quality
Theoretical Construct
9. THE PROBLEM THE PURPOSE
Lack of online ◦ Investigate current
course quality quality status
◦ Relationship to ID
leadership
competencies
◦ Effect on learners
RESEARCH PROBLEM AND
PURPOSE
10. What instructional design leadership competencies are
identified as critical to creating quality online learning
designs?
What are the characteristics of courses created by
participants who have identified critical leadership
competencies?
How do students evaluate the quality of courses by those
using the identified critical leadership competencies?
Research Questions
11. Population/Sample Data Collection
◦ Instructional ◦ In-depth interviews
designers (Masterman et al.,
2009)
◦ AECT members ◦ Course designs
◦ Student evaluations
◦ 6 experts Instrumentation
◦ Work samples
◦ Higher academic ◦ Researcher/interview
roles er
◦ US and Canada • Bracketing
(Kaplan, 2004;
Moustakas, 1994)
Methodology-Qualitative
13. Competencies for Competencies for
Practice Designs
Strategy
Student Engagement
◦ Learner-Centered
Vision ◦ Personal Knowledge
Control
Personality Student Satisfaction
◦ Perception of Quality
Productivity
◦ OASA Alignment
14. RQ1 Analysis
“What instructional design leadership competencies are
identified as critical to creating quality online learning
designs?”
Competencies
Vision
Duties
Ment
Strate
oring
gy
Produc
t-ivity
Emotio
nal
Attributes
Values
Strengt
h
Personal
ity
Themes and Categories
15. Participant Words on Leadership:
P3 ~
Strategy - knowledge of current theories
Productivity - best practices
P1 ~
Personality – collaboration; clear
communication
Vision – convey purpose
P2 ~
Mindset – ready for change
Response
16. Strategies Vision
Contextualized learning Stewards, change
Engagement agents
Research-based strategies Global thinking,
Ethical practices collaboration
Collaboration, connectedness Open-source LO’s
Wikis, Blogs
Personality Productivity
Communication Assumes responsibility
Without it, leaders fail Leaders as producer
Vital to quality Impacts the learner
Implications
17. RQ2 AnalysisRESULTS of DATA ANALYSIS
“What are the characteristics QUESTION 2
FOR RESEARCH of courses created by
participants who have identified critical leadership
competencies? ”
Competencies in Participant Designs
Instructional Strategies by
10%
Category
15% Activities
Authentic Tasks
10% Assessments
65% 8%
Objectives/Outcomes
8% Interaction
Instructional Strategies 31%
7%
Learner-control Tasks
15%
Problem-solving
31%
Theory-based
Instruction
Themes and Categories
18. Course Design Results by Quality Matters
Category Points achieved/ Percentage
Points possible standards met
Learning 8.2 our of 9 91.1%
objectives
Assessment and 11.4 out of 13 87.7%
measurement
Learning 8.6 out of 10 86.0%
engagement
Total 28.2 out of 32 88.1%
Response
20. RQ3 Analysis RESULTS of DATA ANALYSIS
FOR RESEARCH QUESTION 3
“How do students evaluate the quality of courses by those
using the identified critical leadership competencies?”
Student Engagement = 90%
Student Satisfaction = 87%
High Course Quality = 90%
Themes and Categories
21. Student evaluations – are they about the
teacher or the student?
P6 ~
“The purpose for the instructional design
must be the learning
and [the] change of the learners”
Response
23. Research Ideas
Leadership principles in Case studies,
action ethnographies of
designers
Impact of principles on Action research in
learning course rooms
Leadership in design Design and development
process to validate research
models, metrics
Future Research
24. Principles of Leadership Kowch (2009) ~
“New leaders need
(to Possess or Engage In) new ways to
address these
Prescience (Vision) issues beyond the
Preventive/Proactive instructional
leadership
Thinking (Strategy) literature, which
Provision for the Unexpected was found lacking
(Strategy) due to a classical
Personality (Interpersonal focus on the
Skills) supervision of
instructors/teachers
Productivity (Productivity) more than with the
Psychological Toughness leading of the
(Decision-making) instructional
Personal Convictions (Values) process (design and
development, as we
know it).”
The 7 Ps of Leadership Model
25. Online Registrations Globe Unraveling Mobile Learning
Significance to
Instructional Design Field
Editor's Notes
A qualitative, in-depth interview research study was undertaken to explore the phenomenon of quality in online higher education pedagogies. To understand the variables impacting course designs in this genre, expert instructional designers were queried for perceptions and ideas found in practical experiences. (CLICK)
During the past five years of online graduate studies, I have often encountered a skeptical attitude about the quality of internet-based education. What is more, after reading journal articles that frequently decried inferior online instructional strategies, I was inspired to dig deeper into the origin of such negativity. For example, Naidu’s early 21st century work identified a degradation in online pedagogy when designers infused traditional methods into web-based courses. Likewise, Sims & Jones argued with the misaligned approaches they observed---and in 2006 Sims maintained that low quality online designs resulted when teaching strategies and learning activities were, in his opinion, steeped in traditional instructivist methods and theories of learning. From a learning process perspective, Dr. Michael Beaudoin’s work in 2003 and 2007 on designs of technology-based education concluded that (CLICK) a new mode of learning was emerging…and suggested (CLICK) that learning is different when humans interact with computers.Arguments abounded within the design community about the role of computers in the actual learning process, largely borne from comparative research and findings that determined online learning to be the same or not as effective as traditional classroom parallels. In response, Beaudoin, Naidu, and Sims were among those who concluded that leadership was needed in and from the instructional design community in order to address the emerging theories of learning in a new environment. However, Sims and Koszalka questioned whether designers possessed the competencies to advance the field with efficacy and leadership.(CLICK)
To understand how certain competencies would translate into quality online learning pedagogies, it seemed best to explore the phenomenon with practicing instructional designers for what they believed was critical for leading in quality online designs.Therefore, the study took place from several locations where expert designers (CLICK) centered their practices. As all were associated with academic institutions (CLICK) of higher learning in the US and Canada, issues of leadership and quality in online course designing (CLICK) were shared by the larger community of practice. (CLICK)Common concerns for this group were how to provide quality courses within the constraints of (CLICK) current economic shortfalls; how to (CLICK) motivate a team to produce relevant designs; and how to (CLICK) stay current with theories and technologies. (CLICK)
Consequently, the focus of research centered on exploring instructional designer leadership competencies as influencing variables on the central phenomenon of quality, in terms of online course pedagogies. (CLICK)
An extensive literature review and analysis located studies that concluded no significant difference existed between online and traditional classroom instructional outcomes, including a much-cited meta-analysis by Phipps and Merisotis in 1999.Others challenged (CLICK) the veracity of learning with machine intervention, such as Clark’s 1994 arguments that regarded computers as a mere pedagogical tool. On the other hand, research by (CLICK) Dede’s 2007 team found that quality of learning actually changed for the better when immersed in relevant online venues.In response to the contradiction between notions on web-based learning quality, (CLICK) Der-Thanq, Hung, and Wang found traditional pedagogies adapted as online strategies to be the fault of low course effectiveness. Although, the (CLICK) 2008 National Survey of Student Engagement report concluded that online courses provide engagement by stimulating intellectual challenge and that learning was sought on a deeper level, … a low quality perception of online pedagogies persisted. In fact, (CLICK) a 2009 Seaman study confirmed earlier studies by Allen and Seaman that faculty overwhelmingly viewed online curricula to be inferior to traditional affordances.In contrast, (CLICK) Bernard and team concluded that pedagogical excellence was achieved when congruity was struck between strategies and the learning context.(CLICK)
An assumption of this study was that quality is ensured through skillful practice. In the late 20th century, the international board of standards for training, performance, and instructionsought to standardize competencies for instructional designers.While testing for improvement, (CLICK) in 2007 Dooley & Lindner measured instructional designers’ competencies using a new model for practice. Participants were found to increase nearly 2 increments on a novice to expert scale… including in organizational knowledge, skill, and ability - - - established behavior requirements considered necessary to be successful, and, for leadership. (CLICK)A more general observation was conducted by Scott, Coates & Anderson in a 2008 longitudinal study that looked at competencies in various settings and concluded that leadership, per se, is poorly understood in higher education. Likewise, (CLICK) in 2009 Fullan & Scott observed a lack of leadership competency in educational leaders. (CLICK)
On the other hand, a major study by Campbell, Schwier, & Kenny explored perceptions of practicing instructional designers and found that designers perceived themselves as being change agents and leaders. Moreover, leadership competencies were implicated in other studies: (CLICK) One in 2008 by Velletsianos & Miller concluded that leadership is exhibited when contemporary learning tools such as social networking and multi-user-virtual-environments are present in design strategies.(CLICK) Gessick & Derry found collaboration skills and other leadership competencies in play when roles were distributed during asynchronous learning activities. The findings informed Instructional Design teams how to create more effective strategies for online learning.(CLICK) However, it was an earlier study by DeBlois that focused the argument for increased leadership competencies for Instructional designers, when it was found that a majority of faculty and designers shared a perspective of linking success with leadership attributes for advancing the field.It was concluded that researchers and theorists fell short in describing and defining specific leadership competencies critical for designers of online pedagogies. An attempt to fill the gap was made in this study with satisfying results.(CLICK)
Subsequently, a four-fold theoretical framework guided the research study on the central research phenomena of quality in online academic pedagogies.Instructional design theoryCompetency theoryLeadership theory and theories on Quality provided multiple perspectives for determining and testing online pedagogical effectiveness. (CLICK)
The framework provided a foundation from which to explore the problemof an on-going lack of quality in certain online higher education curricula……with a purposetoward exploringthe current status of quality through its relationship to instructional designer leadership competencies and the effect the results have on learners. (CLICK)
Three research questions were asked with one main guiding query:What instructional design leadership competencies are identified as critical to creating quality online learning designs? (Slight pause)(CLICK)
To answer the questions, a qualitative approach was employed. Gelo and others (2008) posited the method as effective for studying a central phenomenonwithin a defined social setting; in this case, (CLICK) a population of instructional designers.From the Design and Development division of the Association for Educational Communications and Technology, six qualified participants, representing instructional and design positions in higher academic institutions, formed a sample group of experts from the US and Canada. (CLICK)The phenomenological approach collected narrative data through a series of semi-structured, in-depth interviews. Secondary data consisted of sample online courses designed and implemented by participants. Additionally, student evaluations of participants’ course designs were used as measurements of student satisfaction.Thus, (CLICK) work samples were the instruments of inquiry. However, accordingto Seidman, interview-based research hinges on the researcher asthe instrument of exploration. Seidman (2006) stressed the unique relationship an interviewer holds with an interviewee , and so, tomitigate undue influence, a limited use of (CLICK)bracketing, as described by Moustakas and Kaplan, helped to set aside personal bias.(CLICK)
Using NVivo9 software, (CLICK) recorded and transcribed interview data were processed using constant comparison analysis. The iterative process involved coding data, or, participants’ words and phrases with descriptors drawn from the research questions. Distilled data resolved into specific categories which were further summarized into general themes.Second, personal documents content analysis (CLICK)was applied to participants’ artifacts. A pre-approved, modified Quality Matters™ rubric evaluated components of the designs for elements, such as engagement and satisfaction. Finally, using frequency of occurrence analysis, assessedstudent course evaluation survey results.Additionally, accuracy of analyzed data (CLICK) was ensured by use of detailed research journals.
In summary, the study results indicated that certain variables of leadership competencies do indeed have a relationship to the quality of online course designs. Major themes deduced from the data implicated four aspects of leadership as critical to producing quality online pedagogies: Strategy VisionPersonality attributes And, Productivity.The identified leadership competencies in practice combined with those located in artifacts (CLICK) to create competent and quality designs. (CLICK)
Specific analysis began with the first research question, “What instructional design leadership competencies are identified as critical to creating quality online learning designs?”For reporting interest and clarification, textual results were converted into integers for tabulation and graphing. Leadership competencies identified by participants were organized under the major themes of Competencies, Attributes, and Duties, While seven categories emerged from the data as shown in the pie graph.Another graph (CLICK)conveyed a categorical-only breakdown and displayed the proportionate complement of each attribute. (CLICK)
In response, participants words captured the essence of the first question:(CLICK) Participant 3 articulated the dominant category of Strategy adding some additional references to Productivity:“…you have to have a knowledge of the latest learning theories and the latest instructional design theories in order to be a leader. You also have to be able to implement those theories, individually and in concert with each other, to achieve best practice, instructional strategies, or instructional strategy design.”Personality (CLICK) and Vision were expressed as critical for leading in quality design practice by Participant 1:“Leadership competencies are…having a process that works; pulling together people, making sure that the interrelationship skills are there and working…and, trying to build a collaborative team who have buy-in and ownership of this course.” And, “using clear communication skills to convey the purpose…” With a summary comment, Participant 2 considered leadership in practice (CLICK) a “mindset” ready for change.(CLICK)
Implications from the first question’s findings was that oversight, foresight, and insight is needed for creating quality online pedagogies. The findings provided a metaphoric interpretation in an umbrella. A distinct leadership mindset will oversee and guide the implementation of effective and satisfying online course designs.The theme (CLICK) Strategies aligned with research such as Sims and Stork’s 2007 study on relevant and contextualized learning,…with Parrish’s 2009 demonstration of how to sustain engagement,…and, with on-going domain-specific research on strategies required for distance learning environment conducted by Bollettino & Bruderlein in 2008 and Hong & Sullivan in 2009. Implications resonate with Campbell’s 2009 study on ethical practices, as well as Sims’s 2009 call for a proactive mindset of collaboration and connectedness with a team of designers, instructors, and students. The (CLICK) Vision theme was in keeping with Sackney & Mergel’s 2007 notion of “stewards of the vision”; with team Campbell’s 2009 findings of instructional designers as change agents; as well as with Reimer’s 2009 implications for global thinking in leadership and Durdu’s team findings for the need of global collaboration among design leaders.Vision further implied leading in the use of nontraditional resources: Caswell’s 2008 team emphasized open-source learning objects; Baggio studied wikis in 2008, and Carmean implied future informed blogs in the same year. By 2011, all of these once-envisioned strategies were in use with more innovation on the horizon as implicated in this study…and, inferred for designers of online academics.(CLICK) Personality, or Communication, skills have long been associated with good leadership including by Sergiovanni and Corbally in 1984. However, in 2008 Howard and Wellins found a failure among leaders due to inferior interpersonal skills, while a 2009 Larson and Lockee study considered it vital. This present study linked interpersonal skills with every aspect of the design effort and considered them a necessity to quality workmanship.Finally, implications from the study results echoed literature that stressed (CLICK) productiveleaders know how to assume responsibility and are expected to do what they say, as described by Argyris & Schön’s 1992 treatise on aligning theory with practice.Further implied was that leaders understand how to work hard to achieve results; in other words, are productive.Indeed, a2008Chen and team study of leadership roles in virtual teams showed that learners prefer a producer, or one who gets things done, to any other style of leader.Lastly, implications of productivity clearly impact the client, which is…the learner.(CLICK)
The second research question What are the characteristics of courses created by participants who have identified critical leadership competencies? …was answered by confirming competencies identified by participants were those actually used in their practice – in specific, those determined to have contributed to the quality of recent artifacts designed and implemented by the participants. The second graph shows an extrapolation of six categories under the more complex theme of Instructional Strategies, which aligned with specific strategies found in the interview data as indicative of competencies in use by leaders of quality online designs.(CLICK)
Triangulating the interview data, participant responses were found to be present in work output during an evaluation of the courses for levels of quality. Using an approved, modified instrument from Quality Matters, the categories relative to quality were applied to course designs. The numeric measures of standards met by the courses of 88.1 percentage of total points possible, exceeded the minimum of 82.9% set by QM, by over 5%. Participant 3 expressed the consistency between leadership and the course design results as:“I guess the biggest challenge ofproviding leadership is, in teaching … that good instructional strategies will lead to effective instruction and learning. Typical lecture, posting readings online, meaningless threaded discussion, recall quizzes and tests, without meaningful real world practice will lead to poor learning and performance.”(CLICK)
The findings implied what Sims and Stork exposed in 2007… that learners are engaged more directly through contextualized and personalized learning during what Naidu called the asynchronous nature of online transactions.In keeping with the concept, participants incorporated opportunities for learners to construct knowledge and meaning through an individualized perception into their designs. Most significantly, the study clarified a quality link between context and environment-relevant Objectives, Assessments, Strategies, and Activities within the course structure. Sims 2011 chapter on simulations stressed the criticality of the alignment of these core essentials for effective learning.Implications were that by applying leadership principles to the design, congruence will be met in online designs.(CLICK)
Lastly, student opinions expressed on terminal surveys were reviewed for common elements focusing on themes of student engagement, student satisfaction, and perception of course quality.The values of 90, 87, and 90 percent, respectively, provided a final corroboration of critical leadership competencies embodied in online course designs.(CLICK)
According to Seok (2009), student evaluations more often measure teacher performance...…however, multiple links between learner-centered criteria in this study’s student course evaluations implied quality of the learning environment to be a major concern for learners.Participant 6 captured the primacy of the design goal: “The purpose for the instructional design must be the learning and the change of the learners.” (CLICK)
Final implications of this study is that personal leadership competencies do drive critical design decisions which ultimately impact online pedagogical quality, and will benefit the designer if understood and applied.Implicated by the student evaluation data was the desire for learner-controlled engagement techniques… coupled with authentic tasks for satisfactory learning. (CLICK)
It is suggested more research is needed to pinpoint those competencies critical to quality practices in the online domain and causes of a lack of understanding for their need and use. For example, studies like this present one are needed to mine the ideas of practitioners. Other research may include case studies to view principles in action for individuals and design teams, and perhaps ethnographic studies will reap data from an in-depth experience into the daily interactions of academic design departments.Second, participatory action research may enlist instructors in what works and what does not in the online environment as design interacts with and impacts learners.Third, design and development research could be used to validate domain-specific models and metrics, such as the one suggested by this researcher: the 7 Ps of Leadership for ID – shown in the next slide.(CLICK)
Support for a new model of leadership rests in researcher intention to answer a call by Kowch (2009) that decried,“New leaders need new ways to address these issues beyond the instructional leadership literature, which was found lacking due to a classical focus on the supervision of instructors/teachers more than with the leading of the instructional process (design and development, as we know it).” (CLICK)With this model, a designer in any capacity, role, or position will improve in quality designing. Each principle is explicated more fully in the dissertation and includes direct application to the process of ID. Note the correlation between the principle and study themes.(CLICK)
What set this study apart from others in the field… was its focus on leadership from a domain-specific perspective. This research served to inform the instructional design community as to what impact critical leadership attributes of practitioners in fact does have on quality online pedagogies. This is important for meeting an urgent demand for web-based educational resources for training in excellent thinking and reasoning skills. The need is evident in the sheer numbers (CLICK) enrolling in online learning as conveyed in the most recent Allen & Seaman survey which reported a 17 to 1 ratio of online enrollment to overall student population growth.Moreover, in a 2010 International Society for Technology in Education conference, Dr. Jean-Francois Rischard, former vice-president of the World Bank, warned of (CLICK) global disaster if educators do not influence learners in critical skills needed for creating solutions to the world’s complex and chaotic problems.A shift in the very nature of education and learning to personalized, autonomous acquisition of knowledge “as-needed”, “when-desired”, and (CLICK) “on-the-go”…….demands leadership to assert agendas for high quality technology-distributed pedagogies through revised standards of practice.And finally, a new model of leadership for guiding the practice of online instructional designers will enhance the contributions of this study to the worthy profession of Instructional Design.THANK YOU FOR YOUR KIND ATTENTION AND CONTRIBUTION TO THE SUCCESS OF MY RESEARCH STUDY AND DISSERTATION!(CLICK)
