This document summarizes two projects focused on research and evaluation in educational settings.
The first project examined how a museum program provided opportunities for parental engagement in their child's schooling. Evaluations looked at participation levels and implementation issues, while research explored the program's ability to engage parents from a low-income urban community.
The second project, called LEAP, piloted and expanded a STEM program across multiple sites. Formative evaluations assessed interest, attitudes, and implementation, as well as individual and institutional capacity. Research goals included identifying learning outcomes and how museums and community centers could address STEM gaps. Key successes included evaluation contributing different perspectives, and lessons about coordinating evaluations across sites. Research implications included examining STE
Connie Sung, PhD
Assistant Professor of Rehabilitation Counseling
Department Counseling, Educational Psychology & Special Education
June Chen, PhD
Former Hegarty Fellow in MSU-DOCTRID Program
The NEW eXtension: Issues. Innovation. Impact.Christine Geith
We unveiled our new go-forward strategy for the NEW eXtension today at the annual meeting of the National Extension Directors and Administrators of the U.S. Cooperative Extension System.
Presentation given by Gail Matthews-DeNatale during a pre-conference gathering of Sloan Blended Learning grant recipients prior to the 2009 Sloan-C Conference on online learning.
Scaling up Innovation: Why Theories of Change MatterBrandon Muramatsu
by Elaine Seymour, University of Colorado at Boulder. Presented at the Workshop on Disseminating CCLI Innovations: Arlington, VA, February 18-19, 2010. Workshop organized by Joe Tront, Flora McMartin and Brandon Muramatsu.
California Community College Faculty Motivation and Reflection on Open Textbo...Una Daly
Interviews were conducted with twelve faculty members at community colleges in California who adopted open textbooks in their teaching practice for one academic term or longer. The interviews queried faculty on motivation to undertake the adoption, pedagogical considerations, student savings and feedback, and support from other campus stakeholders.
Faculty were asked how their teaching and student learning was affected as a result of adopting an open textbook in their course. Specifically they were asked if they were collaborating more with other faculty members and whether they were now using a wider range of instructional materials in their courses. With regards to student learning, they were asked if they believed that student learning had improved or whether student retention had improved as a result of the adoption of an open and free textbook. Any unanticipated outcomes that had resulted from the adoption either in their own practice or with students was also queried.
In addition to the faculty and students, other stakeholders on campus are often involved in the decision and process to adopt an open textbook. College initiatives or pilot programs to increase access and equity were sometimes the instigators for making the change and other times it was strictly a faculty decision. Library, instructional design, and bookstore staff were other stakeholders who played roles in the adoption process.
Attend this presentation to better understand the motivations of college faculty who adopt open textbooks and how it affected their teaching practice. Hear about the challenges they encountered and any unexpected outcomes. Learn what students had to say about using open textbooks in the classroom and how it affected their learning and ability to be successful.
This presentation shared an overview of the Next Generation Science Standards (NGSS), compared NGSS with existing state science standards, and explored differences in science instruction based on the new standards. The new NGSS for grades K-12 aim to prepare students for college, career, and citizenship by emphasizing a deeper understanding and application of science practices, content and cross-cutting concepts. NGSS, developed by a national team of scientists and educators partnering with 26 lead states, were adopted for implementation across Maryland’s public education classrooms beginning in the 2017-2018 academic year. While many aspects of NGSS were not novel concepts, the standards intentionally accentuate students thinking and acting like scientists and engineers by working to explain natural phenomena and solve problems. This approach reduces the content students memorize and underscores skills and knowledge needed to explain and understand natural phenomena. Students entering higher education may be less adept at rote memorization but better at solving problems, constructing explanations and developing deeper comprehension. During the workshop, participants analyzed and compared samples of a student lesson that varied with respect to learned-centeredness and discussed how learner-centered pedagogy supports instruction aligned with NGSS. Participants then discussed the implications of Maryland’s adoption of NGSS on higher education.
Having research grade equipment available for science students is usually prohibitively expensive for a community college or small college. In this panel discussion we described an NSF grant -supported innovative approach to share portable scientific equipment between three institutions: Hood College, Mount Saint Mary’s University, and Frederick Community College. The significant benefits for student learning through access to state-of-the-art instrumentation were discussed. The challenges and experiences from the planning stages of the grant through the implementation were presented. Based on our experiences to date, we recommended best practices for inter-institutional and intra-institutional collaborations. We also described other collaborations that have developed as a result of this inter-institutional cooperation.
"Go boldly forth!" (EG261) says Pope Francis, and we can do this in our parish community - but where do we start? The power of taking action in our local community to reach out to meet those who are poor, vulnerable and oppressed and discover Christ in them is an inspiring for every person who experiences this. Jesus Christ and Pope Francis inspire us with examples of being "border-crossers" of going to 'the margins' of society and loving those we meet. We can follow their example, and doing so as a parish community is a transformational experience of God's evangelising love for the world. In this workshop you will explore how to engage with your local community, the practical steps of beginning a local outreach program and gain an understanding how your Catholic parishioners can grow in their faith through taking action in your community with people and families who are in need of support.
Jenny Collins-White is the National Program Manager for Advocacy and Mission Formation at Catholic Mission. She has studied mission, scripture and theology extensively and holds a Bachelor Degree in Social Work.
Jenny is married and has 2 children and has been involved in leading her parish choir for more than 20 years. Today, the choir boasts more than 45 members and the outreach through music to community of people with disabilities and the elderly bring great joy to all involved. Additionally, Jenny has led Catholic Mission's outreach and advocacy programs for more than 15 years. Jenny's national and local programs engage thousands of children, women and men in outreach and a considered transformational and a powerful tool for evangelisation. They include Village Space, "Get-Involved-Locally" (a homework support and community relationship program with Sudanese refugees), and the campaign to 'End Child Detention.'
Connie Sung, PhD
Assistant Professor of Rehabilitation Counseling
Department Counseling, Educational Psychology & Special Education
June Chen, PhD
Former Hegarty Fellow in MSU-DOCTRID Program
The NEW eXtension: Issues. Innovation. Impact.Christine Geith
We unveiled our new go-forward strategy for the NEW eXtension today at the annual meeting of the National Extension Directors and Administrators of the U.S. Cooperative Extension System.
Presentation given by Gail Matthews-DeNatale during a pre-conference gathering of Sloan Blended Learning grant recipients prior to the 2009 Sloan-C Conference on online learning.
Scaling up Innovation: Why Theories of Change MatterBrandon Muramatsu
by Elaine Seymour, University of Colorado at Boulder. Presented at the Workshop on Disseminating CCLI Innovations: Arlington, VA, February 18-19, 2010. Workshop organized by Joe Tront, Flora McMartin and Brandon Muramatsu.
California Community College Faculty Motivation and Reflection on Open Textbo...Una Daly
Interviews were conducted with twelve faculty members at community colleges in California who adopted open textbooks in their teaching practice for one academic term or longer. The interviews queried faculty on motivation to undertake the adoption, pedagogical considerations, student savings and feedback, and support from other campus stakeholders.
Faculty were asked how their teaching and student learning was affected as a result of adopting an open textbook in their course. Specifically they were asked if they were collaborating more with other faculty members and whether they were now using a wider range of instructional materials in their courses. With regards to student learning, they were asked if they believed that student learning had improved or whether student retention had improved as a result of the adoption of an open and free textbook. Any unanticipated outcomes that had resulted from the adoption either in their own practice or with students was also queried.
In addition to the faculty and students, other stakeholders on campus are often involved in the decision and process to adopt an open textbook. College initiatives or pilot programs to increase access and equity were sometimes the instigators for making the change and other times it was strictly a faculty decision. Library, instructional design, and bookstore staff were other stakeholders who played roles in the adoption process.
Attend this presentation to better understand the motivations of college faculty who adopt open textbooks and how it affected their teaching practice. Hear about the challenges they encountered and any unexpected outcomes. Learn what students had to say about using open textbooks in the classroom and how it affected their learning and ability to be successful.
This presentation shared an overview of the Next Generation Science Standards (NGSS), compared NGSS with existing state science standards, and explored differences in science instruction based on the new standards. The new NGSS for grades K-12 aim to prepare students for college, career, and citizenship by emphasizing a deeper understanding and application of science practices, content and cross-cutting concepts. NGSS, developed by a national team of scientists and educators partnering with 26 lead states, were adopted for implementation across Maryland’s public education classrooms beginning in the 2017-2018 academic year. While many aspects of NGSS were not novel concepts, the standards intentionally accentuate students thinking and acting like scientists and engineers by working to explain natural phenomena and solve problems. This approach reduces the content students memorize and underscores skills and knowledge needed to explain and understand natural phenomena. Students entering higher education may be less adept at rote memorization but better at solving problems, constructing explanations and developing deeper comprehension. During the workshop, participants analyzed and compared samples of a student lesson that varied with respect to learned-centeredness and discussed how learner-centered pedagogy supports instruction aligned with NGSS. Participants then discussed the implications of Maryland’s adoption of NGSS on higher education.
Having research grade equipment available for science students is usually prohibitively expensive for a community college or small college. In this panel discussion we described an NSF grant -supported innovative approach to share portable scientific equipment between three institutions: Hood College, Mount Saint Mary’s University, and Frederick Community College. The significant benefits for student learning through access to state-of-the-art instrumentation were discussed. The challenges and experiences from the planning stages of the grant through the implementation were presented. Based on our experiences to date, we recommended best practices for inter-institutional and intra-institutional collaborations. We also described other collaborations that have developed as a result of this inter-institutional cooperation.
"Go boldly forth!" (EG261) says Pope Francis, and we can do this in our parish community - but where do we start? The power of taking action in our local community to reach out to meet those who are poor, vulnerable and oppressed and discover Christ in them is an inspiring for every person who experiences this. Jesus Christ and Pope Francis inspire us with examples of being "border-crossers" of going to 'the margins' of society and loving those we meet. We can follow their example, and doing so as a parish community is a transformational experience of God's evangelising love for the world. In this workshop you will explore how to engage with your local community, the practical steps of beginning a local outreach program and gain an understanding how your Catholic parishioners can grow in their faith through taking action in your community with people and families who are in need of support.
Jenny Collins-White is the National Program Manager for Advocacy and Mission Formation at Catholic Mission. She has studied mission, scripture and theology extensively and holds a Bachelor Degree in Social Work.
Jenny is married and has 2 children and has been involved in leading her parish choir for more than 20 years. Today, the choir boasts more than 45 members and the outreach through music to community of people with disabilities and the elderly bring great joy to all involved. Additionally, Jenny has led Catholic Mission's outreach and advocacy programs for more than 15 years. Jenny's national and local programs engage thousands of children, women and men in outreach and a considered transformational and a powerful tool for evangelisation. They include Village Space, "Get-Involved-Locally" (a homework support and community relationship program with Sudanese refugees), and the campaign to 'End Child Detention.'
Heartfit365 helps parents in their 40s and 50s gain new perspective so that they can make choices that give them the outcomes they want most in life. This presentation is based on the book Heartfit365 and explores one parents journey from mid-life crisis to the Ironman. The presentation covers the process of wanting more from your life and shares a solution framework for how to manage your expectations to get the outcomes you want in life. This presentation is not about becoming a triathlete, it's about deciding to move forward towards your goals every day.
As a Nerium Brand Partner, you exclusively represent Nerium International’s breakthrough products. What better way to earn a part-time to a full-time income than by sharing products you love?
Ways to earn:
Build a Customer Base and earn:
Immediate Retail Profits by selling our real results products from your personal inventory
$16 - $60 Customer Acquisition Bonuses – get paid weekly bonuses when you enroll new Preferred Customers.
10% - 25% Commissions on Personal Customers reorders.
Free Personal product – Enroll 3 Preferred Customers and receive your product for free!
Free Inventory with our Nerium Gives Back inventory replenishment program.
Build a team of Brand Partners and earn:
Initial Order Bonuses –get paid weekly when you and your team sell new Starter & Upgrade Packs.
10% Coaching Commissions – earn up to a 10% ongoing commissions on your top Brand Partners monthly earnings!
Team Commissions – earn lucrative overrides on global team sales.
Live Better Bonuses – earn $50,000 - $1,500,000 Lifestyle Bonuses.
Lexus Car Bonuses – earn up to $1,000 monthly luxury car bonus.
Incentive Trips – earn all expense paid trips.
Strategically rebuilding the Church of the Nativity involved studying, learning from and adopting successful practices in other Christian communities. Dr Ruth Powell will explore trends in evangelisation and what is working well in Christian communities across Australia. Participants are invited to take big picture ideas, learn from others and apply them in their own parishes.
Ruth Powell is Director of National Church Life Survey (NCLS) Research and an Associate Professor at the Australian Catholic University. She has been a part of the NCLS team since 1991. She has written about many aspects of Australian church life, including church health, denominational differences, and individual attitudes. Her PhD research focused on age differences among church attenders. Some of the publications she has co-authored include Winds of Change, Views from the Pews, Shaping a Future, Build My Church, Taking Stock, and Enriching Church Life.
Colleague #1 - Renee Morris Plum investigated the interactioWilheminaRossi174
Colleague #1 - Renee Morris
Plum investigated the interaction within individualized education program meetings through a collective case study. The qualitative study was aligned through key components. Plum (2008) stated the problem to be within the area of individualized education program collaborative meetings. The actually conversations and interactions of the members were undocumented of how the decisions of an IEP was met. The researchers’ purpose was to “capture and analyze the conversation among participants collaborating in IEP team meetings in urban schools in the greater Detroit area. “ (Plum, 2008). The research was seeking to find answers to what the procedures during an IEP meeting were.
According to Plum (2008), there were five research questions addressed in the case study:
1. What conversational structures are evident in the delivery of information to participants on which decision of placement is based?
2. How are turns allocated and questions asked and answered?
3. How do participants in the IEP make relevant their membership to categories: professional, parent, general education teacher, special education teacher, school psychologist or other qualified examiner, and so forth?
4. How do membership categories function in establishing interaction leading to the decision of placement?
5. How do participants orient to the asymmetries inherent in the institutional setting of the IEP demonstrable through their talk?
Plum (2008) does not specifically state a theoretical framework for this research. The researcher utilized three data collection tools to collect discussions of the IEP meetings. One tool specifically was a recorder to collect the conversations during IEP meetings as he observed.
Analysis
The study was aligned and justified. The qualitative case study was focused on a particular area in the greater Detroit area. Johnson & Christensen, (2008); Lichtman, (2006); Washington, (2015) as cited by Finch (2016), explains that qualitative research is often less generalizable and looks at the problem from a wider lens that examines the breadth of phenomena in a natural environment through interviews and observations. Research questions for this type of research are usually vague. Qualitative research seeks to explore or describe a phenomena. The use of three data collection tools for the discussions assisted in validating the findings. Finch aligned the components and allowed the research to flow and maintain consistency.
References
Finch, S., (2016). A Quantitative Study of Teachers’ Social Emotional Competency and Social
Instructional Practices in Metropolitan Atlanta Preschools – ProQuest. Retrieved 7 June 2021, from https://www.proquest.com/docview/1812535015
Plum, C. (2008). Interaction within individualized education program meetings:Conversation analysis of a collective case study (Doctoral dissertation). Retrieved from http://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=1015&context=hodgkin ...
Researching e-portfolios: The current state of playdcambrid
The first in the Europortfolio project's series of open webinars, from February 7, 2014. Inter/National Coalition for Electronic Portfolio Research co-directors Darren Cambridge, Barbara Cambridge, and Kathleen Yancey present on the philosophy behind and design of the Coalition, how its results illustrate the principle of "scaling out," and the four propositions about assessment with e-portfolios and their non-negotiable core that Coalition members are currently exploring.
Education and Technology Partnerships as Intercultural Communities: An Ethnog...CITE
CITERS2014 - Learning without Limits?
http://citers2014.cite.hku.hk/program-overview/keynote-green/
13 June 2014 (Friday)
09:10 – 10:00
Keynote 1: Education and Technology Partnerships as Intercultural Communities: An Ethnographic Perspective
Speaker: Professor Judith GREEN (Department of Education, University of California, Santa Barbara)
Chair: Dr. Susan BRIDGES (Associate Professor, Faculty of Education, HKU)
Type and Use of Innovative Learning Environments in Australasian Schools .Tec...eraser Juan José Calderón
Type and Use of Innovative Learning Environments in Australasian Schools ILETC Survey 1
Wesley Imms, Marian Mahat, Terry Byers & Dan Murphy
• What types of learning environments are in use
in Australian and New Zealand schools?
• What types of teaching approaches happen in these?
• What types of learning do they facilitate?
Researching ePortfolios: The current state of play- Darren Cambridge, Barbara...EPNET-Europortfolio
#ePortfolios #Webinar
webianr available at https://www.youtube.com/watch?v=AUVTGmLHYmU
Published on Feb 19, 2014
Researching ePortfolios: The current state of play led by Darren Cambridge, Babara Cambridge and Kathleen Blake Yancey
This webinar was held on Friday 7th Febuary 2014 by www.europortfolio.org
This webinar discusses the research on e-portfolios, presenting the work of the Inter/National Coalition for Electronic Portfolio Research as a model for collaborative inquiry embedded within the process of implementation that both generates new knowledge and leads to successful results.
Over more than a decade, the Coalition has worked with nearly 70 further and higher education institutions in the US, Canada, the UK, Australia, and the Netherlands to better understand how e-portfolios can supporting learning, assessment, and institutional change.
The webinar will provide an overview of the Coalition's process, survey some results from cohorts that have completed their work, and discuss current questions it is investigating and how they might apply to cross-sector practice in Europe.
For more information about the Coalition and its work see http://ncepr.org/
Webinar leaders will be: Barbara Cambridge, Director, Washington Office, National Council of Teachers of English, Darren Cambridge, Principal Consultant, Networked Learning Group, American Institutes for Research and Kathi Yancey, Kellogg W. Hunt Professor of English and Distinguished Research Professor, Florida State University.
Europortfolio is a European Network of ePortfolio Experts & Practitioners.
Europortfolio, a not-for profit association established with the support of the European Commission, is, dedicated to exploring how e-portfolios and e-portfolio-related technologies and practices can help us to empower:
1. 'Individuals as reflective learners and practitioners;
2. Organisations as a place for authentic learning and assessment, and
3. Society as a place for lifelong learning, employability and self-realisation."
Europortfolio has a broad agenda, if you would wish to know more, or to get involved, you can do this by visiting our website www.europortfolio.org
Exploring the Scholarship of Teaching and Learning A Comprehensive Overview (...
Itest research evaluation fina lrev-1
1. ITEST Convening - August 2014
Implementing High Quality
Research and Evaluation
Sukey Blanc, PH.D.
Creative Research &
Evaluation, LLC
Principal Researcher
Dale McCreedy, Ph.D.
The Franklin Institute
Director of Gender, Adult
Learning & Community
Engagement
3. Framing Evaluation and
Research
Each project positions evaluation as
critical – both formative and summative.
Essential role of evaluation in
conceptualizing research.
Theoretical frames used for research.
Research grew out of commitment to
contribute to the field.
External evaluators played integral role
during the proposal and research
phases.
5. Evaluation: Participation; Implementation
issues; Progress towards goals.
Research: How does a museum program
provide opportunities for parents in a low-
income, urban community to engage in
their child’s schooling?
Theoretical Frame: – Angie Calabrese
Barton’s Ecologies of Parent Engagement
(EPE)
6. Evaluation:
LEAP Pilot:
Interest, science attitudes,
implementation
LEAP FSD:
Capacity of individuals and
institutions
Sustainability of program at sites
Potential for/interest in scale-up
Research:
• Focus on broader LEAP trajectory
• Scale up (Coburn, 2003) defined with fou
inter-related dimensions (depth,
sustainability, spread, & shift in ownership
8. Evaluation: primarily formative
Research originally conceptualized as:
• identifying learning processes and
outcomes
• exploring how museums and OST
centers could work together to
address gaps in STEM experiences in
underserved communities
9. Key evaluation successes:
• Evaluation is seen by all as key to
project.
• Evaluation data contributes a different
lens on enacted curriculum, site
capacities, and family involvement.
A key evaluation challenge:
• Coordination and communication with
out-of-school sites (this is true for
10. Implications for research questions and
research design
• STEM outcomes for youth and adults
• Partnerships to support STEM capacity
in out-of-school sites
Research topics/approaches will connect to
literature in the field and will build on
knowledge built through process of STEM
3D evaluation.
Editor's Notes
the purpose of evaluation is essentially to improve the existing program for the target population, while research is intended to prove a theory or hypothesis. Although both use similar data collection and analysis methods, the two disciplines diverge again during use and dissemination. This relationship can be visualized using an hourglass shape:
We became interested in the often unrecognized and untapped potential of parents through our collaborative work on a parent involvement program designed and implemented at three urban elementary schools in Philadelphia.
drew upon The Ecologies of Parent Engagement (EPE) framework (Calabrese Barton et al., 2004). This framework posits a definitional shift from parent involvement, typically defined in terms of individual actions and behaviors desired by school staff, to parent engagement, defined in terms of parents’ social and contextual relationships relative to others in their child’s schooling. The EPE framework was compelling to us because it allowed us to think about expectations for and the practice of parent involvement in schools and in supporting children within a larger social and cultural context. In addition, it focused on parent involvement in low-income, urban elementary schools, acknowledging that the realities of these parents’ lives often dictate different forms of involvement than those engaged in by middle-class, Caucasian parents. And it takes into account the “what,” “how,” and “why” of involvement, thus offering a holistic model for understanding the mechanisms through which parent involvement occurs.
Sub-questions under this primary research question guided the development of theoretical propositions; they were as follows:
How are parents engaged in their children’s schooling as a result of participation in the museum program?
What were the conditions that facilitated parents’ engagement through the program?
What were the obstacles that hindered parents’ engagement through the program?
How well does the EPE framework, and specifically the constructs of authoring and positioning, describe the mechanisms through which the museum program facilitated parent engagement?
STEM 3D is most recent project. Started in Spring of 2013 –
First three months – program staff identified 5 out of school time centers that were committed to attending professional development, implementing new STEM curriculum in after school and summer programs , and in nvolving parents in activities about STEM and STEM awareness.
During the first complete year, the centers began implementing propblem based unit themed around STEM and STEM careers
In the upcoming year, the goal is for each center to develop its own problem based units and to increase parent engagement in the project
So far, evaluation has been primarily formative. One of the key things that I’ve been able to do as an evaluator is bring an outsider’s lens to the process of program implementation. The staff on the project is also very reflective and thoughtful about program implementation, but there are things that I can bring as an outsider that are hard to do for staff members focused on developing the project. One useful thing was an analysis of the differences between the intended curriculum and the enacted curriculum. By intended curriculum, I mean the materials that the staff and centers use in planning and professional development. By enacted curriculum, I mean what actually happens in activities in the out of school sites. The program staff is very aware that there are differences between what they expect to appen and what actually happens in sites, but they value having someone else who is not involved wit the program help them look at this systematically and think about how to address problematic gaps.
In this upcoming year, I’ll be continuing formative evaluation, and together with the staff, I’ll also start thinking about how the formative evaluation learnings can help us refine the research questions we identified in our original proposal. As you can see, the research questions originally fell into two big areas –which we’ll narrow down.
One has to do with big questions about learning and the other one has to do with big questions about capacity building and partnership.
When the project was introduced, facilitators and coordinars at the sites were clear that evaluation would be part of this from te beginning. They know me and tey don’t feel like it is an extra burden. I also interact with them, and at times get their input on evaluation issues.
The other success I already mentioned but it bears repeating – that data for formative evaluation is being collected simultaneously with program implementation, but what the evaluation provides is the lens of an outsider who is skilled in assessing educational programs.
One key evaluation challenge is a challenge tht is shared by the program staff, out of school sites are incredibly over-burdened. The project picked high capacity sites, but even these are understaffed and always juggling multiple roles. Concretely, this means that sites are not always good about communicating when they are doing what, which means that both staff and evaluator have to be patient and flexible. Of course this is a problem from my perspecives. As a professional evaluator, I have lots o fdifferent projects, and I can’t necesssarity drop what I am doing if I site gives me short notices about an important activity.
Resolution is flexibility of roles. The project staff does share their data with me, both in the form of informally notes and thoughts and in the form of videos. And sometimes I do the same with them.
.
Moving into research, the fact that we know the sites and the players will let the project staff and me make an informed decisions about research related to outcomes - the most meaningful and realistic outcomes to look at (likely to be about STEM careers, at least for youth). Evaluation experience will also help us frame the theoretical framework we use to talk about building partnerships between the Science Centers and community sites – especially important is something that I will be investigatingin the upcoming year, which has to do with the existence of f knowledge brokers in each site who can mediate between the wolrd view of the project staff and the world view of faciliators at community centers. This is coming up first s an issue for implementation and formative evaluation, but it is very likely that it will be a generative thee that can enrich the field as a whole as we move into the research phase.
Just to summarize, there is a value in having an outside person provide formative evaluation and in this project, like the ones Dale talked about, the formative evaluation will help provide grounding for the research phase that comes later.