Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply



Published on

Bridgette Masters - NZPsS recipient of the Karahipi Tumuaki President's scholarship

Bridgette Masters - NZPsS recipient of the Karahipi Tumuaki President's scholarship

Published in: Education, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Understanding the simplicity & complexity of evaluating kaupapa māori programmes ā i Bridgette Masters-Awatere Waikato University President s President’s Scholarship Recipient 2008 4 September 2009
  • 2. The SIMPLICITY (description) • Evaluation has been: Evolving with humankind Delivered in a multidisciplinary context Strongly influenced by the political climate Engaged culturally from a “one size fits all” approach, which; • Has a homogenising affect • Maintains systems & tension • Ignores challenges in response to power and control issues • Is responsive to constantly changing political agendas (of the dominant group) © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 2
  • 3. Research Approach • Epistemological position of social constructionism • No claims of objectivity “If a person is blind to the values and nature of their own beliefs, they are hardly likely to be in a position to examine the same for another ethnic group.” • Applied research • Anthropology – understanding how Maori maintain their sense of control and ownership of their programmes when being evaluated from a paradigm that is different to their own • Psychology – exploration of how people have experienced and reconciled such differences among themselves • Sociology – engaging ways for Maori social organisations, structures and processes to be maintained and recognised • Critical approach © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 3
  • 4. Indigenous Theory • C lt Culture specific ifi “culture-specific approach, epistemology and methodology emerge from the worldview of the culture in focus rather than from the worldview of the researcher or the academy.” Shreiber, 2000, p.655 • Non-neutral perspective “ …because I do not take who I am for granted, I posses a heightened sense of reflexivity that can make me a better researcher than White colleagues inexperienced with assessing their social and/or professional position I surroundings that place them in the role of ‘minority’ and who often take for granted their privileged status when in the ‘majority’.” Hendrix, 2002, p.168 • Epistemological orientation “ We begin to write ourselves as indigenous peoples as if we really were ‘out there’, the ‘other’, with all the baggage that this entails… [Academic writing] privileges set of texts, views about the history of an idea, what issues count as significant; and by engaging in the same process uncritically, uncritically we too can render indigenous writers invisible or unimportant while reinforcing the validity of other writers.” Smith, 1999, p.36 © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 4
  • 5. What was my research about? “Can evaluation incorporate cultural values when being applied to a Kaupapa Maori programme?” programme? From participants’ perspectives : 1. How is success determined/defined? 2. What do Maori communities understand evaluation to be? 3. How do evaluators & providers perceive cultural factors to relate to their work? l t t th i k? 4. What factors support and/or impede the inclusion of cultural concepts in an evaluation of a Kaupapa Maori p g p p p programme? 5. Does what Maori say (providers, evaluators and communities) align with commentary from international evaluation practice? © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 5
  • 6. The Historical Context - Evaluation • Linked to human evolution A simple explanation... • Formally a young practice “Evaluation is the systematic • Multidisciplinary nature assessment of the worth or merit of some object.” object. • Politically driven agendas A complex explanation... In the US evaluation became federally legislated and has been “Evaluation is both simpler and more complex than their individual influenced over the years by social conclusions; simpler at the meta-level, and political climates of the day more complex i th d t il Th h d l in the detail... The hard work in evaluation theory involves NZ context is similarly aligned to unpacking the way in which political climate A market-driven climate. evaluaiton e al aiton is a per asi e m lti pervasive multi- economy initially saw a decline function, multi-role, multi-player here. But with an increase in By- enterprise: context-dependent here, Maori-For-Maori funding, Maori For Maori funding context independent there biased there, evaluation grew. here, objective there.” © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 6
  • 7. The Historical Context - Māori • Indigenous people around the world have to advocate for the legitimacy & validity of their knowledge because of the privileging of Western Science. • The ability of indigenous knowledge is recognised for its importance to community survival as they are ultimately the people who depend on such knowledge for their survival & the ones capable of sustainably conserving it. • The link between the physical & social envrionment is evident in Maori literature and provides the basis for Kaupapa Maori research methodology; p p gy • Evolved in response to colonisation processes • Theory and practice considered equally valid y y • Derived from a Matauranga Maori ontology © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 7
  • 8. What the people are saying... Objective 1 • In a political environment where efficiency equals success, success which translates as more output for less resource, the ability to attain predetermined contracted outputs influenced decisions regarding success factors: p g g … the way we define success is different compared to our funders… they’re still motivated by the numbers coming in, which means we need to be motivated by the numbers coming in too… It’s the same for our team, we It s spread our networks when we have a hui and we might only have 1 person or 5 people, but if we can change the lives and help those people it’s been a success for us. (kaimahi tumuaki, kaupapa māori provider, R5) The reporting template is based solely on numbers and I think it’s once or twice a year that [we get to] submit a narrative report. Some of our contracts are different, but the onus is on us to back up why the stats are low and all the th th other work th t’ gone i t a programme, and th outcomes we’ve h d k that’s into d the t ’ had that can’t be measured by a number… (kaimahi, iwi, provider, R2) © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 8
  • 9. What the people are saying... Objective 2 • Understanding of the role evaluation has: … Having an external person come and check that the outcomes are being delivered to our service users… [They have a] more objective opinion, and because they don’t have personal contact… they can come in say what’s missing in a programme even if that person who’s running it is a nice person… (kaimahi, māori provider, R5) ...having evaluators come in and they had Maori researchers on their team who came to evaluate our programme was great for me. Because they brought their p g plan and explained how they were g g to evaluate the whole p y going programme. After critically examining all the components, they found that the most successful and least resourced services of the programme were [the Maori services]… Other people who had been trying to get rid of my role were stopped in their tracks with that evaluation I saw [the evaluation] as evaluation. confirmation and recognition of my work… [The evaluation] also provided some useful strategies for moving [the service] forward even more. (former kaimahi, mainstream provider, R2) , p , ) © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 9
  • 10. What the people are saying... Objective 3 • Relation of cultural factors to their work/evaluation: … we should… embrace their [whanau/community] suspicion, embrace their fear – even as one of their own. To avoid and ignore them means you think you know better…(whānau, R6H7) ... [Maori] evaluators have to be fluid in their relationships with either Maori or non-Maori… non Maori because [they] have to negotiate their way amongst a range of non-Maori colleagues who could be informed, or misinformed about Treaty relationship issues. (evaluator, R6H5) The level of effort needed to bring [non-Maori] evaluators firstly to the table and then to engage takes a huge amount of energy. That often has to come from [Māori] at the cost of energy going towards furthering our own f [Mā i] t th t f i t d f th i agendas… (evaluator, H4) © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 10
  • 11. What the people are saying... Objective 4 • Factors that support/impede cultural inclusion in evaluation: • Support factors are: • Partnership relationships • Consultation and active involvement • Recognition of our tino rangatiratanga • Impeding factors are: • Desire of government/funders to “follow suit” internationally • Credibility is attributed off shore • Current political climate / election cycles © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 11
  • 12. What the people are saying... Objective 5 • Congruence with international considerations about cultural considerations in evaluation: • Issues of cultural consideration were severely limited, and i most cases non-existent. While my online d in t i t t Whil li examination of various evaluation bodies returned limited consideration, there did appear to be a g , pp growingg consideration of cultural matters amongst societies. The exceptions were with American and Australia. • Across all of the evaluation bodies indigenous matters did not exist. An exception could be made with regards to one Regional Chapter (Saskatchewan) of the Canadian Evaluation Society (CES) who had created space for 2 native members or their representatives on its executive. © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 12
  • 13. The COMPLEXITY (enactment) • How do you engage a multi-level, political process that satisfies differing needs? g • Recognising diversity • Whose values count? • Power and control contention • International context • Indigenous voice not considered/engaged • American driven knowledge construction • Australian considered second leaders (asking NZ/Māori for direction) • Māori context • Ethical approach fundamental • Autonomy / self determination paramount © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 13
  • 14. WHERE TO FROM HERE? • Emphasis on & influence from Science with its preoccupation with methodolatry, validity and methodolatry validity, maintaining social compliance (Chamberlain, 2000) means that the focus is on change at the lower levels of g importance; eg. changing processes rather than the allocation of resources, ownership, power and control (Tino Rangatiratanga) Rangatiratanga)… …paradoxically, while critical researchers locate the powerful in their analyses of problems, they exclude them from their solutions. The exclusion or bypassing of the powerful is counterproductive, given critical theorists’ own claims that they are frequently partially responsible for the problem, through their direct or indirect control of the economic, political or communicative practices which sustain it. Unless revolutionary change is advocated or it contemplated, social change requires the involvement of the powerful in the process of education and action designed to serve the critically examined interests of all (Robinson, 1993, p.236) © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 14
  • 15. REFERENCES • Guba, E., & Lincoln, Y. (1985). Effective evaluation: Improving the usefulness of evaluation results through responsive and naturalistic approaches (4th edition ed.). San Francisco: Jossey-Bass Inc. • Hendrix, K. G. (2002). "Did being black introduce bias into your study?": Attempting to mute the race-related research of black scholars. The Howard Journal of Communications, 13, 153-171. , , • Henry, E., & Pene, H. (2001). Kaupapa Māori: Locating indigenous ontology, epistemology and methodology in the academy. Organization: Speaking Out, 8(2), 234-242 . • Indīgena, F. S., & Kothari, B. (1997). Rights to the Benefits of Research: Compensating Indigenous Peoples for their Intellectual Contribution. Human Organization, 56(2), 127-137. g , ( ), • Keefe, V., Cram, F., Orsmby, C., & Ormsby, W. (1998). Representing the ‘other’ when they’re our relations. Paper presented at the Cultural Justice and Ethics Symposium: New Zealand Psychological Society Annual Conference, Wellington. • Mathison, S. (1993). Rethinking the evaluator role: Partnerships between organizations and evaluators. Paper presented at the American Evaluation Association Annual Meeting: The promise and practice of evaluation in organizations that learn, Albany, USA. • Moewaka Barnes, H. (2003). Maori and evaluation: some issues to consider. In N. Lunt, C. Davidson & K. McKegg (Eds.), Evaluating Policy and Practice: A New Zealand Reader (pp. 146-151). Auckland: Pearson Education New Zealand Limited. Limited • Neuman, W. (2000). The meanings of methodology. In Social Research Methods (4th ed., pp. 63-88). Boston: Allyn and Bacon. • Patton, M. (1997). Utilization-Focused Evaluation: The new century text (3rd ed.). Thousand Oaks, California: Sage. • Pih Pihama, L., Cram, F & Walker, S. (2002). C ti methodological space: A lit t L C F., W lk S (2002) Creating th d l i l literature review of K i f Kaupapa M i Maori research. Canadian Journal of Native Education, 26(1), 30. • Schreiber, L. (2000). Overcoming methodological elitism: Afrocentrism as a prototypical paradigm for intercultural research. International Journal of Intercultural Relations, 24, 651-671. • Smith L T (1999) Decoloni ing methodologies Research and indigeno s peoples London and D nedin Zed Smith, L. T. (1999). Decolonizing methodologies: indigenous peoples. Dunedin: Books and University of Otago Press. © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 15
  • 16. The University of Waikato Private Bag 3105 Hamilton, New Zealand 0800 WAIKATO © THE UNIVERSITY OF WAIKATO • TE WHARE WANANGA O WAIKATO 4 September 2009 16