1. HEALTH PROMOTION INTERNATIONAL Vol. 16, No. 4© Oxford University Press 2001. All rights reserved Printed in Great BritainSetting standards in the evaluation ofcommunity-based health promotion programmes—a unifying approachJENNI JUDD, C. JAMES FRANKISH1 and GLEN MOULTON1Territory Health Services, Darwin, and Deakin University, Australia and 1Institute of Health PromotionResearch, University of British Columbia, Vancouver, CanadaSUMMARYCommunity-based health promotion often emphasizes A typology of standards is presented. Arbitrary, experientialelements of empowerment, participation, multidisciplinary and utility standards are based on perceived needs andcollaboration, capacity building, equity and sustainable priorities of practitioners, lay participants or professionaldevelopment. Such an emphasis may be viewed as being in decision-makers. Historical, scientific and normativeopposition to equally powerful notions of evidence-based standards are driven by empirical, objective data. Proprietydecision making and accountability, and with funders’ and and feasibility standards are those wherein the primary con-government decision-makers’ preoccupation with measur- cern is for consideration of resources, policies, legislationing outcomes. These tensions may be fuelled when com- and administrative factors. The ‘model’ standards approachmunity practitioners and lay participants feel evaluations is presented as an exemplar of a combined approach thatare imposed upon them in a manner that fails to appreciate incorporates elements of each of the other standards. Wethe uniqueness of their community, its programme, and argue that the ‘optimal’ standard for community-based healthpractitioners’ skills and experience. This paper attempts to promotion depends on the setting and the circumstances.provide an approach that depicts evaluation as being mutu- There is no ‘magic bullet’, ‘one-size-fits-all’ or ‘best’ stand-ally beneficial to both funders/government and practitioners. ard. Further, we argue that standards should be set from anFirst, a values stance for health promotion, termed a ‘salu- inclusive, salutogenic orientation. This approach offers atogenic’ orientation, is proposed as a foundation for the means of creating a situation in which policy-makers andevaluation of community-based health promotion. Secondly, funders are more supportive of evaluation designs thatwe discuss possible objects of interest, the first component fit with community realities, and community stakeholdersof an evaluation. We then discuss the spirit of the times and are more capable and consistent in rigorously evaluatingits implications for community-based health promotion. community-based health promotion programmes andFinally, we address the key question of setting standards. policies.Key words: community-based health promotion; evaluation; salutogenic; standardsINTRODUCTIONCommunity-based health promotion programmes ing and control over personal, social, economicoften emphasize empowerment, participation, and political forces in order to take action tosocial and sustainable development, multidis- improve their life situations (Israel et al., 1994),ciplinary collaboration, capacity building and and capacity building refers to the problem-equity. Empowerment, in its most general sense, solving capability among individuals, organizations,refers to the ability of people to gain understand- neighbourhoods and communities (Hawe, 1994); 367
2. 368 J. Judd et al.in the context of health promotion workers, it The issues raised in this paper are closely relatedrefers to their ability to enhance the capacity of a to the sustainability and improvement of pro-system to prolong and multiply health effects, grammes, and the health of the communitieswhich represents a ‘value added’ dimension to served. Practitioners are often concerned thathealth outcomes offered by any particular health their programmes will not be continued due to apromotion programme (Hawe et al., 1998). This perceived lack of success by decision makers.emphasis is often juxtaposed with equally power- We assert that standards for the evaluationful notions of evidence-based decision making of community-based health promotion are, forand accountability, in that funders and govern- the most part, implicitly defined or assumed.ment decision-makers are frequently more con- Secondly, we assert that standards employed incerned with measuring outcomes and defining the evaluation of community-based health pro-success. Community practitioners and lay partici- motion that are not expressed may succumbpants often feel that evaluations are imposed to the same fate as health promotion indicators;upon them, and that the evaluation process does i.e. they ‘are often predetermined and shaped bynot appreciate the uniqueness of their community, those in political, administrative and economicits programme, and its resources and skills fields’ [(St Leger, 1999), p. 194]. More import-(Labonte and Robertson, 1996; Trussler and antly, they are often ignored or forgotten as aMarchand, 1998). programme proceeds over time. We endorse Portraying these viewpoints in a dichotomized making the use of such standards more explicitmanner may appear overly simplistic. We present and transparent in a collaborative process. Wethem in this manner to assert that the issues raised recognize that the word ‘standard’ may have aby both sides represent legitimate concerns pejorative connotation for some stakeholders inwithin the practice of community-based health community-based health promotion; however,promotion. It is essential to recognize that all we argue that all stakeholders consciously orparties involved are seeking to provide the most unconsciously employ their own standards inworthwhile programmes or policies to a designated assessing programme quality.community. The progression from programme The first section of the paper underscores theobjectives, to strategies employed, to data collected, values and philosophy of health promotion as itto definitions of programme success is often not relates to the evaluation process. Next, we discussoperationally articulated in a transparent, meas- possible objects of interest, the first componenturable fashion. The focus of this paper is on the of an evaluation, in community-based healthlatter stage (i.e. setting of standards), which we promotion. Then, we further explicate the spiritargue is the least developed in health promotion. of the times and its implications for community- While concerns for accountability and outcomes based health promotion. Finally, we present aare part of our current zeitgeist (spirit of the taxonomy of ‘standards’ for evaluating community-times), evaluation should not be a disempower- based health promotion against which suching process. Rather, it can contribute to ‘the pro- objects of interest can be measured.cess of enabling people to increase control over, For our present purposes, we endorse theand to improve their health’ (World Health definition of evaluation proposed by Green andOrganization, 1986). If practitioners are provided Kreuter (Green and Kreuter, 1999), namely thatwith adequate support for conducting an evalu- evaluation involves the comparison of an objectation, they are highly motivated in knowing if of interest against a standard of acceptability.they are making a difference, and how they can We believe all stakeholders have a role in articu-improve their programme. lating the objects of interest (e.g. changes in health This paper provides an approach that depicts status, community development, intersectoralevaluation as being mutually beneficial to all collaboration) and standards of acceptability forstakeholders. Our aim is to make the evaluation a given programme or policy, not just the personprocess more transparent and collaborative so who commissions the evaluation. We endorse thethat all parties will be satisfied, and gain from the use of a comprehensive, diverse set of standardsoutcomes of community-based health promotion that reflects different concerns and forms ofevaluations. It will assist practitioners and evidence related to the evaluation of healthdecision-makers in defining programme success promotion programmes. This approach offers aat the outset of a programme and/or its evalu- means of creating a situation in which policy-ation, rather than being the last issue discussed. makers and funders can be more supportive of
3. Setting standards in health promotion evaluation 369evaluation designs that fit with community real- Social capital can be used to measure the capacityities, and community stakeholders can become of the social linkages and their resilience or fragility.more capable and consistent in evaluating their Social capital is iterative and experientially developed,health promotion programmes and policies. and requires both levels of trust and competence in social interaction’ [(Cox, 1997), p. 2] [see also (Putnam,Evaluation should facilitate understanding by all 1993; Kawachi et al., 1997; Lomas, 1998)].stakeholders. Although we refer to community-based health promotion, the issues raised maybe relevant to health promotion interventions Using this approach, evaluation standards shouldat other levels (i.e. national) in other settings maximize human health, quality-of-life and well-(i.e. workplace- or school-based health promotion) being. This view also recognizes that health hasand other disciplines. an instrumental value rather than being an end in itself. We propose that, from a salutogenic orientation,VALUES AND RELATED ISSUES IN evaluation standards in community-based healthTHE EVALUATION OF COMMUNITY- promotion must consider the values and pertin-BASED HEALTH PROMOTION ent issues of health promotion in appraising the success or failure of a given initiative (Labonte,Community-based health promotion is explicitly 1996; Hancock et al., 1998). While the purpose ofconcerned with a vision of a preferred future this paper is not to reiterate all such values and(Labonte, 1996). This vision includes a viable issues, we highlight those that we deem to benatural environment, a sustainable economic critical.environment, a sufficient economy, an equitable First, power is central to practice; accordingly,social environment, a convivial community and the proposed salutogenic view is consistent witha liveable environment (Labonte, 1993). This an egalitarian approach that rejects professional‘vision’ can be encapsulated in what Antonovsky dominance in the decision making surroundingtermed as a ‘salutogenic’ orientation to health programme evaluations. Programmes are evalu-(Antonovsky, 1979; Antonovsky, 1996). We ated ‘by real people in complex organizations thatpropose salutogenesis as the core or foundational are marked by historically developed and struc-value underlying the development, articulation turally organized power relations and humanand implementation of standards for community- wants and interests’ [(Cervero and Wilson, 1994),based health promotion programmes or policies. p. 249]. Multiple stakeholders (i.e. consumers/ The word salutogenic derives from a com- citizens, practitioners, managers and decision-bination of ‘salus’ meaning health, and ‘genesis’ makers) have a role to play in evaluation. Healthmeaning to give birth. Salutogenesis literally promotion demands coordinated action andmeans ‘that which gives birth to health’. In trad- collaboration among governments, health, socialitional public health and community medicine and economic sectors, non-governmental andapproaches, a ‘pathogenic’ perspective, in which voluntary organizations, local authorities, in-the focus is on disease or illness and its preven- dustry and the media to promote individual andtion or treatment, most often dominates inter- community health. Scientific and local indigen-ventions. Adoption of a salutogenic perspective ous knowledge each have a contribution to makehighlights the importance of starting from a towards the evaluation of programmes andconsideration of how health is created and main- policies.tained through community-based health promotion Secondly, the salutogenic view recognizes that(Cowley and Billings, 1999). Salutogenesis sug- health promotion is people-centred and collect-gests a link to notions of ‘social capital’, capacity ivist (Raeburn and Rootman, 1998). It is at oddsbuilding and citizen engagement in that it focuses with a strong emphasis on individual responsi-on activities that seek to maximize the health bility for health that ignores the impact of social,and quality-of-life of individuals, families and cultural, economic and environmental determin-communities. Social capital is defined here as: ants of health. Health promotion seeks to maximize the inclusion or involvement of individuals or‘the factor that allows collective action in the public groups who have been historically marginalized,sphere and for the common good. It is social cohesion, such as Aboriginal peoples and the poor—thoseand comprises attention, engagement and trust of both with the poorest health status. Participation andnon-familiar people and the institutions of governance. ownership of the programme by the community
4. 370 J. Judd et al.facilitates problem solving, builds community made at an individual level can underestimatecompetence, and creates successful, sustainable the gains that an intervention might make.programmes, rather than programmes that areimposed by outsiders (Eisen, 1994; Camiletti,1996). Stakeholders of programmes and evalu- OBJECTS OF INTEREST IN THEations should recognize that communities are EVALUATION OF COMMUNITY-dynamic, and socially, culturally and economic- BASED HEALTH PROMOTIONally heterogeneous. Therefore, strategies need tobe adapted to local needs and possibilities. Before one can set standards for a community- Thirdly, the proposed salutogenic approach to based health programme there is the need tocommunity-based health promotion clearly ques- articulate relevant ‘objects of interest’ (i.e. thosetions the dominance of economic rationalism and factors or variables that will be tracked andmarket ideology in public policy (Labonte, 1996). assessed in a given evaluation). The objects ofIt is explicitly concerned with the aforemen- interest for community-based health promotiontioned vision of a preferred future that includes a programmes or policies can be quite diverse.viable natural environment and a sustainable Below we discuss several issues related to theeconomic environment. Its emphasis is on equity nature of the community-based health pro-rather than productivity; and health rather than gramme as it relates to the question of ‘objects ofwealth. interest’ for evaluation. Underpinning each of these values and the The diversity of potential objects of interestadoption of a salutogenic approach to setting derives first from the fact that health promotionstandards is the notion of empowerment— interventions can occur at multiple levelsincreasingly recognized as a key element in the (biomedical, lifestyle or behavioural, and socio-evaluation of community-based health pro- environmental) and in diverse settings withinmotion (Fetterman et al., 1996). Empowerment a community. Internationally, community-basedis usually described as a process, but may be health promotion has become a major strategyconsidered an outcome variable (i.e. an object of that has received prominence through majorinterest) when capacity building is a major studies [e.g. Stanford Five City Project, PATCHactivity of a community intervention. Empower- (Planned Approach Towards Community Health),ment encompasses participation, multidisciplin- and various Healthy Cities approaches]. Manyary collaboration, equity, capacity building, and such projects have faced major challenges andsocial and sustainable development (Hawe, 1994). many have not achieved great changes in healthApproaches such as empowerment evaluation (Syme, 1997; Green and Kreuter, 1999; Potvin(Fetterman et al., 1996), participatory research and Richard, 2001). Programmes have not always(Green et al., 1995), participatory evaluation been relevant to those being targeted and inter-(Health Canada, 1996) and ‘responsive construct- vention methods are not always appropriate toive evaluation’ (Guba and Lincoln, 1989) foster those involved. The focus on communities doesthe systematic generation of new knowledge and not always take into account the diverse sub-social capital. This is done through a process that groups and the social context in which people livebuilds upon the skills and experiences of all parties and work (Syme, 1997). Nutbeam and his col-involved, and contributes to quality-of-life and leagues (Nutbeam et al., 1993) concluded that thewell-being. Such evaluations move toward saluto- information gained in large-scale programmesgenesis in that they offer viable possibilities for can disseminate quickly to surrounding jurisdic-the evaluation of community-based health pro- tions and interfere with classic intervention andmotion programmes. They are commensurate evaluation designs through contamination.with the unifying approach to setting standards Secondly, health promotion often employsas detailed in this paper. multiple strategies, including creating healthy With these principles in mind, evaluation is public policy and supportive environments,necessarily a collaborative group activity, funda- fostering individual or group skills and capacities,mentally democratic, participatory, and must strengthening community action and reorientingexamine issues of concern to the community in health services (World Health Organization,an open forum. Evaluations of community-based 1986). It may try to modify the social contexthealth promotion programmes limited to aggre- that influences health behaviours as a means togates of changes in health behaviour or attitudes achieve improved quality-of-life and well-being.
5. Setting standards in health promotion evaluation 371In this regard, health promotion is wholly con- in their evaluations. They should attempt tosistent with what Hamilton and Bhatti termed integrate ‘process’ evaluation (i.e. intervention‘population health promotion’ (Hamilton and activities, staff performance, etc.) with ‘impact’Bhatti, 1996), and with notions of ‘population evaluation (i.e. proximal, intermediate changeshealth’ and the determinants of health that have in behaviour, lifestyle and the environment) andcome to dominate the health discourse in Canada with ‘outcome’ evaluation (i.e. distal, longer-termand elsewhere (Frankish et al., 1999). changes in policy, health status, etc.). While one Community-based health promotion program- need not address every level of evaluation in ames are often large in scope, have extended time single project, it is useful to consider each levelframes and require many resources. We argue from a conceptual and planning perspective. Thethat health promotion programmes are most recent shift toward a ‘population health’ ap-likely to be beneficial when they are flexible and proach in which the objects of interest are distal,responsive to changing realities. Health promotion non-medical determinants of health furtherprogrammes and associated evaluations must complicates the evaluation process.also accommodate diverse definitions of what the The preceding section highlights some of theterm ‘community’ means. Community has all of complexities associated with defining ‘objectsthe following elements—identity, geography and of interest’ for health promotion. Such objects ofpolitics. Finally, they must struggle with issues of interest to be included in an evaluation needrepresentativeness and who can speak for a given to be clearly delineated at the outset of thecommunity (Wiesenfeld, 1996). process. However, in keeping with our definition Evaluations of community-based health pro- of evaluation, the identification of objects ofmotion programmes may be quantitatively and interest is only the first step. Next, programmequalitatively distinguished from typical experi- planners and decision-makers must articulatemental studies to the degree that they embrace ‘standards of acceptability’ for each object of in-a multi-level, multi-strategy vision of individual terest. That is, they must choose where they willand environmental change. In this regard, a ‘set the bar’ and how they will define the successwide spectrum of evaluation approaches has been of each element of a given programme or policy.used in community health promotion. These As Patton states:incorporate highly structured, methodologicallydriven evaluations, including randomized con- ‘Objectives are often set a long time before thetrol trials (RCTs) through to much less rigidly programme is under way or well before an actualstructured, highly participatory forms of research evaluation has been designed. Reviewing objectives and establishing precise standards of … [acceptability]and evaluation as discussed in the previous sec- just before data collection increases the likelihood thattion (Nutbeam, 1998). Little consensus has been judgement criteria will be up to date, realistic, andreached about the most appropriate method meaningful’ [(Patton, 1997), p. 304].of evaluating community-based programmes.Nutbeam encourages an ‘expansionist’ approach If they are unwilling or unable to set standardsto evaluation that would consider the range of before data collection in a calm and deliberatestrategies employed, the different outcomes from manner, there is no reason to believe they canthose strategies and provide a wide range of do so afterward (Patton, 1997). One means ofpotential indicators of success (Nutbeam, 1998). facilitating the setting of standards is to createConsideration of a broad range of measures of speculative or dummy data for the objects ofsuccess or standards of acceptability fits much interest; this makes the process more concrete.more comfortably with modern concepts of The explicit articulation and linking of standardshealth promotion. The development of indicators and objects of interest will facilitate a worthwhileand instruments that measure these changes is evaluation that in turn will improve the pro-equally important [see (Nutbeam, 1998)]. gramme, and the health of communities. Based upon the complexity of health pro-motion programmes delineated in the precedingparagraphs, it is evident that the possible objects THE SPIRIT OF THE TIMESof interest in an evaluation are vast. Ideally,community-based health promotion programmes In recent times, many health promotion program-have a balanced emphasis on processes, impacts mes, their evaluations, and standards of accept-and outcomes that serve as the objects of interest ability seem to be driven more by a concern for
6. 372 J. Judd et al.the electoral cycle than by scientific evidence result in drawing inappropriate conclusionsor community relevance. This drive towards regarding health promotion practice.‘accountability’ stems from a public demanding In summary, the evaluation of community-greater responsiveness of health professionals based health promotion programmes differs inand policy-makers, and concerns regarding allo- substantive ways from controlled experimentalcations of economically pressed health resources studies (see next section). Many of the traditionalby governments, health care providers and organ- assumptions of positivist research either do notizations (Alexander et al., 1995; Solberg et al., hold or are very difficult to apply in a community1997; Morfitt, 1998; Zakus, 1998). This drive is in setting. Having recognized these difficulties doestension with a parallel, increased interest in social not mean that community stakeholders are freecapital and the role of community-level factors in from responsibility for evaluation and/or account-generating healthy communities (Minkoff, 1997; ability. Rather, there is a need for a balancedRose et al., 1997; Eastis, 1998; Lomas, 1998). It is approach to evaluation that accommodates bothoften in conflict with the idea that programmes community realities and decision-makers’ concernsmay be more effective if they emerge from local for evidence and accountability.consensus and priorities (Health and Welfare The next section presents an integrativeCanada, 1990; Tabrizi, 1995; Zakus and Lysack, approach to setting standards in community-1998). based health promotion. Our purpose is to Within the current economic and political frame evaluation as a win-win, collaborative andclimate there is also strong pressure to incorpor- capacity-building exercise.ate what is termed ‘evidence-based practice’—aborrowed medical paradigm that has been appliedmainly to clinical decision making. Evidence- THE USE OF STANDARDS INbased practice is a framework in medicine for EVALUATING COMMUNITY-BASEDasking questions, tracking new types of strong HEALTH PROMOTION INITIATIVESand useful evidence, distinguishing it from weak,irrelevant or useless evidence, and putting it into We recognize that community-based healthpractice. The concern with this approach is the promotion (and its evaluation) is a multi-stageundue emphasis that is placed on RCTs and process involving the setting of objectives, themeta-analysis (Rada et al., 1999). Not all inter- execution of strategies, the collection of data, andventions can be investigated by these methods, an assessment or appraisal of the relative successnor can they, in the case of health promotion prac- or failure of a given intervention. Glasgow, Vogttice, be economically justified. While approaches and Boles proposed a similar comprehensivethat consider clustering issues [see (Simpson evaluation framework, where they argued thatet al., 1995; Thompson et al., 1997; Hayes and multifaceted interventions incorporating policy,Bennett, 1999)] can come closer to addressing the environmental and individual components shouldrealities of conducting community-based evalu- be evaluated with measurements suited to theirations, they are not wholly satisfactory. settings, goals and purpose (Glasgow et al., 1999). Most stakeholders support the need for a con- They proposed the RE-AIM model for evalu-ceptually sound evidence base for interventions ating public health interventions that assessedthat aim to promote health. However, the cur- five dimensions: reach, efficacy, adoption, imple-rent search for evidence using methods and mentation and maintenance.strategies that do not fit with community It is the process of stating objectives and asso-realities is unlikely to succeed. Health prom- ciated standards that is of interest here. Threeotion programmes may be at risk of the appli- elements are central to our proposed approach.cation of inappropriate methods of assessing The first is our strong endorsement and adoptionevidence, an over-emphasis on health status of a salutogenic stance and values base. Theoutcomes and individual behaviour change, and second is our recommendation for the use of aan increased pressure on precious resources comprehensive, diverse set of standards that(Speller et al., 1997b). These emphases may be reflect different concerns and forms of evidence.to the detriment of important considerations The third is for the use of an inclusive, empower-and evidence relating to the building of com- ing process of dialogue that engages all relevantmunity capacity and addressing the broader, stakeholders in the setting of standards for anon-medical determinants of health. It may also given initiative.
7. Setting standards in health promotion evaluation 373 A ‘standard’, as defined by the Webster’s New The following section identifies eight ap-Collegiate Dictionary (1979) is something estab- proaches to setting standards that we arguelished by authority, custom or general consent should be considered in community healthas a model or example. In the health promotion promotion. We discuss the relative strengths,context, standards of acceptability serve to weaknesses and applicability of each approach.identify the desired level of outcome and allow The different approaches to setting standards areall parties to agree on how much change should organized according to what Green and Kreuterbe achieved in return for a given investment term the ‘three world views of population needsof resources. They serve as targets, which, when and planning’ (Green and Kreuter, 1999) (seemet or exceeded, signal success, improvement or Figure 1).growth. Standards can be technically, procedur- Arbitrary, experiential and utility standardsally, system- or outcome-oriented. fall into the upper left circle, in which planning Many fields and disciplines have utilized (and evaluation) is primarily driven by thestandards of acceptability (McKenzie and Jurs, perceived needs, values and expectations of1993; McKenzie and Pinger, 1997; Green and practitioners, lay participants or professionalKreuter, 1999). Similarly, Patton uses the phrase decision-makers. Historical, scientific and norm-‘standards of desirability’ to evaluate program- ative standards fall into the upper right circle,mes (Patton, 1997). The use of standards in where planning and evaluation are driven bycommunity-based health promotion is in keeping empirical, objective data. Finally, propriety andwith the parallel movement toward use of a broad feasibility standards fall into the bottom circle,range of community health indicators identified wherein the primary concern is for considerationthrough a collaborative process (Hancock et al., of available resources, existing policies, legis-1998) [see also a special issue of Health lation and administrative factors. Objective andPromotion International (1988), 3 (1)]. For health policy-related standards (which are like scientificpromotion programmes, the standards will be standards) are often given greater weight bythe expected level of improvement in the social, external decision-makers than those in the uppereconomic, health, environmental, behavioural, left circle. The intersection of the three circleseducational, organizational or policy conditions represents what has been termed ‘model’ stand-stated in the programmes’ objectives and repre- ards (American Public Health Association, 1991).sented in the associated objects of interest for Our position is that there is nothing inherentlyevaluation. superior about any one of the eight types ofFig. 1: Setting standards for evaluation. Adapted from Green and Kreuter (Green and Kreuter, 1999).
8. 374 J. Judd et al.standards. Judgment and discretion are unavoid- evaluation processes helped to document the needable, and to some degree desirable, in decision for, and the effectiveness of their programme.making, which operates within a paradigm orenvironment that shapes the process and the Utility standardsoutcome(s). Decision making is a social process Utility standards are intended to ensure that aand methods are social constructions that are community-based health promotion programmehistorically determined and situated, and build will serve the needs of programme recipients,only on existing knowledge (Potvin et al., 1994; community stakeholders, practitioners andPotvin, 1996). The more important question is: government decision-makers (Joint Committeewhich type of standard fits, with which questions, on Standards for Educational Evaluation, 1994).in what circumstances? This approach may include a priori identification of stakeholders and their needs, and the selection of pertinent evaluation questions. Although needs-Standards based on perceived needs based or utility standards have the potential ad-and priorities vantage of relevance to local circumstances, theyArbitrary standards may be limited in their representativeness.Arbitrary standards are a simply declared orexpected level of change, and are most often put Standards based on objective dataforward by individuals or groups in a position ofauthority. An example of an arbitrary standard is Historical standardsone in which a decision-maker sets the standard Historical standards are based on previousfor a given initiative without sufficient consulta- performance and data. Generally, this methodtion with important stakeholders and/or consid- applies to outcome objectives that can be easilyeration of available relevant information. measured such as attendance at clinics, and birth An advantage of arbitrary standards is the or mortality rates. They are incremental in nature,efficient way in which they are created. Some and are most useful in situations in which datadisadvantages include that such standards may are routinely accessible.be biased in favour of their creator’s point of view The use of historical standards has severaland the process may be perceived as dictatorial potential advantages. Practitioners may be moreand non-inclusive. For communities, arbitrary comfortable with these standards because theystandards are often not realistic, and often little have been previously involved in devising and/orownership or motivation to meet such standards carrying out these standards. Their skills haveexists. Arbitrary standards are not capacity build- been developed, and can build on previous suc-ing, and thus practitioners and the communities cesses. A practitioner’s role in the developmentthey serve are likely to have little commitment of historical standards may be of a technicalto facilitating or participating in such a health- nature, such as collecting and interpreting data.promoting project. Historical standards are not necessarily a single point but may represent several pointsExperiential standards across time, as in trend analysis. The benchmarksExperiential standards involve a community’s may be transparent and repeatedly collected inperceived needs and priorities. They recognize a consistent manner. For example, Serxner andthe value and utility of local, indigenous know- Chung conducted a trend analysis of social andledge and are community-specific. Their use is in economic indicators of mammography use intension with other types of standards (i.e. norm- Hawaii (Serxner and Chung, 1992). Systems likeative, scientific) that are based on external data or the Behavioral Risk Factor Surveillance Systeminformation drawn from other jurisdictions. Some offer the necessary longitudinal data. Similarly,communities may want to emphasize process and/ Hughes and Cox examined breastfeeding initi-or unanticipated outcomes. Some decision makers ation in Tasmania by demographic and socioeco-may perceive these issues as a disadvantage in nomic factors for the period 1981–1995 (Hughesthe use of experiential standards. Rodney et al. and Cox, 1999). They noted that trend data isoffer an example of the use of indigenous an important component of infant health andknowledge as it relates to the evaluation of a nutrition monitoring and surveillance systems. Itcommunity health advocate programme (Rodney is also an important basis for identifying breast-et al., 1998). Their use of three interrelated feeding promotion needs, prioritizing target
9. Setting standards in health promotion evaluation 375groups and strategies, and in evaluating the successfully elsewhere as a standard for evalu-effectiveness of breastfeeding promotion efforts. ation, they may allow for comparative interven- There are several potential limitations or tions across jurisdictions. Qualities of credibility,disadvantages to the use of historical standards. efficiency and feasibility are often associatedThey may be skewed and data may not be attrib- with normative standards and may enhance theutable to a health promotion programme when probability that health promotion planners willunique phenomena occur. Such phenomena may endorse this type of standard.be the result of new policies and/or media cam- There are several potential limitations to thepaigns within an altered socio-political context. use of normative standards. For communitiesThese standards may be flawed if they are based or states, normative standards set in relation toupon inaccurate or biased data. In such incidences, other jurisdictions may be unrealistic and/orhistorical standards only serve to replicate an in- unachievable, and may not represent a priorityherent error. By their nature, historical standards focus for a specific jurisdiction. A further prac-are not appropriate for new programmes since tical difficulty is that of finding an appropriatethere is no pre-existing data. comparison community or jurisdiction. In some cases, the appropriateness of using one com-Normative standards munity’s achievements for another community’sNormative standards, as with historical stand- standard can be questioned. Some communitiesards, are those wherein data such as the state or (i.e. Aboriginal or low socioeconomic groups)national average for a given health behaviour is have become frustrated with evaluation reportsroutinely collected. Normative standards are continually positioning them at the bottom.usually based on what other programmes or Others question the feasibility of generatingorganizations in similar settings have achieved, community-specific data in order to demonstratewith the advantage that these may be used as a normative comparison.benchmarks. In this case, the benchmark is a Finally, there are ethical concerns related tolevel, and may or may not represent a point in the question ‘what makes a fair comparison acrosstime. To use this method, documentation must be communities?’. Fair is a relative term, and isavailable to practitioners. dependent upon the resources at one’s disposal. In Canada, the British Columbia Ministry of In this regard, it is important to make a distinctionHealth (BCMH, 1994) has produced a frame- between responsibility and reliance. Communitieswork and process for screening for local area and health promotion practitioners can only bebenchmarks that involves selective causes of death expected to meet standards that are consistent(eight indicators), lifestyle characteristics (five with available resources and capacities. Ideally,indicators) and birth factors (four indicators). an evaluation process can assist communities toMore recently, many governments have adopted be more self-reliant and responsible with thea ‘report card’ approach that reports on the resources they do have or to acquire additionalhealth status of a given population, usually on a resources.year-to-year basis. Associated with such reportcards is the parallel proliferation of a host of Scientific standardsnational, provincial or state databases. Each is Scientific standards may be empirically and/orintended to provide the requisite data for making theoretically based, and are developed fromnormative comparisons and planning program- outcomes achieved in controlled studies and gen-mes or policies. It is important to note, however, erally based on systematic reviews of availablethat most of these databases are not oriented literature. Such standards place emphasis ontoward health promotion. Furthermore, indicators RCTs and meta-analysis (Rada et al., 1999).of ‘community health’ or community-level indi- Recent examples include the movement towardcators are often excluded (Frankish and Bishop, ‘best practices’ (Sherman, 1999), the development1999). of ‘preventive practice guidelines’ (US Prevent- Normative standards may provide a clear point ive Services Task Force, 1996) and systematicof reference for health promotion planners research syntheses of the type associated with theand are most likely to be based on ‘objective’ Cochrane collaborations and databases.(quantitative) data. These provide a measure The major advantage of scientific standards isof efficiency because practitioners are ‘not that they are viewed as objective, empirical andre-inventing the wheel’. If they have been used unbiased. They align with a dominant view of
10. 376 J. Judd et al.‘evidence’, which suggests that such standards consider practical issues such as existing policies,are more credible and trustworthy than data or regulations and legislation, logistical factors andevidence generated by other means (i.e. quali- the availability of resources.tative methods). From a positivist perspective,this ‘gold’ standard is only achievable through Propriety standardsempirical science of the type associated with Propriety standards are intended to ensure thatRCTs. community-based health promotion programmes Several disadvantages exist in trying to apply are conducted legally, ethically and with regard‘scientific’ standards to community-based health to the welfare of community participants (Jointpromotion programmes or policies. Such settings Committee on Standards for Educational Evalu-make it impossible to randomly assign individ- ation, 1994). Issues such as formal agreements,uals or groups to a particular community, and it is fiscal responsibility and conflict of interest aresometimes difficult to identify appropriate com- relevant in consideration of propriety standardsparison or control communities. When the unit [see (Roman and Blum, 1987; Jacob, 1994;of analysis is an entire community (rather than Starzomski, 1995; Jenkins and Emmett, 1997)].an individual) it is difficult to manifest the level Brown provides an example of propriety standardsof ‘control’ desired in a typical scientific study. in relation to environmental health issues andIn fact, the complexity of factors associated the US Congress debate over a ‘polluter-pay’with community life is a key to the dynamics of approach to dealing with violations of existingcommunity-based health promotion. Attempting legislation (Brown, 1997).to isolate single variables is contrary to notions ofholism, reciprocal interactions and interdepend- Feasibility standardsence associated with communities. Feasibility standards are intended to ensure that The use of scientific standards in community- the programme will be realistic, prudent and frugalbased health promotion may be perceived as (Joint Committee on Standards for Educationalarbitrary, and their ‘goodness of fit’ to the cir- Evaluation, 1994). Feasibility involves consider-cumstances or needs and expectations of a given ations of cost effectiveness, political viability andcommunity is questionable. There are also ethical practical procedures. One advantage of includingquestions inherent in the notion of ‘control’ feasibility standards is that they may serve as acommunities. Holding some components of a ‘reality check’ with respect to available resources.community’s capacities constant is contradictory They may also act as a catalyst for securingto the empowering, skill-developing process of additional resources. One potential disadvantagecommunity-based health promotion. is that a ‘bottom-line’ mentality may undermine Scientific standards, when imposed by external innovation and creativity. Richardson questioneddecision-makers (e.g. government or funders) are the common belief that economic evaluation isa source of tension for most practitioners and hostile to health promotion and that the require-many health promotion theorists. Randomized ment for health programmes to be cost effectivecontrol trials are time-consuming, expensive, will result in a biased allocation of funds in favourand require a skill level many practitioners do not of programmes that can demonstrate short-termpossess. Community practitioners may not have benefits as defined by inadequate outcome meas-access to relevant data, such as the latest pub- ures (Richardson, 1998). He notes the potentiallished evaluations, which are most often contained for economic evaluation to be counter-productivein academic journals. From a policy perspective, if applied to ‘immature’ projects, and the prac-government decision-makers may not be able or tical problems inherent in the measurementwant to wait for ‘scientific’ data to be generated. of outcomes in health promotion programmes. He proposes a four-fold classification based on a distinction between disease cure, individual healthStandards based on available resources promotion, community welfare and systemicand existing policies change designed to promote either individualWhen it comes to setting standards for community- health or social well-being.based health promotion programmes, planners, Van der Weijden and her colleagues analysedpractitioners and government decision-makers the feasibility of using national cholesterolmust consider different options with respect to guidelines in general medical practices (Van derdata, evidence and benchmarks. They may also Weijden, 1999). Their programme was developed
11. Setting standards in health promotion evaluation 377after barriers to working according to the approach, a lead agency, such as the local healthguideline had been investigated. The quality of department, drives the process of articulatingtargeting of cholesterol testing did not improve ‘model’ standards by organizing the effort andfollowing the intervention. This research demon- providing the needed technical expertise instrated that neither simple dissemination nor an relevant public health practice. The use of a leadintensive programme had a measurable impact agency approach may, however, raise issues ofon performance of work according to the chol- control, questions about roles and responsibil-esterol guideline. Stephenson et al. assessed the ities, and has the potential for disempowermentfeasibility of conducting a large RCT of peer-led of the community members.intervention in schools to reduce the risk of HIV/ Model standards have also been used else-STDs and promote sexual health (Stephenson where (Speller et al., 1997a). A project to developet al., 1998). Questionnaire completion rates of a framework for quality assurance in health90% indicated considerable enthusiasm for peer- promotion practice in England has recently beenled education among educators and pupils. developed. Six key functions of health promotionEvaluation of the behavioural intervention was (strategic planning, programme management,shown to be acceptable to schools, pupils and monitoring and evaluation, education and training,parents, and feasible in practice. resources and information, and advice and con- sultancy) were identified. Model standards and criteria were drawn up for each function, togetherA composite approach to setting standards: with guidance on implementation processes.model standards Model standards may be expressed as pro-The section above highlights a variety of gramme processes, risk factors or objectivesapproaches to setting standards that are relevant related to a specific health outcome. Theseto the practice and evaluation of community- standards need to be flexible to accommodatebased health promotion. The presentation of differences in the mix of contexts and servicesthe eight types of standards recognizes that the available. Stakeholders can therefore participatevarious approaches are not mutually exclusive, in determining their own public health prioritiesnor are they independent. The diversity of ap- that are compatible with national objectives andproaches does beg the question of how different targets.approaches might be combined. These standards represent a form of comprom- One method of combining a variety of stand- ise or consensus standards. They are generallyards is the so-called ‘model standards’ approach. established from a consensus of informed opinionsThis approach is an amalgam and incorporates by professionals and experienced others, andelements of each of the other types of standards. may also have the endorsement of professionalThe term ‘model’ standard is associated with a organizations. A disadvantage of these standardsspecific approach developed in the United States may be the time taken to generate them.in response to Healthy People 2000 and Healthy Compromise standards may be political inCommunities 2000 through the cooperation nature and depend on the quality of the peopleof communities, local health agencies and the involved. Individuals or specific stakeholderprivate sector (APHA, 1991). Similar to its groups may come to the table with diverse andoriginal usage, our use of the term ‘model’ is not sometimes competing/hidden agendas. Whileintended to connote that, in and of itself, this ap- ‘model’ standards suggest an optimal mix ofproach represents the ‘optimal’ or best approach standard setting approaches, for some theseto setting standards in community-based health standards may be settling for the lowest commonpromotion. denominator. With the US approach, model standards weredeveloped to plan programmes and to allocateresources. As a companion to the Healthy People MOVING TOWARD ‘OPTIMAL’2000 report, these standards offer community STANDARDS FOR COMMUNITY-implementation strategies for putting objectives BASED HEALTH PROMOTIONinto practice by establishing achievable communityhealth targets. This method adapts national This paper addresses issues related to evaluationtargets for local relevance and suggests an array and the use of standards in community-basedof activity-based objectives. In the American health promotion. These issues include the
12. 378 J. Judd et al.definition and measurement of relevant out- views of evidence and definitions of success becomes and the use of participatory, empowering examined. In the end, ‘optimal’ standards forevaluation methodologies that assess both the community-based health promotion will be thoseoutcomes achieved and the processes by which that engage diverse stakeholders in a process ofthey are accomplished. collaborative dialogue and decision making. They We recognize that considerable progress has will maximize the fit of the evaluation processbeen made in understanding the complexity of and targets with community capacities, perspect-undertaking evaluations in community settings. ives and resources. Finally, optimal standards willWe acknowledge the corresponding need for tools, help to yield new knowledge that will contributemeasures and evaluation designs that accommodate to health, well-being and quality of life of indi-this complexity. viduals, families and communities. Finally, we recognize two realities. First, that Our hope is that collaborative evaluations willgood science poorly applied will not advance the take into account the varying nature of com-quality and utility of community-based evaluations. munities while building social capital, communityThere is little benefit to be gained from forcing capacity, economic viability and well-being. WellRCT-type designs to be used in circumstances formulated evaluations can assist funders, policy-where they do not fit. Both the process and makers, practitioners and communities in linkingoutcomes of community-based evaluations must the success of specific programmes or policies tobe relevant to community stakeholders, policy- broader contextual economic, environmental ormakers and/or funders. Secondly, the ‘balloons social issues.and t-shirts’ approach to community-based healthpromotion programmes, in which there is little or Address for correspondence: Ms J. Juddno attention paid to evaluation, is equally inap- Territory Health Servicespropriate. Policy-makers, funders and taxpayers PO Box 40596have a right to demand accountability and some Casuarina 0812 NTmeasure of the success of health promotion Australiainitiatives. Our taxonomy of standards, grounded in asalutogenic values stance, is offered as a potential ACKNOWLEDGEMENTSmeans of bridging these ‘two solitudes’. The hopeis to create a win-win situation in which policy- This work was completed while the first authormakers and funders are more supportive of was a Visiting Student at the Institute of Healthevaluation designs (i.e. processes and outcome Promotion Research (IHPR), University ofmeasures) that fit with community realities, and British Columbia and a Doctoral student incommunity stakeholders are more capable and Health Science at Deakin University, Australia.consistent in evaluating community-based health The authors wish to acknowledge the support ofpromotion programmes and policies. Territory Health Services—Long Service Leave We advocate a shift away from a view of (Darwin, Australia) and Health Canada. They alsoevaluation that is dominated by a pathogenic, wish to recognize the support of their colleaguesrisk factor and outcomes-oriented perspective in the Institute of Health Promotion Research.toward a more balanced menu of possible targets From Australia, Penny Hawe, Lawry St Legerfor change and accompanying standards for and Sandy Gifford provided valuable commentsdefining success. This suggestion is not at odds on an earlier version of this manuscript.with standards that are systematic and supportiveof accountability. We conclude by recommendingthat each of our eight types of standards [arbitrary, REFERENCESexperiential (community), utility, historical,scientific, normative, propriety and feasibility] Alexander, J., Zuckerman, H. and Pointer, D. (1995) Thebe considered in planning the evaluation of challenges of governing integrated health care systems.community-based health promotion programmes Health Care Management Review, 20, 69–92.or policies. Explicit consideration of this diverse American Public Health Association (APHA) (1991) Healthy Communities 2000: Model Standards. Americanset of standards may be used to engage all stake- Public Health Association, Washington, DC.holders in inclusive, empowering dialogue. It Antonovsky, A. (1979) Health, Stress and Coping. Jossey-demands that stakeholders’ respective concerns, Bass, San Francisco.
13. Setting standards in health promotion evaluation 379Antonovsky, A. (1996) The salutogenic model as a theory to Hancock, T., Labonte, R. and Edwards, R. (1998) Indicators guide health promotion. Health Promotion International, that Count!—Measuring Population Health at the Com- 11, 11–18. munity Level. Health Canada, Ottawa, ON.British Columbia Ministry of Health and Ministry Hawe, P. (1994) Capturing the meaning of community in Responsible for Seniors (1994) Processes, Benchmarks community intervention evaluation: some contributions and Responsibilities for Developing Community Health from community psychology. Health Promotion Inter- Councils and Regional Health Boards: Meeting the Chal- national, 9, 199–210. lenge, Action for a Healthy Society. BC Ministry of Health Hawe, P., King, L., Noort, M., Gifford, S. M. and Lloyd, B. and Ministry Responsible for Seniors, Victoria, BC. (1998) Working invisibly: health workers talk aboutBrown, K. S. (1997) Off the hook: what Olin might mean. capacity building in health promotion. Health Promotion Environmental Health Perspectives, 105, 44–47. International, 13, 285–294.Camiletti, Y. A. (1996) A simplified guide to practicing Hayes, R. J. and Bennett, S. (1999) Simple sample size community-based/community development initiatives. calculation for cluster-randomized trials. International Canadian Journal of Public Health, 87, 244–247. Journal of Epidemiology, 28, 319–326.Cervero, R. M. and Wilson, A. L. (1994) The politics of Health and Welfare Canada (1990) Prevention through responsibility: A theory of program planning practice Partnership: Collaborating for Change. National Strategy for adult education. Adult Education Quarterly, 45, for Enhancing Preventive Practices of Health Professionals. 249–268. Report of a National Workshop, October 28–31, 1990,Cowley, S. and Billings, J. R. (1999) Resources revisited: Ottawa. Health and Welfare Canada, Ottawa, ON. salutogenesis from a lay perspective. Journal of Advanced Health Canada Population Health Directorate (1996) Nursing, 29, 994–1004. Guide to Project Evaluation: a Participatory Approach.Cox, E. (1997) Building social capital. Health Promotion Available at: http://www.hc-sc.gc.ca/hppb/phdd/guide/ Matters, 4, 1–4. introduction.htm. Accessed on October 21, 1999.Eastis, C. (1998) Organizational diversity and the produc- Hughes, R. and Cox, S. (1999) An analysis of breastfeeding tion of social capital. American Behavioral Scientist, 42, initiation in Tasmania by demographic and socioeconomic 66–77. factors for the period 1981–1995. Breastfeeding Review, 7,Eisen, A. (1994) Survey of neighbourhood-based compre- 19–23. hensive community empowerment initiatives. Health Israel, B., Checkoway, B., Schulz, A. and Zimmerman, M. Education Quarterly, 21, 235–252. (1994) Health education and community empowerment:Fetterman, D. M., Kaftarian, S. J. and Wandersman, A. (eds) conceptualising and measuring perceptions of individual, (1996) Empowerment Evaluation: Knowledge and Tools organisational, and community control. Health Education for Self-Assessment and Accountability. Sage Publications, Quarterly, 21, 149–170. Thousand Oaks, CA. Jacob, F. (1994) Ethics in health promotion: freedom orFrankish, C. J. and Bishop, A. (1999) Background Paper and determinism? British Journal of Nursing, 3, 299–302. Plan for Inclusion of Community Health Indicators in the Jenkins, D. and Emmett, S. (1997) The ethical dilemma of Canadian Community Health Survey. Prepared for the health education. Professional Nurse, 12, 426–428. Canadian Consortium of Health Promotion Research Joint Committee on Standards for Educational Evaluation Centres and the Advisory Committee on the Canadian (1994) The Program Evaluation Standards: How to Assess Community Health Survey, Ottawa, ON. Evaluations of Educational Programs. Sage Publications,Frankish, C. J., Veenstra, G. and Moulton, G. (1999) Popu- Thousand Oaks, CA. lation Health in Canada: a Working Paper. Prepared for Kawachi, I., Kennedy, B., Lochner, K. and Prothrow-Stith, D. the National Conference on Shared Responsibility for (1997) Social Capital, Income inequality and mortality. Health and Social Impact Assessments: Advancing the American Journal of Public Health, 87, 1491–1498. Population Health Agenda. Institute of Health Promotion Labonte, R. (1993) Health Promotion and Empowerment: Research, University of British Columbia, Vancouver, BC. Practice Frameworks. Participaction Series. CentreGlasgow, R. E., Vogt, T. M. and Boles, S. M. (1999) for Health Promotion, University of Toronto, Toronto, Evaluating the public health impact of health promotion ON. interventions: the RE-AIM framework. American Labonte, R. (1996) Health promotion and population Journal of Public Health, 89, 1322–1327. health: what do they have to say to each other? CanadianGreen, L. W. and Kreuter, M. W. (1999) Health Promotion Journal of Public Health, 86, 165–167. Planning: an Educational and Ecological Approach, 3rd Labonte, R. and Robertson, A. (1996) Delivering the goods, edn. Mayfield Publishing Company, Mountain View, CA. showing our stuff: the case for a constructivist paradigmGreen, L. W., George, M. A., Daniel, M., Frankish, C. J., for health promotion research and practice. Health Herbert, C. J., Bowie, W. R. and O’Neill, M. (1995) Study Education Quarterly, 23, 431–447. of Participatory Research in Health Promotion: Review Lomas, J. (1998) Social capital and health: implications and Recommendations for the Development of Partici- for public health and epidemiology. Social Science and patory Research in Health Promotion in Canada. Institute Medicine, 47, 1181–1188. of Health Promotion Research, University of British McKenzie, J. F. and Jurs, J. L. (1993) Planning, Implementing Columbia and the BC Consortium for Health Promotion and Evaluating Health Promotion Programs: a Primer. Research for The Royal Society of Canada, Vancouver, Macmillan Publishing Company, New York. BC. McKenzie, J. F. and Pinger, R. R. (1997) An IntroductionGuba, E. G. and Lincoln, Y. S. (1989) Fourth generation to Community Health. Jones and Bartlett Publishers, evaluation. Sage, Newbury Park, CA. Sudbury, MA.Hamilton, N. and Bhatti, T. (1996) Population Health Pro- Minkoff, D. (1997) Producing social capital—national social motion: an Integrated Model of Population Health and movements and civil society. American Behavioral Health Promotion. Health Canada, Ottawa, ON. Scientist, 40, 606–619.
14. 380 J. Judd et al.Morfitt, G. (1998) Report of the Auditor General on trials, 1900 through 1993. American Journal of Public Regionalization, Accountability and Governance. Auditor Health, 85, 1378–1383. General’s Office, Victoria, BC. Solberg, L., Mosser, G. and Mcdonald, S. (1997). TheNutbeam, D. (1998) Evaluating health promotion—progress three faces of performance measurement—improvement, problems and solutions. Health Promotion International, accountability and research. Joint Commission Journal on 13, 27–44. Quality Improvement, 23, 135–147.Nutbeam, D., Smith, C., Murphy, S. and Catford, J. (1993) Speller, V., Evans, D. and Head, M. (1997a) Developing Maintaining evaluation designs in long term community quality assurance standards for health promotion practice based health promotion programmes: Heartbeat Wales in the UK. Health Promotion International, 12, 215–224. case study. Journal of Epidemiology and Community Speller, V., Learmouth, A. and Harrison, D. (1997b) The Health, 47, 127–133. search for evidence of effective health promotion. BritishPatton, M. Q. (1997) Utilization-Focused Evaluation: The Medical Journal, 315, 361–363. New Century Text, 3rd edn. Sage, Thousand Oaks, CA. Starzomski, R. (1995) What do ethics have to do withPotvin, L. (1996) Methodological challenges in evaluation lifestyle change? Canadian Journal of Cardiology, of dissemination programs. Canadian Journal of Public 11[Suppl. A], 4A–7A. Health, 87, S79–S83. Stephenson, J., Oakley, A., Charleston, S., Brodala, A.,Potvin, L. and Richard, L. (2001) The evaluation of Fenton, K., Petruckevitch, A. and Johnson, A. M. (1998) community health promotion programs. In Rootman, I., Behavioural intervention trials for HIV/STD prevention Goodstadt, M., Hyndman, B., McQueen, D. V., Potvin, L., in schools: are they feasible? Sexually Transmitted Springett, J. and Ziglio, E. (eds) Evaluation in Health Infections, 74, 405–408. Promotion: Principles and Perspectives. World Health St Leger, L. (1999) Health promotion indicators. Coming Organization, Copenhagen. out of the maze with a purpose. Health Promotion Inter-Potvin, L., Paradis, G. and Lessard, R. (1994) Le paradoxe national, 14, 193–195. de l’évaluation des programmes communautaires multiples Syme, S. L. (1997) Individual vs community interventions de promotion de la santé. Ruptures, 1, 45–57. in public health practice: Some thoughts about a newPutnam, R. (1993) Making Democracy Work: Civic approach. Health Promotion Matters, 2, 2–9. Traditions in Modern Italy. Princeton University Press, Tabrizi, S. (1995) Effective Public Participation in Health Princeton, NJ. Decision Making: Vancouver Health Board’s PopulationRada, J., Ratima, M. and Howden-Chapman, P. H. Health Advisory Committees. Unpublished manuscript, (1999) Evidence based purchasing of health promotion: University of British Columbia, Vancouver, BC. methodology for reviewing evidence. Health Promotion Thompson, S. G., Pyke, S. D. and Hardy, R. J. (1997) The International, 14, 177–187. design and analysis of paired cluster randomized trials:Raeburn, J. and Rootman, I. (1998) People-Centred Health an application of meta-analysis techniques. Statistics in Promotion. John Wiley & Sons, Toronto, ON. Medicine, 16, 2063–2079.Richardson, J. (1998) Economic evaluation of health Trussler, T. and Marchand, R. (1998) Knowledge from promotion: friend or foe? Australian and New Zealand Action: Community-based Research in Canada’s HIV Journal of Public Health, 22, 247–253. Strategy. AIDS Vancouver/Health Canada, Ottawa, ON.Rodney, M., Clasen, C., Goldman, G., Markert, R. and US Preventive Services Task Force (1996) Guide to Clinical Deane, D. (1998) Three evaluation methods of a com- Preventive Services, 2nd edn. Williams & Wilkins, munity health advocate program. Journal of Community Baltimore. Health, 23, 371–381. van der Weijden, T., Grol, R. and Knottnerus, J. (1999)Roman, P. M. and Blum, T. C. (1987) Ethics in worksite Feasibility of a national cholesterol guideline in daily health programming: who is served? Health Education practice. A randomized controlled trial in 20 general Quarterly, 14, 57–70. practices. International Journal for Quality in HealthRose, R., Mishler, W. and Haerpfer, C. ( 1997) Social capital Care, 11, 131–137. in civic and stressful societies. Studies in Comparative Wiesenfeld, E. (1996) The concept of we—a community International Development, 32, 85–111. social psychology myth. Journal of Community Psychology,Serxner, S. and Chung, C. S. (1992) Trend analysis of social 24, 337–346. and economic indicators of mammography use in Hawaii. World Health Organization (WHO) (1986) Ottawa Charter American Journal of Preventive Medicine, 8, 303–308. for Health Promotion. Health Promotion, 1, iii–v.Sherman, V. C. (1999) Raising Standards in American Zakus, J. (1998) Resource dependency and community Health Care: Best People, Best Practices, Best Results. participation in primary health care. Social Science and Jossey-Bass, San Francisco, CA. Medicine, 46, 475–494.Simpson, J. M., Klar, N. and Donnor, A. (1995) Accounting Zakus, J. and Lysack, C. (1998) Revisiting community for cluster randomization: a review of primary prevention participation. Health Policy and Planning, 13, 1–12.