The current issue and full text archive of this journal is available at www.emeraldinsight.com/0144-3577.htm The effectiveness Factors inﬂuencing the of PMSs effectiveness of performance measurement systems 1287 Amy Tung, Kevin Baird and Herbert P. Schoch Department of Accounting and Corporate Governance, Received June 2010 Macquarie University, Sydney, Australia Revised November 2010 Accepted April 2011AbstractPurpose – The purpose of this paper is to examine the association between the use ofmultidimensional performance measures and four organizational factors with the effectiveness ofperformance measurement systems (PMSs).Design/methodology/approach – Data were collected by mail survey questionnaire from arandom sample of 455 senior ﬁnancial ofﬁcers in Australian manufacturing organizations.Findings – The results reveal that the use of multidimensional performance measures is associatedwith two dimensions of the effectiveness of PMSs (performance and staff related outcomes). Theresults also reveal that organizational factors were associated with the effectiveness of PMSs.Speciﬁcally, top management support was found to be associated with the effectiveness of PMSs inrespect to the performance related outcomes, and training was associated with the staff relatedoutcomes.Practical implications – The ﬁndings provide managers with an insight into the desirable PMScharacteristics and the speciﬁc organizational factors that they can focus on in order to enhance theeffectiveness of their performance measurement system.Originality/value – This study contributes to the limited empirical research examining theeffectiveness of PMSs regarding the extent to which organizational processes are achieved. In addition,the study provides an empirical analysis of the association between the ﬁve perspective (ﬁnancial,customer, internal business process, learning and growth, and sustainability) BSC model and fourorganizational factors with the effectiveness of PMSs.Keywords Australia, Manufacturing industries, Performance measures,Performance measurement system, Multidimensional performance measures, Top management support,Training, Employee participation, Link of performance to rewardsPaper type Research paper1. IntroductionTo survive in today’s rapidly changing environment, organizations must identify theirexisting positions, clarify their goals, and operate more effectively and efﬁciently.Performance measurement systems (PMSs) assist organizations in achieving suchobjectives. Neely et al. (1995, p. 81) deﬁnes a PMS as “a set of metrics used toquantify both the efﬁciency and effectiveness of actions”. An effective PMS enablesan organization to assess whether goals are being achieved, and facilitates theimprovement of the organization as a whole (Lebas, 1995) by identifying their position,clarifying goals, highlighting areas requiring improvement, and facilitating reliable International Journal of Operationsforecasts (Neely et al., 1996). Hence, an effective PMS enables an organization to & Production Management Vol. 31 No. 12, 2011measure and control its performance in line with the deﬁned strategy. pp. 1287-1310 q Emerald Group Publishing Limited While the recent PMS literature has focused on the shift from traditional PMSs, which 0144-3577focus on ﬁnancial measures, to multidimensional PMSs such as the performance DOI 10.1108/01443571111187457
IJOPM pyramid (Lynch and Cross, 1991), the balanced scorecard (BSC) (Kaplan and Norton,31,12 1992), and the performance prism system (Neely and Adams, 2000), there is limited empirical evidence examining the effectiveness of such PMSs. Furthermore, the majority of these studies assess PMS effectiveness in relation to overall organizational performance (Crabtree and DeBusk, 2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al., 2003; Hoque and James, 2000), thereby assuming a direct association1288 between the PMS and performance. This approach is inconsistent with Hamilton and Chervany’s (1981) claim that the impact of the PMS on performance is indirectly inﬂuenced by the effect on improvements in organizational processes. In other words, organizational objectives such as sales revenue, proﬁt contribution and customer satisfaction will not be realized unless speciﬁc organizational objectives (e.g. motivating performance, developing individual’s skills and knowledge, providing useful feedback to employees, and providing an accurate assessment of business unit performance) are achieved. Accordingly, the ﬁrst objective of this study is to contribute to the limited empirical research (Malina and Selto, 2001; Whorter, 2003) examining the effectiveness of PMSs based on the extent to which organizational processes are achieved. The measurement of performance is an on-going task, hence, in order to achieve system effectiveness, organizations need to devote time and effort to managing the system (Neely et al., 2000). Hence, in an attempt to provide practitioners with an insight into how to achieve and maintain effectiveness, the second objective of the study is to contribute to the contingency literature by examining the factors associated with the effectiveness of PMSs. The ﬁrst factor examined, the use of multidimensional performance measures, has been advocated by both academics and practitioners in order to complement the limitations of traditional ﬁnancial PMSs and to increase the effectiveness of PMSs (Van der Stede et al., 2006; Kaplan and Norton, 2001, 1996, 1992). While many multidimensional frameworks have been advocated, and the beneﬁts of using multidimensional performance measures have received wide publicity in the literature (Van der Stede et al., 2006; Bryant et al., 2004), there is considerable variation in the adoption rates reported for the most common multidimensional approach, the BSC (Rigby and Bilodeau, 2009 (53 percent); Chung et al., 2006 (31 percent); Ittner et al., 2003 (20 percent); Speckbacher et al., 2003 (26 percent)). The variation in the adoption of multidimensional performance measures raises concerns regarding the contribution of such measures towards the effectiveness of PMSs. Accordingly, this study aims to contribute to the literature by examining the association between the use of multidimensional performance measures and the effectiveness of PMSs. The study also aims to provide an empirical analysis of the association between speciﬁc organizational factors (top management support, training, employee participation and the link of performance to rewards) with the effectiveness of PMSs. While these organizational factors do not represent a comprehensive list of all relevant factors, they were chosen for two reasons. First, they have been widely cited as factors contributing to the success of various management accounting practices such as activity-based costing (ABC) (Baird et al., 2007; Shields, 1995), enterprise resource planning (Motwani et al., 2002; Rao, 2000), and management information system (MIS) (Raghunathan et al., 1999; Doll, 1985; Schultz and Ginzberg, 1984). Second, while they have been identiﬁed in previous studies as the main contingency factors associated with the effectiveness of PMSs (Burney et al., 2009; Hoque and Adams, 2008; Cheng et al., 2007; Kleingeld et al., 2004; Chan, 2004), this was in isolation, and no study has analysed
all four factors together. Hence, this study is motivated to ﬁll this gap in the literature by The effectivenessexamining the link between all four organizational factors and the effectiveness of PMSs of PMSswithin Australian manufacturing organizations. In addition, given the majority of previous studies examining the inﬂuence oforganizational factors on PMS effectiveness have used the case study approach(Kleingeld et al., 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002;Kaplan, 2001), there is a gap in the literature empirically examining this association. 1289Hence, the current study is motivated to ﬁll this gap by using the survey method in anattempt to enhance the generalizability of the ﬁndings. The remainder of this paper is structured as follows. Section 2 provides the literaturereview and develops the relevant hypotheses. Sections 3 and 4 then discuss the methodand results. Finally, Section 5 provides the conclusion, limitations, and future directionsfor research.2. Literature review2.1 Performance measurement systemsPMSs have become a ﬁeld of interest over the last two decades with many studiesdiscussing various aspects of performance measurement such as: the purpose andusage (Marchand and Raymond, 2008; Horngren et al., 2005; Simons, 2000), design(Bhasin, 2008; Kennerley and Neely, 2002; Neely and Adams, 2000; Kaplan and Norton,1996, 1992; Lynch and Cross, 1991), and implementation (Ratnasingam, 2009; Othman,2008; Speckbacher et al., 2003; Kaplan, 2001). An effective PMS, which is deﬁned as the achievement of the objectives set fora task (Clinquini and Mitchell, 2005), is important for a number of reasons. First,it can encourage goal congruence. For example, an appropriate PMS can be used tocommunicate the strategy and goals of an organization and align employees’ goals withorganizational goals. Second, an effective PMS can provide accurate information toenable managers to track their own performance and evaluate employees’ performancein an effective and efﬁcient manner. Finally, an effective PMS can provide organizationswith an indication of their current market position and assist them in developing futurestrategies and operations (Langﬁeld-Smith et al., 2009). This study operationalises aneffective PMS as the extent to which 16 desired PMS outcomes are achieved. Traditionally, PMSs have focused mainly on ﬁnancial measures such as proﬁt, cashﬂow and return on investment to evaluate the performance of employees (Chan, 2004).This focus has a number of shortcomings. First, these outcome-oriented measures donot allow managers to assess how well employees perform across the full range ofstrategically important areas, such as quality and service delivery. Second, traditionalﬁnancial measures describe consequences rather than causes, hence they are notactionable. Such measures provide limited guidance for future actions since they do nottell managers what needs to be ﬁxed (Langﬁeld-Smith et al., 2009). Third, the focus onaggregate ﬁnancial outcomes may encourage managers to engage in “gaming” behaviorto maximize short-term results at the expense of long-term effectiveness (Chow andVan der Stede, 2006). Finally, traditional ﬁnancial measures can conﬂict with strategy andthey are not externally focused (Chow and Van der Stede, 2006; Kaplan and Norton, 1996). The limitations of traditional PMSs, together with intense competitive pressuresand changing external demands, have led to the increased advocacy of non-ﬁnancialmeasures (Neely, 1999). Such contemporary PMSs have been espoused by both
IJOPM academics and practitioners in order to address the limitations of traditional ﬁnancial31,12 performance measures and to assist organizations to build competitive advantage under changing economic conditions (Kaplan and Norton, 2006, 2004, 2001, 1996, 1992). The common characteristics of contemporary systems include the linking of strategies, objectives and measures, and the incorporation of both ﬁnancial and non-ﬁnancial measures that cover a range of perspectives (Langﬁeld-Smith et al., 2009). Since the1290 BSC is the most recognized and utilized contemporary PMS (Rigby and Bilodeau, 2009; Chang et al., 2008; Jusoh et al., 2008; Bedford et al., 2006; Pike and Roos, 2004; Atkinson et al., 1997), it is used in this study to exemplify the use of multidimensional performance measures. 2.2 The BSC The ﬁrst-generation BSC was mainly a PMS which proposed a speciﬁc structure to measure tangibles and intangibles (Speckbacher et al., 2003; Kaplan and Norton, 1992). The framework complemented the ﬁnancial perspective measures with non-ﬁnancial operational measures emphasizing three other perspectives: customer satisfaction, internal processes and learning and growth. It provided a more balanced view of organizational performance by capturing both leading (e.g. customer satisfaction, on-time delivery, employee training, etc.) and lagging (e.g. sales revenue, ROI, cash ﬂows, etc.) performance measures (Kaplan and Norton, 1996, 1992). In 1996, Kaplan and Norton advocated the causal links between the perspectives included within the BSC. The reﬁned model communicated the organization’s desired outcomes and hypothesized the means by which the desired outcomes could be achieved. For instance, if organizations trained their employees well, then the quality of service would be improved as well as customer satisfaction; if customer satisfaction improved, then customers would purchase more, thereby improving the overall proﬁtability of the organization. Hence, the second-generation BSC was proposed as a multidimensional PMS which describes strategy through cause and effect relationships (Speckbacher et al., 2003; Kaplan and Norton, 1996). It enabled organizational units and employees to understand the strategy and identify how they can contribute to its achievement by becoming aligned with the strategy. Consequently, today’s BSC has become a strategic management system that implements strategy through communication, action plans and incentives (Speckbacher et al., 2003; Kaplan and Norton, 2001). As a further development, the BSC included additional perspectives (Kaplan and Wisner, 2009; Kaplan and Norton, 2006, 2004, 2001). With sustainability becoming a major concern for various stakeholders (e.g. customers, investors, and the government) and affecting the organizational “bottom line”, a sustainability BSC was subsequently advocated (Langﬁeld-Smith et al., 2009; Epstein, 2008; Figge et al., 2002). Epstein (2008) suggested that the inclusion of the sustainability perspective is appropriate where sustainability is considered a part of the business core strategy and important to creating competitive advantage. To provide a more comprehensive account of the use of multidimensional performance measures, this study adopts the ﬁve perspective (ﬁnancial, customer, internal business process, learning and growth, and sustainability) BSC model. 2.2.1 Adoption and use of the BSC. Silk (1998) estimated that 60 percent of the Fortune 1000 companies in the USA have had experience with a BSC. In the UK, 57 percent of businesses were reported to use a BSC and 53 percent of non-users
were discussing possible implementation. In contrast, Speckbacher et al. (2003) reported The effectivenessthat more than 60 percent of the companies in their study had not considered the BSC. of PMSsSimilarly, Ittner et al. (2003) indicated that only 20 percent of the ﬁrms in their study useda BSC, while 50 percent of the ﬁrms had not even considered implementing it. Use of the BSC however does not guarantee satisfaction with De Geuser et al. (2009)referring to the literature highlighting the gap between the use of the BSC and evidenceof its effectiveness (Davis and Albright, 2004; Norreklit, 2003; Speckbacher et al., 2003; 1291Otley, 1999). Thus, while the Management Tools and Trends Survey (Rigby andBilodeau, 2009) showed that in 2008, 53 percent of organizations globally used the BSCand by the end of 2009, the usage rate was expected to reach 69 percent, it was found that51 percent of user organizations were not satisﬁed with their BSC. Similarly, Ittner et al.(2003) revealed that organizations were only moderately satisﬁed with the measurementsystem with 37.2 percent of respondents rating it as not meeting expectations.Bedford et al. (2006) also concluded that while respondents agreed that the BSC hadhelped in achieving some objectives, the extent to which the proclaimed beneﬁts of theBSC were achieved was still fairly low. Given the mixed ﬁndings with respect to thesuccess of the BSC, this study investigates the association between the use ofmultidimensional performance measures and the effectiveness of PMSs.2.3 The association between the use of multidimensional performance measures and theeffectiveness of PMSsMultidimensional PMSs assist organizations by enhancing the likelihood that allrelevant performance dimensions are considered (Ittner et al., 2003). Furthermore, suchsystems allow managers to focus on the “means to the end”, while also enabling themto demonstrate strong performance in a variety of areas (Baird, 2010). Hoque andAdams (2008) suggest that multidimensional PMSs are capable of providing signalsand motivating improvement in crucial activities. Similarly, Van der Stede et al. (2006)found that regardless of strategy, organizations with more extensive PMSs, especiallythose that included objective and subjective non-ﬁnancial measures, have betteroverall performance. Van der Stede et al. (2006) also demonstrated that non-ﬁnancialperformance measures are better than ﬁnancial measures in helping organizationsimplement and manage new initiatives. A growing stream of literature provides evidence that the use of multidimensionalperformance measures contributes to the effectiveness of PMSs (Crabtree and DeBusk,2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al., 2003; Whorter,2003; Malina and Selto, 2001; Hoque and James, 2000). Most of these studies examinedthe effectiveness of PMSs from the perspective of their contribution to the company’sﬁnancial performance. For example, Davis and Albright (2004) applied aquasi-experimental study in a US banking organization to investigate the relationshipbetween BSC implementation and the ﬁnancial performance of bank branches.The study supports the theory that the BSC can be used to improve ﬁnancialperformance, with bank branches that implemented the BSC outperforming otherbranches on key ﬁnancial measures. Similarly, Braam and Nijssen (2004) suggest thatBSC usage, which is aligned to company strategy, positively inﬂuences overall companyperformance. Ittner et al. (2003) found that while BSC usage was associated with highermeasurement system satisfaction, there was no evidence that BSC usage was related
IJOPM to stock returns. However, Crabtree and DeBusk (2008) extended this study to31,12 investigate the contribution of the BSC to shareholder returns in different public sector companies, and found that BSC usage was associated with higher stock returns. Malina and Selto (2001) and Whorter (2003) assessed the effectiveness of PMSs based on organizational processes (e.g. communicating strategic objectives, creating strategic alignment, motivating employees and serving as a management control device)1292 as opposed to ﬁnancial performance. Malina and Selto (2001) found that the BSC was an effective device for evaluating corporate strategy. Their results also show evidence of casual relations between motivation, strategic alignment and effective management control with the BSC. Similarly, Whorter (2003) showed that BSC users consistently reported higher agreement about having the information needed for making the best work-related decisions. Whorter (2003) also concluded that the BSC not only provides useful performance feedback to employees but is also an aid in the accurate assessment of employee performance: H1. The extent of use of multidimensional performance measures is associated with the effectiveness of the PMS. 2.4 The association between organizational factors and the effectiveness of PMSs Prior studies have identiﬁed top management support (Hoque and Adams, 2008; Johanson et al., 2006; Bourne, 2005; Chan, 2004; Bourne et al., 2002; Kennerley and Neely, 2002; Kaplan, 2001), training (Chan, 2004; Emerson, 2002), employee participation (Hoque and Adams, 2008; Kleingeld et al., 2004), and the link of performance to rewards (Burney et al., 2009; Chan, 2004) as key organizational factors associated with the effectiveness of PMSs. 2.4.1 Top management support. Top management support has been highlighted as an important contingency factor in supporting various management accounting practices such as ABC (Baird et al., 2007; Shields, 1995) and MISs (Doll, 1985). The impact of top management support on PMS effectiveness has been referred to in a number of studies (Bourne, 2005; Chan, 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002). For example, Bourne et al. (2002) investigated the success of the redesign of PMSs. They found that top management support was inﬂuential in the successful implementation and on-going usage of the new PMS. This study also indicated that the continuous involvement by top management was invaluable in resolving problems when crises and conﬂicts arose. Chan (2004) and Emerson (2002) also reported that top management commitment and leadership buy in are key factors in enhancing PMS effectiveness. Similarly, Kennerley and Neely (2002) found that top-level management support was critical for PMS design and implementation, while the availability of management time to reﬂect on measures was a major contributor to the effectiveness of PMSs: H2. The extent of top management support is associated with the effectiveness of the PMS. 2.4.2 Training. Training is deﬁned as “a planned effort by an organization to facilitate the learning of job-related behavior” (Wexley, 1984, p. 13). The importance of training in relation to the development and implementation of a successful PMS is highlighted in a number of studies. Cavaluzzo and Ittner (2004, p. 249), for example, found that performance measurement development and outcomes are positively associated with the extent of related training provided to the manager. The provision of training
resources indicates that an organization is willing to provide sufﬁcient resources to The effectivenesssupport the development and implementation of PMSs. of PMSs Chan (2004) cites training as a crucial factor for PMSs to be effective. All performancemeasures need to have a clearly communicated purpose and be perceived as bothrelevant and reliable so that managers can access useful information for decisionmaking. Without training, managers may perceive the PMS measures as less useful andignore them when making decisions. Similarly, Emerson (2002) concluded that training 1293is the key to maintaining the usefulness and the effectiveness of PMSs. Training not onlyallows users to understand performance measurement concepts and principles, but alsoprovides both employees and managers with an opportunity to operate the system.Hence, the better that users understand the purposes of the system and how tooperationalise it, the more likely they will commit to it, thereby enhancing the likelihoodthat the desired results will be achieved: H3. The extent of PMS-related training provided is associated with the effectiveness of the PMS.2.4.3 Employee participation. Many studies have referred to the beneﬁts of employeeempowerment (Morrell and Wilkinson, 2002; Koberg et al., 1999; Chiles and Zorn,1995) and employee involvement and participation (Cox et al., 2007, 2006; Pun et al.,2001; Wimalasiri and Kouzmin, 2000). These studies tend to operationalise theseconcepts in terms of employees’ involvement in decision making. Similarly, employeeparticipation refers to the “involvement of managers and their subordinates ininformation processing, decision making, or problem solving endeavors” (Wagner,1994, p. 312). This study operationalises employee participation in terms of the extentto which lower level employees participate in designing the PMS. The association between employee participation and the effectiveness of PMSs hassupport from prior studies (Chan, 2004; Kleingeld et al., 2004; Kaplan and Norton,2001). These studies report that a higher level of employee participation contributed tothe effectiveness of PMSs. For instance, Kleingeld et al. (2004) found that on averagethe improvement in performance was signiﬁcantly greater for those employees in ahigh participation situation as opposed to those in a low participation situation. Thisperformance improvement was attributed to both cognitive mechanisms (includingincreased communication, better utilization of knowledge, increased understanding ofthe job) and motivational mechanisms (less resistance to change, commitment to thesystem, acceptance of feedback and goals). Similarly, Kaplan and Norton (2001) maintained that in order to achieve an effectiveBSC, employees at lower levels in the organizational hierarchy should be involved inthe establishment of performance measures. This bottom-up participation approachallows employees to take the initiative in deﬁning their responsibilities as well as theassociated performance indicators. Therefore, employees will commit to the systemand desired outcomes can be achieved to a greater extent: H4. The extent of employee participation in designing the PMS is associated with the effectiveness of the PMS.2.4.4 The link of performance to rewards. The link of performance to rewards is avital contingency factor in motivating employees (Rynes et al., 2005; McShane andTravaglione, 2003; Bonner and Sprinkle, 2002; PA Consulting Group, 1998).
IJOPM A survey of 500 companies reported that companies that link performance to pay31,12 showed twice the shareholder returns as those who did not (PA Consulting Group, 1998). McShane and Travaglione (2003) suggested that companies need to align rewards with performance that is within the employee’s control. Hence, the more employees see a “line of sight” between their daily actions and the reward, the more motivated they will be to improve performance.1294 Linking performance to rewards has also been identiﬁed as a crucial factor inﬂuencing the effectiveness of PMSs (Burney et al., 2009; Johanson et al., 2006; Chan, 2004). For instance, in Chan’s (2004) study of municipal governments in the USA and Canada, it was found that the linkage of the PMS to compensation was uncommon, and “the lack of linkage of the BSC to rewards” was considered to be a barrier to the systems’ effectiveness. While there is a lack of empirical evidence examining the link of performance to rewards on the effectiveness of PMSs, given the importance of the link of performance to rewards and the increasing number of large businesses rewarding both employees and managers based on BSC performance (Epstein and Manzoni, 1998), H5 is stated as follows: H5. The extent of the link of performance to rewards is associated with the effectiveness of the PMS. 3. Method A survey questionnaire was mailed to the senior ﬁnancial ofﬁcer of a random sample of 445 Australian manufacturing business units identiﬁed from the Kompass Australia (2009) directory. The manufacturing industry was selected as a number of prior studies on PMSs suggest that manufacturing organizations are more likely to have a mature and comprehensive PMS in place (Malina and Selto, 2001; Simons, 2000; Kaplan and Norton, 1996, 1992). Business units were chosen as the unit of analysis because PMS characteristics may differ across business units within an organization. Senior ﬁnancial ofﬁcers were chosen as they were expected to have a sound understanding of their business unit’s PMS. The Dillman (2007) tailored design method was employed to administer the survey. In total, 141 responses were received for a response rate of 30.9 percent. In total, 23 of the questionnaires were incomplete, hence 118 questionnaires were used for the data analysis. As was the case in Robert (1999), non-response bias was assessed by comparing the independent and dependent variable values across early and late respondents. No signiﬁcant differences were detected. 3.1 Variable measurement 3.1.1 The effectiveness of the PMS. The effectiveness of PMSs is measured by assessing the extent to which 16 desired outcomes of PMSs have been achieved. The 16 measures (the Appendix) were developed based on a review of the literature relating to the effectiveness of PMS (Lawler, 2003) with minor modiﬁcations made to ﬁt the context of the study. Respondents were required to indicate the extent to which their PMS had achieved each of the 16 perceived outcomes using a ﬁve-point Likert scale with anchors of 1 “not at all” and 5 “to a great extent”. Factor analysis (principal components with varimax rotation) using a cutoff point of 0.60 revealed that the 16 outcomes loaded onto two dimensions, with the factor structure consistent with Baird (2010). The ﬁrst dimension included nine items which all refer to the achievement of organizational goals and objectives, hence, this dimension
was labeled “performance-related outcomes”. The second dimension included seven The effectivenessitems which are more concerned with employees, hence this dimension was labeled of PMSs“staff-related outcomes”. These two dimensions were subsequently scored as theaverage score of the items loading on to each dimension with higher (lower) scoresrepresenting a more (less) effective PMS. 3.1.2 The usage of multidimensional performance measures. The extent to whichrespondents were using multidimensional performance measures was measured using 1295two approaches. The ﬁrst approach required respondents to simply indicate if they wereusing a BSC (“yes” or “no”). Since this approach is reliant on respondents understandingof the nature of a BSC, a more comprehensive approach which focuses on the performancemeasures employed within organizations, was also adopted. This approach requiredrespondents to indicate the extent to which they were using 26 different performancemeasures (the Appendix) to assess their business units’ performance, on a ﬁve-pointLikert scale with anchors of 1 “not at all” to 5 “to a great extent”. These measures werederived primarily from the BSC literature and were mainly designed for manufacturingorganizations (Epstein, 2008; Jusoh et al., 2008; Van der Stede et al., 2006; Bryant et al.,2004; Ittner et al., 2003; Kaplan and Norton, 2001, 1996). Factor analysis (principal components with varimax rotation) using a cutoff point of0.6 revealed that the 26 items loaded onto six speciﬁc dimensions covering the followingperspectives: ﬁnancial, customer, internal business, learning, growth and sustainability.These ﬁndings are in line with Figge et al. (2002), except that the learning and growthperspectives were separated. These two perspectives were subsequently combined inaccordance with the ﬁve perspectives BSC model. Each of the ﬁve perspectives were scored as the sum of the items loading onto eachperspective with higher (lower) scores indicating the PMS focused on each perspectiveto a greater (lesser) extent. Since a different number of items loaded onto each ofthe perspectives, average scores were calculated with the use of multidimensionalperformance measure scored as the sum of the averages across the ﬁve perspectiveswith higher (lower) scores indicating that multidimensional performance measureswere used to a greater (less) extent. 3.1.3 Organizational factors. Each of the four organizational factors was measuredusing a summated ﬁve-point Likert scale with anchors of 1 “strongly disagree” and5 “strongly agree”. Top management support was measured using a three-item summatedscale (the Appendix) with respondents required to indicate the extent to which topmanagement provided adequate resources (Krumwiede, 1998), communicated effectively(Grover, 1993) and exercised its authority in support of the PMS. Top managementsupport was measured as the average score for the three items, with higher (lower) scoresindicating a higher (lower) level of top management support. The level of related training was measured using three items (the Appendix) drawnfrom Baird et al. (2007), with minor adjustments made to ﬁt the context of the currentstudy. Speciﬁcally, respondents were required to indicate if adequate training had beenprovided to develop, to implement and to ensure employees understood the PMS.Training was measured as the average score for the three items, with higher (lower)scores indicating a higher (lower) level of related training provided by the organization. In the absence of speciﬁc measures in the literature on employee participation in aPMS context, two self-developed items (the Appendix) were adopted following a review
IJOPM of the employee participation/involvement literature (Sinclair et al., 2005; Harel and31,12 Tzafrir, 1999; Huselid, 1995; Wagner, 1994). Speciﬁcally, respondents were required to indicate the extent to which lower level employees participated in designing the PMS and were involved in selecting performance measures. The perceived level of employee participation was subsequently scored as the average score for the two items with higher (lower) scores indicating a higher (lower) level of employee participation.1296 The link of performance to rewards was assessed using two items (the Appendix) based on the literature on performance and rewards (Rynes et al., 2005; Lawler, 2003; Huselid, 1995). Respondents were required to indicate the extent to which performance is linked to ﬁnancial rewards such as pay or bonus, and non-ﬁnancial rewards such as recognition or service awards in their organization. The analysis revealed that the two questions were measuring different factors: the extent to which performance is linked to ﬁnancial rewards and to non-ﬁnancial rewards. These measures are analyzed as separate independent variables, with higher (lower) scores indicating a stronger (weaker) link of performance to rewards. 4. Results Table I shows summary statistics for the dependent and independent variables. For the multi-item scales, the actual range was comparable with the theoretical range, and the Cronbach’s a coefﬁcients met or exceeded the 0.70 threshold generally considered acceptable in regard to scale reliability (Nunnally, 1978, p. 245). The mean scores of the effectiveness of PMSs for both the performance-related outcomes (3.50) and the staff-related outcomes (3.26) are slightly higher than the mid-point of the range, indicating that on average the respondents assessed their PMS to be moderately effective. The performance-related outcomes were achieved to a greater extent, with the mean scores of all nine items equal to or greater than the seven staff-related outcomes. The performance-related outcomes that were achieved Minimum Maximum Variables n a Mean SD (theoretical) (theoretical) Cronbach’s a Independent variables Use of multidimensional performance measures 118 2.94 0.70 1.17 (1) 4.67 (5) Top management support 117 3.51 1.02 1 (1) 5 (5) 0.915 Training 117 3.11 1.07 1 (1) 5 (5) 0.963 Employee participation 117 2.41 1.02 1 (1) 5 (5) 0.761 Link of performance to ﬁnancial rewards 117 3.50 1.16 1.00 (1) 5.00 (5) Link of performance to non- ﬁnancial rewards 117 2.93 1.13 1.00 (1) 5.00 (5) Dependent variables Effectiveness of PMS (performance-related outcomes) 117 3.50 0.81 1 (1) 5 (5) 0.932 Effectiveness of PMS (staff-related outcomes) 117 3.26 0.93 1 (1) 5 (5) 0.924 aTable I. Note: The number of responses (n) varies due to the fact that not all survey items were completed byDescriptive statistics respondents
to the greatest extent included: assisting in achieving the goals (mean score of 3.68); The effectivenessproviding useful performance feedback to employees (mean score of 3.64); developing of PMSsa performance-oriented culture (mean score of 3.59); and providing an accurateassessment of business unit performance (mean score of 3.59). The staff-relatedoutcomes that were achieved to the greatest extent included: developing individual’sskill and knowledge (mean score of 3.38), identifying talented employees (mean scoreof 3.36), and rewarding talented employees (mean score of 3.31). 1297 In regard to the four organizational factors, while the mean score of most of thefactors lie on the higher end of the scale, the mean value of the link of performance tonon-ﬁnancial rewards (2.93) was slightly below the mid-point of the range indicating arelative weak link between performance and non-ﬁnancial rewards. As discussed in the method section, two approaches were used to assessthe use of multidimensional performance measures. Table II reveals that 39 respondents(33.1 percent) indicated that they were using a BSC in their business unit.The more comprehensive approach to measuring the use of multidimensionalperformance measures focused on the extent to which business units were employing26 performance measures covering the ﬁve perspectives of the BSC. Table I reveals thatthe mean score for the use of multidimensional performance measures (2.94) was slightlylower than the mid-point of the range, indicating a moderate use of multidimensionalperformance measures in Australian manufacturing organizations. Table III provides a more detailed analysis of the extent to which measures relatingto each of the ﬁve perspectives were employed. The greatest emphasis was placed onthe ﬁnancial perspective (3.59) followed by the customer (3.43), learning and growth(3.11), and internal business process (3.06) perspectives. The mean score of thesustainability perspective (2.19) was below the mid-point of the range indicating arelatively low level of usage of this perspective.4.1 Analysis of the association between the use of multidimensional performancemeasures and organizational factors with the effectiveness of PMSsTable IV presents the results of the one-way analysis of variance (ANOVA) used toexamine the difference in the level of PMS effectiveness based on whether respondentswere using a BSC. Respondents using a BSC reported a signiﬁcantly higher level ofPMS effectiveness with respect to both performance- and staff-related outcomes.BSC usage Frequency Adjusted percentageYes 39 33.1 Table II.No 79 66.9 BSC usageBSC perspectives n Minimum Maximum Mean RankFinancial 118 1.00 (1) 5.00 (5) 3.59 1Customer 118 1.00 (1) 5.00 (5) 3.43 2Internal business process 118 1.00 (1) 5.00 (5) 3.06 4 Table III.Learning and growth 118 1.17 (1) 5.00 (5) 3.11 3 Use of multidimensionalSustainability 118 1.00 (1) 5.00 (5) 2.19 5 performance measures
IJOPM These results provide preliminary evidence that the use of multidimensional31,12 performance measures is associated with the effectiveness of PMSs, thereby providing support for H1. The association between the use of multidimensional performance measures and PMS effectiveness was also analyzed using a more comprehensive approach based on the extent of use of multidimensional performance measures. Stepwise regression was1298 used to examine the association between both the use of multidimensional performance measures and organizational factors with PMS effectiveness, with the results presented in Table V. For the effect on performance-related outcomes, the model was statistically signiﬁcant (F ¼ 63.812, p ¼ 0.000) with an R 2 of 0.530 indicating that 53 percent of the variance in the achievement of performance-related outcomes can be explained by the explanatory factors. The model reveals that the use of multidimensional performance measures ( p ¼ 0.000) was signiﬁcantly associated with the effectiveness of PMSs. In addition, top management support ( p ¼ 0.000) was signiﬁcantly associated with the performance-related outcomes. Table V also provides the ﬁndings for staff-related outcomes, with the model found to be statistically signiﬁcant (F ¼ 38.535, p ¼ 0.000) with an R 2 of 0.405 indicating that 40.5 percent of the variance in the achievement of staff-related outcomes can be explained by the explanatory factors. The model reveals that the use of multidimensional performance measures was found to be signiﬁcantly associated with the achievement of staff-related outcomes ( p ¼ 0.000). The level of training ( p ¼ 0.000) was also signiﬁcantly associated with PMS effectiveness. The ﬁndings provide further support for H1 and partially support H2 and H3. The importance of the use of multidimensional performance measures in explaining the level of PMS effectiveness prompted further exploratory analysis to investigate the association between each of the ﬁve perspectives of the BSC with the effectiveness of PMSs. These ﬁndings are presented in Section 4.2.Table IV. Performance-related outcomes Staff-related outcomesResults of the one-way BSC usage n Mean F-statistic Signiﬁcance Mean F-statistic SigniﬁcanceANOVA comparing thelevel of PMS effectiveness BSC user 39 3.88 14.297 0.000 3.71 15.869 0.000based on BSC usage Non-BSC user 78 3.31 3.03 Performance-related outcomes Staff-related outcomes Variables Coefﬁcient t-statistics Signiﬁcance Coefﬁcient t-statistics SigniﬁcanceTable V.Results of stepwise Multidimensional PMS 0.343 4.512 0.000 0.374 4.465 0.000regression analysis Top managementof the association support 0.487 6.411 0.000between the use Training 0.362 4.325 0.000of the multidimensional F-value 63.812 38.535performance measures p-value 0.000 0.000and organizational R2 0.530 0.405factors with the Adjusted R 2 0.522 0.395effectiveness of PMSs n 115 115
4.2 Analysis of the association between the ﬁve perspectives of the BSC with the The effectivenesseffectiveness of PMSs of PMSsTable VI reveals the stepwise regression analysis ﬁndings. The performance-relatedoutcomes model was statistically signiﬁcant (F ¼ 63.847, p ¼ 0.000) with an R 2 of0.528 indicating that 52.8 percent of the variance in the achievement of theperformance-related outcomes can be explained by the two perspectives of the BSCfound to be signiﬁcantly associated with the performance-related outcomes: the internal 1299business process ( p ¼ 0.000) and learning and growth ( p ¼ 0.000) perspectives. The staff-related outcomes model was also statistically signiﬁcant (F ¼ 56.768,p ¼ 0.000) with an R 2 value of 0.499 indicating that 49.9 percent of the variance in theachievement of the staff-related outcomes can be explained by the two perspectives ofthe BSC found to be signiﬁcantly associated with the staff-related outcomes: thelearning and growth ( p ¼ 0.000) and sustainability ( p ¼ 0.000) perspectives.5. Conclusion5.1 DiscussionThe ﬁrst objective of the study was to examine the effectiveness of PMSs in respect totheir impact on organizational processes. The study evaluated the effectiveness of PMSsbased on the extent to which 16 desired outcomes were achieved. By focusing on theoutcomes achieved, the study contributes to the empirical body of knowledge on PMSssince the majority of previous studies have only assessed PMS effectiveness based onoverall organizational performance. This approach provides managers with a moredetailed insight into the ability of the PMS to assist their organization in achievingspeciﬁed desired outcomes. Factor analysis revealed that these items reﬂected twodimensions of PMS effectiveness: performance- and staff-related outcomes. The resultsrevealed that the mean score for the effectiveness of PMSs for both dimensions wasslightly above the mid-point of the range, indicating that the PMSs of Australianmanufacturing organizations were only moderately effective. This ﬁnding highlightsthe signiﬁcance of the study’s investigation of the contingency factors associated withthe effectiveness of PMSs. The results also showed that organizations were more successful in achievingthe performance-related outcomes than the staff-related outcomes. This suggeststhat PMSs have mainly been used as a managerial tool to assist the organization inmotivating performance, implementing the organizational strategy and achieving goals. Performance-related outcomes Staff-related outcomesVariables Coefﬁcient t-statistics Signiﬁcance Coefﬁcient t-statistics SigniﬁcanceInternal businessprocess 0.277 3.830 0.000 Table VI.Learning and growth 0.558 7.730 0.000 0.539 7.445 0.000 Results of stepwiseSustainability 0.289 4.001 0.000 regression analysisF-value 63.847 56.768 of the associationp-value 0.000 0.000 between each of the ﬁveR2 0.528 0.499 perspectives of the BSCAdjusted R 2 0.520 0.490 with the effectivenessn 116 116 of PMSs
IJOPM Less emphasis is being placed on achieving staff-related outcomes such as addressing31,12 the concerns of staff, ensuring staff time is used efﬁciently, and managing poorly performing staff. The latter ﬁnding is of concern given that survival in today’s rapidly changing world is dependent on the achievement of both staff- and performance-related outcomes. Harel and Tzafrir (1999, p. 185) highlighted the importance of focusing on employees,1300 suggesting that an organization’s staff are its strategic assets which “form a system of resources and rare abilities that cannot easily be copied, and provide the company with its competitive edge”. Hence, organizations which view staff as potential partners and important assets enhance the likelihood of achieving better organizational performance. There is also evidence that the achievement of staff-related outcomes can assist in the achievement of performance-related outcomes. If organizations adequately address the concerns of their employees, they are more likely to be emotionally attached to a particular organization, and hence more willing to assist in the achievement of organizational goals (Myer and Allen, 1991). Accordingly, we suggest that managers place greater emphasis on the achievement of staff-related outcomes. This should be embodied in the design of the PMS so as to incorporate both contributions from employees as well as reﬂecting their personal needs. The second objective of the study was to examine the association between the use of multidimensional performance measures and four organizational factors with the effectiveness of the PMS. The initial analysis focused on ascertaining the extent to which organizations were using multidimensional performance measures. Results revealed that only 33.1 percent of organizations were using the BSC, which is consistent with previous ﬁndings (Crabtree and DeBusk, 2008 (35 percent); Chung et al., 2006 (31 percent); Speckbacher et al., 2003 (26 percent); Whorter, 2003 (35 percent)). A more comprehensive analysis of the use of multidimensional performance measures revealed that Australian manufacturing organizations placed the greatest emphasis on measures relating to the ﬁnancial perspective of the BSC, followed by the customer, learning and growth internal business process, and sustainability perspectives. This ﬁnding is consistent with the majority of the BSC literature which suggests that ﬁnancial measures are still used to the greatest extent (Crabtree and DeBusk, 2008; Hoque and Adams, 2008; Davis and Albright, 2004; Braam and Nijssen, 2004; Ittner et al., 2003; Hoque and James, 2000; Lipe and Salterio, 2000; Ittner and Larcker, 1998). The ﬁndings indicate that while organizations may be enticed to use a BSC, and even claim to use the BSC, the reality is that the greatest emphasis is still placed on the traditional ﬁnancial-based perspective. Therefore, if organizations are to reap the beneﬁts of using multidimensional PMSs such as the BSC, it is crucial that they do not just pay lip service to the inclusion of measures covering the other perspectives. Rather they need to acknowledge the importance of the other perspectives of the BSC and place increasing emphasis on using measures relating to each of the perspectives. Analysis of the association between the use of multidimensional performance measures and organizational factors with the effectiveness of PMSs revealed that the use of multidimensional performance measures, as operationalized by the BSC, and two organizational factors (top management support, and training) exhibited a signiﬁcant association with the effectiveness of PMSs. The use of multidimensional performance measures was positively associated with both the performance- and staff-related outcomes. This ﬁnding is in line with previous
studies (Chow and Van der Stede, 2006; Van der Stede et al., 2006; Bryant et al., 2004) The effectivenesswhich have advocated that organizations should incorporate both ﬁnancial and of PMSsnon-ﬁnancial measures in the PMS. Similarly, the ﬁndings reinforce the literatureadvocating the beneﬁts of the BSC (Langﬁeld-Smith et al., 2009; Epstein, 2008; Kaplanand Norton, 2006; Speckbacher et al., 2003). The ﬁndings highlight the need formanagers to evaluate the inherent characteristics of their PMS and their impact on theachievement of such outcomes. In particular managers need to focus on the extent to 1301which diversiﬁed performance measures reﬂecting the ﬁve perspectives of the BSC areincorporated in their PMS. Additional exploratory analysis revealed that the internal business and learning andgrowth perspectives were associated with the effectiveness of PMSs regarding theperformance-related outcomes, while the learning and growth and sustainabilityperspectives were signiﬁcantly associated with the staff-related outcomes. While thisﬁnding highlights the importance of adopting a BSC, it also provides managers withinsight into the speciﬁc BSC dimensions which warrant their attention in order toenhance PMS effectiveness. Managers are therefore encouraged to ensure that their PMSemphasises the use of performance measures in relation to the internal business process(e.g. productivity, usage of resources, cycle time and number of product returns),learning and growth (e.g. hours of training provided, improvements made to employeefacility, number of new product produced and percentage of revenue from newapplications) and sustainability (e.g. investment in environmental management,promotion of environmental causes and investment in community services) perspectivesin order to enhance the effectiveness of their PMS. Analysis of the association between the organizational factors and the effectivenessof the PMS provides an insight into the prevailing organizational conditions thatcould enhance/prohibit PMS effectiveness. Top management support was found to beassociated with the performance-related outcomes, and the level of training wasassociated with the staff-related outcomes. While top management support has beenfound to be a critical success factor for PMS implementation (Bourne, 2005; Chan, 2004;Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002), the ﬁndings highlight theimportance of the continued involvement and support from top management. Hence, inorder to achieve the desired performance-related outcomes, a concentrated effort by topmanagement aimed at continuous improvement, open communication and consistentsupport is required (Kaynak, 2003). Top management is therefore encouraged topersonally commit to the PMS and ensure that enough time and resources are dedicatedon an on-going basis to properly develop and manage the existing PMS. In addition,organizations which provide more related training to their staff are able to achieve thedesired staff-related outcomes. This supports Harel and Tzafrir’s (1999) suggestion thatmoving knowledge information and power to lower levels of the organization is a way tosustain competitive advantage. Organizations could therefore employ appropriatetraining with respect to the use of PMSs across different business levels to enhance theknowledge and skill of employees in developing and implementing the systems. The study contributes to the literature by examining PMS effectiveness in terms ofthe effect on organizational processes. The two dimensions of PMS effectiveness,performance- and staff-related outcomes, serve to make management more aware of theneed to focus on different aspects of PMS effectiveness as well as providing researcherswith a new measure which can be used to evaluate its effectiveness. In addition,
IJOPM the association between the use of multidimensional performance measures31,12 and organizational factors with the effectiveness of the PMS provides managers of organizations with an insight into the desirable characteristics of an effective PMS and the prevailing organizational conditions which can support the PMS. Hence, managers need to focus on using multidimensional performance measures, and increase the level of top management support and related training in relation to their PMS.1302 5.2 Limitations and future research The study is subject to the usual limitations of the survey method. While the survey method is useful in ascertaining associations rather than causal relationships between variables (Singleton and Straits, 2005), this approach generates potential threats as respondents may answer questions in accordance with social desirability bias. Future studies may incorporate face-to-face interviews in order to provide richer descriptions into the hypothesised associations. Future studies could also collect data from multiple respondents across different management levels. This may assist in overcoming the common method bias associated with the single respondent approach. The study also used a number of self-developed measures. For instance, the measures of PMS effectiveness, the usage of multidimensional performance measures, and two organisational factors, employee participation and the link of performance to rewards, were self-developed. The face validity of these measures was enhanced through a pilot study of ten academics with relevant expertise, and their content validity was enhanced by developing the measures based on an extensive review of the relevant literature. However, while factor analysis provides evidence of the construct validity of the ﬁrst three measures, the validity of these measures still needs to be conﬁrmed in future studies, especially given the sample size is considered small for performing factor analysis. In addition while the Cronbach’s a scores conﬁrm the reliability of the ﬁrst three of these measures, the two items used to measure link of performance to rewards were found to be measuring separate constructs. Hence, there is concern as to the reliability of this measure and future studies may explore alternative ways of measuring this factor. In addition, the current study only provides empirical evidence in relation to the association between four organizational factors (top management support, training, employee participation and link of performance to rewards) and the effectiveness of PMS. Future studies may consider the association between other organizational factors such as organizational structure, and management style, with PMS effectiveness. To enhance the generalizability of the ﬁndings, future studies could be conducted using similar parameters in other industries such as service and the non-proﬁt sector. Notes 1. Cohen’s (1988) formulae which considers the number of independent variables, statistical signiﬁcance and power, and effect size, was used to determine the required number of valid responses (91). Assuming a conservative response rate of 20 percent, a sample size of 445 was determined. 2. The Kompass Australia business directory provides details of manufacturing businesses in Australia. It is assumed that a random sample taken from this directory is representative of the Australian manufacturing industry.
3. The Dillman (2007) Tailored Design Method provides guidelines in respect to the format and The effectiveness style of questions, personalisation, and distribution procedures. There is evidence that this approach leads to improved response rates. of PMSsReferencesAtkinson, A., Balakrishnan, A.R., Booth, P., Cote, J.M., Groot, T., Malmi, T., Roberts, H., Uliana, E. and Wu, A. (1997), “New directions in management accounting research”, Journal of 1303 Management Accounting Research, Vol. 9, pp. 79-108.Baird, K. (2010), “The effectiveness of strategic performance measurement systems”, unpublished working paper, Macquarie University, Sydney.Baird, K., Harrison, G. and Reeve, R. (2007), “Success of activity management practices: the inﬂuence of organizational and cultural factors”, Accounting and Finance, Vol. 47 No. 1, pp. 47-67.Bedford, D.S., Brown, D.A. and Malmi, T. (2006), “Balanced scorecard content, use, and performance impacts: some Australian evidence”, paper presented at the European Accounting Association, 29th Annual Conference, Dublin.Bhasin, S. (2008), “Lean and performance measurement”, Journal of Manufacturing Technology Management, Vol. 19 No. 5, pp. 670-84.Bonner, S.E. and Sprinkle, G.B. (2002), “The effects of monetary incentives on effort and task performance: theories, evidence, and a framework for research”, Accounting, Organizations and Society, Vol. 27 Nos 4/5, pp. 303-45.Bourne, M.C. (2005), “Researching performance measurement system implementation: the dynamics of success and failure”, Production Planning and Control, Vol. 16 No. 2, pp. 101-13.Bourne, M.C., Neely, A.D., Platts, K.W. and Mills, J.F. (2002), “The success and failure of performance measurement initiatives: the perceptions of participating managers”, International Journal of Operations and Production Management, Vol. 22 No. 11, pp. 1288-310.Braam, G.J.M. and Nijssen, E.J. (2004), “Performance effects of using the balanced scorecard: a note on the Dutch experience”, Long Range Planning, Vol. 37 No. 4, pp. 335-49.Bryant, L., Jones, D.A. and Widener, S.K. (2004), “Managing value creation within the ﬁrm: an examination of multiple performance measures”, Journal of Management Accounting Research, Vol. 16 No. 1, pp. 107-31.Burney, L.L., Henle, C.A. and Widener, S.K. (2009), “A path model examining the relations among strategic performance measurement system characteristics, organizational justice, and extra- and in-role performance”, Accounting, Organizations and Society, Vol. 34 Nos 3/4, pp. 305-21.Cavaluzzo, S.K. and Ittner, C.D. (2004), “Implementing performance measurement innovations: evidence from government”, Accounting, Organisations and Society, Vol. 29 No. 1, pp. 243-67.Chan, Y.C.L. (2004), “Performance measurement and adoption of balanced scorecards: a survey of municipal governments in the USA and Canada”, The International Journal of Public Sector Management, Vol. 17 No. 3, pp. 204-21.Chang, C.J., Lee, H.I. and Chen, W.C. (2008), “A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan”, Expert Systems with Applications, Vol. 34 No. 1, pp. 96-107.Cheng, M.I., Dainty, A. and Moore, D. (2007), “Implementing a new performance management system within a project-based organisation: a case study”, International Journal of Productivity and Performance Management, Vol. 56 No. 1, pp. 60-75.
IJOPM Chiles, A.M. and Zorn, T.E. (1995), “Empowerment in organizations: employees’ perceptions of the inﬂuences on empowerment”, Journal of Applied Communication Research, Vol. 2331,12 No. 1, pp. 1-25. Chow, C.W. and Van der Stede, W.A. (2006), “The use and usefulness of nonﬁnancial performance measures”, Management Accounting Quarterly, Vol. 7 No. 3, pp. 1-8. Chung, L.H., Gibbons, P.T. and Schoch, H.P. (2006), “The management of information and1304 managers in subsidiaries of multinational corporations”, British Journal of Management, Vol. 17 No. 2, pp. 153-65. Clinquini, L. and Mitchell, F. (2005), “Success in management accounting: lessons from the activity-based costing/management experience”, Journal of Accounting and Organizational Change, Vol. 1 No. 1, pp. 63-78. Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd ed., Lawrence-Erlbaum, Mahwah, NJ. Cox, A., Marchington, M. and Sutter, J. (2007), Embedding the Provision of Information and Consultation in the Workplace: A Longitudinal Analysis of Employee Outcomes in 1998 and 2004, DTI Employee Relations Research Series, No. 72, Department of Trade and Industry, London. Cox, A., Zagelmeyer, S. and Marchington, M. (2006), “Embedding employee involvement and participation at work”, Human Resource Management Journal, Vol. 16 No. 3, pp. 250-67. Crabtree, A.D. and DeBusk, G.K. (2008), “The effects of adopting the balanced scorecard on shareholder returns”, Advances in Accounting, Vol. 24 No. 1, pp. 8-15. Davis, S. and Albright, T. (2004), “An investigation of the effect of balanced scorecard implementation on ﬁnancial performance”, Management Accounting Research, Vol. 15 No. 2, pp. 135-53. De Geuser, F., Mooraj, S. and Oyon, D. (2009), “Does the balanced scorecard add value? Empirical evidence on its effect on performance”, European Accounting Review, Vol. 18 No. 1, pp. 93-122. Dillman, D.A. (2007), Mail and Internet Surveys: The Tailored Design Method, 2nd ed., Wiley, New York, NY. Doll, W.J. (1985), “Avenues for top management involvement in successful MIS development”, MIS Quarterly, Vol. 9 No. 1, pp. 17-35. Emerson, B. (2002), “Training for performance measurement success: an effective training program can help get performance measurement off the ground and sustain the system as it matures into a catalyst for government accountability and improvement”, Government Finance Review, April, available at: www.thefreelibrary.com/Trainingþ forþperformanceþmeasurementþsuccess%3aþAnþeffectiveþtraining-a085048611 (accessed 1 May 2009). Epstein, M. (2008), Making Sustainability Work: Best Practices in Managing and Measuring Social and Environmental Impacts, Greenleaf, Shefﬁeld. Epstein, M. and Manzoni, J.F. (1998), “Implementing corporate strategy: from tableaux de bord to balanced scorecard”, European Management Journal, Vol. 16 No. 2, pp. 190-203. Figge, F., Hahn, T., Schaltegger, S. and Wagner, M. (2002), “The sustainability balanced scorecard – linking sustainability management to business strategy”, Business Strategy and the Environment, Vol. 11 No. 5, pp. 269-84. Grover, V. (1993), “An empirical derived model for the adoption of customer-based inter-organizational systems”, Decision Sciences, Vol. 24 No. 3, pp. 603-39. Hamilton, S. and Chervany, N.L. (1981), “Evaluating information system effectiveness – part I: comparing evaluation approaches”, MIS Quarterly, Vol. 5 No. 3, pp. 55-69.
Harel, H.G. and Tzafrir, S.S. (1999), “The effect of human resource management practices on the The effectiveness perceptions of organizational and market performance of the ﬁrm”, Human Resource Management, Vol. 38 No. 3, pp. 185-200. of PMSsHoque, Z. and Adams, C. (2008), Measuring Public Sector Performance: A Study of Government Departments in Australia, CPA Australia, Melbourne.Hoque, Z. and James, W. (2000), “Linking balanced scorecard measures to size and market factors: impact on organizational performance”, Journal of Management Accounting Research, 1305 Vol. 12 No. 1, pp. 1-17.Horngren, C.T., Bhimani, A., Datar, S.M. and Foster, G. (2005), Management and Cost Accounting, 3rd ed., Pearson Education, Harlow.Huselid, M.A. (1995), “The impact of human resource management practices on turnover, productivity, and corporate ﬁnancial performance”, Academy of Management Journal, Vol. 38 No. 3, pp. 535-672.Ittner, C.D. and Larcker, D.F. (1998), “Innovations in performance measurement: trends and research implications”, Journal of Management Accounting Research, Vol. 10, pp. 205-38.Ittner, C.D., Larcker, D.F. and Randall, T. (2003), “Performance implications of strategic performance measurement in ﬁnancial services ﬁrms”, Accounting, Organizations and Society, Vol. 28 Nos 7/8, pp. 715-41.Johanson, U., Skoog, M., Backlund, A. and Almqvist, R. (2006), “Balancing dilemmas of the balanced scorecard”, Accounting, Auditing & Accountability Journal, Vol. 19 No. 6, pp. 842-57.Jusoh, R., Ibrahim, D.N. and Zainuddin, Y. (2008), “The performance consequence of multiple performance measures usage: evidence from the Malaysian manufacturers”, International Journal of Productivity and Performance Management, Vol. 57 No. 2, pp. 119-36.Kaplan, R.S. (2001), “Strategic performance measurement and management in nonproﬁt organizations”, Nonproﬁt Management and Leadership, Vol. 11 No. 3, pp. 353-70.Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard-measures that drive performance”, Harvard Business Review, Vol. 70, January/February, pp. 71-9.Kaplan, R.S. and Norton, D.P. (1996), “Linking the balanced scorecard to strategy”, California Management Review, Vol. 39 No. 1, pp. 53-79.Kaplan, R.S. and Norton, D.P. (2001), The Strategy-focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment, Harvard Business School Press, Boston, MA.Kaplan, R.S. and Norton, D.P. (2004), “Measuring the strategic readiness of intangible assets”, Harvard Business Review, February, pp. 52-63.Kaplan, R.S. and Norton, D.P. (2006), Alignment: Using the Balanced Scorecard to Create Corporate Synergies, Harvard Business School Press, Boston, MA.Kaplan, S.E. and Wisner, P.S. (2009), “The judgmental effects of management communications and ﬁfth balanced scorecard category on performance evaluation”, Behavioral Research in Accounting, Vol. 21 No. 2, pp. 37-56.Kaynak, H. (2003), “The relationship between total quality management practices and their effects on ﬁrm performance”, Journal of Operations Management, Vol. 21 No. 4, pp. 405-35.Kennerley, M. and Neely, A. (2002), “A framework of the factors affecting the evolution of performance measurement systems”, International Journal of Operations and Production Management, Vol. 22 No. 11, pp. 1222-45.Kleingeld, A.D., Tuijl, H.V. and Algera, J. (2004), “A participation in the design of performance management systems: a quasi-experimental ﬁeld study”, Journal of Organizational Behavior, Vol. 25 No. 7, pp. 831-51.
IJOPM Koberg, C.S., Boss, R.W., Senjem, J.C. and Goodman, E.A. (1999), “Antecedents and outcomes of empowerment: empirical evidence from the health care industry”, Group and Organsiation31,12 Management, Vol. 24 No. 1, pp. 71-91. Kompass Australia (2009), Kompass Australia, Peter Isaacson Publications, Elsternwick. Krumwiede, K.R. (1998), “The implementation stages of activity-based costing and the impact of contextual and organizational factors”, Journal of Management Accounting Research,1306 Vol. 10, pp. 239-77. Langﬁeld-Smith, K., Thorne, H. and Hilton, R. (2009), Management Accounting: An Australian Perspective, 6th ed., McGraw-Hill, Sydney. Lawler, E.E. (2003), “Reward practices and performance management system effectiveness”, Organizational Dynamics, Vol. 32 No. 4, pp. 396-404. Lebas, M.J. (1995), “Performance measurement and performance management”, International Journal of Production Economics, Vol. 41 Nos 1-3, pp. 23-35. Lipe, M.G. and Salterio, S.E. (2000), “The balanced scorecard: judgmental effects of common and unique performance measures”, Accounting Review, Vol. 75 No. 3, pp. 283-9. Lynch, R.L. and Cross, K.F. (1991), Measure Up! Yardsticks for Continuous Improvement, Blackwell, Cambridge. McShane, S. and Travaglione, T. (2003), Organizational Behaviour on the Paciﬁc Rim, McGraw-Hill, Sydney. Malina, M.A. and Selto, F.H. (2001), “Communicating and controlling strategy: an empirical study of the effectiveness of the balanced scorecard”, Journal of Management Accounting Research, Vol. 13, pp. 47-90. Marchand, M. and Raymond, L. (2008), “Researching performance measurement systems: a information systems perspective”, International Journal of Operations and Production Management, Vol. 28 No. 7, pp. 663-86. Morrell, K. and Wilkinson, A. (2002), “Empowerment: though the smoke and past the mirrors?”, Human Resource Development International, Vol. 5 No. 1, pp. 119-30. Motwani, J., Mirchandani, D., Madan, M. and Gunasekaran, A. (2002), “Successful implementation of ERP projects: evidence from two case studies”, International Journal of Production Economics, Vol. 75 Nos 1/2, pp. 83-96. Myer, J.P. and Allen, N.J. (1991), “A three-component conceptualization of organizational commitment”, Human Resource Management Review, Vol. 1, pp. 61-89. Neely, A. (1999), “The performance measurement revolution: why now and what next?”, International Journal of Operations and Production Management, Vol. 19 No. 2, pp. 205-28. Neely, A. and Adams, C. (2000), Perspectives on Performance: The Performance Prism, Gee Publishing, London. Neely, A., Gregory, M. and Platts, K. (1995), “Performance measurement system design: a literature review and research agenda”, International Journal of Operations and Production Management, Vol. 25 No. 12, pp. 1228-63. Neely, A., Mills, J., Platts, K., Gregory, M. and Richards, H. (1996), “Performance measurement system design: should process based approaches be adopted?”, International Journal of Production Economics, Vol. 46, pp. 423-31. Neely, A., Mills, J., Platts, K., Richards, H., Gregory, M., Bourne, M. and Kennerley, M. (2000), “Performance measurement system design: developing and testing a process-based approach”, International Journal of Operations and Production Management, Vol. 20 No. 10, pp. 1119-45.
Norreklit, H. (2003), “The balanced scorecard: what is the score? A rhetorical analysis of the The effectiveness balanced scorecard”, Accounting, Organizations and Society, Vol. 28 No. 6, pp. 591-619. of PMSsNunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY.Othman, R. (2008), “Reﬂective practice: enhancing the effectiveness of the balanced scorecard with scenario planning”, International Journal of Productivity and Performance Management, Vol. 57 No. 3, pp. 259-66.Otley, D. (1999), “Performance management: a framework for management control systems 1307 research”, Management Accounting Research, Vol. 10 No. 4, pp. 282-362.PA Consulting Group (1998), “An incentive scheme that works: pay purview, how one knowledge-intensive company has overhauled its incentives”, The Economist, August, available at: www.paconsulting.com/introducing-pas-media-site/archive/an-incentive- scheme-that-works-pay-purview-how-one-knowledge-intensive-company-has-overhauled- its-incentives/ (accessed 10 April 2009).Pike, S. and Roos, G. (2004), “Mathematics and modern business management”, Journal of Intellectual Capital, Vol. 5 No. 2, pp. 243-56.Pun, K., Chin, K. and Gill, R. (2001), “Determinants of employee involvement practices in manufacturing enterprises”, Total Quality Management, Vol. 12 No. 1, pp. 95-109.Raghunathan, B., Raghunathan, T. and Tu, Q. (1999), “Dimensionality of the strategic grid framework: the construct and its measurement”, Information Systems Research, Vol. 10 No. 4, pp. 343-55.Rao, S.S. (2000), “Enterprise resource planning: business needs and technologies”, Industrial Management and Data Systems, Vol. 100 No. 1, pp. 81-8.Ratnasingam, P. (2009), “Service quality management applying the balanced scorecard: and exploratory study”, International Journal of Commerce and Management, Vol. 19 No. 2, pp. 127-36.Rigby, D. and Bilodeau, B. (2009), Management Tools and Trends 2009, Bain & Company, Boston, MA.Robert, E.S. (1999), “In defence of the survey method: an illustration from a study of user information satisfaction”, Accounting and Finance, Vol. 39 No. 1, pp. 53-77.Rynes, S.L., Gerhart, B. and Parks, L. (2005), “Personnel psychology: performance evaluation and pay for performance”, Annual Review of Psychology, Vol. 56, pp. 571-600.Schultz, R. and Ginzberg, M.J. (1984), “Implementation research-third generation”, Applications of Management Science, Vol. 4, pp. 1-83.Shields, M.D. (1995), “An empirical analysis of ﬁrms’ implementation experiences with activity-based costing”, Journal of Management Accounting Research, Vol. 7, pp. 148-66.Silk, S. (1998), “Automating the balanced scorecard”, Management Accounting, Vol. 79 No. 11, pp. 38-44.Simons, R. (2000), Performance Measurement and Control Systems for Implementing Strategy, Prentice-Hall, Englewood Cliffs, NJ.Sinclair, R.R., Leo, M.C. and Wright, C. (2005), “Beneﬁt system effects on employees’ beneﬁt knowledge, use, and organizational commitment”, Journal of Business and Psychology, Vol. 20 No. 1, pp. 3-29.Singleton, R.A. and Straits, B.C. (2005), Approaches to Social Research, Oxford University Press, New York, NY.
IJOPM Speckbacher, G., Bischof, J. and Pfeiffer, T. (2003), “A descriptive analysis on the implementation of balanced scorecard in German speaking countries”, Management Accounting Research,31,12 Vol. 14 No. 4, pp. 361-88. Van der Stede, W.A., Chow, C.W. and Lin, T.W. (2006), “Strategy, choice of performance measures, and performance”, Behavioral Research in Accounting, Vol. 18, pp. 185-205. Wagner, A.J. (1994), “Participation’s effects on performance and satisfaction: a reconsideration of1308 research evidence”, Academy of Management Review, Vol. 19, pp. 312-30. Wexley, K.N. (1984), “Personnel training”, Annual Reviews, Vol. 35, pp. 519-51. Whorter, L.B.M. (2003), “Does the balanced scorecard reduce information overload?”, Management Accounting Quarterly, Vol. 4 No. 4, pp. 23-7. Wimalasiri, J. and Kouzmin, A. (2000), “A comparative study of employee involvement initiatives in Hong Kong and the USA”, International Journal of Manpower, Vol. 21 No. 8, pp. 614-34. Appendix. Variable measurement Effectiveness of PMS (adapted from Lawler (2003), Ittner et al. (2003) and Kaplan and Norton (2001, 1996)) Please indicate the extent to which your business unit’s PMS assists your business unit in achieving each of these. Performance-related outcomes: Motivating performance. Assisting in the achievement of goals. Developing a performance-oriented culture. Supporting change efforts. Providing useful performance feedback to employees. Implementing the organizational strategy. Providing an accurate assessment of business. Ensuring staff commitment to organizational objectives. Linking individual performance to business unit performance. Staff-related outcomes: Developing individual’s skill and knowledge. Addressing the concerns of staff. Ensuring the concerns of staff. Identifying talented employees. Rewarding talented employees. Identifying poor performing staff. Managing poor performing staff. The use of multidimensional performance measures Financial perspective: Sales revenue. Return on investment. Improvement in net assets/liabilities.
Customer perspective: The effectiveness On-time product delivery. of PMSs Number of new customers. Quality of products. Number of product returns.Internal business process perspective: 1309 Usage/wastage of resources. Productivity. Cycle time. Expenditure on warranty claims.Learning and growth perspective: Hours of training provided. Improvements made to employee facilities. Number of employee suggestions implemented. Number of new products produced. Time to market for new products. Percentage of revenue from new products/new applications.Sustainability: Investment in environmental management. Promotion of environmental causes. Investment in community services. Community connectedness services. Promotion of community causes.Organizational factorsTop management support: Top management has provided adequate resources to support the PMS. Top management has effectively communicated its support for the PMS. Top management exercises its authority in support of the PMS.Training: Adequate training has been provided to ensure employees understand the PMS. Adequate training has been provided to develop the PMS. Adequate training has been provided to implement the PMS.Employee participation: Lower level employees participated in designing the PMS. Lower level employees were involved in selecting performance measures.Link of performance to rewards: Performance is linked to ﬁnancial rewards (pay, bonuses, etc.) in your business unit. Performance is linked to non-ﬁnancial rewards (recognition, service awards, etc.) in your business unit.
IJOPM About the authors Amy Tung has taught both undergraduate and postgraduate subjects in the management31,12 accounting area. Her research interests include performance measurement systems, environmental management and employee organizational commitment. She is undertaking her PhD in sustainability, with focus on environmental management systems and environmental performance. Amy Tung is the corresponding author and can be contacted at: manamy. email@example.com Kevin Baird has taught both undergraduate and postgraduate subjects in the management accounting area for 16 years. He has also supervised Honours and PhD students across many different topic areas within the management accounting discipline including activity-based management practices, total quality management, performance measurement systems, management control systems, outsourcing, employee organizational commitment and employee empowerment. Herbert P. Schoch has taught both undergraduate and post-graduate courses, primarily in Management Accounting and he has supervised PhD and Honours students. He has also taught Financial Accounting, Business Strategy and Entrepreneurship and Entrepreneurial Management. He has taught in Australia, Singapore, Hong Kong, Canada and the USA. His research interests include management control systems, management accounting, outsourcing, accounting education and entrepreneurship. He has published numerous journal articles, book chapters and monographs. He also has experience in working in manufacturing, public accounting and has managed and operated his own business. To purchase reprints of this article please e-mail: firstname.lastname@example.org Or visit our web site for further details: www.emeraldinsight.com/reprints