Ejkm volume1-issue1-article8


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ejkm volume1-issue1-article8

  1. 1. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 1http://www.ejkm.com ©MCIL All rights reservedIMPaKT: A Framework for Linking Knowledge Managementto Business PerformancePatricia M Carrillo, Herbert S Robinson, Chimay J Anumba and Ahmed M Al-Ghassani,Department of Civil and Building Engineering, Loughborough University, UKP.M.Carrillo@lboro.ac.ukAbstract: A number of organisations have recognised the importance of managing their organisation knowledgein a more structured manner. However, the question arises as to how to evaluate the benefits of a KnowledgeManagement (KM) strategy and its associated initiatives on the performance of the organisation. This paperpresents a framework for the assessment of the likely impact of KM and discusses findings from an evaluationworkshop held to critique the framework.Keywords: Knowledge management, business performance, evaluation1. IntroductionBusiness organisations are becomingincreasingly aware of the need for innovativeapproaches to responding more effectively toclients demands and changes in the marketplace. Knowledge Management (KM) is centralto this and is increasingly recognised as anintegral part of an organisations strategy toimprove business performance. A key issue inthe implementation of KM strategies is theevaluation of the likely impact. The difficulty formany organisations stems from the fact thatthe implementation of KM initiatives has oftenbeen ad hoc, without a coherent framework forperformance evaluation. A KnowledgeManagement (KM) initiative can be developedto improve the performance of a simple taskand its impact easily evaluated. However, aswe move away from simple tasks toorganisation-wide systemic problems, KMinitiatives become more complex andintertwined. This makes it difficult to evaluatethe impact of these initiatives on businessperformance. There is therefore a need for aperformance-based approach to KM thatexplicitly shows the interactions between KMinitiatives and a set of measures for evaluatingtheir effectiveness and efficiency. TheKnowledge Management for ImprovedBusiness Performance (KnowBiz) project is athree-year research project, sponsored by theEPSRC and industrial collaborators, aimed atinvestigating the relationship between KM andbusiness performance. As part of the project,an initial concept of an KM framework wasdeveloped (Robinson et al, 2001). Thisconcept has now evolved into an operationalframework refined through a follow-uptechnical workshop with the projects industrialcollaborators.This paper presents the development of aframework for Improving ManagementPerformance through KnowledgeTransformation (IMPaKT) and discussesfindings from the application of the frameworkbased on an evaluation workshop held withindustrial partners. Two distinct types ofperformance measures are identified toevaluate KM initiatives - an effectivenessmeasure, which relates to the degree ofrealisation of the strategic objectives and anefficiency measure reflecting the nature of theprocess used to implement KM initiatives.2. Linking KM strategy to businessperformanceKnowledge within the business context can fallwithin the spectrum of tacit (implicit)knowledge and explicit (codified) knowledge.Tacit knowledge is stored in peoples headsand is difficult to share. Explicit knowledge iscaptured or stored in an organisation’smanuals, procedures, databases, and istherefore, more easily shared with otherpeople or parts of an organisation.Organisational knowledge is a mixture ofexplicit and tacit knowledge and the role of KMis to unlock and leverage the different types ofknowledge so that it becomes available as anorganisational asset. However, a key issue inKM is the evaluation of the likely benefits. KMstrategies are more likely to be successfullyimplemented if a performance-based approachis adopted that explicitly shows the interactionsbetween KM initiatives and a set ofperformance measures for evaluating theireffectiveness and efficiency.Carrillo et al (2000) suggested that KM couldbe integrated into key performance indicators
  2. 2. 2 Patricia Carrillo et alhttp://www.ejkm.com ©MCIL All rights reserved(KPIs), and other performance measurementapproaches. There is evidence that someorganisations are now implementing varioustypes of business performance measurementmodels such as the Balanced Scorecard(Kaplan and Norton, 1996) and the ExcellenceModel (EFQM, 1999). A recent survey ofconstruction organisations shows that about40% already have a KM strategy and another41% plan to have a strategy within a year(Carrillo et al, 2003). About 80% alsoperceived KM as having the potential toprovide benefits to their organisations, andsome have already appointed a senior personor group of people to implement their KMstrategy.However, a major problem in KM is evaluatingits likely impact on business performance.Performance is therefore a key issue andperformance measurement models provide abasis for developing a structured approach toKM. Business performance measurementmodels are being used increasingly toencourage organisations to focus onmeasuring a wider range of businessperformance issues relating to processes,people and product. A recent surveyconducted by the KnowBiz research teamshows that over 35% of constructionorganisations are using either the BalancedScorecard (BSC) or the Excellence Model(EM) and about a quarter (26.4%) are usingother measurement systems, mainly the EganKPIs or bespoke models. Over 90% oforganisations using the BSC or the EM alsohave or plan to have a KM strategy in the shortterm (within a year). However, a significantfactor identified in the case studies is the lackof co-ordination between businessimprovement and KM (Robinson et al, 2003).Linking KM to business performance couldmake a strong business case in convincingsenior management about the need to adopt aKM strategy, particularly when the ability todemonstrate benefits of KM is becoming moreimportant in the competition for funding.3. Research methodologyThis framework is a deliverable for an ongoingUK government EPSRC-sponsored researchproject supported by a number of industrialcollaborators. A variety of research methodswere used including literature review,questionnaire survey, industry case studiesand semi-structured interviews for thedevelopment of the framework. A literaturereview identified the key issues in knowledgemanagement and performance measurement.A questionnaire survey and case studies withindustrial collaborators were undertaken toidentify practices, motivation, barriers andenablers in the application of KM and businessperformance measurement. The initial conceptof the framework was developed based on thefindings from the literature review,questionnaire survey and case studies. Theframework was further developed, reviewedand refined through a follow-up technicalworkshop and the applicability of the KMframework was validated through pilot studiesand an evaluation workshop with industrialcollaborators.4. The IMPaKT FrameworkA knowledge management strategy should notonly facilitate the transformation of the varioustypes of knowledge within an organisation butshould provide an evaluation mechanism tomeasure the effectiveness and efficiency ofany strategy. A three-stage framework forImproving Management Performance throughKnowledge Transformation (IMPaKT) hasbeen developed to link KM to performancemeasurement (see Figure 1).The framework recognises that to be able toassess the impact of knowledge management,KM initiatives have to be aligned to anorganisations strategic objectives. Key issuesat each stage are further explored throughTemplates (illustrated in the subsequentsections) supported by detailed guidelines. Foreach stage, there are steps or thoughtprocesses required to structure businessproblems.4.1 Stage 1 - Developing a BusinessImprovement StrategyThe aim of Stage 1 is to provide a structure forformulating a strategic business plan byidentifying the external (business) drivers,defining strategic objectives or goals,identifying critical success factors, anddeveloping measures for monitoringperformance improvement. The outcome ofStage 1 is a business improvement plan withperformance targets and measurableindicators to assess performance. Table 1shows a condensed version of the template fordeveloping a business improvement strategy.
  3. 3. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 3http://www.ejkm.com ©MCIL All rights reservedDefinition of Business GoalsIdentify business aim/strategic objectivess anddevelop measuresAssess Implications for the OrganisationalKnowledge BaseIdentify existing knowledge from a process,people and product perspectiveDevelop KM initiatives and align toimprovement measuresAssess results on key performance measuresAssessment of the Impact onManagement PerformanceDetermine knowledge gap from a process,people and product perspectiveAssess results from process, people and productmeasuresBusinessStrategyKnowledgeManagementStrategyEvaluationofKnowledgeManagementStrategyFigure 1: IMPaKT FrameworkTable 1: Developing a business improvementstrategySTAGE ONE STEPS1.1 Choose a business problem with a knowledgedimension. This is achieved by asking whetherthere is any knowledge the organisation oughtto have to improve the situation or solve theproblem1.2 Place the business problem in a strategiccontext by relating it to your external businessdrivers, strategic objectives and criticalsuccess factors1.3 Select an appropriate set of measures tomonitor progress towards achieving yourstrategic objectives, and identify the businessprocesses they relate to1.4 Identify previous, current, target andbenchmark scores for various performancemeasures and establish the performance gapsSteps 1.1 to 1.4 are supported by detailedguides such as a glossary to facilitate theunderstanding of key terms of the Framework,a sample of performance measures with theirmetric definitions and examples of possiblebenefits arising from improvement in keyperformance measures.The first step in Stage 1 is to choose abusiness problem and to analyse theknowledge dimension of the problem. KMproblems are business problems that areassociated with, related to, or caused by adysfunction in the processes ofobtaining/capturing, locating/accessing,sharing or the application of knowledge. Thenext step involves putting the businessproblem in its strategic context by identifyingthe organisations external and internal forces.For example, the external business drivers(external forces) are the key issues influencingan organisation to achieve or cope with radicalchanges in the business environment. Theseissues could, for example, be technological(e.g. the need for innovation), market orstructural factors (e.g. expansion/ downsizing),etc. The selection of measures forperformance monitoring is also a crucialaspect of Stage 1. The improvement measuresare driven by the firm’s strategy and willtherefore reflect the strategic objectives of theorganisation.
  4. 4. 4 Patricia Carrillo et alhttp://www.ejkm.com ©MCIL All rights reserved4.2 Stage 2 - Developing a KM StrategyThe aim of Stage 2 is to clarify the whether thebusiness problem has a knowledge dimensionand to develop specific KM initiatives toaddress the business problem. The outcome ofStage 2 is a KM strategic plan with a set ofinitiatives and implementation tools to supportbusiness improvement. Table 2 shows acondensed version of the steps involved inidentifying the knowledge implications of abusiness strategy and for developingknowledge management initiatives forbusiness improvement.Table 2: Identifying KM problems andinitiativesSTAGE TWO STEPS2.1 Clarify the knowledge dimension of yourbusiness problem by identifying the KMprocess(es) involved2.2 Develop specific KM initiatives to address thebusiness problem2.3 Select possible tools to support the KMprocess(es) identified in the context of yourbusiness problem2.4 Identify possible relationships between KMinitiatives and performance measures and showhow they relate to the strategic objectives (theCause-and-Effect Map)2.5 Prepare an Action Plan and identify changemanagement and resources requiredSteps 2.1 to 2.5 are also supported by detailedguides such as a questionnaire to identify theKM sub-processes involved, a matrix for theselection of the most appropriate KM tools anda checklist to identify possible barriers andfacilitators prior to implementation.Identifying the KM sub-processes associatedwith the business problem is the first step inclarifying a KM problem. Knowledgemanagement consists of distinct butinterrelated processes that are not linear butcan be cyclical and iterative. Examples of KMprocesses are generate, propagate, transfer,locate and access, maintain and modify(Anumba et al, 2001). Others have useddifferent classifications of the KM life cycle e.g.generate, codify and transfer (Ruggles, 1997);creation, location, capture, share and use ofknowledge (Tiwana, 2000); discovery andcapturing; organisation and storage;distribution and sharing; creation and leverage,retirement and archiving (Robinson et al,2001). The next step is to identify the KMinitiatives required. KM initiatives aresystematic goal-directed efforts for addressinga KM problem in order to achieve businessimprovement. For example, a KM problemassociated with client satisfaction could beimproved by utilising more effectivelyinformation that already exists within theorganisation about clients. It may also includeother initiatives such as setting-up a post-tender forum with clients or project closuremeetings to share information. A set of KMinitiatives identified should align with the KMstrategy. However, KM tools are required forthe implementation of initiatives. A range oftools can be selected including both IT-based(hardware and software) and non-IT-basedsystems (Robinson et al, 2001). The hardwaretools comprise the platform required to supportan organisations knowledge managementstrategy. The software tools vary from simpledatabases and groupware to intelligentdecision support systems such as expertsystems and business intelligence tools. Thenon-IT-based systems will focus on such toolsas informal dialogue, mentoring, communitiesof practice, formal network meetings andresearch collaboration forum to harvest newideas. It is also vital to assess anorganisations readiness before a KM strategyis implemented. An appropriate knowledgemanagement context should be developed andits readiness assessed against the reformneeded, resources required and resultsmonitoring mechanism in place prior to theimplementation of KM. KM is useful but thereis a need to have the necessary reform inplace, have adequate resources and to be ableto demonstrate the benefits through a result-oriented approach.4.3 Stage 3 - Developing a KMEvaluation Strategy and anImplementation PlanThe aim of Stage 3 is, therefore, to provide astructured approach for evaluating the impactof KM initiatives on business performance. Theoutcome of Stages 1 and 2 of the IMPaKTFramework is a business improvementstrategy underpinned by KM. The outcome ofstage 3 is a KM strategy and animplementation plan with priorities and anappreciation of likely impact of various KMinitiatives on business performance or keyperformance measures. This stage is the mostchallenging, as the justification of KM initiativesdepends on the expected benefits (e.g.performance improvement). Two distinct typesof performance measures are identified;measures of effectiveness and measures of
  5. 5. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 5http://www.ejkm.com ©MCIL All rights reservedefficiency. Measures of Effectiveness areoutcome-based measures relating to thedegree to which target performance measuresare achieved but does not take account of thecost of implementation. Measures of Efficiencyare process -based measures relating to thenature of the KM system used inimplementation and are a ratio of expectedbenefit or utility per unit of KM investment. It is,however, recognised that organisations at theembryonic stage of KM may not have a full-scale measurement framework but may needto start with basic qualitative performancemeasures to demonstrate the benefits (APQC,2001). More concrete measures may have tobe developed as an organisation progresses toa transformation stage where KMimplementation is mature and well co-ordinated. Table 3 is a condensed version ofthe steps for KM Evaluation.Steps 3.1 to 3.5 are supported by variousguides developed such as a KM cost andbenefit component checklists, and a KM guideto evaluation/assessment techniques.4.3.1 Measures of EffectivenessKM strategies need to be aligned to strategicobjectives. These links will enable anassessment of the effectiveness of KM interms of the degree to which strategicobjectives are realised. The Cause-and-EffectMap (Figure 2), showing possible relationshipsbetween KM initiatives, performance measuresand the strategic objectives they relate to,forms the basis for determining the contributionof each KM initiative to the performancemeasures.Table 3: Developing KM evaluation strategyand an implementation planSTAGE THREE STEPS3.1 Use the Cause-and-Effect Map developed in2.4 to assess the likely contribution of the KMinitiatives to the performance measures3.2 Assess the probability of success of your KMinitiative in improving your performancemeasures (the effectiveness measure)3.3 Identify the cost components for implementingeach KM initiative and the possible benefits (theefficiency measure)3.4 Choose an appropriate method to assess (ex-ante) the impact of each KM initiative on yourbusiness performance3.5 Prioritise your KM initiatives based on the twomeasures of performanceSTRATEGIC OBJECTIVES KM INITIATIVES PERFORMANCE MEASURES(Cause) (Effect)To handover our projectswith zero defect(A) Number of defects athandover(1) Zero DefectsUse existing knowledge toavoid defectsTo increase profitability (B) Customer satisfaction(2) Best PracticeConstructionMaterials/MethodsTo increase customersatisfaction(C) ProfitTo reduce material wastefrom site(D) Defects inproducts(3) Best Practice LegalIssuesTo reduce corporate risk(E) Number ofcases/arbitration/disputesFigure 2: Cause-and-Effect MapThe first step in the assessment process is toidentify the cause-effect relationships. Thesecond step is to evaluate the impactquantitatively or qualitatively. A KM initiativemay have varying impact on performancemeasures i.e. the impact of an initiative may begreater in some than others. The impacts orcontributions are determined using directweighting techniques such as ranking andrating.
  6. 6. 6 Patricia Carrillo et alhttp://www.ejkm.com ©MCIL All rights reserved4.3.2 Measures of EfficiencyDetermining the contribution of a KM initiativein improving performance score is notsufficient on its own to make strategicdecisions, particularly where there are othercompeting initiatives and resource constraints.The efficiency of the process used toimplement the KM initiatives should also beevaluated. This will also help uncover the realcosts of KM initiatives. Information technology(IT) tools and technologies form one third ofthe time, effort, and money that is required todevelop and use a KM system (Tiwana, 2000).Information technology costs is the mostobvious, but the bigger, and often, hiddencosts are associated with people (APQC,1997) and the related time/cost associatedwith setting up human interactive systems andprocess reengineering or adjustments to coreand supporting business processes. There arevarious inputs or cost components of a KMinitiative as outlined below:KM team component represents the costassociated with both the core (e.g. knowledgemanagers) and support team (e.g. ITpersonnel) required for implementingknowledge management initiatives.KM infrastructure component represents thecosts associated with providing the setting upIT and non-IT systems to provide knowledgecreation and sharing capability.There are different types of cost associatedwith KM such as staff costs, organisational or(re)organisational costs, hardware andsoftware costs. As different KM hardware andsoftware tools are used for the implementationof KM initiatives, consideration should be givennot only to their appropriateness in terms offunctionality (i.e. ease of use, integration, focusand maturity) but, also cost. Costs could bedirect or indirect, one-off/lump sum (e.g.purchase and initial installation cost ofhardware and software, consultants fee etc.)or recurrent/periodic (e.g. hardware/ softwaremaintenance costs, staff costs etc) oroccasional costs (e.g. hardware upgrades,support staff costs etc).There are also different types of benefits to beexpected such as:People e.g. direct labour saving, reduction instaff turnover;Processes e.g. direct cost savings, increasedproductivity;Products e.g. direct cost savings, increasedsales; andOther e.g. repeat customers, new customers.4.3.3 Evaluation MethodsThe aim of evaluation is to identify the input i.e.the nature of KM initiatives and their output i.e.the consequences (both positive and negative)in terms of changes in performance orcontribution to business benefits or losses.Table 4 shows the various evaluationtechniques included in the framework.Table 4: Evaluation TechniquesEvaluation Technique When to use itCost minimisation analysis: This involves a simple costcomparison of KM initiatives as it is assumed that the consequences(outputs) are identical or differences between the outputs areinsignificant. It does not therefore take account of the monetaryvalue of the consequences (outputs).When output of KM initiatives are identical inwhatever unit of measurement is used.Cost effectiveness analysis: This involves the comparison ofKM initiatives where the consequences (output) are measured usingthe same natural or physical units. The assumption is that the outputis worth having and the only question is the cost of the input todetermine the most cost-effective solution.When output of KM initiatives are measured in thesame natural or physical units e.g. number ofaccidents prevented, reduction in absenteeism orwaste, training man-hours, etc.Cost utility analysis: This involves a comparison of KMinitiatives (inputs) which are measured in monetary units with theconsequences (outputs) measured using utility or a preferencescale. Utility refers to the value or worth of a specific level ofimprovement measured by the preferences of individuals, teams ororganisation with respect to a particular outcome.When a significant component of the output cannotbe easily measured, quantified or expressed inmonetary unitsUseful in making internal comparison betweendivisions when there is, for example, a decision tointroduce a pilot project within an organisation.Cost benefit analysis: This approach provides a comparison ofthe value of input resources used up by the KM initiative comparedto the value of the output resources the KM initiative might save orcreate. Consequences of KM initiatives are measured in monetaryterms so as to make them commensurate with the costs.When a significant component of the output can beeasily measured, quantified or expressed inmonetary unitsUseful in determining return on investment (ROI),Internal Rate of Return (IRR), Net Present Value(NPV) or Payback Period of KM investments.A number of these techniques can therefore berecommended in the framework depending on(a) the existing techniques used by the
  7. 7. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 7http://www.ejkm.com ©MCIL All rights reservedorganisation and (b) the level of detail requiredin evaluating the KM initiative.5. Framework evaluationAs part of the research programme, two one-day workshops were planned for thedevelopment of the IMPaKT framework. Thefirst workshop was conducted at the end of thefirst year of the research project to assess andrefine the initial concept and to provide ideasfor the detailed development of the framework.A second workshop was held at the end of thesecond year. This was an evaluation workshopaimed at assessing the robustness of theframework that has evolved. Participantsfamiliar with KM and business improvementissues were invited to the workshop following aconsultation with the projects industrialcollaborators. There were thirteen participantsincluding a director of technical services,senior business improvement and knowledgemanagers, account manager, businesssystems and IT managers, both from theconstruction and manufacturing industries.Over three-quarters of the participants has ahigh level of awareness on KM (76.9%) andbusiness improvement (84.6%) issues.5.1 Evaluation MethodologyThe evaluation workshop started with apresentation of the outline of the framework, aworkshop brief and workshop manualconsisting of tasks list with supportingdiagrams and guidelines. Participants wereorganised into four teams of three to fourpeople, with research team members acting asworkshop facilitators. The workshop wasdivided into two sessions. Session 1 wasbased on the use of Template 1 to develop abusiness improvement strategy with a KMresponse, and covered Stages 1 and 2 of theframework. Session 2 was based on usingTemplate 2 to develop a KM evaluationstrategy and an implementation plan andcovered Stage 3 of the framework.Each group was asked to choose a businessproblem with a knowledge dimension and tostructure the problem using a templateprovided by the IMPaKT framework. Eachteam went through the evaluation exerciseusing different examples of business problems.At the end of the workshop, a group discussionwas held to identify issues regarding the use ofthe templates and an evaluation questionnairewas given to participants to complete. Theevaluation questionnaire consisted ofstatements reflecting key aspects of theframeworks capabilities. Participants wereasked to rate each statement with respect tothe degree to which they agreed or disagreedwith it and to provide suggestions for improvingthe framework.5.1.1 Findings and FeedbackThe results based on the analysis of theevaluation questionnaires completed by theworkshop participants are shown in Figures 3to 6. The questionnaire used a rating scalefrom 1 (strongly disagree) to 5(strongly agree).Linking KM tobusiness strategyMonitoringbusiness strategyMonitoring KMstrategyEvaluationcomponentGlossary of termsAverage Ratings5. 3: Average ratings of key components of overall frameworkThe overall approach of the IMPaKTframework was rated well in terms of theframeworks capabilities to link businessimprovement and knowledge management,and for developing and monitoring businessimprovement and knowledge managementstrategies (see Figure 3). The glossary ofterms accompanying the framework was
  8. 8. 8 Patricia Carrillo et alhttp://www.ejkm.com ©MCIL All rights reservedconsidered helpful in the evaluation process.However, there were some concerns about theterminology. It was suggested that simplifyingor refining some definitions could help as someof the terms used could mean different thingsto different people or organisations. Theevaluation component was also found to beuseful, although, it was rated slightly lowerthan the other aspects of the framework. It wasnoted that the development of the frameworkrepresents a significant attempt to conduct astructured approach to assessing the benefitsof KM to be able to convince senior managers,Detailed findings of the components of theframework are presented in subsequentsections. Figure 4 is a summary of the averageratings for Stage 1 of the framework.Strategic contextLink strategicobjectives toperformancemeasuresLink performancemeasures tobusiness processesTypes ofperformancescoresAverage Ratings5. 4: Average ratings of key components in Stage 1All of the participants strongly agreed that theframework allows an organisation to be able toput its KM/business problems into a strategiccontext. The need to align the strategicobjectives of an organisation to performancemeasures, and to be able to relateperformance measures to the businessprocesses they impact on, was also found tobe useful aspects of the framework, as theratings for both are high. The performancemonitoring aspect encapsulating the differenttypes of performance scores (previous,current, target and benchmark scores) werealso considered to be important, although theaverage rating of 4.00 is not as high as otheraspects.Figure 5 is a summary of the average ratingsfor key aspects of Stage 2 of the framework.The KM clarification process was found to beuseful, so are the KM tools required in theimplementation as part of a businessimprovement strategy.KM clarificationprocessKM toolsLevel oforganisationalreadinessReadiness auditchecklistAverage Ratings5. 5: Average ratings of key components in Stage 2
  9. 9. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 9http://www.ejkm.com ©MCIL All rights reservedAnother key issue is the level of organisationalreadiness to implement KM. This is relevantsince, regardless of the enthusiasm andresources directed towards improving KM,these efforts may not be successful becausethere may be fundamental technical and socialissues that need to be addressed. Forexample, installing a skills yellow page couldbring benefit but if employees are not willing toprovide updates of their experience then theskills yellow pages will rapidly becomeoutdated. Participants agreed thatorganisational readiness is not only a verysignificant factor to consider prior toimplementing a KM strategy, but theaccompanying checklist provided was useful inidentifying the barriers and facilitators to KM.Template 1 for stages 1 and 2 of theframework was considered quite clear andvery useful in working through the issues orproblems selected. One participantcommented that it was useful thought processto go through, well focussed and easy to use.Other participants noted that the template alsoprovides a link with the external environment(external drivers) of an organisation and is agood template for a general business problem.However, it was suggested that in dealing withsome of the issues arising, it is important forsenior management to be involved especiallyfor key strategic issues. Although theworkshop was based on structuringhypothetical business problems, it wasacknowledged that the framework could bemore easily implemented in a company set upwere real data is widely available. Figure 6 is asummary of the ratings for key aspects ofStage 3 of the framework.L i n k s K M a n db u s i n e s s s t r a t e g yK M im p a c t o n b u s i n e s sp e r f o r m a n c eI d e n t i f i c a t i o n o fc o s t s a n d b e n e f i t sM e a s u r e o fe f f e c t i v e n e s sM e a s u r e o fe f f i c i e n c yE v a lu a t i o n G u i d eA v e r a g e R a t i n g s5 . 04 . 03 . 02 . 01 . 0Figure 6: Average ratings of key components in Stage 3Average ratings of slightly over 4 shows thatparticipants agreed that the framework doesfacilitate an understanding of the links betweenbusiness improvement and KM, and alsoprovides a basis to be able to assess theimpact of KM. It was also found to be helpful inidentifying the potential cost and benefits ofKM initiatives. The measures of effectivenessand efficiency are thought to be goodapproaches for assessing the impact of KM onbusiness performance. However, the ratingsfor the evaluation guide to help identify themost suitable techniques to assess the impactof KM initiatives was only slightly aboveaverage. Some participants simply focused oncost benefit analysis. This is, in part, due totheir familiarity with, or popularity of costbenefit analysis (CBA) compared to otherevaluation techniques.6. DiscussionTemplate 2 was relatively more difficult to usecompared to Template 1, although it wasacknowledged that the intention is clear.Template 1 had undergone a number ofiterations and pilot testing before beingpresented at the evaluation workshop.However Template 2 was relatively new thusthe workshop proved a valuable exercise inproviding areas for improvement. Someparticipants found it quite complex asevaluation is considered a difficult area.However, the Cause-and-Effect Map wasfound to be very useful as the starting point forthe evaluation. It was also found to be useful tofacilitate a structured way of thinking about aproblem and a good way to explain tomanagement how everything is related -performance measures, initiatives andstrategic objectives. There were suggestions
  10. 10. 10 Patricia Carrillo et alhttp://www.ejkm.com ©MCIL All rights reservedthat the Cause-and-Effect Map could also beused as a summary of the first session of theworkshop based on Template 1. The checklistfor identifying costs was found to be well laidout and helpful, although it was noted that theapproach to costing might be differentdepending on the cost models used inindividual organisations. The benefit side wasmore difficult to address, however, it wasagreed that the checklist does help in providingsome structure in the evaluation of benefits.Further refinement of the cost and benefitevaluation checklists will continue. But it wassuggested that putting more details into itcould probably make it more complicated andpossibly renders it less credible. There wasalso some concern about the repetition onTemplate 2. However, due to the paper-basedversion being used in the workshop it was feltthat certain aspects had to be repeated toassist participants but this problem will beovercome in an electronic/ automated versionof the framework, which would also enhancedelivery. Other suggestions include clarifyingsome of the headings to reflect the tasks list,and simplifying Template 2. Issues were alsoraised about how the framework could beintroduced to senior management and the levelof details of a KM implementation plan to beprovided to senior executives.All the recommendations made at theevaluation workshop have been addressed inthe version of the template described in Tables1, 2 and 3. The framework therefore provides asolid basis for developing KM strategies thatare not only coherent but also consistent withthe overall strategic objectives of anorganisation. The next stage of the researchinvolves refining the IMPaKT frameworkfurther, and developing an automated versionand an IT architecture to facilitate theimplementation of KM strategies, andintegrating it into an existing KM tool calledCLEVER. CLEVER helps organisations toidentify specific KM problems and guides usersthrough providing solutions to these problems(Anumba et al., 2001).7. ConclusionsThe development of a three-stage KnowledgeManagement framework (IMPaKT) to enablethe impact of KM on business performancehas been presented and discussed. Therobustness of the framework was assessedthrough a technical workshop with industrialcollaborators and a post-workshop evaluationquestionnaire. The findings based on thequestionnaires analysed and the discussionsprovide sufficient evidence of the potential ofIMPaKT as a structured framework fordeveloping a KM evaluation strategy as part ofbusiness improvement. The two measures ofperformance proposed to determine theeffectiveness and efficiency of KM initiativesdoes not only ensure that appropriateinitiatives are selected but enables the rankingof KM initiatives in terms of level of impact onbusiness performance and on specificperformance measures. The increasingnumber of organisations now implementing theBalanced Scorecard and the Excellence Modelmeans that KM can be readily linked toperformance measures. The initial focus of thework reported and the evaluation is based onanalysis from both construction andmanufacturing organisations. However,IMPaKT is a generic framework applicable toother sectors as well. Further development andfine-tuning of the framework will continue aspart of the on-going KnowBiz ResearchProject.ReferencesAPQC (American Productivity & QualityCenter) (1997), Using InformationTechnology to Support KnowledgeManagement: Consortium BenchmarkingStudy (Final Report) Texas, USAAPQC (American Productivity & QualityCenter) (2001) Measurement forKnowledge Management;http://www.apqc.org/free/articles/dispArticle.cfm?ProductID=1307 on 6 March 2002Anumba, CJ, Carrillo PM, Backhouse, CJ,Brookes, NJ and Sinclair MA (2001)Cross Sectoral Learning in the VirtualEnvironment – EPSRC Final Report,Loughborough University, UK.Carrillo, P.M, Anumba, C.J. and Kamara, J.M.(2000). Knowledge Management Strategyfor Construction: Key IT and ContextualIssues, Proceedings of CIT 2000,Reykjavik, Iceland, 28-30 June,Gudnason, G. (ed.), 155-165Carrillo, P.M, Robinson, H.S, Al-Ghassani, A.Mand Anumba, C.J (2002), KnowledgeManagement in Construction: Drivers,Resources and Barriers (in press)EFQM (1999) Introducing Excellence,European Foundation for QualityManagement Brussels, Belgium.Kaplan, R.S. and Norton, D. P. (1996) "TheBalanced Scorecard - Measures thatDrive Performance", Harvard BusinessReview, 70(1), 71-79.Robinson, H.S., Carrillo, P.M, Anumba, C.Jand Al-Ghassani A.M. (2001) KnowledgeManagement: Towards an IntegratedStrategy for Construction Project
  11. 11. Electronic Journal of Knowledge Management, Volume 1 Issue 1 (2003) 1-12 11http://www.ejkm.com ©MCIL All rights reservedOrganisations, Proceedings of the FourthEuropean Project Management (PMI)Conference, London, June 6-7, Paperpublished on CD.Robinson, H.S., Carrillo, P.M, Anumba, C.Jand Al-Ghassani A.M. (2002) KnowledgeManagement in Large ConstructionOrganisations: Measures, Misconceptionsand Missing Links (to be submitted forpublications)Ruggles, R. (1997) Knowledge ManagementTools. Butterworth-Heinemann.Tiwana, A. (2000) The KnowledgeManagement Toolkit: PracticalTechniques for Building a KnowledgeManagement System, Prentice-Hall, NewJersey, USA.
  12. 12. 12http://www.ejkm.com ©MCIL All rights reserved