Tallinn 2008 anttila


Published on

Published in: Business, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Tallinn 2008 anttila

  1. 1. ”You get what you measure” or not? Challenges for fact-based quality management Juhani Anttila Venture Knowledgist – Quality Integration www.QualityIntegration.biz , juhani.anttila@telecon.fi 15.3.2008 These pages are licensed under Creative Commons Nimeä 2.5 lisenssi1 http://creativecommons.org/licenses/by/2.5/deed.fi
  2. 2. ”You get what you measure (*)” or not? Challenges for fact-based quality management 1. Searching a balanced position to statistical quality management 2. Factual business performance 3. Facts, data, information and knowledge for business management 4. Using measurements, analyses, and knowledge within business management 5. Example: Using ZEF methodology for business performance evaluations 6. A topical problem: Quality of networks 7. Summary and conclusions2 (*) 32500 Google hits xxxx/2.3.2008/jan
  3. 3. What am I striving for? Statistics Quality Mathematical science pertaining to the Degree to which a set of inherent collection, analysis, interpretation or characteristics fulfils needs and explanation, and presentation of data expectations Quality management (Quality of management) Degree to which coordinated activities of management to direct and control the organization fulfill the needs and expectations of organization’s all stakeholders Principles of good management Factual approach to decision making (ISO 9000 - Quality management principles) Management by fact (Malcolm Baldrige - Core values and concepts) Management by processes and facts (EFQM - Fundamental concepts of excellence) Business methodology Business performance management Measuring business process performance3 3462/2.3.2008/jan
  4. 4. Positioning to statistics in quality management Quality management Constraints: traditions: - Theoretical / -Statistical process mathematical control difficulties -SixSigma - Problems -Problem solving - Warnings -Taguchi techniques Walter Shewhart How to find balance? Darrell Huff 1931 19544 3461/16.2.2008/jan
  5. 5. Information problems and danger of empirism • Data hell – Chaotic data disturbance • Data bulimia or data anorexia • Data mechanization – overly emphasized information technology and impoverished skills • Knowledge splintering into objective information pieces • Data pedantry and data naiveté • Popularization – Complex realities overly simplified and information contents impoverished • Information insecurity • Information capitalism or information proletariat – misuse of information power5 • Information terror and data crime 3465/14.2.2008/jan (Ref.. Jaana Venkula, Huff, etc.)
  6. 6. Warning about the danger of information (*) Information is not knowledge. Knowledge comes from theory. Without theory, there is no way to use the information that comes to us on the instant. There is no true value of any characteristic, state, or observation. Warning about people rewarding based on quantitative measurements, deadly disease #3 of leadership: “Evaluation of performance, merit rating, or annual review nourish short-term performance, annihilates long-term planning, builds fear, demolishes teamwork, nourishes rivalry and politics. It leaves people bitter, crushed, bruised, battered, desolate, despondent, dejected, feeling inferior, some even depressed, unfit for work for weeks after receipt of rating, unable to comprehend why they are inferior. It is unfair, as it ascribes to the people in a group differences that may be caused totally by the system that they work in.” More important than to create scrupulous and even harmful business measurement systems, is to get tacit knowledge of business leaders and workers effectively interact with each other for creating extensive profound knowledge within an organization.6 3466/14.2.2008/jan (*) W. Edwards Deming (e.g. in "The new economics")
  7. 7. Pragmatic (*) genuine knowledge Pragmatism (C.S. Peirce): Action based information / knowledge approach: • Genuine knowledge makes possible to act in a new meaningful way. • Knowledge can be genuine only if it has been tried and justified in operation. • Knowledge is a basis of an organization and a community and their practical operations. • Knowledge cannot be objective. Charles Sanders Peirce ”The knowledge you take is equal to the knowledge you make.” (**) (*) Ref. the other paradigms of knowledge theories: - Heuristic (internal comprehension based knowledge) - Empirical (external observation and experience based knowledge)7 2475/4.3.2008/jan (Ref.: Jaana Venkula) (**) Lennon, McCartney
  8. 8. Quantitative approach to driving business performance Quantitative indication of performance: - Phenomenon to be considered of a business interest and its character – especially in business processes - Measure / indicator, characteristic, quantity - Metrics, unit of measure - Numerical values and value range of the quantity Practicalities of performance management: - Target / required values, observed values - Meter, gauge, means to measure / observe the quantity - Measurement, assessment technique - Uncertainty - Conclusions, decisions and actions based on measurements - Approach and practices for performance improvement8 1613/10.3.2008/jan
  9. 9. Performance measurements for business management – Need of business information Management areas: Purpose of measurements: - Operational management - Research activities for getting new - Strategic management knowledge - Acquisition of information for planning business or operations Scope of measurements: - Controlling operations and processes - Performance of business processes !Performance monitoring - Performance of products (goods and !Determination measurements services) !Verification measurements, hypothesis testing (AQL / LQ) !Validation testing Related aspects: - Measurements for problem solving and performance improvement: Kaizen, - Measurements management breakthrough and learning - Business performance management - Measurements for quality assurance9 3467/2.2.2008/jan
  10. 10. From business activities to overall performance of an organization and to performance excellence Overall business performance References: } Competitive performance Customer- Operational Product Financial and focused performance performance marketplace performance - processes - goods performance PERFORMANCE - workforce - services EXCELLENCE! Excellence is originated from the business activity not only Financial Non-financial being superficially outstanding measures (data) measures (data) DOMAIN OF ACTION - BUSINESS ACTIVITY (Operational business processes and projects)10 0877/22.3.2008/jan (Ref.: Malcolm Baldrige Criteria)
  11. 11. Interesting information on business performance • Information available from all key business areas and processes and related requirements: – Consistent and balanced quantitative data through measures and indicators of business related facts • Level of performance • Rate of improvement, or slope of trends in data demonstrating development of business performance – ”Three generation reporting”: Past, now, future – Pattern of development: Linear, non-linear, spiral • Comparisions of performance levels and their development with own targets, competitors, and World-Class benchmarks • Relationships of achieved quantitative business results with management and process activities: Cause-effect linkages11 3468/2.2.2008/jan (Ref.: Malcolm Baldrige Criteria)
  12. 12. Temporal dimensions of the present moment “Now” Time The ‘Now’ of future events and performance The ‘Now’ of present events and performance The ‘Now’ of past events and performance } Kairos Kronos The present “Now” always includes the impacts of the past events and the challenges of the future events!12 2583/5.3.2008/jan
  13. 13. What is information security all about? Basic concepts characterizing security of the information consist of: • Integrity – information that one is using for his/her actions is accurate • Availability – one has access to the relevant information when and so long it is needed • Confidentiality – information one is using is not manipulated by anybody else. Additionally authenticity (information is authentic, trustworthy, or genuine) and authority (one has right to use the information) aspects are significant especially when using the means of electrical communication. The most essential information security aspect is privacy!13 2476/11.2.2008/jan (Ref.: ISO/IEC 27001 / 27002)
  14. 14. You don’t get necessarily what you measure Measurements in all kinds of business activities are important for management but in practice: • Measurements are often excessive, unfocused or wrong (*) • Business results you get don’t necessarily relate to what you measure • What you get through measurements is not necessarily meaningful or good for organization or people being managed • Business leaders may not know what they don’t know (“Tacit ignorance”) Measurements results represent explicit knowledge of the business performance but business intentions and real business results are principally of tacit knowledge of business leaders and workers in the organization. No measurements can be objective but they are always affected by the intension and awareness of somebody – even in physics objectivity is impossible. What is being measured, by what kinds of means or methodology, what is obtained through measurements, and how the measurements results are understood – they all depend on somebody’s intention and awareness.14 (*) Dr. Juran: “Vital few” / “Trivial many” 3469/5.3.2008/jan
  15. 15. Facts & knowledge for managing business activities Wisdom - myths - values Knowledge Intervention - explicit records Reflecting and deciding - tacit knowledge Plan / Act (know-how, competence) Information ”Ba” Analysing You get what A P you measure Data C D Environments Measuring ... Facts Effects15 The performance reality of the company business processes 0609/25.3.2006/jan
  16. 16. PDCA model (*) for a good management: Coordinated activities to direct and control an organization (**) P = Planning D = Doing ACTING: PLANNING: C = Checking • Preventing actions • Business and A = Acting • Improving actions management models • Re-engineering • Business plan • Communicating Applying PDCA model: • Approaches and • Recognizing and methodology • Rational control (operational) rewarding • Continual rational small step A P improvement (operational), C D CHECKING: DOING: “Kaizen” approach • Assessing the • Deploying the approach • Innovative breakthrough performance and achieving the results changes (strategic) (***) • Reviewing the • Controlling operational performance performance All three PDCA loops modes and • Corrective actions four PDCA phases are strongly16 information / knowledge dependent 2343x/15.8.2007/jan (*) Deming / Shewhart Cycle (**) ISO 9000 (***) Dr. Shiba
  17. 17. Control of the business activities - feedback Business Operations Business output requirement (Business processes) input Top Notes: Business • Requirements for control : control Throughput times Tfb<<Top input • ”A decision must be made at lowest possible level in Feedback the organization and as close Tfb as possible to where its outcome will be executed.” (Peter Drucker) Target17 1847/26.2.2008/jan (Ref.: Bode, Asby)
  18. 18. Traditional quality tools Traditional ”quality tools” have a long history and are very strongly related with data prosessing and management of business operations. Assortment of the methods may seem confusing, despite that they have been arranged into different groups such as Seven basic quality tools, Seven new management and planning tools, 127 tools, Kaizen tools, SixSigma tools, etc. tools. These tools include e.g.: Simple basic tools: Process management tools: Inspection, testing or •Data collection tools •Control charts acceptance methods: •Idea creation, brainstorming •IDEF0 •Attribute testing tools, nominal group technique •Process capability methods •Variable testing •Check lists / sheets •Flow charts Data analysis methods: •Survey methods •Statistical tests •Affinity diagram Project management tools: •Matrix diagram •Pert / Gantt chart •Variance analysis •Histogram / bar graph Reliability (dependability) tools: •Regression analysis •Scatter plots •Failure mode effects and •Design of experiments •Relations diagram criticality analysis (FMECA) •Z-score analysis •Cause-effect, fishbone or •Fault tree analysis (FTA •Value analysis Ishikawa diagram •Reliability prediction •Pareto chart •Reliability testing •Evaluation and decision- making tools, decision three, Recommendation: Organizations should establish and maintain18 force-field diagram their tool sets for quality management, “Business excellence tool •Voting methods kit”, as well as the measurement and information management at 3470/26.2.2008/jan large.
  19. 19. Using measurements, analyses, and knowledge within business management (*) Enablers criteria Category 4: Measurement, analysis, and knowledge management: !Measurement, analysis, and improvement of organizational performance (Assessment area 4.1) !4.2 Management of information, information technology, and knowledge (Assessment area 4.1) Results criteria: Category 7: Results !Product outcomes (Assessment area 7.1) !Customer-focused outcomes (Assessment area 7.2) !Financial and market outcomes (Assessment area 7.3) !Workforce-focused outcomes (Assessment area 7.4) !Process effectiveness outcomes (Assessment area 7.5) !Leadership outcomes (Assessment area 7.6)19 3472/.20.3.2008/jan (*) Malcolm Baldrige Criteria 2008
  20. 20. Using measurements, analyses, and knowledge within business management (*) Enablers criteria: !Criterion 2: Policy and strategy - Policy and strategy are based on information from performance measurement, research, learning and external related activities (Criterion part 2b) !Criterion 4: Partnership and resources - Information and knowledge are managed (Criterion part 4c) Results criteria: !Customer results (Criterion 6) !People results (Criterion 7) !Society results (Criterion 8) !Key performance results (Criterion 9)20 3473/.20.3.2008/jan (*) EFQM Criteria
  21. 21. Information technology serves our knowing and learning as an extension of the body Impact of technology: The form of a medium imbeds itself in the message, creating a symbiotic relationship by which the medium influences how the message is perceived, creating subtle change over time. We shape our tools and afterwards our tools shape us. Use of technology: – Collecting data – Processing data and information Marshall McLuhan 1911-1980 – Storing data and information – Sharing / communicating information and knowledge (IT = Interactive technology)21 3474/2.1.2008 /jan
  22. 22. Collaborative knowledge working / learning environment with Web 2.0 tools, “Enterprise 2.0” Collaborative knowledge working environment for activity XXX at organization XYZ: Group- Common shared knowledge-base accumulated, members discussed, and updated by the group-members (Wiki) Knowledge feeds from group-members findings, observations, and reflection, and comments (Blog) Discussion forums Automatic information feeds from external Documents files sources, e.g. Internet (Aggregators) " Links to • Other tools (calendar, to-do list, diary,22 Skype, virtual meeting room, etc.) • Administration of the environment 3372/15.9.2007/jan
  23. 23. Web 2.0 challenges Web 2.0: Second-generation of Internet-based services - such as social networking sites, wikis, communication tools, and folksonomies - that let people collaborate and share information online in ways previously unavailable RSS: (Rich Site Summary) Really Simple Syndication Mashup: A website or application that combines content from more than one source into an integrated experience Tag cloud: A visual depiction of content tags used on a website Facebook: a social utility that connects people with friends and others who work, study and live around them RTE – Real Time Economy: Speeding up information flow between companies, each having capabilities to monitor their businesses continuously and quickly react to changes and exceptions23 3460/2.3.2008/jan (Ref.: http://www.oreilly.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html )
  24. 24. Example: ZEF evaluation methodology ZEF is an advanced interactive evaluation methodology and tool that is originated from strong core competences of evaluation technique and human interface technology. Origin: • ZEF methodology was developed in Finland at Oulu University as Z-scored Electronic Feedback (= ZEF) tool. Effectiveness of its two dimensional evaluation structure was validated by University of Lapland. • Commercial ZEF solution is a product of Oulu based company ZEF Solutions Inc. Usage: • ZEF is an all-purpose tool that is suitable for all kinds of organizations. • ZEF evaluation services are web-based solutions operating as SaaS (Software as a Service). With ZEF on-line tool one may collect, compare and evaluate data effectively and efficiently from even a big group of people. • ZEF tool has been used by a lot of organizations for business information and hundreds thousands of ordinary citizens in several survey-cases of general societal interest.24 3336/.20.3.2008/jan (Ref.: http://zefsolutions.com/www/english , http://qiblog.blogspot.com/2007/06/carry-out-appropriate-surveys.html )
  25. 25. ZEF evaluations ZEF tool is applicable for: • All kinds of quality-related performance evaluation, feedback and survey applications !E.g. 3-In-1 assessment scheme that combines Malcolm Baldrige model, EFQM model and ISO 9000 and is easy to tailor to specific business aspects or organizational strategic priorities. Examples include modifications for !Health services quality management (CEN/TS 15224) !Food safety management (ISO 22000) !Information technology service management (ISO/IEC 20000) !Process performance evaluations !Customer / workforce satisfaction • Other business related assessments needs including !Innovations evaluation !Risk, vulnerability and threat analyses !Strategy analyses !Customer lead surveys • Other general e.g. social, entertainment, etc. applications including !National innovation policy surveys !Parliamentary / presidential election comparisons !Education and school evaluations25 !Idols rating 3476/20.3.2008/jan
  26. 26. Improving effectiveness / efficiency of self-assessments with an advanced tool (ZEF)26 3288/20.3.2008/jan (*) ZEF Solutions Inc., Finland: www.zef.fi
  27. 27. Performance evaluation for excellence through a multi-faceted and consistent approach Overall business performance (e.g. 3-In-1 criteria) 2. Strategic 5. Human management resource focus 7. Business 1. Leadership results 3. Customer and 6. Process market focus management 4. Measurement, analysis,and knowledge management Leadership performance Customer satisfaction Information security Social responsibility Market communication Product Innovations performance IT Governance Board operations Strategic performance Process performance Corporate security People satisfaction Project performance Risk management 360 feedback Supplier performance27 Networking capability OHS assessment Quality Assurance: ISO 9001, AQAP 3325/26.3.2008/jan Environmental management
  28. 28. Results of ZEF evaluations (“3-in-1”) Assessesment scheme “3in1”: EFQM, Malcom Baldrige and ISO9000 combined in one scheme Category 4: Measurement, analysis, and knowledge management 1. We follow up fulfilment of targets and development of performance on company and process levels with adequate measures and indicators. 2. We follow up effectiveness and efficiency of our business processes. 3. We have easily available analyzed and current information on performance of our company and its development against targets and other references. 4. Our real-time internal accounting operates in advance. 5. Our employees measure, analyze and evaluate quality of their own work in order to make decisions and initiate improvement actions for their work in line with our28 company-wide plans. … … Etc. other assessment items 3475/20.3.2008/jan
  29. 29. Results of ZEF evaluations29 3334/.20.8.2007/jan
  30. 30. Statistical inferences from ZEF evaluations Detailed business information is obtained from ZEF raw data through statistical treatments. All individual respondent answers to the query items in the ZEF evaluations are recorded as separate data pieces that are then used for statistical analyses as needed: ! The basic ZEF report shows the means and standard deviations of all respondents’ answers per query items in one or two dimensions as needed. ! The result data may be normalized according Z-scored transformation that is useful to emphasize relative differences among query items. ! Answers from respondents may be grouped in arbitrary way or presented as individual answers. ! Results of different ZEF evaluations, e.g. of different points in time, may be compared by using ZEF’s “comparison engine” function. ! Raw data of evaluations may be presented in the formats of Excel (.xls) or "Concurrent Versioning System" (.cvs) in order to facilitate the use of more sophisticated statistical tools, e.g. SPSS, for data analyses. ZEF tool’s interactive user-interface enhances user’s deliberation when he/she is responding to the query items and also enhances understanding the relative differences among query items. That makes answering more motivated and justified.30 3471/.20.3.2008/jan
  31. 31. Z-score transformation Z-score for an item indicates how far and in what direction that item deviates from its distributions mean expressed in units of its distributions standard deviation. The z-score transformation is such that if every item in a distribution is converted to its z score, the transformed scores will have a mean of zero and a standard deviation of one. Z-score transformation is especially useful when seeking to compare the relative standings of items from distributions with different means or different standard deviations.31 3478/20.3.2008/jan (Ref.: http://www.sysurvey.com/tips/statistics/zscore.htm )
  32. 32. Z-scored normalized evaluation results (ZEF)32 3477/.20.3.2008/jan
  33. 33. Summary and conclusions 1. Competitiveness and success of all organizations are based on right business related information and knowledge on time. 2. Impressive business phenomena and their nature including stochastic features should be taken into account in planning business performance measurements. 3. Business related measurements and information may have many different purposes. 4. One should recognize vital few valid subjects for business performance measurements. 5. Measurements should be based on sound methodologies and effective measuring tools. 6. Observed data should be understood in the business context and also data’s statistical viewpoints should be noted. 7. Tacit knowledge and explicit information should be brought together in making conclusions and activating managerial measures. 8. Interactive information technology gives remarkable advantage and serves as an extension of our body and senses both in making observations and in using information within business communities. 9. Cooperation of quality and statistics experts is obviously useful.33 3479/.20.3.2008/jan
  34. 34. j Overall quality of a network Win / Win in network Overall quality of a network Qvm: Degree to which the network fulfils needs and S S expectations of all network members: i - m members in the network A A - Each member gets benefits (Si) but also loses something (Ai). Network Network member i member j Number of knots with k links Knots with few links Power law of a scale-free network: Knots with many links A topical problem: How to measure in practice the overall quality of a network?34 3464/15.2.2008/jan Number of links (k)
  35. 35. Please, evaluate my presentation and material and give your feedback with ZEF: http://www.zef.fi/service/user/?q=597 Thank you!35 xx/15.2.2008/jan