“In Desert Storm, an aerial observer located an enemy unit and sent a bombing request to the artillery headquarters. Using the enemy location’s coordinate received from the artillery headquarters, the Navy ship off the coast fired two rounds, but both missed the target by 527 meters, a distance way greater than expected precision! What went wrong? It turned out that the artillery headquarters and the Navy used different geo-coordinate systems with which the same coordinates represent different locations on earth.” (Zhu & Fu, 2009)“The NASA Mars Climate Orbiter was lost after messages between two different systems were misinterpreted: the first was sending values in US Customary units (lbf-s), the second assumed that the values arriving were measured in SI units (Ns); the result was an initial orbit 170km lower than planned—23km below survivable height” (Davies, Harris, Crichton, Shukla, & Gibbons, 2008). The result: the loss of $125 million. And interoperability problems can be very costly, eg. US automotive industry have additional costs of about $1 billion dollar due to low interoperability (Brunnermeier & Martin, 2002)Triage voorbeeld
10052012 erwin folmer tno quality of semantic standards
Title: to modify choose Insert then Heater and footer Date: to modify
WAT IS EEN SEMANTISCHE STANDAARD? Gaat over betekenis van informatie Uitgewisseld door computer systemen (Tussen organisaties) De E-Portfolio standaard! Soort van taal die computers spreken om de uitgewisselde informatie van elkaar te kunnen begrijpen
WAT IS HET PROBLEEM? Puzzelstukjes: De uitwisseling gaat niet optimaal, dat noemen we beperkte interoperabiliteit. Semantiek is een probleem. Beschikbaarheid, adoptie en kwaliteit van semantische standaarden. Wat kost dat? Automobiel industrie VS: 5 miljard Electro industrie VS: 3.9 miljard Capital Facilities industrie VS: 15.8 miljard Zorg VS: 29 miljard (98.000 levens) Rampen: Volendam ramp
NASA Mars Climate Orbiter After a 286-day journey, the probe fired its engine on September 23 1999 to push itself into orbit. The engine fired as the spacecraft came within 60 km of the planet But that was about 100 km closer than planned and about 25 km beneath the level at which the it could function properly. NASA lost the $125 million spacecraft Mars Climate Orbiter because Lockheed Martin used imperial http://mars.jpl.nasa.gov/msp98/news/mco990930.html http://edition.cnn.com/TECH/space/9909/30/mars.metric.02/ units whilst NASA used metric.Source:http://www.interoperabilityconference.org/microsoft.ppt
SEMANTIEK – VOORBEELD 2 TRIAGE “The German triage system also uses four, sometimes five colour codes to denote the urgency of treatment.” http://en.wikipedia.org/wiki/Triage 7/7/12 10
MIJN ONDERZOEK Aangetoond: Probleem met de kwaliteit van semantische standaarden Aangetoond: “Onderzoekshiaat” Mijn ontwerpbijdrage: 1. Meer inzicht in semantische standaarden 2. Meer transparantie in kwaliteit 3. Kwaliteit verbeterbaar maken
UITSTAPJE: PROFIELEN / INTERNATIONALESTANDAARDENMAAR WE HEBBEN TOCH UBL? (OF OAGIS, XBRL, HR-XML, ETC) Standaarden waren geen doel op zich toch? Voor interopabiliteit is strakheid nodig: dus profielen. Lastig is de juiste balans (met internationale standaarden, flexibel en strakheid, etc.) Architectuur – Strategie (codelijstjes,.localizations, etc.)
HET KWALITEITSMODEL Kwaliteitsmodel van Semantische Standaarden (Quality Model of Semantic Standard)A. Product Kwaliteit B. Proces Kwaliteit C. Kwaliteit in Gebruik Ontwikkeling en Inhoud van de De toepassing van beheer van de standaard de standaard standaard (de specificatie) (de implementaties) (de organisatie)
Measurable Concept Definition Remarks A. Product Quality Measurable Concept A. Product Quality B. Process Quality Definition The total attributes of a standard that This includes both internal and external for mea Remarks/Synonyms/Direction The qualityits ability to satisfy stated and quality in ISO terms. determine of the sequence of interdependent and Organizational quality. Measurable implied needs when used under in the standard. Remarks/Synonyms linked procedures that resulted specified Definition B1. Development & conditions. (ISO 9126) how the standard Concept The professionalism of This concept is based on BOMOS. (Folm Maintenance in A1. Functionality C. Quality The capabilityto which maintenance are organized. Punter,fulfills the functional development and the a standardprovide used specification 2011) The extent of the standard to can be The Quality in use. B1.1 Practice D&M Process functions which meet stated theirimpliedto needs of the intended job. guide developers in The capability users to meet and needs a by specified of the D&M process to suit The D&M should needs when the environment. effectiveness, standardization standardwith under achieve specified goals is used achieving quality. B1.1.1 specified conditions. (ISO 9126) specifiedthe The availability of a document describing efficiency and satisfaction in a Including both the initial development B. Process Quality A1.1 Completeness The extent to and(ISO 14598) is of Documented C1. development which a standard process. This includes other terms like relevancy context of use. maintenance The adoption of depth, and scope for approach and change procedure. A pate Process Acceptance sufficient breadth,the standard within the and suitability, and is the functional view Adoption/acceptancepart of the process, as check should be in practice.A1. Functionality A2. Usability A3. Durability the task at hand. (Wand & Wang, 1996) domain. on the contentsigned IPR statementTheworkgroup as a of the specification. of C1.1 Solution participants. The extent to which solutions providers have at hand isproviders providean task Solution aimed at solving products B1.1.2 Providers The time needed for changes to take place, interoperability problem. used by the end standa adopted the standard. and servicethe version history of the Check that are A1.1.1 Changes Time for The level of functions specified in themaintenance if the standardofby solution beginning with the status of the new Indicates the overview covers all users. The adoption the maintenance reque Covered Functions specification in relation to the standard. functionality required to solve the request until the release of the history. providers is a multiplier for adoption. C. Quality in Practice B1.1.3 C1.1.1 A1.1.2 interoperability problem. fixes that solves major Is it possible way is the process agile enough The ability to release bug Implementations and The level of implementations in the products Unplanned Changes The specified the standard. In what for end interoperability problem. users to use the errors services offered by solution providersWhen information using products off thethe stand within level of information enable a quick next version of standards by elements are missingB1. Development & A1.1 Completeness A2.1 Understandability A3.1 Adaptability and to to support the interoperability or when too which information elements Covered Information elementsbe used by end users. shelfin many the bug is fixed? Are bug fixes B2. Communication B3. Organization in End User provided by solution providers? have beenreleased?5 of products and bug was Maintenance Products/Servicesproblem. Checkadded, it How longa negative the top will have after the impact on identified? interoperability. service in the market on their standard B.1.1.4 A1.2 Accuracy The capability of a thorough and documented review of needed specificity andmeasured are t The presence of the standard to provide The level Aspects that need to be usage. A1.1.1 Covered Functions A2.1.1 Availability of A3.1.1 Modularity Review Procedure true data with the needed degree of process. C1.1.2 The availability of tools and components that Check if toolsof review cycles and the numb number or components and precision in both semantic meaningare Knowledge Representations Availability of precision. (ISO to simplify implementations of can be used 9126 & ISO 25012) technical syntax. (This does nottypes ofbut reviewers. Different cover, stakehold available, for example open source Implementation the standard. relates to,reviewers? Public review round? Passiv components. of the content: the quality A1.1.2 Covered Information A3.1.2 Dynamic Content consistency (A1.3)) active (like testing) reviews? reviews or B1.1 D&M Process B2.1 Support B3.1 Governance Support Tools C1. Acceptance C2. A2.1.2 Structure of the Interoperability B1.1.5 A1.2.1 C1.1.3 The level of detailof consultants, and language, The standard address a outside the to mod usage of a methodology, including The availability and in-depth of the The Does the expertise methodology will A proper available specific lead Use of Methodology scope. the process. Availability of within Specificity implementation partners as a support for problem or a generic problem? Is there an explic SSO. that are maintainable. Specification A3.1.3 Extensibility A1.2.2 Implementation The match between the unambiguously implementation. Syntactic and semantic for the provide How choice organizations methodology? many made accuracy. (For B1.1.1 Documented Process B2.1.1 Helpdesk B3.1.1 Decision Making B1.1.6 Precision Support requested of advanced tooling within the process. surnameon the standard? and the too The usage and provided precision. (ISO example Automation reduces errors. List consultancy (instead of name, A1.2 Accuracy Use of Tooling The extent to which the end users have not limited to 10use of the standard. developm C1.2 End Users 25012) that are normally used in the The actual digits)) A2.1.3 Readability of the A1.3 Consistency The extent theconsistency in using the adopted of standard. process. The degree of coherence and freedom of B1.1.2 Time for Changes Specification B2.1.2 Champion B1.1.7 C1.2.1 The extent to which the D&M process is organized Thethe percentage of use? Of total will The level of usage within the intended contradiction within the standard (ISOgroups same values (vocabulary control) and end What is absence of stakeholder C1.1 Solution Providers C2.1 MaturityA3.2 B3.2 Fitness Maintainability Open Process Market elements to convey similar concepts and 25012). The quality of the contentmeans that no in user audience. it is accessible for all. openness, i.e. an impact. Open access of the organizations? Of total transactions? A1.2.1 Specificity Penetration meaning in a standard. (Stvilia et al., 2007) different models. bygroup is excluded. Differen stakeholder Differentiated different user groups. B1.1.3 Unplanned Changes A1.3.1 Recognition The level of ambiguity thethe information The quality of the structuringstakeholders? (e.g. C1.3 The extent to which of standards receive rates for different and The credibility. A2.1.4 Conditions Specified Information elements, and consistency of use. external recognition. definition academia and SME’s have lower of the information elements. A1.2.2 Precision C1.1.1 Implementations in C2.1.1 Stability B3.2.1 Reputation of SSO ambiguity C1.3.1 The external formal recognition of the Bothparticipation fees than large industry external status and reputation in B2.2 Adoption Strategy A3.2.1 Seperation of B1.1.4 Review Procedure Products/Services A2.1.5 Learning Time End User Concerns A1.3.2 Recognition The level of ambiguity of the function standard. The quality of the IsOpen meeting: Are the meeti players). the standard formally the domain. structuring and Function ambiguity elements and consistency of use. Achievements definition locations accessible? Telco’s and e-mail acclaimed functions, processes of the(eg. ISO status) Is the and B3.2.2 Expertise of SSO C2.1.2 Changes per Release business rules. acclaimed by governments? cale standard to limit meetings/costs? Is the used C1.1.2 Availability of B1.1.5 Use of Methodology B2.2.1 Adoption Plan A1.4 Compliancy The capability of the standard to adhere The compliancy when(Lammers et the 2010) (e.g. published?explain standards are comply or other list in al., A3.2.2 Localisations B1.2 Versioning The capability of the standard toor to other standards, conventions have versioning Netherlands) howdoesconformance required in Explicit version management is implemented, and But the it also measure A1.3 Consistency Implementation Support C2.1.3 Versions in B3.2.3 Quality of Active Use regulations in laws, but also to define possibility to standardthe appropriate expectations. place that combines stability and the the raise can be assured. to this fame/reputation of the standard? Tools make changes. A2.2 Testability Community what compliancy implies for this standard. Is it above or under par? (Chase, 1995) B1.1.6 Use of Tooling B2.2.2 Certification B1.2.1 A3.2.3 Dependability C2. (ISO organization25012) The 9126 & ISOtwo orprocedures of version The ability of and more systems or The capability of the standard to policy on ve Documented and published Version Interoperability The compliance version management, also into achieve meaningful communication A1.4.1 numbering andto exchange information andCompliancy with other standards an two components level to other standards, management including at approach for A1.3.1 Information Ambiguity C1.1.3 Availability of C2.1.4 Life Cycle Management relation to backwards compatibility. External compliance conventions, or regulationshaslaws and use the information that in been levels: 1. Standards used to create this minor version numbering, major and between systems. B1.1.7 Open Process Implementation Support A2.2.1 Test Services A3.2.4 Financial B3.3 Version Continuance similar prescriptions. & Lebreton, 2007) standard (e.g. UML). example with the numberi exchanged. (Legner releases, for A1.3.2 Function Ambiguity C2.1 Maturity The capability of the standard to be a stableStandards onstandard will positively new versio 2. A mature different levels of a What is the trigger to start and proven solution. interoperabilityinteroperability. of maintenance Based on the number influence (e.g. laws, or technical C2.1.1 A stable release schema means ample time in Count the numberreleases are within in num standards). requests? Or of versions limited C2.2 Correctness Advancedness Orientation A3.3 B3.3.1 Profit A1.4.2 Stability The availability of a strict set of testable between releases. Is there a strict formulationthere tool support to d on time-basis? Is if versions several years. Too many the B1.2 Versioning C1.2 End Users A2.3 Openness Compliance defined rules that define compliancy with the within a short time will temper and mainten with version management both implementation is to be conformant to the standard. standard? requests? Is backwards compatibility This supports strict adoption and interoperability. A A1.4 Compliancy B3.3.2 Revenue Model implementations. onefor a certain period? guaranteed major release per maximum of C2.2.1 Interoperable Installed Base A3.3.1 B1.2.2 A2. Usability The capability of the standard has to be The ability within the D&M process to deal with contains terms like implementabilityrequests Is an overview of maintenance Also year is recommended. Is there a fixed B1.2.1 Version Management Implementations Maintenance maintenance requests. release process (e.g. a new as to the MRs present on the website? Are understood, learnt, used and attractive to and readability, and is needed soversion will C1.2.1 Market Penetration A2.3.1 One World Requests the user, when used under specified estimate the efforts required for Jan. 1How many traceable, including history? st)? become available yearly on A1.4.1 External Compliance A3.3.2 Technical C2.1.2 The number of changes that have been made Count the number Requests, and how often conditions. (ISO 9126) Maintenance of changes per implementation. B1.2.2 Maintenance Request Advancedness A2.1 Changes per The capability of the standard to enable to the standard. To enable these processed, interpreted by do the release. be read and and how often it to A2.3.2 Availability C2.2.2 Fault Tolerance Understandability Release the user to understand the standard for Too manyto changes? lead changes might indicate that users. (ISO 25012) A1.4.2 Compliance Defined usage for particular tasks and conditions current all stakeholders might to submit M Arequality is low andinvited have C1.3 Recognition B2. Communication of use. (ISO 9126) The totality of communication activities related to impact on interoperability. an The presentation of the standard to the A2.3.3 Use / Re-Use A3.3.3 Business Processes C2.2.3 Completeness A2.1.1 The standard. the level of available knowledge of the The availability theworld. outside adaptations needed in Calculate of representations other B2.1 Support Availability of standard in different represented support. than the specification, such as based on The availability of knowledgeable forms. Support aimed at helping stakeholders software implementations knowledge implementation guides, “how to’s”, “forthe stand (the choice of) implementing function points for a new version. How C1.3.1 Recognition C2.2.4 Relevancy Conceptual A3.3.4 B2.1.1 representations The availability of knowledgeable support from the isItthe oldest version of the dummies”, training, etc. implementations, and old will improve Advancedness Helpdesk A2.1.2 The structuremaintaining the standard. organization of the specification contains Understandable and complete structure of requ valuable feedback on maintenance standard in use? And how many Achievements A2.4 Technical Complexity Structure of the all needed and expected subjects in a the specification between the current or any othe versions in document. forum, Is there a helpdesk, Specification logical manner. (Hyatt & Rosenberg, version? for asking questions and receivin means C2.3 Cost & Benefits C2.1.3 1996) number of versions that are concurrently Too many versions will tamper many chan The appropriate answers? How
VOORBEELDEN VAN METINGEN Produkt Kwaliteit: Leesbaarheid van de specificatie: “15 jaar en ouder” Eenduidige Definities: “Naam = de naam van een persoon” Kwaliteit in Gebruik: Relevantie: Onderdelen in standaard die in implementaties niet gebruikt worden.
UITSTAPJE: QUALITY IN PRACTICE Compleetheid en Relevantie (uit: Quality in Practice) Zhu and Wu: Completeness and Relevance voor XBRL Ui S Compleetheid: Zit alles er in? Completenessi Ui Relevantie: Wordt alles gebruikt? Ui S Relevancyi S Resultaat: Schokkend!
RAZEND INTERESSANT VOOR ELKE STANDAARD Aanvullend voor SETU
DE RESULTATEN Verrassend en inspirerend! Completeness and Relevance behoorlijk goed Zeer waardevolle input voor nieuwe versie van een standaard. Beroerde implementatie kwaliteit: Wat is de reden? En oplossing? (cursus?)
IQMSS Situationeel ingezet: dus niet het hele kwaliteitsmodel. Niet alles is extreem helder/concreet. Wel gevalideerd (cases / workshops) Garantie op nieuwe inzichten is er! Zelfs voor de meest ervaren standaardisatie-ontwikkelaar.