Cost Estimation of Ontologies Using ONTOCOMElena Simperl, Tobias Bürger, Igor Popov, UIBK
Motivation: A typical business scenario                               How do I                               identify     ...
Methods and approaches to cost estimation                                  Bottom-up estimation                 Top-down e...
ONTOCOM- Overview               ONTOCOM – A cost estimation model for building ontologies               ONTOCOM uses top...
ONTOCOM               How ONTOCOM works:   Define lifecycle phases   •Ontology building   •Ontology reuse   •Ontology mai...
Top down breakdownProject ACTIVEDate: 18.06.2008,Dubrovnik
The parametric equation               PM: effort in person months               A : baseline multiplicative constant (in...
Effort multipliers               Each process stage is characterized by a specific set of cost drivers               The...
Cost drivers               Product drivers account for the influence ontology characteristics have on                cost...
ONTOCOM               ONTOCOM Model Calibration                                     Input from experts                   ...
Using ONTOCOM: An example               Exemplary ontology with 600                concepts, 100 relations and 50        ...
Data collection using an online survey                         We need your data – please visit the survey here:          ...
Data collection and model calibration in SALERO               55 identified multimedia ontologies, 15                repl...
New web site            http://ontocom.sti-innsbruck.atProject ACTIVEDate: 18.06.2008,Dubrovnik
Outlook and future plans               Development of a family of ONTOCOM models                    -   ONTOCOM-Ultra Lit...
Goal: Web 2.0 and semantic technologies’ economic  measurements – cost estimation               Produce methods to assess...
Subgoal: Benefit estimation methods for ontologies     Central question: What are the benefits gained from the introducti...
First proposal: A multiple gap model for user informationsatisfaction analysis     User Information Satisfaction (UIS) is...
Sources               Elena Paslaru Bontas Simperl, Christoph Tempich, Malgorzata Mochol                "Cost estimation ...
Thank you for your attentionProject ACTIVEDate: 18.06.2008,Dubrovnik
Upcoming SlideShare
Loading in …5
×

ONTOCOM

574 views

Published on

Short tutorial on ONTOCOM (ontology cost model) for the ACTIVE research project.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
574
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ONTOCOM

  1. 1. Cost Estimation of Ontologies Using ONTOCOMElena Simperl, Tobias Bürger, Igor Popov, UIBK
  2. 2. Motivation: A typical business scenario How do I identify How much Ontologies relevant does it ? expeditures? cost? What do I gain from the introduction of What do we the system ? need to build them? How do the gains materialize ?Project ACTIVEDate: 18.06.2008,Dubrovnik
  3. 3. Methods and approaches to cost estimation Bottom-up estimation Top-down estimation Experts estimate the costs of Experts estimate the total costs Expert Judgment low-level components or of a product or a project activities Costs are calculated using Cost are estimated using a Analogy Method analogies between low-level or global similarity function for activities products or projects Costs are calculated as an Decomposition average sum of the costs of lower-level units, whose Method development are known in advance Costs are calculated using a Costs are calculated using a statistic model which predicts statistic model which is the costs of lower-level units on calibrated using historical data Parametric Method the basis of historical data about and predicts the current value the costs of developing such of the total development costs unitsProject ACTIVEDate: 18.06.2008,Dubrovnik
  4. 4. ONTOCOM- Overview  ONTOCOM – A cost estimation model for building ontologies  ONTOCOM uses top-down, parametric and expert-based methods to form its basis for cost estimation of ontology building  ONTOCOM is realized using a combination of methods: - Top-down breakdown of ontology engineering processes to reduce complexity (Decomposition method) - Parametric method to create a-priori statistical prediction model - Validation and calibration of model according to existing project data and experts estimations lead to a-posteriori model (Expert judgmentProject ACTIVEDate: 18.06.2008,Dubrovnik
  5. 5. ONTOCOM  How ONTOCOM works: Define lifecycle phases •Ontology building •Ontology reuse •Ontology maintenance Specify cost drivers •Ontology building •Ontology reuse •Ontology maintenance Refine the model •Evaluate cost drivers Top-down methodology •Specify start values •Calibrate the model Parametric methodology Parametric methodology Expert-based methodologyProject ACTIVEDate: 18.06.2008,Dubrovnik
  6. 6. Top down breakdownProject ACTIVEDate: 18.06.2008,Dubrovnik
  7. 7. The parametric equation  PM: effort in person months  A : baseline multiplicative constant (in person months)  Size : expected size of ontology (distinction between different entitiy types e.g. classes, properties, axioms´and size of ontology building/reuse/maintenance)  α : acknowledges non-linear behavior wrt. to size  EM : effort multiplier (correspond to cost drivers)Project ACTIVEDate: 18.06.2008,Dubrovnik
  8. 8. Effort multipliers  Each process stage is characterized by a specific set of cost drivers  The cost drivers are associated to rating levels  The rating level (from very low to very high) expresses the impact of each cost driver on the development effort  Each rating level of each cost driver is associated to a weight (quantitative analysis) - effort multiplier (EM)  The values of effort multiplier are subject of further calibration on the basis of the statistical analysis of real-world project data.Project ACTIVEDate: 18.06.2008,Dubrovnik
  9. 9. Cost drivers  Product drivers account for the influence ontology characteristics have on costs - e.g. Complexity of the Domain Analysis, Required Reusability, Documentation Needs  Project drivers account for the influence of project setting characteristics on the overall development - E.g. Support Tools, multi-site development  Personnel drivers emphasize the role of team experience, ability and continuity w.r.t. the effort invested in the process - E.g. Ontologist/Domain Expert Experience, Language/Tool Experience  Total amount of cost drivers: 20  Identification of cost drivers through literature survey, analysis of empiricial data and expert interviews  Overview of the cost drivers: http://ontocom.sti-innsbruck.at/ontocom.htmProject ACTIVEDate: 18.06.2008,Dubrovnik
  10. 10. ONTOCOM  ONTOCOM Model Calibration Input from experts Calibration Linear Regression a-priori method Correlation Analysis a-posteriori method Bayesian Analysis Input from gathered dataProject ACTIVEDate: 18.06.2008,Dubrovnik
  11. 11. Using ONTOCOM: An example  Exemplary ontology with 600 concepts, 100 relations and 50 axioms.  Cost drivers: - domain analysis complexity (DCPLX): high - Evaluation of the results (OE) has a high influence on the effort - Instantiation complexity (ICPLX) has a low impact on the effort - Remaining cost drivers: nominal effort  Constant A and α: values 2.58 and 0.15 as resulting from the calibrationProject ACTIVEDate: 18.06.2008,Dubrovnik
  12. 12. Data collection using an online survey We need your data – please visit the survey here: http://survey.sti2.at/public/survey.php?name=OntocomSurveyJune13Project ACTIVEDate: 18.06.2008,Dubrovnik
  13. 13. Data collection and model calibration in SALERO  55 identified multimedia ontologies, 15 replies (30 %)  Survey results - Main application of multimedia ontologies: Annotation (47%) - Total size between 35-10000 - Development effort between 0.5 and 130 PM - Many ontologies were built from scratch (45%) - Most ontologies in OWL-DL (53%)  Calibration using linear regression and Bayesian analysis resulted in new effort multipliers  Prediction quality improved!Project ACTIVEDate: 18.06.2008,Dubrovnik
  14. 14. New web site http://ontocom.sti-innsbruck.atProject ACTIVEDate: 18.06.2008,Dubrovnik
  15. 15. Outlook and future plans  Development of a family of ONTOCOM models - ONTOCOM-Ultra Lite for the estimiation of folksonomies - ONTOCOM-Lite for the estimation of lightweight ontologies - ONTOCOM (Standard) for the estimation of heavyweight ontologies  Tool support for ONTOCOM - Automatic calibration and addition / removal of data points - Form based use of ONTOCOM for cost prediction  Benefit estimation of ontologiesProject ACTIVEDate: 18.06.2008,Dubrovnik
  16. 16. Goal: Web 2.0 and semantic technologies’ economic measurements – cost estimation  Produce methods to assess costs of core Web2.0 and semantic technological solutions  Demonstrate their tangible and measurable benefits within an enterprise for their adoption  Cost prediction for development, maintenance and usage of Web2.0 and semantic technological components  How to reach this goal: - Develop a general model of Semantic Web based applications - Develop a catalogue of cost drivers for distributed, collaborative applications based on Web2.0 and semantic technologies - Using literature analysis, expert interviews and knowledge elicitation (use case partners) - Collect cost-benefit related data to calibrate the model & improve prediction quality  Expected outcome: - Tool suite for effort estimation, planning and controlling - Prototypical methods to integrate cost/benefit rationals into collaborative knowledge creation / elicitation tasksProject ACTIVEDate: 18.06.2008,Dubrovnik
  17. 17. Subgoal: Benefit estimation methods for ontologies  Central question: What are the benefits gained from the introduction of an ontology based application?  Typical distinction: tangible / intangible benefits  Different methods have a quantitative, qualitative or financial output  Requirements – the nature of benefits of ontologies 1. Most expected benefits from typical uses are intangible - For Communication: to ensure interoperability, for disambiguation (unique identification), or for knowledge transfer (by excluding unwanted interpretations through informal semantics). - For Computational Inference: for browsing / searching (automatic inferring of implicit facts), for automation / code generation or to spot logical inconsistencies. - For Reuse and organisation of knowledge: for knowledge reuse or for structuring of information and knowledge. 2. As the main impact of the use of ontologies is to improve information communication, the method should not have a financial output 3. Ontologies and applications using them should be assess simultaneously as an ontology typically only acquires value when used in combination with an application (analogously to information systems)
  18. 18. First proposal: A multiple gap model for user informationsatisfaction analysis  User Information Satisfaction (UIS) is a method to measure intangible benefits  UIS can be measured through a comparison of user expectations with perceived performance on a number of different facets  Multiple gap models are useful for assessing how systems are viewed at various stages of their design, implementation, and use  UIS = f(gap1,…Gapn, Influencing-factors)
  19. 19. Sources  Elena Paslaru Bontas Simperl, Christoph Tempich, Malgorzata Mochol "Cost estimation for ontology development: applying the ONTOCOM model" In W. Abramowicz and H.C. Mayr, Technologies for Business Information Systems. Springer-Verlag Berlin Heidelberg , 2006.  Elena Paslaru Bontas Simperl, Christoph Tempich, York Sure "ONTOCOM: A Cost Estimation Model for Ontology Engineering" In: Proceedings of the International Semantic Web Conference ISWC 2006  Tobias Bürger "A Benefit Estimation Model for Ontologies" In: Poster Proceedings of the 5th European Semantic Web Conference (ESWC), 2008.  Further information: see http://ontocom.sti-innsbruck.at/info.htmProject ACTIVEDate: 18.06.2008,Dubrovnik
  20. 20. Thank you for your attentionProject ACTIVEDate: 18.06.2008,Dubrovnik

×