Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Design and Evaluation   of the  Goal-oriented Design Knowledge Library Framework iConference 2012 <ul><li>Andrew Hilts </l...
The GO-DKL Framework <ul><li>A model and method for extracting, codifying and storing relational excerpts of design knowle...
Siloed e-Democracy Design <ul><li>Emerging scholarly community </li></ul><ul><li>“ Design knowledge ”  expressed textually...
Study objectives <ul><li>Unified model of scholarly findings for accessible and efficient retrieval. </li></ul><ul><li>Sup...
Design Knowledge Management <ul><li>How to effectively manage information systems design knowledge? </li></ul><ul><ul><li>...
Design Knowledge: a metamodel
Design Knowledge: a metamodel <ul><li>Conceptualize </li></ul><ul><ul><li>What is  design knowledge? </li></ul></ul><ul><u...
Design Knowledge: a metamodel <ul><li>Codify </li></ul><ul><ul><li>How do we obtain and store this knowledge? </li></ul></...
Design Knowledge: a metamodel <ul><li>Retrieve </li></ul><ul><ul><li>How can this knowledge be accessed? </li></ul></ul>ht...
Design Knowledge: a metamodel <ul><li>Represent </li></ul><ul><ul><li>How is retrieved knowledge depicted? </li></ul></ul>...
Design Knowledge: a metamodel <ul><li>Situate </li></ul><ul><ul><li>How to contextualize? </li></ul></ul>
Design Knowledge: a metamodel <ul><li>Evaluate </li></ul><ul><ul><li>How to determine if appropriate for usage context? </...
Methodology
Hevner et al ’ s model of I.S. research
A goal-oriented view of design knowledge <ul><li>Design knowledge as the relationship between I.S. design features and sta...
Building a design knowledge base <ul><li>How to  codify  design knowledge expressed in narrative textual form? Our approac...
Library and Project Data
Codification and Analysis
Retrieval <ul><li>How to effectively navigate these relationships? </li></ul><ul><li>Goal-oriented browsing - start from g...
Application Example <ul><li>Facebook DSS </li></ul><ul><ul><li>Design goals: </li></ul></ul><ul><ul><li>Participation be i...
Goal association Users ’   project goal Aggregate goals associated with project goals
Goal browsing Description of a library goal
Relationally guided retrieval Impacted & Related  library goals
Tree-list representation and contextualization of project model Alter contributions to parent Deselect nodes to negate the...
A generated goal model slice
A closer look
Generated Design Feature Report
Generated Publication Report
e-Democracy design practitioner  work practices: methods <ul><li>Semi-structured interviews with 6 individuals. </li></ul>...
e-Democracy design practitioner  work practices: findings <ul><li>Primarily rely on tacit experience; little on codified g...
Practitioner Evaluation of GO-DKL <ul><li>Relevance </li></ul><ul><ul><li>Useful as a heuristic -  “ did I miss anything? ...
Future Work <ul><li>Develop web-based community platform for goal-oriented design knowledge sharing. </li></ul><ul><ul><li...
Questions? <ul><li>[email_address] </li></ul><ul><li>[email_address] </li></ul>
References <ul><li>Ahmed, S. (2005). Encouraging reuse of design knowledge: a method to index knowledge. Design Studies, 2...
Upcoming SlideShare
Loading in …5
×

Design & Evaluation of the Goal-Oriented Design Knowledge Library

918 views

Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Design & Evaluation of the Goal-Oriented Design Knowledge Library

  1. 1. Design and Evaluation of the Goal-oriented Design Knowledge Library Framework iConference 2012 <ul><li>Andrew Hilts </li></ul><ul><li>Eric Yu </li></ul><ul><li>Faculty of Information, </li></ul><ul><li>University of Toronto </li></ul>February 9, 2012
  2. 2. The GO-DKL Framework <ul><li>A model and method for extracting, codifying and storing relational excerpts of design knowledge from scholarly publications. </li></ul><ul><li>A method for analyzing such a knowledge base to support information systems design. </li></ul>
  3. 3. Siloed e-Democracy Design <ul><li>Emerging scholarly community </li></ul><ul><li>“ Design knowledge ” expressed textually in journal articles or white papers. </li></ul><ul><li>A shared knowledge base could accelerate and strengthen advances in the domain. </li></ul>
  4. 4. Study objectives <ul><li>Unified model of scholarly findings for accessible and efficient retrieval. </li></ul><ul><li>Support designers / analysts to leverage past work to apply to current problem contexts. </li></ul><ul><li>Highlight related areas of application domain that may be relevant to a designer ’ s project. </li></ul>
  5. 5. Design Knowledge Management <ul><li>How to effectively manage information systems design knowledge? </li></ul><ul><ul><li>No one solution -- domain specific. </li></ul></ul>
  6. 6. Design Knowledge: a metamodel
  7. 7. Design Knowledge: a metamodel <ul><li>Conceptualize </li></ul><ul><ul><li>What is design knowledge? </li></ul></ul><ul><ul><li>What is not? </li></ul></ul>Fludd, R. (1619). Description of perception.
  8. 8. Design Knowledge: a metamodel <ul><li>Codify </li></ul><ul><ul><li>How do we obtain and store this knowledge? </li></ul></ul>Stohlman, T. (2008). Touching Rosetta . http://www.flickr.com/photos/namlhots/227505138/
  9. 9. Design Knowledge: a metamodel <ul><li>Retrieve </li></ul><ul><ul><li>How can this knowledge be accessed? </li></ul></ul>http://www.examiner.com/sites/default/files/styles/image_full_width/hash/StanfordLibrary-JustinSullivan_Getty%20Images-12-17-2004.jpg
  10. 10. Design Knowledge: a metamodel <ul><li>Represent </li></ul><ul><ul><li>How is retrieved knowledge depicted? </li></ul></ul>Magritte, R. (1935). The Human Condition .
  11. 11. Design Knowledge: a metamodel <ul><li>Situate </li></ul><ul><ul><li>How to contextualize? </li></ul></ul>
  12. 12. Design Knowledge: a metamodel <ul><li>Evaluate </li></ul><ul><ul><li>How to determine if appropriate for usage context? </li></ul></ul>
  13. 13. Methodology
  14. 14. Hevner et al ’ s model of I.S. research
  15. 15. A goal-oriented view of design knowledge <ul><li>Design knowledge as the relationship between I.S. design features and stakeholder objectives. </li></ul><ul><li>Features may help or hurt goals. </li></ul><ul><li>Assess potential designs based on potential goal satisfaction. </li></ul>
  16. 16. Building a design knowledge base <ul><li>How to codify design knowledge expressed in narrative textual form? Our approach: </li></ul><ul><ul><li>Qualitative, inductive coding scheme based on our notion of goal-oriented design knowledge. </li></ul></ul><ul><ul><li>Coding scheme applied to academic papers; instances of scheme concepts tagged and converted into database records and relationships. </li></ul></ul><ul><ul><li>29 codified publications. </li></ul></ul>
  17. 17. Library and Project Data
  18. 18. Codification and Analysis
  19. 19. Retrieval <ul><li>How to effectively navigate these relationships? </li></ul><ul><li>Goal-oriented browsing - start from goals, end up with design feature recommendations </li></ul><ul><ul><li>Directly browsing long lists of goals is not scalable. </li></ul></ul><ul><ul><ul><li>Solution - aggregate goal classifications </li></ul></ul></ul><ul><li>User builds a custom project model , a collection of relevant goals from the retrieved information. </li></ul>
  20. 20. Application Example <ul><li>Facebook DSS </li></ul><ul><ul><li>Design goals: </li></ul></ul><ul><ul><li>Participation be incentivized </li></ul></ul><ul><ul><li>External information be considered </li></ul></ul><ul><ul><li>Degree of Consensus be obtained </li></ul></ul><ul><ul><li>Social Pressures be reduced </li></ul></ul>
  21. 21. Goal association Users ’ project goal Aggregate goals associated with project goals
  22. 22. Goal browsing Description of a library goal
  23. 23. Relationally guided retrieval Impacted & Related library goals
  24. 24. Tree-list representation and contextualization of project model Alter contributions to parent Deselect nodes to negate their contributions Export “ model slice ” Label colour calculated based on received contributions Drill down to reveal high-level goals ’ contributions
  25. 25. A generated goal model slice
  26. 26. A closer look
  27. 27. Generated Design Feature Report
  28. 28. Generated Publication Report
  29. 29. e-Democracy design practitioner work practices: methods <ul><li>Semi-structured interviews with 6 individuals. </li></ul><ul><li>Interviews transcribed and qualitatively analyzed. </li></ul>
  30. 30. e-Democracy design practitioner work practices: findings <ul><li>Primarily rely on tacit experience; little on codified guidelines. </li></ul><ul><li>Inspired by popular websites; tacit evaluations thereof. </li></ul><ul><li>Design goals very similar to those codified in the GO-DKL knowledge base. </li></ul>“ The things that inform me are not documentation about projects, but the projects themselves. ”
  31. 31. Practitioner Evaluation of GO-DKL <ul><li>Relevance </li></ul><ul><ul><li>Useful as a heuristic - “ did I miss anything? ” </li></ul></ul><ul><ul><li>Anticipate problems in advance </li></ul></ul><ul><li>Suggestions </li></ul><ul><ul><li>Link to “ off-the-shelf ” solutions </li></ul></ul><ul><ul><li>Goal Prioritization </li></ul></ul><ul><ul><li>Designers can contribute </li></ul></ul><ul><li>Critiques </li></ul><ul><ul><li>Trustworthiness </li></ul></ul><ul><ul><li>Vocabulary </li></ul></ul><ul><ul><li>Usability </li></ul></ul>GO-DKL: A source of inspiration for potential solutions to design problems that must be primarily considered in their own context.
  32. 32. Future Work <ul><li>Develop web-based community platform for goal-oriented design knowledge sharing. </li></ul><ul><ul><li>User-contributed design knowledge; less focussed on scholarly publications. </li></ul></ul><ul><ul><li>Peer assessments of design relationship trustworthiness and contextual dependence. </li></ul></ul><ul><li>Extensive user testing and interface redesign. </li></ul>
  33. 33. Questions? <ul><li>[email_address] </li></ul><ul><li>[email_address] </li></ul>
  34. 34. References <ul><li>Ahmed, S. (2005). Encouraging reuse of design knowledge: a method to index knowledge. Design Studies, 26(6):565–592. </li></ul><ul><li>Ahmed, S. and Wallace, K. M. (2004). Understanding the knowledge needs of novice designers in the aerospace industry. Design Studies, 25(2):155–173. </li></ul><ul><li>Alexander, I. and Kiedaisch, F. (2002). Towards recyclable system requirements. In Proceedings Ninth Annual IEEE International Conference and Workshop on the Engineering of Computer-Based Systems, pages 9–16. IEEE Comput. Soc. </li></ul><ul><li>Babar, M., Gorton, I., and Kitchenham, B. (2006). A framework for supporting architecture knowledge and rationale management. Rationale Management in Software. </li></ul><ul><li>Davies, T. (2009). The Blossoming Field of Online Deliberation. In Davies, T. R. and Pena Gangadharan, S., editors, Online Deliberation: Design, Research, and Practice, pages 1–20, San Francisco, CA. CSLI Publications. </li></ul><ul><li>Eilouti, B. H. (2009). Design knowledge recycling using precedent-based analysis and synthesis models. Design Studies, 30(4):340–368. </li></ul><ul><li>Esfahani, H. and Yu, E. (2010). A Repository of Agile Method Fragments. In Munch, J., Yang, Y., and Schafer, W., editors, LNCS 6195: New Modeling Concepts for Todays Software Processes, pages 163–174, Berlin / Heidelberg. Springer. </li></ul><ul><li>Muller, W. (1996). Typology and the organization of design knowledge. Design Studies, 17(2):111–130. </li></ul><ul><li>Ormerod, T., Mariani, J., Ball, L., and Lambell, N. (1999). Desperado: Three-in-one indexing for innovative design. In Sasse, A. and Johnson, C., editors, 13th International Conference on Human-Computer Interaction, page 336, Edinburgh, UK. Ios PR Inc. </li></ul><ul><li>Oxman, R. E. and Planning, T. (1994). Precedents in design: a computational model for the organization of precedent knowledge. Design Studies, 15(2):141–157. </li></ul><ul><li>Wahid, S., Mccrickard, D. S., Chewar, C. M., and Chong, J. (2006). Entangled Design Knowledge : Relationships as an Approach to Claims Reuse. </li></ul>

×