Design and Evaluation   of the  Goal-oriented Design Knowledge Library Framework iConference 2012 <ul><li>Andrew Hilts </l...
The GO-DKL Framework <ul><li>A model and method for extracting, codifying and storing relational excerpts of design knowle...
Siloed e-Democracy Design <ul><li>Emerging scholarly community </li></ul><ul><li>“ Design knowledge ”  expressed textually...
Study objectives <ul><li>Unified model of scholarly findings for accessible and efficient retrieval. </li></ul><ul><li>Sup...
Design Knowledge Management <ul><li>How to effectively manage information systems design knowledge? </li></ul><ul><ul><li>...
Design Knowledge: a metamodel
Design Knowledge: a metamodel <ul><li>Conceptualize </li></ul><ul><ul><li>What is  design knowledge? </li></ul></ul><ul><u...
Design Knowledge: a metamodel <ul><li>Codify </li></ul><ul><ul><li>How do we obtain and store this knowledge? </li></ul></...
Design Knowledge: a metamodel <ul><li>Retrieve </li></ul><ul><ul><li>How can this knowledge be accessed? </li></ul></ul>ht...
Design Knowledge: a metamodel <ul><li>Represent </li></ul><ul><ul><li>How is retrieved knowledge depicted? </li></ul></ul>...
Design Knowledge: a metamodel <ul><li>Situate </li></ul><ul><ul><li>How to contextualize? </li></ul></ul>
Design Knowledge: a metamodel <ul><li>Evaluate </li></ul><ul><ul><li>How to determine if appropriate for usage context? </...
Methodology
Hevner et al ’ s model of I.S. research
A goal-oriented view of design knowledge <ul><li>Design knowledge as the relationship between I.S. design features and sta...
Building a design knowledge base <ul><li>How to  codify  design knowledge expressed in narrative textual form? Our approac...
Library and Project Data
Codification and Analysis
Retrieval <ul><li>How to effectively navigate these relationships? </li></ul><ul><li>Goal-oriented browsing - start from g...
Application Example <ul><li>Facebook DSS </li></ul><ul><ul><li>Design goals: </li></ul></ul><ul><ul><li>Participation be i...
Goal association Users ’   project goal Aggregate goals associated with project goals
Goal browsing Description of a library goal
Relationally guided retrieval Impacted & Related  library goals
Tree-list representation and contextualization of project model Alter contributions to parent Deselect nodes to negate the...
A generated goal model slice
A closer look
Generated Design Feature Report
Generated Publication Report
e-Democracy design practitioner  work practices: methods <ul><li>Semi-structured interviews with 6 individuals. </li></ul>...
e-Democracy design practitioner  work practices: findings <ul><li>Primarily rely on tacit experience; little on codified g...
Practitioner Evaluation of GO-DKL <ul><li>Relevance </li></ul><ul><ul><li>Useful as a heuristic -  “ did I miss anything? ...
Future Work <ul><li>Develop web-based community platform for goal-oriented design knowledge sharing. </li></ul><ul><ul><li...
Questions? <ul><li>[email_address] </li></ul><ul><li>[email_address] </li></ul>
References <ul><li>Ahmed, S. (2005). Encouraging reuse of design knowledge: a method to index knowledge. Design Studies, 2...
Upcoming SlideShare
Loading in …5
×

Design & Evaluation of the Goal-Oriented Design Knowledge Library

709 views
668 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
709
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Design & Evaluation of the Goal-Oriented Design Knowledge Library

  1. 1. Design and Evaluation of the Goal-oriented Design Knowledge Library Framework iConference 2012 <ul><li>Andrew Hilts </li></ul><ul><li>Eric Yu </li></ul><ul><li>Faculty of Information, </li></ul><ul><li>University of Toronto </li></ul>February 9, 2012
  2. 2. The GO-DKL Framework <ul><li>A model and method for extracting, codifying and storing relational excerpts of design knowledge from scholarly publications. </li></ul><ul><li>A method for analyzing such a knowledge base to support information systems design. </li></ul>
  3. 3. Siloed e-Democracy Design <ul><li>Emerging scholarly community </li></ul><ul><li>“ Design knowledge ” expressed textually in journal articles or white papers. </li></ul><ul><li>A shared knowledge base could accelerate and strengthen advances in the domain. </li></ul>
  4. 4. Study objectives <ul><li>Unified model of scholarly findings for accessible and efficient retrieval. </li></ul><ul><li>Support designers / analysts to leverage past work to apply to current problem contexts. </li></ul><ul><li>Highlight related areas of application domain that may be relevant to a designer ’ s project. </li></ul>
  5. 5. Design Knowledge Management <ul><li>How to effectively manage information systems design knowledge? </li></ul><ul><ul><li>No one solution -- domain specific. </li></ul></ul>
  6. 6. Design Knowledge: a metamodel
  7. 7. Design Knowledge: a metamodel <ul><li>Conceptualize </li></ul><ul><ul><li>What is design knowledge? </li></ul></ul><ul><ul><li>What is not? </li></ul></ul>Fludd, R. (1619). Description of perception.
  8. 8. Design Knowledge: a metamodel <ul><li>Codify </li></ul><ul><ul><li>How do we obtain and store this knowledge? </li></ul></ul>Stohlman, T. (2008). Touching Rosetta . http://www.flickr.com/photos/namlhots/227505138/
  9. 9. Design Knowledge: a metamodel <ul><li>Retrieve </li></ul><ul><ul><li>How can this knowledge be accessed? </li></ul></ul>http://www.examiner.com/sites/default/files/styles/image_full_width/hash/StanfordLibrary-JustinSullivan_Getty%20Images-12-17-2004.jpg
  10. 10. Design Knowledge: a metamodel <ul><li>Represent </li></ul><ul><ul><li>How is retrieved knowledge depicted? </li></ul></ul>Magritte, R. (1935). The Human Condition .
  11. 11. Design Knowledge: a metamodel <ul><li>Situate </li></ul><ul><ul><li>How to contextualize? </li></ul></ul>
  12. 12. Design Knowledge: a metamodel <ul><li>Evaluate </li></ul><ul><ul><li>How to determine if appropriate for usage context? </li></ul></ul>
  13. 13. Methodology
  14. 14. Hevner et al ’ s model of I.S. research
  15. 15. A goal-oriented view of design knowledge <ul><li>Design knowledge as the relationship between I.S. design features and stakeholder objectives. </li></ul><ul><li>Features may help or hurt goals. </li></ul><ul><li>Assess potential designs based on potential goal satisfaction. </li></ul>
  16. 16. Building a design knowledge base <ul><li>How to codify design knowledge expressed in narrative textual form? Our approach: </li></ul><ul><ul><li>Qualitative, inductive coding scheme based on our notion of goal-oriented design knowledge. </li></ul></ul><ul><ul><li>Coding scheme applied to academic papers; instances of scheme concepts tagged and converted into database records and relationships. </li></ul></ul><ul><ul><li>29 codified publications. </li></ul></ul>
  17. 17. Library and Project Data
  18. 18. Codification and Analysis
  19. 19. Retrieval <ul><li>How to effectively navigate these relationships? </li></ul><ul><li>Goal-oriented browsing - start from goals, end up with design feature recommendations </li></ul><ul><ul><li>Directly browsing long lists of goals is not scalable. </li></ul></ul><ul><ul><ul><li>Solution - aggregate goal classifications </li></ul></ul></ul><ul><li>User builds a custom project model , a collection of relevant goals from the retrieved information. </li></ul>
  20. 20. Application Example <ul><li>Facebook DSS </li></ul><ul><ul><li>Design goals: </li></ul></ul><ul><ul><li>Participation be incentivized </li></ul></ul><ul><ul><li>External information be considered </li></ul></ul><ul><ul><li>Degree of Consensus be obtained </li></ul></ul><ul><ul><li>Social Pressures be reduced </li></ul></ul>
  21. 21. Goal association Users ’ project goal Aggregate goals associated with project goals
  22. 22. Goal browsing Description of a library goal
  23. 23. Relationally guided retrieval Impacted & Related library goals
  24. 24. Tree-list representation and contextualization of project model Alter contributions to parent Deselect nodes to negate their contributions Export “ model slice ” Label colour calculated based on received contributions Drill down to reveal high-level goals ’ contributions
  25. 25. A generated goal model slice
  26. 26. A closer look
  27. 27. Generated Design Feature Report
  28. 28. Generated Publication Report
  29. 29. e-Democracy design practitioner work practices: methods <ul><li>Semi-structured interviews with 6 individuals. </li></ul><ul><li>Interviews transcribed and qualitatively analyzed. </li></ul>
  30. 30. e-Democracy design practitioner work practices: findings <ul><li>Primarily rely on tacit experience; little on codified guidelines. </li></ul><ul><li>Inspired by popular websites; tacit evaluations thereof. </li></ul><ul><li>Design goals very similar to those codified in the GO-DKL knowledge base. </li></ul>“ The things that inform me are not documentation about projects, but the projects themselves. ”
  31. 31. Practitioner Evaluation of GO-DKL <ul><li>Relevance </li></ul><ul><ul><li>Useful as a heuristic - “ did I miss anything? ” </li></ul></ul><ul><ul><li>Anticipate problems in advance </li></ul></ul><ul><li>Suggestions </li></ul><ul><ul><li>Link to “ off-the-shelf ” solutions </li></ul></ul><ul><ul><li>Goal Prioritization </li></ul></ul><ul><ul><li>Designers can contribute </li></ul></ul><ul><li>Critiques </li></ul><ul><ul><li>Trustworthiness </li></ul></ul><ul><ul><li>Vocabulary </li></ul></ul><ul><ul><li>Usability </li></ul></ul>GO-DKL: A source of inspiration for potential solutions to design problems that must be primarily considered in their own context.
  32. 32. Future Work <ul><li>Develop web-based community platform for goal-oriented design knowledge sharing. </li></ul><ul><ul><li>User-contributed design knowledge; less focussed on scholarly publications. </li></ul></ul><ul><ul><li>Peer assessments of design relationship trustworthiness and contextual dependence. </li></ul></ul><ul><li>Extensive user testing and interface redesign. </li></ul>
  33. 33. Questions? <ul><li>[email_address] </li></ul><ul><li>[email_address] </li></ul>
  34. 34. References <ul><li>Ahmed, S. (2005). Encouraging reuse of design knowledge: a method to index knowledge. Design Studies, 26(6):565–592. </li></ul><ul><li>Ahmed, S. and Wallace, K. M. (2004). Understanding the knowledge needs of novice designers in the aerospace industry. Design Studies, 25(2):155–173. </li></ul><ul><li>Alexander, I. and Kiedaisch, F. (2002). Towards recyclable system requirements. In Proceedings Ninth Annual IEEE International Conference and Workshop on the Engineering of Computer-Based Systems, pages 9–16. IEEE Comput. Soc. </li></ul><ul><li>Babar, M., Gorton, I., and Kitchenham, B. (2006). A framework for supporting architecture knowledge and rationale management. Rationale Management in Software. </li></ul><ul><li>Davies, T. (2009). The Blossoming Field of Online Deliberation. In Davies, T. R. and Pena Gangadharan, S., editors, Online Deliberation: Design, Research, and Practice, pages 1–20, San Francisco, CA. CSLI Publications. </li></ul><ul><li>Eilouti, B. H. (2009). Design knowledge recycling using precedent-based analysis and synthesis models. Design Studies, 30(4):340–368. </li></ul><ul><li>Esfahani, H. and Yu, E. (2010). A Repository of Agile Method Fragments. In Munch, J., Yang, Y., and Schafer, W., editors, LNCS 6195: New Modeling Concepts for Todays Software Processes, pages 163–174, Berlin / Heidelberg. Springer. </li></ul><ul><li>Muller, W. (1996). Typology and the organization of design knowledge. Design Studies, 17(2):111–130. </li></ul><ul><li>Ormerod, T., Mariani, J., Ball, L., and Lambell, N. (1999). Desperado: Three-in-one indexing for innovative design. In Sasse, A. and Johnson, C., editors, 13th International Conference on Human-Computer Interaction, page 336, Edinburgh, UK. Ios PR Inc. </li></ul><ul><li>Oxman, R. E. and Planning, T. (1994). Precedents in design: a computational model for the organization of precedent knowledge. Design Studies, 15(2):141–157. </li></ul><ul><li>Wahid, S., Mccrickard, D. S., Chewar, C. M., and Chong, J. (2006). Entangled Design Knowledge : Relationships as an Approach to Claims Reuse. </li></ul>

×