The document discusses a system for managing knowledge and informal learning to support decision making in businesses. It includes a personal training environment, portfolio system, competence model and organizational environment. A decision support system analyzes informal learning activities and competencies to provide tools and visualizations to facilitate dialog and decisions around training, promotion and more. Experts evaluated the system and provided feedback on improvements including more filters, guidance and data representations. The system aims to help both employees showcase informal learning and organizations make better training and hiring decisions based on analyzed knowledge.

1. Knowledge Management and decision
making based on informal learning
activities in business
Dr. Francisco José García Peñalvo (fgarcia@usal.es)
Dr. Miguel Ángel Conde González (mconde@usal.es)
Grupo GRIAL – University of Salamanca, Spain
Valencia, Spain, July 9-11, 2013

2. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

3. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

4. Introduction (I)
GRIAL – University of Salamanca
• Information is a key element for decision-making
processes
• It is necessary to manage and exploit that information
and ICT makes it possible
• One issue to take into account is informal learning,
specially relevant in the workplace
• Recognition of informal learning
• Enhances employability
• Benefits companies and managers
• Helps to develop task-oriented skills and know-how
• Gives employees the opportunity to learn and keep their skills
up-to-date

5. Introduction (II)
GRIAL – University of Salamanca
• ICT facilitates such recognition
• TENCompetence Project, FREE, IBAK, Open Budgets
• Focused on the validation of informal learning and particularly
issues relating to certification and qualification
• How to obtain a strategic advantage of such informal
learning for organizations and employees?

6. Introduction (and III)
GRIAL – University of Salamanca
• Create a dialog about informal learning evidences
between institutions and employees, based on
• An ICT ecosystem
• A methodology to articulate such dialog
• Implemented as a proof of concept in TRAILER
project
!
h"p://trailerproject.eu/

7. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

8. System description overview (I)
GRIAL – University of Salamanca
• Employee identifies learning + Institutions
recognizes it => Make decisions
• ICT ecosystem for informal learning recognition
• PTE (Personal Training Environment) or PKN
(Personal Knowledge Network)
• Portfolio System
• Competence Model
• Organizational Environment

9. System description overview (II)
GRIAL – University of Salamanca

10. System description overview
(and III)
GRIAL – University of Salamanca

11. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

12. DSS (I)
GRIAL – University of Salamanca
• Defined in the organizational environment
• Critical for decision-making processes
• Two main elements
• Analysis layer
• Gathers public information from the portfolio
• PersonalPortfolioView
• Competence API
• Makes a preliminary analysis to present the
information
• Decision support tools

13. DSS (II)
GRIAL – University of Salamanca
• What is the information considered?
• Informal Learning Activities
• User-Organization Competences
• User-Organization relationships
• Competences/Working Areas/Tags
• How is the information represented to facilitate
decision-making and dialog processes?
• Text-based
• Chart-based
• Data visualization techniques

14. DSS (III)
GRIAL – University of Salamanca

15. DSS (IV)
GRIAL – University of Salamanca

16. DSS (and V)
GRIAL – University of Salamanca

17. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

18. Evaluation (I)
• Implementation as a proof of concept
• Expert testing
 Cognitive Walkthrough complemented with Think
Aloud
• Screen recording of the experts when they used the system
• Classification of breakdowns by Nielsen’s model
GRIAL – University of Salamanca

19. Evaluation (II)
• Expert Testing
• System Usability Scale (SUS)
– 49.6 score
– Far from 68 recommended satisfaction
• Perceived ease of use (PEOU)
– Crombach Alpha to check the validity of the survey
⊳ High level of consistency 0.934
– Average value near 4 (the neutral value of the
scale)
GRIAL – University of Salamanca

20. Evaluation (and III)
• Qualitative Evaluation
• Expert’s opinion about the components of the system
GRIAL – University of Salamanca
Easy to Understand Helpful Improvements
E1 Quite easy Very useful It is ok as is
E2 More explanation
needed
Quite useful Look and Feel and Descriptions
E3 - Lot of information at a glance More options
E4 Need more tips - More filters
E5 Wizard Facilitate making decisions Create a wizard to guide the search
E6 More filters It can be more still more
useful
New filters about competences and
ILAs
E7 Good Ok -
E8 - Useful Other kind of charts
E9 Clear - -
E10 Very easy Great -
E11 - Solve my needs -
E12 Straightforward Easy -
E13 - Too much data Description of data, other kind of
representations
E14 Teaching needed Not sure More info about the data shown

21. Outline
• Introduction
• System description overview
• The Decision Support System
• Evaluation
• Conclusions
GRIAL – University of Salamanca

22. Conclusions (I)
• Methodology to make decisions based on informal
learning
 Technological framework to support it
• Knowledge management for both employees and
organizations
• Employees
 Identify and make visible informal learning to organizations
• Organizations
 Training related decisions
 Promotion decisions
• Dialog between organizations and employees
GRIAL – University of Salamanca

23. Conclusions (and II)
• Each organization has different needs
• Applying the proposed methodology makes possible
to manage knowledge based on informal learning
evidences
• Future work must allow improving the DSS in order
to work with atomic indicators
GRIAL – University of Salamanca

24. Acknowledgements
With the support of the Lifelong Learning Program of
the EU. Project Reference: 519141-LLP-1-2011-1-ES-
KA3-KA3MP
This project has been funded with support from the
European Commission. This publication only reflects
the views of the authors, and the Commission cannot
be held responsible for any use that may be made of
the information contained therein
GRIAL – University of Salamanca

25. Knowledge Management and decision
making based on informal learning
activities in business
Dr. Francisco José García Peñalvo (fgarcia@usal.es)
Dr. Miguel Ángel Conde González (mconde@usal.es)
Grupo GRIAL – University of Salamanca, Spain
Valencia, Spain, July 9-11, 2013