Exploring and Examining Assessment Data via a Matrix Visualisation


Published on

Presentation at AVI 2004 (Gallipoli, Italy) to accompany short paper - doi: 10.1145/989863.989886

Basically a matrix of scores, assessors v candidates, where rows/columns can be sorted and scores grouped on the crosshair axes to see them more easily in context

Also a matrix where the cells (not rows/columns) are hierarchical

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Exploring and Examining Assessment Data via a Matrix Visualisation

  1. 1. Exploring & Examining Assessment Data via a Matrix Visualisation Mar tin Graham & Jessie Kennedy Napier University, Edinburgh
  2. 2. Background • Work part of OPAL – Online Partner Lens • Prospective business partners, employers, employees assess each other on various characteristics • ‘Lazy’ users would like to search these assessments rather than perform their own AVI 2004 - Gallipoli, Italy 2 of 13
  3. 3. Why Visualisation? • Why not just search the numbers? • Simple search – may just pick out lenient assessors rather than quality candidates • Statistical analysis – can give averages, but not in context of related assessments • Recommender system - only possible if user has previously performed assessments of their own • Lack of feedback & freedom to browse • Use a visualisation to convey context of users and assessments AVI 2004 - Gallipoli, Italy 3 of 13
  4. 4. Why Matrix Visualisation? • Node-link visualisations give precedence to nodes • Also clutter rapidly when edges >>> nodes • In our case, the interesting data is primarily the assessments – the edges • Matrix visualisations have edges/links at the centre of attention • Directed nature of edges mean assessors and candidates map naturally to axes AVI 2004 - Gallipoli, Italy 4 of 13
  5. 5. Initial Layout AVI 2004 - Gallipoli, Italy 5 of 13
  6. 6. Assessment Context • Investigating single assessments doesn’t tell us much as score is product of both assessment and candidate • Showing related assessments could reveal context of assessment, and of the participants AVI 2004 - Gallipoli, Italy 6 of 13
  7. 7. Assessment Context • Reveal context by overlaying related assessments as ordered bars • For example, say the candidate crosshair intersects two assessments, both coloured blue • The assessors who gave these ratings have their other evaluations collected and ordered around these points • In this case, the bars show the candidate got the worst scores that each assessor handed out • Not only are the scores poor on an ‘absolute’ scale, they are poor ‘relatively’ too AVI 2004 - Gallipoli, Italy 7 of 13
  8. 8. Assessment Context • This candidate has been involved in 7 evaluations in total, all of them poor • Furthermore, they have received the lowest score each assessor has handed out AVI 2004 - Gallipoli, Italy 8 of 13
  9. 9. Assessment context • Brushing a point in the matrix will show value bars for assessors and candidates • Here, we see a low score obtained even though the candidate has a very high average • The bars along the vertical crosshair show that this assessor has a history of handing out harsh evaluations. AVI 2004 - Gallipoli, Italy 9 of 13
  10. 10. Filtering / Focusing • Candidates and assessors may be filtered by position in matrix • I.e. remove all assessors with < n assessments • Assessments are hierarchical in nature • User may filter matrix to include only attributes they are interested in • User may zoom on portions of matrix to see assessment details AVI 2004 - Gallipoli, Italy 10 of 13
  11. 11. Specific Attributes AVI 2004 - Gallipoli, Italy 11 of 13
  12. 12. Conclusions • Collating and overlaying related assessments acts as a context for verifying a candidate’s or assessor’s associated evaluations • Allows users to see whether a candidate’s scores are consistent or not given the assessors who have applied them AVI 2004 - Gallipoli, Italy 12 of 13
  13. 13. Acknowledgements • OPAL – EU Project IST-2001-33288 • http://www.opal-tool.net AVI 2004 - Gallipoli, Italy 13 of 13