Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Exploring and Examining Assessment Data via a Matrix Visualisation


Published on

Presentation at AVI 2004 (Gallipoli, Italy) to accompany short paper - doi: 10.1145/989863.989886

Basically a matrix of scores, assessors v candidates, where rows/columns can be sorted and scores grouped on the crosshair axes to see them more easily in context

Also a matrix where the cells (not rows/columns) are hierarchical

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Exploring and Examining Assessment Data via a Matrix Visualisation

  1. 1. Exploring & Examining Assessment Data via a Matrix Visualisation Mar tin Graham & Jessie Kennedy Napier University, Edinburgh
  2. 2. Background • Work part of OPAL – Online Partner Lens • Prospective business partners, employers, employees assess each other on various characteristics • ‘Lazy’ users would like to search these assessments rather than perform their own AVI 2004 - Gallipoli, Italy 2 of 13
  3. 3. Why Visualisation? • Why not just search the numbers? • Simple search – may just pick out lenient assessors rather than quality candidates • Statistical analysis – can give averages, but not in context of related assessments • Recommender system - only possible if user has previously performed assessments of their own • Lack of feedback & freedom to browse • Use a visualisation to convey context of users and assessments AVI 2004 - Gallipoli, Italy 3 of 13
  4. 4. Why Matrix Visualisation? • Node-link visualisations give precedence to nodes • Also clutter rapidly when edges >>> nodes • In our case, the interesting data is primarily the assessments – the edges • Matrix visualisations have edges/links at the centre of attention • Directed nature of edges mean assessors and candidates map naturally to axes AVI 2004 - Gallipoli, Italy 4 of 13
  5. 5. Initial Layout AVI 2004 - Gallipoli, Italy 5 of 13
  6. 6. Assessment Context • Investigating single assessments doesn’t tell us much as score is product of both assessment and candidate • Showing related assessments could reveal context of assessment, and of the participants AVI 2004 - Gallipoli, Italy 6 of 13
  7. 7. Assessment Context • Reveal context by overlaying related assessments as ordered bars • For example, say the candidate crosshair intersects two assessments, both coloured blue • The assessors who gave these ratings have their other evaluations collected and ordered around these points • In this case, the bars show the candidate got the worst scores that each assessor handed out • Not only are the scores poor on an ‘absolute’ scale, they are poor ‘relatively’ too AVI 2004 - Gallipoli, Italy 7 of 13
  8. 8. Assessment Context • This candidate has been involved in 7 evaluations in total, all of them poor • Furthermore, they have received the lowest score each assessor has handed out AVI 2004 - Gallipoli, Italy 8 of 13
  9. 9. Assessment context • Brushing a point in the matrix will show value bars for assessors and candidates • Here, we see a low score obtained even though the candidate has a very high average • The bars along the vertical crosshair show that this assessor has a history of handing out harsh evaluations. AVI 2004 - Gallipoli, Italy 9 of 13
  10. 10. Filtering / Focusing • Candidates and assessors may be filtered by position in matrix • I.e. remove all assessors with < n assessments • Assessments are hierarchical in nature • User may filter matrix to include only attributes they are interested in • User may zoom on portions of matrix to see assessment details AVI 2004 - Gallipoli, Italy 10 of 13
  11. 11. Specific Attributes AVI 2004 - Gallipoli, Italy 11 of 13
  12. 12. Conclusions • Collating and overlaying related assessments acts as a context for verifying a candidate’s or assessor’s associated evaluations • Allows users to see whether a candidate’s scores are consistent or not given the assessors who have applied them AVI 2004 - Gallipoli, Italy 12 of 13
  13. 13. Acknowledgements • OPAL – EU Project IST-2001-33288 • AVI 2004 - Gallipoli, Italy 13 of 13