Presentation at the Blackboard T&L Conference, 2012. Antwerp. Examines local and national drivers of VLE adoption and the importance of understanding the student (and staff) user experience. Reflects on how evaluation of technology and innovation has changed over time and focuses on the annual EHU student elearning survey which is now providing longitudinal data of use adn expectations. Describes how the survey data is used to inform change.

1. In their own words: Understanding
and Enhancing Our Students’
Experience of Blackboard
Lindsey Martin, Edge Hill University

2. Overview
Drivers: National, Institutional, personal
Tools to aid understanding
‘In their own words’: what makes a positive EHU
student experience of Blackboard?
Making use of our student data within EHU
Discussion: what can we learn from each other?

3. Drivers: National
New business
model
Student fees, student as consumer and
assumptions of their expectations as
digital ‘natives’
Institutional Audit
2004-2008
Strong messages from QAA outcomes
reports around the need for a strategic
approach to VLE deployment
National Student
Survey (NSS)
League tables based on student
satisfaction – increasingly looking to the
VLE for enhancements to organisation
and management, communications and
learning resources

4. Drivers: Edge Hill University
Rapid
expansion
Institution has doubled in size in past 5 years
Institutional
Audits 2005
and 2010
QAA 2005 recommendation for a more strategic
use of VLE resulted in an institutional ‘minimum
entitlement’ to the VLE - phased in 2008-2010
which mainstreamed adoption
Much change
around the
VLE
2008 moved from WebCT to Blackboard CE8
and Managed Hosting
VLE Review 2009-2010
2011 upgraded from CE8 to Blackboard Learn
9.1 and rebranded Learning Edge

5. VLE: adoption on undergraduate
programmes
2005 5%
2006/07 60%
2008/09 71.47%
2009/10 90.56%

6. Key tools to aid evaluation and understanding
Baseline data CE8 activity. Drilling down to Faculty and year of
study, active use of Blackboard tools
Blackboard
reporting tools
Usage statistics – users, live courses
Google Analytics Supplements Blackboard reports. Provides
valuable user information – geographic location,
browsers, mobile access
Annual eLearning
Survey
Online survey, began in 2008/09. Mixture of
closed questions supplemented with free text
NSS/Student Union
surveys/focus
groups
NSS and SU qualitative data is limited with regard
to the VLE. Focus groups exploring ‘student
experience’ only offer a glimpse of VLE

7. Evolving … evaluation for decision-making
Assumptions Satisfaction Expectations Impact
Rule of thumb
Experience
Similarities
Perceptions
Emotional
Behavioural
Will recommend to others?
Intentions KPIs/ performance
measurement
Return on
investment
Engagement
Access
‘You said…
…we did’
Checklists
Scale
Benchmarking
Value for money
L&T Literature
‘early adopters’ mainstream

8. The EHU Student eLearning Survey
A snapshot in
time
An indicator of the extent that the VLE is integral
to the learning experience of EHU students
Longitudinal 4 years worth of data
Not just a VLE
survey
What other technologies do our students use (or
want to use) in their studies – and what would
they like to see more of? Where and how do
they study?
But is also a
VLE ‘health
check’
Frequency of use, importance to study, impact
on their learning, what works and what doesn’t

9. Survey Headlines
0
10
20
30
40
50
60
70
80
90
Never Rarely Sometimes A lot Never heard of this
09/10
10/11
11/12
How often do you use Learning Edge in your studies? Frequency

10. Survey Headlines
How often do you use Learning Edge in your studies? Importance
0
10
20
30
40
50
60
70
80
90
Very important Quite important Quite unimportant Unimportant
09/10
10/11
11/12

11. Survey Headlines
Learning Edge has enhanced the knowledge and understanding I get from
lectures, tutorials and practical sessions
0
10
20
30
40
50
60
70
80
90
08/09 09/10 10/11 11/12
Agree/strongly agree
neutral
disagree/strongly disagree

12. Survey Headlines
My tutors regularly update Learning Edge with course information and materials
0
10
20
30
40
50
60
70
80
90
09/10 10/11 11/12
Agree/strongly agree
neutral
Diasagree/ strongly disagree

13. Key themes from the 2011/12 survey
Mobile
technologies
55.8% participants own a smartphone (36.1% in
2010/11). Picking up notifications, grades, lecture
notes ‘on the go’ often in preference to bringing a
laptop
Importance of
Learning Edge
Used frequently, rated important, enhancing
learning, online submission, communication
2 sides of the
same coin
4 themes identified: access (technical), access
(content and resources), communication, design
and layout
Social networking
for informal
learning
Facebook widely used for informal learning –
Facebook ‘Hub’ and Learning Edge ‘classroom’ –
largely student owned and managed

14. • “This is the first time I have used eLearning and I
love it. It gives me the freedom to log on whenever
I have time and continue my study and it is fast
and effective. “
• “Consistent format across all modules means I
know where to look for information. “
• “If used properly it has the capability to improve
everyone’s learning experience – even just using
the basic features. “
Student quotes from the 4th EHU Student eLearning Survey, 2011/12

15. • “It can sometimes be slow. “
• “Layout of Blackboard modules can be
confusing, not always sure where to find
information and material. “
• “Learning Edge is massively underused and
handing in hard copies of work seems
pointless when the facility is available. “
Student quotes from the 4th EHU Student eLearning Survey, 2011/12

16. Using our survey findings …
You said … we
did activities
Identifying and rectifying touch points (build your
‘ideal’ VLE responses, needs and issues)
Influencing
academic
practice
Getting the message out that the student experience
of the VLE is largely determined by how it is set up
and managed by tutors – consistency, lecture notes,
communications
Raising
visibility and
awareness
Demonstrating that we understand the needs of our
key stakeholders. Disseminating through
deliberative structures and informal channels, audit
trail
Supporting
business
cases
Blackboard 9.1, Complex Managed Hosting, Mobile
Learn, Mobile Central, Assignment Handler,

17. Lessons learned …
Have a prize
draw!
No brainer!
Make it mean
something
Take any negative comments on the chin and do
something about them. It has to inform real and
visible improvement/enhancement
Communicate!
Communicate!
Communicate!
Before, during, after! Student email, Blackboard
announcements, Learning Edge blog, Facebook,
Student Union, Committees …
Learn from
mistakes
Allocate lots of time – it generates a lot of data.
Listen to students – they tell you if questions don’t
work. Time your survey release for maximum
uptake. Get some headlines out quickly afterwards

18. • Outcomes from Institutional audit and review, QAA,
http://www.qaa.ac.uk/ImprovingHigherEducation/Pages/
Outcomes.aspx
• Kyriaki Anagnostopoulou, University of Bath. Evaluating
the Role of Technology in Supporting Students. Follow
the Sun, 13-15 April 2011 http://vimeo.com/35129348
• EHU Student eLearning Survey Collection
http://www.eshare.edgehill.ac.uk/1348/
References

19. Contact
Lindsey Martin Lindsey.Martin@edgehill.ac.uk