• Like


Uploaded on



Kim Comer, University of Manchester
Paul Coulthard, University of Manchester

Since moving to Bb9.1, the University of Manchester has decided to carry out one major Blackboard service pack upgrade per calendar year. As part of this decision, the University has agreed to use the Easter vacation period for the upgrade. Until the recent announcement of changes to the delivery of the service packs, Manchester was operating on the principle of applying the latest odd-numbered service pack, usually released in December, during the spring upgrade window. This presentation will provide some background on the rationale behind the institution’s decisions to upgrade over Easter and apply odd-numbered service packs. The presentation will also describe the testing approach that has been adopted by Manchester. The testing is divided into three separate areas: technical, functional and integration. The technical testing is carried out by the central eLearning Applications Team (i.e. system administration) working closely with Blackboard Managed Hosting; the functional testing is managed by our Functional Evaluation and Testing group (FEAT), which consists of eLearning Technologists from across the institution; and the integration testing is handled by our Web Services Team. The presentation will focus mainly on the technical and functional testing and provide examples of the test plans used at Manchester. The talk will also address some of the dilemmas that come with testing:

What to test?
How much can you test?
Regression or black-box testing?
How long should testing take?
Who should carry out the work?

The session will end with a discussion of how Manchester will have to respond to the changes in the release cycle of services packs and in light of the institution’s move to the new Blackboard Enterprise SIS framework.

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Throughout the testing FEAT are always in communication with eLAT about an issues; eLAT can then escalate immediately to MH for resolution.


  • 1. ICEBREAKER Blackboard Quiz!
  • 2. Service Pack Upgrades: The Testing Dilemma PAUL COULTHARD, Senior VLE Analyst KIM COMER, Teaching & Learning Adviser University of Manchester
  • 3. ABOUT US PAUL COULTHARD Senior VLE Analyst paul.r.coulthard@manchester.ac.uk KIM COMER Teaching & Learning Adviser kim.comer@manchester.ac.uk
  • 5. ABOUT MANCHESTER Map of UK & Westeros
  • 6. ABOUT MANCHESTER • One campus located in Manchester city centre • 38,000 students (27,000 undergraduates and 11,000 PGT/PGR) • 11,000 staff (including 6,000 academic and research staff)
  • 7. ABOUT MANCHESTER eLAT/TLSO Faculty of Humanities Faculty of Medical and Human Sciences Faculty of Engineering and Physical Sciences Faculty of Life Sciences
  • 8. ABOUT MANCHESTER • Four Faculty eLearning Teams (approx. 80 staff) • eLearning Applications Team (6 staff) • Web Services (2 staff) • Change Advisory Board (CAB) • Associate Vice-President for Teaching, Learning & Students (Prof Richard Reece) • Online Education Strategy Group • Teaching & Learning Support Office (Kim Comer)
  • 9. ABOUT MANCHESTER • Upgraded to Service Pack 14 over 10th – 13th April 2014 • 87 Building Blocks (65 available) • Managed Hosting Clients • Three servers: Test& Development, Staging and Production environments • 10 nodes and 1 database • 30-minute incremental update and nightly full snapshot • Shibboleth and CAS • Campus Solutions • Blackboard Mobile Learn and Central
  • 10. WHAT WE ARE GOING TO LEARN TODAY • To share good practice and lessons learned; • To initiate a dialogue with other institutions about their approaches and experiences; • To share our experiences of working with Blackboard Managed Hosting as part of the upgrade process.
  • 11. OUR RATIONALE • Two dedicated maintenance windows per year: 1) Easter vacation period and 2) beginning of summer • One system upgrade per year carried out over Easter • Service pack available in December • Upgrade date agreed one year before • Communications begin summer before • Planning begins in November
  • 12. SPRING UPGRADE Advantages Disadvantages Activity levels on Blackboard are lower during the Easter vacation period relative to usage before and after. Availability of staff required to carry out testing. The Service Pack upgrade introduces fixes that users may benefit from in Semester 2. Shorter period of time to implement upgrade. A Spring upgrade will give users more time to adjust to any system changes and extend the User Acceptance Testing period. Two separate upgrade windows helps to spread out the workload, e.g. planning, evaluation and testing.
  • 13. SUMMER UPGRADE Advantages Disadvantages The upgrade would take place after the summer exam period, meaning that online exams would remain unchanged from the January exams. Availability of staff required to carry out testing. Blackboard activity levels are lower over the summer than during the standard academic year. Users will not benefit from any fixes or enhancements until the next academic year. The User Acceptance Testing period will be very short as academics do not prepare their courses over the summer.
  • 14. OUR APPROACH Three types of testing 1. Technical – eLearning Applications Team 2. Functional – Faculty eLearning Teams (FEAT) 3. Integration – Web Services
  • 15. OUR APPROACH Update test plans and evaluate service pack T&D server Conduct integrity check of copy of production to staging. Test staging server (double run through) Test production server (single run through) Dissemination to: Comms group KB group (report) eLTs (presentation) Dec-Feb Feb-March March April April T&D Staging Staging Production
  • 16. OUR APPROACH Technical testing • System administration testing of eLearning applications • Regression, comparative, versioning, etc. • Partnership with Blackboard Managed Hosting
  • 19. OUR APPROACH Functional testing • Functional, Evaluation and Testing (FEAT) group • Functional evaluation and testing of changes to tools and features in eLearning applications, including fixes and enhancements. • 12 testers
  • 20. OUR APPROACH Functional testing • Testing timetable • Testing plan • Final report
  • 21. OUR APPROACH Integration testing • Web Services - integration between eLearning applications and other University systems (e.g. Campus Solutions). • Feed file, snapshot, new courses, role assignments • Course rollover • Moving to new SIS Framework
  • 22. OUR DILEMMA • What to test? • How much can you test? How long should it take? • Regression or black-box testing? • Who should carry out the work? • How do you manage expectations?
  • 23. YOUR EXPERIENCES We would like to hear about your testing approaches and experiences.
  • 24. YOUR EXPERIENCES • How often do you upgrade? • When do you upgrade? • Who carries out your testing? • What approach do you take?
  • 25. WHAT NEXT? • How can we share best practice? Test plans? • Experiences with upgrades, e.g. issues, enhancements, etc.?
  • 26. QUIZ ANSWERS • In what Service Pack was negative marking introduced? Service Pack 8 • How many language packs are there? 18 • Where is Blackboard’s headquarters based? Washington, DC • What does xpLor stand for? Cross-platform Learning Object Repository
  • 27. QUIZ ANSWERS • What are the standard course and organisation roles? Course Builder, Grader, Instructor, Student and Teaching Assistant • In SP14, the Retention Centre does not work in which browsers? IE9/10 and Chrome 33
  • 28. QUIZ ANSWERS • Can you identify these Blackboard employees? Jay Bhatt, President and CEO, Blackboard Matt Small, Senior VP and Managing Director, International, Bl ackboard
  • 29. THANK YOU! PAUL COULTHARD paul.r.coulthard@manchester.ac.uk KIM COMER kim.comer@manchester.ac.uk