SCORM Learner Assessment Generator (SLAG) A note about SLAG. While SLAG is a good way to hide the assessments and their answers within the <dataFromLMS> tag in the manifest file, it does not stop anyone from directly entering the score into the LMS via the use of bookmarklets. The question is “Do I need the answers to the test to set the score?” The answer depends on the way the exam is coded.
For high stakes assessments students should be proctered otherwise they can, fairly easily, find very non-technical ways to cheat the system. This is not the focus of the presentation, but it is worth mentioning because of 2 reasons (1) These are probabily the first ideas students think of when they are going to cheat and (2) these are the hardest to stop from a pureley techincal stand point.
I use the term hacker to describe anyone that tries to circumvent the code of the course in order to modify the flow of data. I use the term cheater to mean anyone who tries to misrepresent their knowledge on the subject matter of any given course. So a hack may or may not be used to cheat and a cheater, in most cases, has no reason to hack the system.
First Line of Defense Easy things to implement that will detour the majority of weekend hackers. These are easily implemented and easily circumvented by savvy programmers. In order to break through the first line of defense it may require the use of browser plugins and tools that most users are not aware of. Second Line of Defense These things are a bit more difficult to implement and will stop most savvy programmers that do not have a deep knowledge of the SCORM API and the LMS SCORM API Adapter. Third Line of Defense These are things that require significant time and skill to implement properly and will stop most attacks or at least take quite some time and testing for a hacker to circumvent. This defense comes down to “Is it worth it?” not “Can it be hacked?” Can these defenses be hacked, yes. Is it worth it? Probably not. Couple the Third Line of Defense with intrusion detection and prosecution policy and there is virtually no reason for anyone to take the risk.
With these things in mind there is a fairly small percentage of people that meet all of the criteria to even have the opportunity and need to hack a SCORM course.
These precautions will stop your weekend hackers. The hackers that are scanning code for answers or implementing some code they got off the internet of from a friend.
These items are a bit harder to implement, but are more effective against mid level hackers. These measures will also stop most bookmarklet attacks.
Most security in daily life focuses on Detection and NOT Prevention. Most people have an alarm system on their house and car and many have video cameras watching their valuables at all times. And these measures can cost upwards of $2,000 while the lock on the door cost 25.00 at Home Depot and nothing other than a rock is needed to break into most houses. The rock through the window trick is an all time favorite. This is clearly a focus on detecting the intruder and not preventing them from intruding. Legal deterrents and the risk of detection is what keeps most people safe most of the time. The same basic principle is in effect with Credit Cards. All someone needs to charge items to your credit card is, your credit card (or at least the information written on the outside of the card in plain view). The prevention is that there is a “special” number on the back and you may need to know the address of the owner of the credit card. Not much to prevent me from using the card without permission. However, the Detection is an entirely different story. Credit Card companies employ very sophisticated detection algorithms and hundreds of employees to watch for unauthorized or “suspicious” transactions. And lawmakers give the Credit Card companies the right to prosecute those that are detected. The deterrent for most people is “The amount of money that can be gained is not worth the possible loss of time that could be suffered in jail”. The moral of the story is for high stakes assessments you need to have instructors watching the students and code in the course and in the LMS looking to Detect cheating / hacking and you need to have the authority in place to prosecute the offenders. More on the subject can also be found at http://www.scorm.com/blog/2009/04/scorm-security-some-perspective/
Security Issues In SCORM
The "Recent" History• Dr. Bill Blackmon introduces SLAG andAssessment Security (Aug. 23rd, 2006)• Claude Ostyn addresses Assessment Security in"In the Eye of the SCORM" (Last Revised March2007)• Philip Hutchison posted WORKING bookmarkleton his blog PiPwerks.com (April 2nd, 2009)• SCORM Community reacts– Mike Rustici (scorm.com) responds (April 3rd, 2009)– Tom King (mobilemind.net) responds (April 5th, 2009)• ADL responds (May 31st, 2009)• ADL Implementation Fest Developers’ Workshop :Security Issues in SCORM
Physical Security Issues• Open Book / Open System– Search the web– Search a book• Student Validation– Who is taking the test?• Plain Old Cheating– Taking screenshots of the exams– Paying / bartering for the answer keys• The list goes on….Cheaters Should Be Proctored
Technical Security Issues• Who will be trying to hack the course or assessment?– Students that do not know the material well enough to pass the course– Students that want to see if they can beat the system.– Developers that are testing vulnerabilities.• What will the hacker be trying to accomplish?– Modify the score, status, session time, interactions, objectives …• Most students will only think to modify score and status.• What is the benefit and risk to the hacker?– Is the hacker confident that the solution they have (created or found onthe Web) will work on the system they intend to hack? And if it fails, orif they are detected, what is the risk to them personally?• Prevention vs. Detection– Is the goal of the implemented security measure to prevent hacking ordetect hacking once it has occured?Framing the Issues
• First Line of Defense– Easy to implement. Stop most weekend hackers.– Easily circumvented by savvy hackers• Second Line of Defense– A bit more difficult to implement– Stops programmers that do not have knowledge of SCORM• Third Line of Defense– Take significant time and skill to implement– This defense comes down to “Is it worth it to hack?” not “Can it behacked?”– Coupled with Detection and Prosecution there is virtually no reasonfor someone to take the risk.• Fourth Line of defense– Proctor the exam!– If the stakes are that high that someone will try to even break the 3rdline of defense, then there may be a need to have an instructorpresent.PreventionStop Them in Their Tracks
Lines of Defense• Must be logged into the LMS as a student and be able toaccess the course to be hacked.– Remind students not to share passwords and make a strong dis-incentive for doing so.• That means if you can detect an attempted hack, you can locate theaccount and person responsible.• Must have the knowledge to perform the hack.– Gained the knowledge from their own knowledge of programming• Highest risk to the course– Gained the knowledge from another hacker (e.g., the Web, word ofmouth)• Medium risk to the course• Must have the willingness / incentive to perform the hack– In most cases, if it is easier to pass the course than implement a hack,The Basics
Lines of Defense• Use FLASH– FLASH is compiled into .swf files, so it is harder to see the code fromwithin the browser• FLASH can be decomplied, so obfuscating the code is also an option• Use more than score and status to determine grade– Make sure that the course uses objectives, interactions and sessiontime, along with score and status, to calculate the overall grade.• The "Fake" API– Add an API object local to your course, so hackers will find that API andset values there instead of the real API.• Set and Terminate– Set all important data at the LAST possible moment before Terminate()is called.Second Line of Defense
Lines of Defense• Secret SCO– Create a manifest file with a hidden SCO that must be completed in order forthe course to be passed. Have the assessment SCO set an objective on thehidden SCO equal to the score on the quiz SCO.• Hide and Go Seek– Have the assessment SCO set an encrypted cookie or Flash Shared Object andthen the last SCO of the course looks at this cookie to determine the finalscore. The final SCO will not set any score that does not match the score in thecookie.• Rollup With a Twist– Have a combinaton of random objective scores that must be set, and add up tothe proper total, in order to mark the course completed.• SCO1 : Objective 1 score = .10• SCO2 : Objective 1 score = .17• SCO3 : Objective 1 score = .28• If the total for objectives = .55 then course is passed.Third Line of Defense
Lines of Defense• Don’t use SCORM for assessments– Use a 3rdparty server side assessment service.• Proctor The Exam!– This could also be your first and only Line of Defense as well,depending on the stakes involved.Fourth Line of Defense
Detection• On the LMS Side– Session Time Comparison• Compare the session time to the actual time the student spent in the SCO.– Compare results to known averages• Compare student "X" SCORM data with the average students’ SCORM data.– Make sure that there is not too much or too little data reported and that it fitswithin the "normal" parameters for that course.• On the SCO Side– The "Fake" API• Using the fake API, you can detect when a hack is being attempted on thefake API, which will be the first one that the would-be hacker will locate.• Once a hack is detected, the SCO should write some data to the LMS tosignal a breach has occured.– Set an objective, set an interaction, set suspend_data. You would want to set adata model element that can be reported on by the LMS.Trespassers will be prosecuted!
Examples• First Line of Defense– Disable Right Click– Open new window and disable F11• Second Line of Defense– Fake the API– Set and Terminate• Third Line of Defense– Rollup With a Twist"Show Me"
Conclusion• High stakes assessments should be proctored• SCORM may not be a good technology choice for non-proctored high stakes exams.• There are ways to make courses more difficult to hack, butthis does not prevent physical types of cheating.• In the end, any type of technical secutiry measure should beseen as a deturent and not as the ultimate solution. And anysecurity measure implementation should ultimately beweighed against the effect the security measure has onusability.
Resources• SCORM Security – Some Perspective– http://www.scorm.com/blog/2009/04/scorm-security-some-perspective/• PiPwerks Blog– http://pipwerks.com/journal/2009/04/02/scorm-security-two-kinds-of-scorm-people/• ADL’s Answer– http://www.adlnet.org/Technologies/scorm/Lists/Announcements/DispForm2.aspx?List=025c59e6-41d3-4496-b4cb-7a3033dea50d&ID=6• SCORM Vulnerabilities + IMS Spec withdrawal = Excitement– http://mobilemind.net/2009/04/scorm-vulnerabilities-ims-spec.html• In the Eye of The SCORM : Assessment Security– http://www.ostyn.com/standards/docs/Eye_Of_The_SCORM_draft.pdfFurther Research
SCORM Security• Instructor Lead On-Line Training Seminar• One-Day of lecture and labs• Discusses security risks and pros and cons of the solutions• Learn how the hackers think– For every security solution demonstrated you will learn thevulerabilityes so you can assess the risk for yourself.Hands-On TrainingJCA Solutions Hands-On Guide to SCORM Securityhttp://www.jcasolutions.com/training.php