Methods to test an e-learning Web application.

Methods to test an e-learning Web application. [email_address]

Problematic and Motivations

LMS are rapidly increasing its complexity

Learning standards

Security policies

Suggestions of users

Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Problematic and Motivations

The real problem of an Open Source LMS

It is a hard task to assure the stability because

Daily changes (new features)

Hundred of different developers

Different thinking

Working separately

Conformance testing Model Implementation in Java Implementation in C++ Conforms? standard ISO 9646 Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Regression testing File a.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Execute conformance tests File c.tcl (version 0.2) Feature 1 Feature 2 Feature 3 Feature 4 Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Regression testing File a.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Feature 1 Feature 2 Feature 3 Feature 4 Execute conformance tests File c.tcl (version 0.2) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Feature 5 ?

Regression testing Feature 1 Feature 2 Feature 3 Feature 4 Version 0.1 Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Version 0.2 ? Regression bugs detected in Features 2 and 3 Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Automatic Model Based Testing Verdict System ? Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Automatic Model Based Testing Automatic testing Verdict System Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Automatic Model Based Testing Automatic testing Verdict Test cases System ?? Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Automatic Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Automatic Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System ???? Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Model

Model

An abstraction of the real thing

We abstract away all the irrelevant details

We abstract away all the confusing details

We make a Simplification of a real system

Visibility of a system to be understood evaluated and criticized

Model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Finit States Machine

Ready to model? Prolearn Summer school 09 Slovakia 2 June 2009 F: (usr,pass) L:register

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Our Login Page

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 http://mymoodleinstance.com http://mymoodleinstance.com/signup http://mymoodleinstance.com/home

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Task 1 Build a model of…

Jtel Summer school 09 Terchova, Slovakia 2 June 2009 User Password OK Signout Welcome Harry Potter My Frequently asked Questions My Faq 1 Q: Where is Terchova? A: In Slovakia 2 Q Where is Gerardo? A: In Terchova Add an Q&A Signout Howards School LMS Add an Q&A Question: Answer: OK Signout

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3

Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3 q1 q2 q3 q5 q7 q8

PRESENTATION OF METHODOLOGIES Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Simulator of the model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g

Simulation of the model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g Trace 1: [s0, x, y,s1] 2: [s1, w,y,s5] Inputs x,w Expected outputs y,y Expected final state s5

Output of the test generator Jtel Summer school 09 Terchova, Slovakia 2 June 2009 ?login(user1, password1) / ! Text (Welcome) ?frequently_asked_questions() / ! Text (My Faq) . . . . ?login(user1,user2) Twt:: form find « login » Twt:: field fill « user1 » Twt:: fiend find « password » Twt:: field fill « password1 » Twt:: form submit Translation to tclwebtest

Conformance test case (Automatic Model Based testing) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 System Specs Formal Model (Functional +Security) Abstract Test Suites Test Scripts Test Execution Trace Report Editor Test Generator Test Transformer Executor Analyzer 1. Modeling 2. Generation 3. Translation 5. Analysis 4. Execution Test Purposes

USE CASE Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Test Preamble Test

Conformance test case Jtel Summer school 09 Terchova, Slovakia 2 June 2009

In the example of FAQ edition

If the page we got after the action is the corret one

Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009

In the example of FAQ edition

If the page we where redirect is the correct one

If the new name of the faq is founded in the faq page

Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Acs-automated testing Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Conformance test case