Methods to test an e-learning Web application. [email_address]
Problematic and Motivations LMS are rapidly increasing its complexity Learning standards Security policies Suggestions of users Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Problematic and Motivations The real problem of an Open Source LMS It is a hard task to assure the stability because Daily changes (new features) Hundred of different developers Different thinking Working separately  Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Conformance testing Model Implementation in Java Implementation in C++ Conforms? standard ISO 9646  Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Regression testing File a.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Execute conformance tests File c.tcl (version 0.2) Feature 1 Feature 2 Feature 3 Feature 4 Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Regression testing File a.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Feature 1 Feature 2 Feature 3 Feature 4 Execute conformance tests File c.tcl (version 0.2) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Feature 5 ?
Regression testing Feature 1 Feature 2 Feature 3 Feature 4 Version 0.1 Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Version 0.2 ? Regression bugs detected in  Features 2 and 3 Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Automatic Model Based  Testing Verdict System ? Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Automatic  Model Based  Testing Automatic testing Verdict System Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Automatic  Model Based  Testing Automatic testing Verdict Test cases System ?? Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Automatic   Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Automatic   Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System ???? Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Model
Model An abstraction of the real thing We abstract away all the irrelevant details We abstract away all the confusing details We make a  Simplification  of a real system Visibility of a system to be understood evaluated and criticized Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Finit States Machine
Ready to model? Prolearn Summer school 09 Slovakia 2 June 2009 F: (usr,pass) L:register
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Our Login Page
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 http://mymoodleinstance.com http://mymoodleinstance.com/signup http://mymoodleinstance.com/home
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Task 1 Build a model of…
Jtel Summer school 09 Terchova, Slovakia 2 June 2009 User Password OK Signout Welcome Harry Potter My Frequently asked Questions My Faq 1 Q: Where is Terchova? A: In Slovakia 2 Q Where is Gerardo? A: In Terchova Add an Q&A   Signout Howards School LMS Add an Q&A Question: Answer: OK Signout
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3
Signup model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3 q1 q2 q3 q5 q7 q8
PRESENTATION OF METHODOLOGIES Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Simulator of the model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g
Simulation of the model Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g Trace 1: [s0, x, y,s1] 2: [s1, w,y,s5] Inputs x,w Expected outputs y,y Expected final state s5
Output of the test generator Jtel Summer school 09 Terchova, Slovakia 2 June 2009 ?login(user1, password1) / ! Text (Welcome) ?frequently_asked_questions() / ! Text (My Faq) . . . . ?login(user1,user2) Twt:: form find « login » Twt:: field fill « user1 » Twt:: fiend find « password » Twt:: field fill « password1 » Twt:: form submit Translation to  tclwebtest
Conformance test case (Automatic Model Based testing) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 System Specs Formal Model (Functional +Security) Abstract Test Suites Test Scripts Test Execution  Trace Report Editor Test Generator Test Transformer Executor Analyzer 1. Modeling 2. Generation 3. Translation 5. Analysis 4. Execution Test Purposes
USE CASE Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Test Preamble   Test Conformance test case Jtel Summer school 09 Terchova, Slovakia 2 June 2009
In the example of FAQ edition If the page we got after the action is the corret one Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error  "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009
In the example of FAQ edition If the page we where redirect is the correct one If the  new   name of the faq  is founded in the faq page Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error  "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Acs-automated testing Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Conformance test case Extract the semantic of the LMS Generating a static trace while interacting with the Web application  Jtel Summer school 09 Terchova, Slovakia 2 June 2009
Conformance test case Extract the semantic of dotLRN Generating a static trace while interacting with the Web application  Jtel Summer school 09 Terchova, Slovakia 2 June 2009

Methods to test an e-learning Web application.

  • 1.
    Methods to testan e-learning Web application. [email_address]
  • 2.
    Problematic and MotivationsLMS are rapidly increasing its complexity Learning standards Security policies Suggestions of users Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 3.
    Problematic and MotivationsThe real problem of an Open Source LMS It is a hard task to assure the stability because Daily changes (new features) Hundred of different developers Different thinking Working separately Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 4.
    Conformance testing ModelImplementation in Java Implementation in C++ Conforms? standard ISO 9646 Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 5.
    Regression testing Filea.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Execute conformance tests File c.tcl (version 0.2) Feature 1 Feature 2 Feature 3 Feature 4 Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 6.
    Regression testing Filea.tcl (version 0.1) File b.tcl (version 0.1) File c.tcl (version 0.1) File d.tcl (version 0.1) Feature 1 Feature 2 Feature 3 Feature 4 Execute conformance tests File c.tcl (version 0.2) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Feature 5 ?
  • 7.
    Regression testing Feature1 Feature 2 Feature 3 Feature 4 Version 0.1 Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 Version 0.2 ? Regression bugs detected in Features 2 and 3 Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 8.
    Automatic Model Based Testing Verdict System ? Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 9.
    Automatic ModelBased Testing Automatic testing Verdict System Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 10.
    Automatic ModelBased Testing Automatic testing Verdict Test cases System ?? Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 11.
    Automatic Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 12.
    Automatic Model Based Testing Automatic testing Verdict Test cases FORMAL Model of system System ???? Jtel Summer school 09 Terchova, Slovakia 2 June 2009 Model
  • 13.
    Model An abstractionof the real thing We abstract away all the irrelevant details We abstract away all the confusing details We make a Simplification of a real system Visibility of a system to be understood evaluated and criticized Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 14.
    Model Jtel Summerschool 09 Terchova, Slovakia 2 June 2009 Finit States Machine
  • 15.
    Ready to model?Prolearn Summer school 09 Slovakia 2 June 2009 F: (usr,pass) L:register
  • 16.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009 Our Login Page
  • 17.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009
  • 18.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009 http://mymoodleinstance.com http://mymoodleinstance.com/signup http://mymoodleinstance.com/home
  • 19.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009 Task 1 Build a model of…
  • 20.
    Jtel Summer school09 Terchova, Slovakia 2 June 2009 User Password OK Signout Welcome Harry Potter My Frequently asked Questions My Faq 1 Q: Where is Terchova? A: In Slovakia 2 Q Where is Gerardo? A: In Terchova Add an Q&A Signout Howards School LMS Add an Q&A Question: Answer: OK Signout
  • 21.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3
  • 22.
    Signup model JtelSummer school 09 Terchova, Slovakia 2 June 2009 q1 q1 q2 q3 q1 q2 q3 q5 q7 q8
  • 23.
    PRESENTATION OF METHODOLOGIESJtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 24.
    Simulator of themodel Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g
  • 25.
    Simulation of themodel Jtel Summer school 09 Terchova, Slovakia 2 June 2009 s0 s1 s2 s3 s4 s0 s5 s6 s7 TP: Action=input w State= s5 ck1= d x/y a/b c/d a/d d/e w/y a/b c/d f/g Trace 1: [s0, x, y,s1] 2: [s1, w,y,s5] Inputs x,w Expected outputs y,y Expected final state s5
  • 26.
    Output of thetest generator Jtel Summer school 09 Terchova, Slovakia 2 June 2009 ?login(user1, password1) / ! Text (Welcome) ?frequently_asked_questions() / ! Text (My Faq) . . . . ?login(user1,user2) Twt:: form find « login » Twt:: field fill « user1 » Twt:: fiend find « password » Twt:: field fill « password1 » Twt:: form submit Translation to tclwebtest
  • 27.
    Conformance test case(Automatic Model Based testing) Jtel Summer school 09 Terchova, Slovakia 2 June 2009 System Specs Formal Model (Functional +Security) Abstract Test Suites Test Scripts Test Execution Trace Report Editor Test Generator Test Transformer Executor Analyzer 1. Modeling 2. Generation 3. Translation 5. Analysis 4. Execution Test Purposes
  • 28.
    USE CASE JtelSummer school 09 Terchova, Slovakia 2 June 2009
  • 29.
    Test Preamble Test Conformance test case Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 30.
    In the exampleof FAQ edition If the page we got after the action is the corret one Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 31.
    In the exampleof FAQ edition If the page we where redirect is the correct one If the new name of the faq is founded in the faq page Verdict assignation if {[string match "*admin/one-faq*" $response_url] } { if { [catch {tclwebtest::form find ~n "faq_add_edit"} errmsg] || [catch\\ {tclwebtest::field find ~v "$faq_new_name"} errmsg] } { aa_error "faq::twt::edit_one failed $errmsg : Dont't Edit a Faq" } else { aa_log "Faq Edited“} Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 32.
    Acs-automated testing JtelSummer school 09 Terchova, Slovakia 2 June 2009
  • 33.
    Conformance test caseExtract the semantic of the LMS Generating a static trace while interacting with the Web application Jtel Summer school 09 Terchova, Slovakia 2 June 2009
  • 34.
    Conformance test caseExtract the semantic of dotLRN Generating a static trace while interacting with the Web application Jtel Summer school 09 Terchova, Slovakia 2 June 2009