Ho Chi Minh City University of Technology, Vietnam
Faculty of Computer Science and Engineering
Lab for Systems Analysis an...
Outline
1. Student Program Correctness Problem
2. Our approach
3. Framework
4. How it works
5. Case study
6. Experiment
7....
Verification and Group Testing Techniques

1. Student Program Correctness Problem
• Traditional method: read all programs t...
Verification and Group Testing Techniques

2. Our approach
• Static methods & Automation

Mai H. Dinh | SAVE | Ho Chi Minh ...
Verification and Group Testing Techniques

3. Framework

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vi...
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

6/24
Verification and Group Testing Techniques

• Exercise list

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology,...
Verification and Group Testing Techniques

• Bubble sort problem

Mai H. Dinh | SAVE | Ho Chi Minh City University of Techn...
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

9/24
Verification and Group Testing Techniques

4. How it works
• Component point of view
• System point of view

Mai H. Dinh | ...
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

11/24
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

12/24
Verification and Group Testing Techniques

5. Case study

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, V...
Verification and Group Testing Techniques
• Bubble sort algorithm with the 10th line is logically wrong
• The 10th line bel...
Verification and Group Testing Techniques
• The testing matrix for above program has 1343 rows and 51 columns
• Loop struct...
Verification and Group Testing Techniques

• The error block is highlighted with red color
⇒ programmer can easily know whe...
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

17/24
Verification and Group Testing Techniques

6. Experiment
• 6 well-known algorithms, with some different implementations
• Ea...
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

19/24
Verification and Group Testing Techniques

Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam

20/24
Verification and Group Testing Techniques

• The result of our experiment is shown in the table and figure below
• The figure...
Verification and Group Testing Techniques
• Successfully 19 times in total 22 implementations
• 3 remained implementations ...
Verification and Group Testing Techniques

7. Issues & Future works
• Path explosion
• Missing code

Mai H. Dinh | SAVE | H...
Thanks for your listenning.
Question?
Upcoming SlideShare
Loading in …5
×

Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques

278 views

Published on

Published in: Education, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
278
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques

  1. 1. Ho Chi Minh City University of Technology, Vietnam Faculty of Computer Science and Engineering Lab for Systems Analysis and VErification (SAVE) Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques Long H. Pham, Nam P. Mai, Mai H. Dinh, Tho T. Quan, Hung Q. Ngo November 5, 2013
  2. 2. Outline 1. Student Program Correctness Problem 2. Our approach 3. Framework 4. How it works 5. Case study 6. Experiment 7. Issues & Future works
  3. 3. Verification and Group Testing Techniques 1. Student Program Correctness Problem • Traditional method: read all programs to verify ⇒ 999 999 999 LOC ? • Automated assessment system: execute, pass all test cases ⇒ test suite cover all possible errors ? ⇒ real execution is potentially dangerous for the system ? Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 3/24
  4. 4. Verification and Group Testing Techniques 2. Our approach • Static methods & Automation Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 4/24
  5. 5. Verification and Group Testing Techniques 3. Framework Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 5/24
  6. 6. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 6/24
  7. 7. Verification and Group Testing Techniques • Exercise list Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 7/24
  8. 8. Verification and Group Testing Techniques • Bubble sort problem Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 8/24
  9. 9. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 9/24
  10. 10. Verification and Group Testing Techniques 4. How it works • Component point of view • System point of view Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 10/24
  11. 11. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 11/24
  12. 12. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 12/24
  13. 13. Verification and Group Testing Techniques 5. Case study Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 13/24
  14. 14. Verification and Group Testing Techniques • Bubble sort algorithm with the 10th line is logically wrong • The 10th line belongs to block S5 ⇒ S5 is expected as the error block. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 14/24
  15. 15. Verification and Group Testing Techniques • The testing matrix for above program has 1343 rows and 51 columns • Loop structure ⇒ compact testing matrix to 11 rows and 9 columns • The test outcome after running test cases is {0 1 0 1 1 1 1 1 1 1 0} Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 15/24
  16. 16. Verification and Group Testing Techniques • The error block is highlighted with red color ⇒ programmer can easily know where the problem is and fix it without too much effort. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 16/24
  17. 17. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 17/24
  18. 18. Verification and Group Testing Techniques 6. Experiment • 6 well-known algorithms, with some different implementations • Each implementation has exactly one bug ⇒ test whether the system can localize these bugs or not Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 18/24
  19. 19. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 19/24
  20. 20. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 20/24
  21. 21. Verification and Group Testing Techniques • The result of our experiment is shown in the table and figure below • The figure shows the chart comparing right localizations with wrong localizations in each algorithm. • The detailed numbers of right/wrong localizations are in the table. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 21/24
  22. 22. Verification and Group Testing Techniques • Successfully 19 times in total 22 implementations • 3 remained implementations are not localized successfully because the generated test suite is not good enough Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 22/24
  23. 23. Verification and Group Testing Techniques 7. Issues & Future works • Path explosion • Missing code Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 23/24
  24. 24. Thanks for your listenning. Question?

×