Your SlideShare is downloading. ×
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques

103

Published on

Published in: Education, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
103
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Ho Chi Minh City University of Technology, Vietnam Faculty of Computer Science and Engineering Lab for Systems Analysis and VErification (SAVE) Assisting Students in Finding Their Own Bugs in Programming Exercises using Verification and Group Testing Techniques Long H. Pham, Nam P. Mai, Mai H. Dinh, Tho T. Quan, Hung Q. Ngo November 5, 2013
  • 2. Outline 1. Student Program Correctness Problem 2. Our approach 3. Framework 4. How it works 5. Case study 6. Experiment 7. Issues & Future works
  • 3. Verification and Group Testing Techniques 1. Student Program Correctness Problem • Traditional method: read all programs to verify ⇒ 999 999 999 LOC ? • Automated assessment system: execute, pass all test cases ⇒ test suite cover all possible errors ? ⇒ real execution is potentially dangerous for the system ? Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 3/24
  • 4. Verification and Group Testing Techniques 2. Our approach • Static methods & Automation Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 4/24
  • 5. Verification and Group Testing Techniques 3. Framework Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 5/24
  • 6. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 6/24
  • 7. Verification and Group Testing Techniques • Exercise list Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 7/24
  • 8. Verification and Group Testing Techniques • Bubble sort problem Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 8/24
  • 9. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 9/24
  • 10. Verification and Group Testing Techniques 4. How it works • Component point of view • System point of view Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 10/24
  • 11. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 11/24
  • 12. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 12/24
  • 13. Verification and Group Testing Techniques 5. Case study Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 13/24
  • 14. Verification and Group Testing Techniques • Bubble sort algorithm with the 10th line is logically wrong • The 10th line belongs to block S5 ⇒ S5 is expected as the error block. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 14/24
  • 15. Verification and Group Testing Techniques • The testing matrix for above program has 1343 rows and 51 columns • Loop structure ⇒ compact testing matrix to 11 rows and 9 columns • The test outcome after running test cases is {0 1 0 1 1 1 1 1 1 1 0} Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 15/24
  • 16. Verification and Group Testing Techniques • The error block is highlighted with red color ⇒ programmer can easily know where the problem is and fix it without too much effort. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 16/24
  • 17. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 17/24
  • 18. Verification and Group Testing Techniques 6. Experiment • 6 well-known algorithms, with some different implementations • Each implementation has exactly one bug ⇒ test whether the system can localize these bugs or not Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 18/24
  • 19. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 19/24
  • 20. Verification and Group Testing Techniques Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 20/24
  • 21. Verification and Group Testing Techniques • The result of our experiment is shown in the table and figure below • The figure shows the chart comparing right localizations with wrong localizations in each algorithm. • The detailed numbers of right/wrong localizations are in the table. Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 21/24
  • 22. Verification and Group Testing Techniques • Successfully 19 times in total 22 implementations • 3 remained implementations are not localized successfully because the generated test suite is not good enough Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 22/24
  • 23. Verification and Group Testing Techniques 7. Issues & Future works • Path explosion • Missing code Mai H. Dinh | SAVE | Ho Chi Minh City University of Technology, Vietnam 23/24
  • 24. Thanks for your listenning. Question?

×