SlideShare a Scribd company logo
Auto-Grading Parallel Programs
Building plugins for WebCAT
Team Lead: Max Grossman
Team Member: Maha Aziz, Anant Tibrewal, Anna Chi
● Background & Motivation
● Related Work
● What is WebCAT?
● Workflow
● Feature table
● Our Progress
● Future Plans
Overview
● Current grading method & Shorts
● A tool for testing and analyzing parallel programs automatically
○ Transparency of the grading process
○ Speed up and simplify the grading process
○ Focus on application performance
○ Possibility of parallel programming MOOC to be created
Background & Motivation
● Mooshak
○ Intended for hosting coding competitions online
○ Simple configuration, minimal requirements (apache, linux,
tcl)
○ Not very pretty UI
● Marmoset
○ Not very extensive Java support
Related Work
What is WebCAT?
What is WebCAT?
from May.11th till now
● Research about multiple autograding systems
Related Work
● Target on WebCat
WebCAT
● Install and set up
● Hackathon plug-in structure
● Feature table
Workflow
Description Priority (1-5, 1
being the highest)
Viability (1-5, 1
being the highest)
Workload (1-5, 1
being the highest)
Assignee
Code styling/checkstyle grading 2 1 2 Anant
Local grading of HJ-lib programs
running in a single thread for
correctness
1 1 1 Anant
Remote grading of HJ-lib
programs running on multiple
cores in a compute cluster
2 3 1 Anna
needs to commit student file to
svn before/after running the
autograder
1 1 5 Anna
Feature Table
Description Priority (1-5, 1
being the highest)
Viability (1-5, 1
being the highest)
Workload (1-5, 1
being the highest)
Assignee
give a way for student provided
tests to be graded with feedback
3/4 once we have
instructor-provided, this
is a 2
3
instructor provided tests with
limited feedback
1 1 1
Ranking of student submissions
based on performance, make it
visible to students to turn every
homework into a competition
(with names anonymized?)
4 4 2 Maha
Statistics on how many students
are passing a given test, to give
feedback on the relative
difficulty of different tests
(maybe part of the
leaderboard?)
3 4 2
Feature Table
Description Priority (1-5, 1
being the highest)
Viability (1-5, 1
being the highest)
Workload (1-5, 1
being the highest)
Assignee
Student peer reviews 5 5 1
Different types of static code
analysis tools
2 once we have
checkstyle, this should
be easier (3)
2
NetID authentication 1 3 ? Maha
Feature Table
Architecture Students
WebCAT
STIC Leaderboard
Homework
Submission
Performance Test
Results
Student View of
Leaderboard
POST Request for
New Submissions
SVN
● Anant
● Maha
● Anna
Our Progress
Architecture Students
WebCAT
STIC Leaderboard
Homework
Submission
Performance Test
Results
Student View of
Leaderboard
POST Request for
New Submissions
SVN
● Anant
○ Local grading of HJ-lib programs running in a single thread for
correctness
○ Built HJ program with JUnit tests to test WebCAT
■ correctness, performance tests
○ Incorporated the FindBugs static analysis tool to WebCAT
■ tool to detect bugs in code
Our Progress
Architecture Students
WebCAT
STIC Leaderboard
Homework
Submission
Performance Test
Results
Student View of
Leaderboard
POST Request for
New Submissions
SVN
● Maha
○ “Leaderboard”
○ created a database which contains all student submissions from
the svn
■ includes test names, execution times, core count, etc.
○ database is displayed to webpage for students to see a ranking
of how well their classmates are doing
■ all information on webpage is anonymous
Our Progress
Leaderboard
Architecture Students
WebCAT
STIC Leaderboard
Homework
Submission
Performance Test
Results
Student View of
Leaderboard
POST Request for
New Submissions
SVN
student-runs
Handled backup of students files using SVN
student ID
before
grading
num of
submission
after grading
student ‘s
code
Results
Remote grading on multiple cores in cluster
WebCAT
STIC
(cluster)
Build folder structure
Run tests
Student code &
Performance test
Send performance
results back
Through admin interface
Slurm personal configuration
DEMO
● Feature table
● Provide documentations for students and instructors
● EduHPC
Future Plans
● Current inconvenient grading process
● Research about autograders
● Good points and properties of WebCAT
● Feature Table
● Our progress
● Next steps
Summary

More Related Content

What's hot

Xp conf-tbd
Xp conf-tbdXp conf-tbd
Xp conf-tbd
XP Conference India
 
Cqrs
CqrsCqrs
MBLTDev15: Brigit Lyons, Soundcloud
MBLTDev15: Brigit Lyons, SoundcloudMBLTDev15: Brigit Lyons, Soundcloud
MBLTDev15: Brigit Lyons, Soundcloud
e-Legion
 
A Report on Web Application Framework Benchmarks and Perl's Performance
A Report on Web Application Framework Benchmarks and Perl's PerformanceA Report on Web Application Framework Benchmarks and Perl's Performance
A Report on Web Application Framework Benchmarks and Perl's Performance
Crimson Thompson
 
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
Vassilis Korkas
 
Measuring LibreOffice Interoperability
Measuring LibreOffice InteroperabilityMeasuring LibreOffice Interoperability
Measuring LibreOffice Interoperability
Dushyant Bhalgami
 
99translations For Developers
99translations For Developers99translations For Developers
99translations For Developers
99translations
 
Lap around visual studio alm 2013
Lap around visual studio alm 2013Lap around visual studio alm 2013
Lap around visual studio alm 2013
Angela Dugan
 
Fast end-to-end-tests
Fast end-to-end-testsFast end-to-end-tests
Fast end-to-end-tests
Rikke Veng Petersen
 
Extreme Programming - to the next-level
Extreme Programming - to the next-levelExtreme Programming - to the next-level
Extreme Programming - to the next-level
Lars Thorup
 
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
Blackboard APAC
 
Introduction to Automated Testing
Introduction to Automated TestingIntroduction to Automated Testing
Introduction to Automated Testing
Lars Thorup
 
Introduction to lambda behave
Introduction to lambda behaveIntroduction to lambda behave
Introduction to lambda behave
RichardWarburton
 
Automated Performance Testing
Automated Performance TestingAutomated Performance Testing
Automated Performance Testing
Lars Thorup
 
Unit testing
Unit testingUnit testing
Unit testing
Nikhil Wanpal
 
Unit Testing in Swift
Unit Testing in SwiftUnit Testing in Swift
Unit Testing in Swift
GlobalLogic Ukraine
 
Functional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNextFunctional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNext
Unfold UI
 
It meet up 1 0 лебедева, лоханов
It meet up 1 0 лебедева, лохановIt meet up 1 0 лебедева, лоханов
It meet up 1 0 лебедева, лоханов
Victoria Astapenko
 
Php Inspections (EA Extended): realtime static code analysis
Php Inspections (EA Extended): realtime static code analysisPhp Inspections (EA Extended): realtime static code analysis
Php Inspections (EA Extended): realtime static code analysis
Vladimir Reznichenko
 
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Binary Studio
 

What's hot (20)

Xp conf-tbd
Xp conf-tbdXp conf-tbd
Xp conf-tbd
 
Cqrs
CqrsCqrs
Cqrs
 
MBLTDev15: Brigit Lyons, Soundcloud
MBLTDev15: Brigit Lyons, SoundcloudMBLTDev15: Brigit Lyons, Soundcloud
MBLTDev15: Brigit Lyons, Soundcloud
 
A Report on Web Application Framework Benchmarks and Perl's Performance
A Report on Web Application Framework Benchmarks and Perl's PerformanceA Report on Web Application Framework Benchmarks and Perl's Performance
A Report on Web Application Framework Benchmarks and Perl's Performance
 
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
Bringing linguistic QA to the cloud - Vassilis Korkas (lexiQA)
 
Measuring LibreOffice Interoperability
Measuring LibreOffice InteroperabilityMeasuring LibreOffice Interoperability
Measuring LibreOffice Interoperability
 
99translations For Developers
99translations For Developers99translations For Developers
99translations For Developers
 
Lap around visual studio alm 2013
Lap around visual studio alm 2013Lap around visual studio alm 2013
Lap around visual studio alm 2013
 
Fast end-to-end-tests
Fast end-to-end-testsFast end-to-end-tests
Fast end-to-end-tests
 
Extreme Programming - to the next-level
Extreme Programming - to the next-levelExtreme Programming - to the next-level
Extreme Programming - to the next-level
 
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
Keeping up with the changes: Automating UAT - Damian Sweeney, Student and Aca...
 
Introduction to Automated Testing
Introduction to Automated TestingIntroduction to Automated Testing
Introduction to Automated Testing
 
Introduction to lambda behave
Introduction to lambda behaveIntroduction to lambda behave
Introduction to lambda behave
 
Automated Performance Testing
Automated Performance TestingAutomated Performance Testing
Automated Performance Testing
 
Unit testing
Unit testingUnit testing
Unit testing
 
Unit Testing in Swift
Unit Testing in SwiftUnit Testing in Swift
Unit Testing in Swift
 
Functional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNextFunctional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNext
 
It meet up 1 0 лебедева, лоханов
It meet up 1 0 лебедева, лохановIt meet up 1 0 лебедева, лоханов
It meet up 1 0 лебедева, лоханов
 
Php Inspections (EA Extended): realtime static code analysis
Php Inspections (EA Extended): realtime static code analysisPhp Inspections (EA Extended): realtime static code analysis
Php Inspections (EA Extended): realtime static code analysis
 
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
Introduction to DevOps. Continuous Integration by Myroslav Dmytrus
 

Similar to Autograder presentation

Integration testing - A&BP CC
Integration testing - A&BP CCIntegration testing - A&BP CC
Integration testing - A&BP CC
JWORKS powered by Ordina
 
Are we there yet?
Are we there yet?Are we there yet?
Are we there yet?
Johann Höchtl
 
Test automation proposal
Test automation proposalTest automation proposal
Test automation proposal
Mihai-Cristian Fratila
 
Abhilash Alwandi resume
Abhilash Alwandi resumeAbhilash Alwandi resume
Abhilash Alwandi resume
Abhilash Alwandi
 
Enter the mind of an Agile Developer
Enter the mind of an Agile DeveloperEnter the mind of an Agile Developer
Enter the mind of an Agile Developer
BSGAfrica
 
Resume (1)
Resume (1)Resume (1)
Resume (1)
alka yadav
 
Keeping code clean
Keeping code cleanKeeping code clean
Keeping code clean
Brett Child
 
Agile Testing 2020
Agile Testing 2020Agile Testing 2020
Agile Testing 2020
arzu TR
 
End-end tests as first class citizens - SeleniumConf 2020
End-end tests as first class citizens - SeleniumConf 2020End-end tests as first class citizens - SeleniumConf 2020
End-end tests as first class citizens - SeleniumConf 2020
Abhijeet Vaikar
 
SPRING BOOT BASED TASK HANDLER.pptx
SPRING BOOT BASED TASK HANDLER.pptxSPRING BOOT BASED TASK HANDLER.pptx
SPRING BOOT BASED TASK HANDLER.pptx
SWETHAABIRAMIM
 
Sakar Patnaik_1.5_testing_Manual_Automation_Selenium
Sakar Patnaik_1.5_testing_Manual_Automation_SeleniumSakar Patnaik_1.5_testing_Manual_Automation_Selenium
Sakar Patnaik_1.5_testing_Manual_Automation_Selenium
SAKAR PATNAIK
 
Test automation - Building effective solutions
Test automation - Building effective solutionsTest automation - Building effective solutions
Test automation - Building effective solutions
Artem Nagornyi
 
ShobhaResume
ShobhaResumeShobhaResume
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
dcieslak
 
Abhilash Alwandi_Resume
Abhilash Alwandi_ResumeAbhilash Alwandi_Resume
Abhilash Alwandi_Resume
Abhilashalwandi
 
Rana_3Years_PerformanceEngineer
Rana_3Years_PerformanceEngineerRana_3Years_PerformanceEngineer
Rana_3Years_PerformanceEngineer
RANA VISHAL SINGH
 
Ranjit Show_Resume_Testing
Ranjit Show_Resume_TestingRanjit Show_Resume_Testing
Ranjit Show_Resume_Testing
Ranjit Show
 
Online Examination Java Projectreport.docx
Online Examination Java Projectreport.docxOnline Examination Java Projectreport.docx
Online Examination Java Projectreport.docx
TanishaPatil4
 
Building environment of #UserDevOps and not only DevOps
Building environment of #UserDevOps and not only DevOpsBuilding environment of #UserDevOps and not only DevOps
Building environment of #UserDevOps and not only DevOps
Rajnish Chauhan
 
Software Project Management - NESDEV
Software Project Management - NESDEVSoftware Project Management - NESDEV
Software Project Management - NESDEV
Krit Kamtuo
 

Similar to Autograder presentation (20)

Integration testing - A&BP CC
Integration testing - A&BP CCIntegration testing - A&BP CC
Integration testing - A&BP CC
 
Are we there yet?
Are we there yet?Are we there yet?
Are we there yet?
 
Test automation proposal
Test automation proposalTest automation proposal
Test automation proposal
 
Abhilash Alwandi resume
Abhilash Alwandi resumeAbhilash Alwandi resume
Abhilash Alwandi resume
 
Enter the mind of an Agile Developer
Enter the mind of an Agile DeveloperEnter the mind of an Agile Developer
Enter the mind of an Agile Developer
 
Resume (1)
Resume (1)Resume (1)
Resume (1)
 
Keeping code clean
Keeping code cleanKeeping code clean
Keeping code clean
 
Agile Testing 2020
Agile Testing 2020Agile Testing 2020
Agile Testing 2020
 
End-end tests as first class citizens - SeleniumConf 2020
End-end tests as first class citizens - SeleniumConf 2020End-end tests as first class citizens - SeleniumConf 2020
End-end tests as first class citizens - SeleniumConf 2020
 
SPRING BOOT BASED TASK HANDLER.pptx
SPRING BOOT BASED TASK HANDLER.pptxSPRING BOOT BASED TASK HANDLER.pptx
SPRING BOOT BASED TASK HANDLER.pptx
 
Sakar Patnaik_1.5_testing_Manual_Automation_Selenium
Sakar Patnaik_1.5_testing_Manual_Automation_SeleniumSakar Patnaik_1.5_testing_Manual_Automation_Selenium
Sakar Patnaik_1.5_testing_Manual_Automation_Selenium
 
Test automation - Building effective solutions
Test automation - Building effective solutionsTest automation - Building effective solutions
Test automation - Building effective solutions
 
ShobhaResume
ShobhaResumeShobhaResume
ShobhaResume
 
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
RandomTest - Random Software Integration Tests That Just Work for C/C++, Java...
 
Abhilash Alwandi_Resume
Abhilash Alwandi_ResumeAbhilash Alwandi_Resume
Abhilash Alwandi_Resume
 
Rana_3Years_PerformanceEngineer
Rana_3Years_PerformanceEngineerRana_3Years_PerformanceEngineer
Rana_3Years_PerformanceEngineer
 
Ranjit Show_Resume_Testing
Ranjit Show_Resume_TestingRanjit Show_Resume_Testing
Ranjit Show_Resume_Testing
 
Online Examination Java Projectreport.docx
Online Examination Java Projectreport.docxOnline Examination Java Projectreport.docx
Online Examination Java Projectreport.docx
 
Building environment of #UserDevOps and not only DevOps
Building environment of #UserDevOps and not only DevOpsBuilding environment of #UserDevOps and not only DevOps
Building environment of #UserDevOps and not only DevOps
 
Software Project Management - NESDEV
Software Project Management - NESDEVSoftware Project Management - NESDEV
Software Project Management - NESDEV
 

Autograder presentation

  • 1. Auto-Grading Parallel Programs Building plugins for WebCAT Team Lead: Max Grossman Team Member: Maha Aziz, Anant Tibrewal, Anna Chi
  • 2. ● Background & Motivation ● Related Work ● What is WebCAT? ● Workflow ● Feature table ● Our Progress ● Future Plans Overview
  • 3. ● Current grading method & Shorts ● A tool for testing and analyzing parallel programs automatically ○ Transparency of the grading process ○ Speed up and simplify the grading process ○ Focus on application performance ○ Possibility of parallel programming MOOC to be created Background & Motivation
  • 4. ● Mooshak ○ Intended for hosting coding competitions online ○ Simple configuration, minimal requirements (apache, linux, tcl) ○ Not very pretty UI ● Marmoset ○ Not very extensive Java support Related Work
  • 7. from May.11th till now ● Research about multiple autograding systems Related Work ● Target on WebCat WebCAT ● Install and set up ● Hackathon plug-in structure ● Feature table Workflow
  • 8. Description Priority (1-5, 1 being the highest) Viability (1-5, 1 being the highest) Workload (1-5, 1 being the highest) Assignee Code styling/checkstyle grading 2 1 2 Anant Local grading of HJ-lib programs running in a single thread for correctness 1 1 1 Anant Remote grading of HJ-lib programs running on multiple cores in a compute cluster 2 3 1 Anna needs to commit student file to svn before/after running the autograder 1 1 5 Anna Feature Table
  • 9. Description Priority (1-5, 1 being the highest) Viability (1-5, 1 being the highest) Workload (1-5, 1 being the highest) Assignee give a way for student provided tests to be graded with feedback 3/4 once we have instructor-provided, this is a 2 3 instructor provided tests with limited feedback 1 1 1 Ranking of student submissions based on performance, make it visible to students to turn every homework into a competition (with names anonymized?) 4 4 2 Maha Statistics on how many students are passing a given test, to give feedback on the relative difficulty of different tests (maybe part of the leaderboard?) 3 4 2 Feature Table
  • 10. Description Priority (1-5, 1 being the highest) Viability (1-5, 1 being the highest) Workload (1-5, 1 being the highest) Assignee Student peer reviews 5 5 1 Different types of static code analysis tools 2 once we have checkstyle, this should be easier (3) 2 NetID authentication 1 3 ? Maha Feature Table
  • 11. Architecture Students WebCAT STIC Leaderboard Homework Submission Performance Test Results Student View of Leaderboard POST Request for New Submissions SVN
  • 12. ● Anant ● Maha ● Anna Our Progress
  • 13. Architecture Students WebCAT STIC Leaderboard Homework Submission Performance Test Results Student View of Leaderboard POST Request for New Submissions SVN
  • 14. ● Anant ○ Local grading of HJ-lib programs running in a single thread for correctness ○ Built HJ program with JUnit tests to test WebCAT ■ correctness, performance tests ○ Incorporated the FindBugs static analysis tool to WebCAT ■ tool to detect bugs in code Our Progress
  • 15. Architecture Students WebCAT STIC Leaderboard Homework Submission Performance Test Results Student View of Leaderboard POST Request for New Submissions SVN
  • 16. ● Maha ○ “Leaderboard” ○ created a database which contains all student submissions from the svn ■ includes test names, execution times, core count, etc. ○ database is displayed to webpage for students to see a ranking of how well their classmates are doing ■ all information on webpage is anonymous Our Progress
  • 18. Architecture Students WebCAT STIC Leaderboard Homework Submission Performance Test Results Student View of Leaderboard POST Request for New Submissions SVN
  • 19. student-runs Handled backup of students files using SVN student ID before grading num of submission after grading student ‘s code Results
  • 20. Remote grading on multiple cores in cluster WebCAT STIC (cluster) Build folder structure Run tests Student code & Performance test Send performance results back
  • 21. Through admin interface Slurm personal configuration
  • 22. DEMO
  • 23. ● Feature table ● Provide documentations for students and instructors ● EduHPC Future Plans
  • 24. ● Current inconvenient grading process ● Research about autograders ● Good points and properties of WebCAT ● Feature Table ● Our progress ● Next steps Summary