I nvolving Students in the F/OSS Learning Environment: Encounters and Experiences FLOSSCOM Thessaloniki, Greece. November 13 th – November 14 th , 2006 Sulayman K. Sowe [email_address]

FLOSSCOM

Thessaloniki, Greece.

November 13 th – November 14 th , 2006

Sulayman K. Sowe

[email_address]

Introduction Software Engineering Education (SE) Now Inspired by closed-source software development in lecture formats. Teaching specific SE skills in semesters. Project-based coursework. Students work in industry to gain practical experience. Challenges Expose students to the SE principles and techniques we teach them. Give students practice in large and long-term projects. Will industry scarify their software (code) to students?

Software Engineering Education (SE) Now

Inspired by closed-source software development in lecture formats.

Teaching specific SE skills in semesters.

Project-based coursework. Students work in industry to gain practical experience.

Challenges

Expose students to the SE principles and techniques we teach them.

Give students practice in large and long-term projects.

Will industry scarify their software (code) to students?

What we want to do Run a small pilot study in F/OSS mode. Students as volunteers in software testing in F/OSS projects. Give students opportunity to work on what they considered interesting. Give students ‘real-world’ experience in dealing with F/OSS projects. Roles Students: find and report bugs. Lecturer: provide useful guidance and support.

What we want to do

Run a small pilot study in F/OSS mode.

Students as volunteers in software testing in F/OSS projects.

Give students opportunity to work on what they considered interesting.

Give students ‘real-world’ experience in dealing with F/OSS projects.

Roles

Students: find and report bugs.

Lecturer: provide useful guidance and support.

F/OSS Teaching & Learning Framework Implemented in the Introduction to Software Engineering (ISE) course. Duration: 12.5 weeks in 5 th Semester in 2005. 15 students volunteered 1 lecturer + 1 adviser.

Implemented in the Introduction to Software Engineering (ISE) course.

Duration: 12.5 weeks in 5 th Semester in 2005.

15 students volunteered

1 lecturer + 1 adviser.

F/OSS Teaching & Learning Framework– Phase 1 Introduction to F/OSS (8hrs) What is F/OSS?, projects, activities, testing, forums, etc. Exploratory process Students browse projects portals ( sourceforge.net ) Projects selection criteria Operating system, # of developers, development status ( α/β) , active mailing lists/forums. 1 st class presentation Brief history of project, bug reporting procedure, testing tools used.

Introduction to F/OSS (8hrs)

What is F/OSS?, projects, activities, testing, forums, etc.

Exploratory process

Students browse projects portals ( sourceforge.net )

Projects selection criteria

Operating system, # of developers, development status ( α/β) , active mailing lists/forums.

1 st class presentation

Brief history of project, bug reporting procedure, testing tools used.

F/OSS Teaching & Learning Framework– Phase 2 [Students Involvement in Testing] 2 nd class presentation (Week 5) Types of bugs found, how they were found, what caused them, how they were reported, any responses?

2 nd class presentation (Week 5)

Types of bugs found, how they were found, what caused them, how they were reported, any responses?

F/OSS Teaching & Learning Framework – Phase 3 3 rd class presentation (Week 12) student particulars (Project name, login id, website, history, screenshots) List of testing activities: bugs found ( bfn ), bugs reported ( brp ), bugs fixed ( bfx ), replies ( rep ) received, Urls of brp , bfx , and rep . Likes, dislikes and future plans (if any) in the project.

3 rd class presentation (Week 12)

student particulars (Project name, login id, website, history, screenshots)

List of testing activities:

bugs found ( bfn ), bugs reported ( brp ), bugs fixed ( bfx ), replies ( rep ) received, Urls of brp , bfx , and rep .

Likes, dislikes and future plans (if any) in the project.

F/OSS Teaching & Learning Framework – Phase 3 Grading: Coursework (50%) Presentations = 10% 3 pts for each presentation in p hase 1 & 2 . 4 pts for presentation in phase 3 . Project participation = 12% # of emails exchanged with the student about his project. Working with testing tools = 13% using and understanding bug tracking systems/bug databases. Testing activities = 15% measured by 3 variables ( bfn , brp , and rep )

Grading: Coursework (50%)

Presentations = 10%

3 pts for each presentation in p hase 1 & 2 .

4 pts for presentation in phase 3 .

Project participation = 12%

# of emails exchanged with the student about his project.

Working with testing tools = 13%

using and understanding bug tracking systems/bug databases.

Testing activities = 15%

measured by 3 variables ( bfn , brp , and rep )

F/OSS Framework Evaluation - 1 Evaluation Approach = Testing activities + Online Surveys Testing activities . Students as bug seekers bfn > brp , some bugs found were already reported. Not required but some students did well in bfx . 43 15 68 72 Sum 2.175 1.281 2.743 3.017 Std. Dev. 1 0 4.0 3.0 Mode 3.00 1.00 4.00 4.00 Median 3.31 1.15 5.23 5.54 Mean 13 13 13 13 No. of students rep bfx brp bfn

Testing activities . Students as bug seekers

bfn > brp , some bugs found were already reported.

Not required but some students did well in bfx .

F/OSS Framework Evaluation - 2 Correlation analysis ( Bivariate) bfx uncorrelated with bfn, brp, rep . bfn vs brp ( r =0.690, p =0.001 ). brp vs rep ( r =0.490 , p = 0.089 ). Interaction with community is moderate. .490 .353 p .281 r bfx .089 .911 p -.035 r brp .024 .730 .001 p .620 .106 .960 r bfn rep bfx brp

Correlation analysis ( Bivariate)

bfx uncorrelated with bfn, brp, rep .

bfn vs brp ( r =0.690, p =0.001 ).

brp vs rep ( r =0.490 , p = 0.089 ). Interaction with community is moderate.

Factor Analysis (Principal Component Analysis & Varimax rotation) F/OSS Framework Evaluation - 3 Students’ behaviour explained in terms of 2 latent factors. f 1 : bfn, brp and to some extent rep f 2 : bfx The factor scores estimated by the Anderson-Rubin method

Factor Analysis (Principal Component Analysis & Varimax rotation)

Students’ behaviour explained in terms of 2 latent factors.

f 1 : bfn, brp and to some extent rep

f 2 : bfx

Student #2 high score in f 1 & f 2 …good performance Student #6 high score in f 1 low score in f 2 Students #3, #5, #8, #9, #10, and #12…moderate performance Students #4 and #11 high score in f 2 ……bug fixers F/OSS Framework Evaluation – 4. Factor Analysis Cont.

Student #2 high score in f 1 & f 2 …good performance

Student #6 high score in f 1 low score in f 2

Students #3, #5, #8, #9, #10, and #12…moderate performance

Students #4 and #11 high score in f 2 ……bug fixers

Online Surveys 2 online surveys (Week 6 & 13). N=11 for both surveys 7 questions in Survey 1 were repeated in Survey 2 McNemer (NA) (100%) McNemer (p = 0.549), Not significant McNemer (p = 0.754), Not S ignificant McNemer (p = 0.039), Some s ignificance McNemer (p = 0.002), S ignificant McNemer (p = 0.065), S ignificant McNemer (p = 0.008), Not significant F/OSS Framework Evaluation - 5

Online Surveys

2 online surveys (Week 6 & 13). N=11 for both surveys

7 questions in Survey 1 were repeated in Survey 2

Online Survey Results - 2 Interesting project Friendly Response

Experiences & Lessons Learnt - 1 Advantages Practical experience software testing in the bazaar. how large and complex F/OSS projects work. writing good bug reports. communicate ideas to ‘virtual colleagues’. working with tools they might use when they graduate. Flexible Schedules: test whenever and wherever you want. Learning Opportunities: learn new tools (e.g. Mantis Bug Tracker). Sample size: 13, effective interaction with the students. Face-to-face contact in phase 1 + presentations (brainstorming).

Advantages

Practical experience

software testing in the bazaar.

how large and complex F/OSS projects work.

writing good bug reports.

communicate ideas to ‘virtual colleagues’.

working with tools they might use when they graduate.

Flexible Schedules: test whenever and wherever you want.

Learning Opportunities: learn new tools (e.g. Mantis Bug Tracker).

Sample size: 13, effective interaction with the students.

Experiences & Lessons Learnt - 2 Disadvantages Specialization: one aspect of the ISE course (software testing). Collaboration: encouraged but not required to work in groups/pairs. Evaluation problem: “Overachievers”. How to reward them? Sample size: 15 out of 150 volunteered. Lecturer as project manager more than preparing and delivering 2hrs lectures lecturer as projects manager Difficulties Email/work overload: keeping pace with students emails. When to stop scoring points: students “Now stop finding and reporting bugs!” ?

Disadvantages

Specialization: one aspect of the ISE course (software testing).

Collaboration: encouraged but not required to work in groups/pairs.

Evaluation problem: “Overachievers”. How to reward them?

Sample size: 15 out of 150 volunteered.

Lecturer as project manager

more than preparing and delivering 2hrs lectures

lecturer as projects manager

Email/work overload: keeping pace with students emails.

When to stop scoring points: students “Now stop finding

and reporting bugs!” ?

Experiences & Lessons Learnt - 3 Communication IM/Web-based discussion forum. Problems: Timing, archiving, searching, etc. Contradict F/OSS Freedom Norm pre-select and present students diversity of projects. Group work/Community formation group students around specified projects . Identify task-leader based on his contribution. Grading/Evaluation # views a bug report received from the project’s community. Problems: Available in few projects. Students view their own submission many times. Possible Improvements

Communication

IM/Web-based discussion forum.

Problems: Timing, archiving, searching, etc.

Contradict F/OSS Freedom Norm

pre-select and present students diversity of projects.

Group work/Community formation

group students around specified projects .

Identify task-leader based on his contribution.

Grading/Evaluation

# views a bug report received from the project’s community.

Problems: Available in few projects. Students view their own submission many times.

Work in progress: Conduct a full experiment involving larger sample of students and extend and use the statistical analysis method used in this pilot study. F/OSS & SE Education- Projects as Bazaars of learning : How can we blend the F/OSS teaching and learning environment with the formal SE teaching and learning context in colleges and universities? A broad understanding of the F/OSS pedagogy More research of F/OSS evaluation and assessment methodologies How F/OSS can improve the quality of teaching and learning. How to create a partnership between students and F/OSS developers, projects and industry? Conclusion

Work in progress:

Conduct a full experiment involving larger sample of students and extend and use the statistical analysis method used in this pilot study.

F/OSS & SE Education- Projects as Bazaars of learning :

How can we blend the F/OSS teaching and learning environment with the formal SE teaching and learning context in colleges and universities?

A broad understanding of the F/OSS pedagogy

More research of F/OSS evaluation and assessment methodologies

How F/OSS can improve the quality of teaching and learning.

How to create a partnership between students and F/OSS developers, projects and industry?

SELECTED PUBLICATIONS Sowe, S. K., Karoulis, A., Stamelos I. (2005). A constructivist view of knowledge management in open source virtual communities . In Figueiredo, D. A., Paula, A. (eds), Managing Learning in Virtual Settings: The Role of Context . Idea Group, Inc, pp.290-308. Sowe, S. K., Karoulis, A., Stamelos I., Bleris G.L. (2004). F ree-Open Source learning community and web-based technologies . IEEE Learning Technology Newsletter , Vol. 6 (1), 2004. pp.26-29. Sowe, S.K., I. Stamelos. (2005). Identification of Knowledge Brokers in OSS Projects Mailing Lists through Social and Collaborative Networks. In Proceedings of 10th Panhellenic Conference in Informatics , Volos, Greece, pp. 132-141. Sowe, S.K., Stamelos, I., Angelis, I. 2006, Identifying knowledge brokers that yield software engineering knowledge in OSS projects, Information and Software Technology , Vol. 48, 2006, pp 1025-1033 Sowe, S.K., Stamelos, I., Deligiannis, I. 2006, A Framework for Teaching Software Testing using F/OSS Methodology , in IFIP International Federation for Information Processing, Vol. 203, Open Source Systems , eds. Damiani, E., Fitzerald, B., Scacchi, W., Scott, M., Succi, G., (Boston: Springer), pp. 261-266. Sowe, S.K., I. Stamelos, L. Angelis. 2006, An Empirical Approach to Evaluate Students Participation in Open Source Software Projects . The IADIS CELDA 2006 conference , Barcelona, Spain, 8-10 December 2006. Sowe, S.K., Stamelos . 2006, Involving Software Engineering Students in Free and Open Source Software Projects: Encounters and Experiences. Journal of Information Systems Education , To appear. This work was partially funded through the European Commission, DG Education and Culture, Socrates programme, Minerva action line, project ref: 229405 - CP -1-2006-1- PT - MINERVA – M ( http://www.flosscom.net/ ).

Sowe, S. K., Karoulis, A., Stamelos I. (2005). A constructivist view of knowledge management in open source virtual communities . In Figueiredo, D. A., Paula, A. (eds), Managing Learning in Virtual Settings: The Role of Context . Idea Group, Inc, pp.290-308.

Sowe, S. K., Karoulis, A., Stamelos I., Bleris G.L. (2004). F ree-Open Source learning community and web-based technologies . IEEE Learning Technology Newsletter , Vol. 6 (1), 2004. pp.26-29.

Sowe, S.K., I. Stamelos. (2005). Identification of Knowledge Brokers in OSS Projects Mailing Lists through Social and Collaborative Networks. In Proceedings of 10th Panhellenic Conference in Informatics , Volos, Greece, pp. 132-141.

Sowe, S.K., Stamelos, I., Angelis, I. 2006, Identifying knowledge brokers that yield software engineering knowledge in OSS projects, Information and Software Technology , Vol. 48, 2006, pp 1025-1033

Sowe, S.K., Stamelos, I., Deligiannis, I. 2006, A Framework for Teaching Software Testing using F/OSS Methodology , in IFIP International Federation for Information Processing, Vol. 203, Open Source Systems , eds. Damiani, E., Fitzerald, B., Scacchi, W., Scott, M., Succi, G., (Boston: Springer), pp. 261-266.

Sowe, S.K., I. Stamelos, L. Angelis. 2006, An Empirical Approach to Evaluate Students Participation in Open Source Software Projects . The IADIS CELDA 2006 conference , Barcelona, Spain, 8-10 December 2006.

Sowe, S.K., Stamelos . 2006, Involving Software Engineering Students in Free and Open Source Software Projects: Encounters and Experiences. Journal of Information Systems Education , To appear. This work was partially funded through the European Commission, DG Education and Culture, Socrates programme, Minerva action line, project ref: 229405 - CP -1-2006-1- PT - MINERVA – M ( http://www.flosscom.net/ ).

Thanks! Questions? Comments?