FLOSS Pilot Studies


Published on

FLOSS projects are excellent Bazaars of learning.

Published in: Economy & Finance, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

FLOSS Pilot Studies

  1. 1. I nvolving Students in the F/OSS Learning Environment: Encounters and Experiences <ul><ul><li>FLOSSCOM </li></ul></ul><ul><ul><li>Thessaloniki, Greece. </li></ul></ul><ul><ul><li>November 13 th – November 14 th , 2006 </li></ul></ul><ul><ul><li>Sulayman K. Sowe </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>
  2. 2. Introduction <ul><li>Software Engineering Education (SE) Now </li></ul><ul><ul><li>Inspired by closed-source software development in lecture formats. </li></ul></ul><ul><ul><li>Teaching specific SE skills in semesters. </li></ul></ul><ul><ul><li>Project-based coursework. Students work in industry to gain practical experience. </li></ul></ul><ul><li>Challenges </li></ul><ul><ul><li>Expose students to the SE principles and techniques we teach them. </li></ul></ul><ul><ul><li>Give students practice in large and long-term projects. </li></ul></ul><ul><ul><li>Will industry scarify their software (code) to students? </li></ul></ul>
  3. 3. <ul><li>What we want to do </li></ul><ul><ul><li>Run a small pilot study in F/OSS mode. </li></ul></ul><ul><ul><li>Students as volunteers in software testing in F/OSS projects. </li></ul></ul><ul><ul><li>Give students opportunity to work on what they considered interesting. </li></ul></ul><ul><ul><li>Give students ‘real-world’ experience in dealing with F/OSS projects. </li></ul></ul><ul><li>Roles </li></ul><ul><ul><li>Students: find and report bugs. </li></ul></ul><ul><ul><li>Lecturer: provide useful guidance and support. </li></ul></ul>
  4. 4. F/OSS Teaching & Learning Framework <ul><li>Implemented in the Introduction to Software Engineering (ISE) course. </li></ul><ul><li>Duration: 12.5 weeks in 5 th Semester in 2005. </li></ul><ul><li>15 students volunteered </li></ul><ul><li>1 lecturer + 1 adviser. </li></ul>
  5. 5. F/OSS Teaching & Learning Framework– Phase 1 <ul><li>Introduction to F/OSS (8hrs) </li></ul><ul><ul><li>What is F/OSS?, projects, activities, testing, forums, etc. </li></ul></ul><ul><li>Exploratory process </li></ul><ul><ul><li>Students browse projects portals ( sourceforge.net ) </li></ul></ul><ul><li>Projects selection criteria </li></ul><ul><ul><li>Operating system, # of developers, development status ( α/β) , active mailing lists/forums. </li></ul></ul><ul><li>1 st class presentation </li></ul><ul><ul><li>Brief history of project, bug reporting procedure, testing tools used. </li></ul></ul>
  6. 6. F/OSS Teaching & Learning Framework– Phase 2 [Students Involvement in Testing] <ul><li>2 nd class presentation (Week 5) </li></ul><ul><ul><li>Types of bugs found, how they were found, what caused them, how they were reported, any responses? </li></ul></ul>
  7. 7. F/OSS Teaching & Learning Framework – Phase 3 <ul><li>3 rd class presentation (Week 12) </li></ul><ul><ul><li>student particulars (Project name, login id, website, history, screenshots) </li></ul></ul><ul><ul><li>List of testing activities: </li></ul></ul><ul><ul><ul><li>bugs found ( bfn ), bugs reported ( brp ), bugs fixed ( bfx ), replies ( rep ) received, Urls of brp , bfx , and rep . </li></ul></ul></ul><ul><ul><li>Likes, dislikes and future plans (if any) in the project. </li></ul></ul>
  8. 8. F/OSS Teaching & Learning Framework – Phase 3 <ul><li>Grading: Coursework (50%) </li></ul><ul><ul><li>Presentations = 10% </li></ul></ul><ul><ul><ul><li>3 pts for each presentation in p hase 1 & 2 . </li></ul></ul></ul><ul><ul><ul><li>4 pts for presentation in phase 3 . </li></ul></ul></ul><ul><ul><li>Project participation = 12% </li></ul></ul><ul><ul><ul><li># of emails exchanged with the student about his project. </li></ul></ul></ul><ul><ul><li>Working with testing tools = 13% </li></ul></ul><ul><ul><ul><li>using and understanding bug tracking systems/bug databases. </li></ul></ul></ul><ul><ul><li>Testing activities = 15% </li></ul></ul><ul><ul><ul><li>measured by 3 variables ( bfn , brp , and rep ) </li></ul></ul></ul>
  9. 9. F/OSS Framework Evaluation - 1 Evaluation Approach = Testing activities + Online Surveys <ul><li>Testing activities . Students as bug seekers </li></ul><ul><li>bfn > brp , some bugs found were already reported. </li></ul><ul><li>Not required but some students did well in bfx . </li></ul>43 15 68 72 Sum 2.175 1.281 2.743 3.017 Std. Dev. 1 0 4.0 3.0 Mode 3.00 1.00 4.00 4.00 Median 3.31 1.15 5.23 5.54 Mean 13 13 13 13 No. of students rep bfx brp bfn
  10. 10. F/OSS Framework Evaluation - 2 <ul><li>Correlation analysis ( Bivariate) </li></ul><ul><li>bfx uncorrelated with bfn, brp, rep . </li></ul><ul><li>bfn vs brp ( r =0.690, p =0.001 ). </li></ul><ul><li>brp vs rep ( r =0.490 , p = 0.089 ). Interaction with community is moderate. </li></ul>.490 .353 p .281 r bfx .089 .911 p -.035 r brp .024 .730 .001 p .620 .106 .960 r bfn rep bfx brp
  11. 11. <ul><li>Factor Analysis (Principal Component Analysis & Varimax rotation) </li></ul>F/OSS Framework Evaluation - 3 <ul><li>Students’ behaviour explained in terms of 2 latent factors. </li></ul><ul><ul><ul><ul><li>f 1 : bfn, brp and to some extent rep </li></ul></ul></ul></ul><ul><ul><ul><ul><li>f 2 : bfx </li></ul></ul></ul></ul>The factor scores estimated by the Anderson-Rubin method
  12. 12. <ul><li>Student #2 high score in f 1 & f 2 …good performance </li></ul><ul><li>Student #6 high score in f 1 low score in f 2 </li></ul><ul><li>Students #3, #5, #8, #9, #10, and #12…moderate performance </li></ul><ul><li>Students #4 and #11 high score in f 2 ……bug fixers </li></ul>F/OSS Framework Evaluation – 4. Factor Analysis Cont.
  13. 13. <ul><li>Online Surveys </li></ul><ul><ul><li>2 online surveys (Week 6 & 13). N=11 for both surveys </li></ul></ul><ul><ul><li>7 questions in Survey 1 were repeated in Survey 2 </li></ul></ul>McNemer (NA) (100%) McNemer (p = 0.549), Not significant McNemer (p = 0.754), Not S ignificant McNemer (p = 0.039), Some s ignificance McNemer (p = 0.002), S ignificant McNemer (p = 0.065), S ignificant McNemer (p = 0.008), Not significant F/OSS Framework Evaluation - 5
  14. 14. Online Survey Results - 2 Interesting project Friendly Response
  15. 15. Experiences & Lessons Learnt - 1 <ul><li>Advantages </li></ul><ul><li>Practical experience </li></ul><ul><ul><li>software testing in the bazaar. </li></ul></ul><ul><ul><li>how large and complex F/OSS projects work. </li></ul></ul><ul><ul><li>writing good bug reports. </li></ul></ul><ul><ul><li>communicate ideas to ‘virtual colleagues’. </li></ul></ul><ul><ul><li>working with tools they might use when they graduate. </li></ul></ul><ul><li>Flexible Schedules: test whenever and wherever you want. </li></ul><ul><li>Learning Opportunities: learn new tools (e.g. Mantis Bug Tracker). </li></ul><ul><li>Sample size: 13, effective interaction with the students. </li></ul>Face-to-face contact in phase 1 + presentations (brainstorming).
  16. 16. Experiences & Lessons Learnt - 2 <ul><li>Disadvantages </li></ul><ul><li>Specialization: one aspect of the ISE course (software testing). </li></ul><ul><li>Collaboration: encouraged but not required to work in groups/pairs. </li></ul><ul><li>Evaluation problem: “Overachievers”. How to reward them? </li></ul><ul><li>Sample size: 15 out of 150 volunteered. </li></ul><ul><li>Lecturer as project manager </li></ul><ul><ul><li>more than preparing and delivering 2hrs lectures </li></ul></ul><ul><ul><li>lecturer as projects manager </li></ul></ul>Difficulties <ul><li>Email/work overload: keeping pace with students emails. </li></ul><ul><li>When to stop scoring points: students “Now stop finding </li></ul><ul><li>and reporting bugs!” ? </li></ul>
  17. 17. Experiences & Lessons Learnt - 3 <ul><li>Communication </li></ul><ul><ul><li>IM/Web-based discussion forum. </li></ul></ul><ul><ul><ul><li>Problems: Timing, archiving, searching, etc. </li></ul></ul></ul><ul><li>Contradict F/OSS Freedom Norm </li></ul><ul><ul><li>pre-select and present students diversity of projects. </li></ul></ul><ul><li>Group work/Community formation </li></ul><ul><ul><li>group students around specified projects . </li></ul></ul><ul><ul><li>Identify task-leader based on his contribution. </li></ul></ul><ul><li>Grading/Evaluation </li></ul><ul><ul><li># views a bug report received from the project’s community. </li></ul></ul><ul><ul><ul><li>Problems: Available in few projects. Students view their own submission many times. </li></ul></ul></ul>Possible Improvements
  18. 18. <ul><li>Work in progress: </li></ul><ul><ul><li>Conduct a full experiment involving larger sample of students and extend and use the statistical analysis method used in this pilot study. </li></ul></ul><ul><li>F/OSS & SE Education- Projects as Bazaars of learning : </li></ul><ul><ul><li>How can we blend the F/OSS teaching and learning environment with the formal SE teaching and learning context in colleges and universities? </li></ul></ul><ul><ul><li>A broad understanding of the F/OSS pedagogy </li></ul></ul><ul><ul><li>More research of F/OSS evaluation and assessment methodologies </li></ul></ul><ul><ul><li>How F/OSS can improve the quality of teaching and learning. </li></ul></ul><ul><ul><li>How to create a partnership between students and F/OSS developers, projects and industry? </li></ul></ul>Conclusion
  19. 19. SELECTED PUBLICATIONS <ul><li>Sowe, S. K., Karoulis, A., Stamelos I. (2005). A constructivist view of knowledge management in open source virtual communities . In Figueiredo, D. A., Paula, A. (eds), Managing Learning in Virtual Settings: The Role of Context . Idea Group, Inc, pp.290-308. </li></ul><ul><li>Sowe, S. K., Karoulis, A., Stamelos I., Bleris G.L. (2004). F ree-Open Source learning community and web-based technologies . IEEE Learning Technology Newsletter , Vol. 6 (1), 2004. pp.26-29. </li></ul><ul><li>Sowe, S.K., I. Stamelos. (2005). Identification of Knowledge Brokers in OSS Projects Mailing Lists through Social and Collaborative Networks. In Proceedings of 10th Panhellenic Conference in Informatics , Volos, Greece, pp. 132-141. </li></ul><ul><li>Sowe, S.K., Stamelos, I., Angelis, I. 2006, Identifying knowledge brokers that yield software engineering knowledge in OSS projects, Information and Software Technology , Vol. 48, 2006, pp 1025-1033 </li></ul><ul><li>Sowe, S.K., Stamelos, I., Deligiannis, I. 2006, A Framework for Teaching Software Testing using F/OSS Methodology , in IFIP International Federation for Information Processing, Vol. 203, Open Source Systems , eds. Damiani, E., Fitzerald, B., Scacchi, W., Scott, M., Succi, G., (Boston: Springer), pp. 261-266. </li></ul><ul><li>Sowe, S.K., I. Stamelos, L. Angelis. 2006, An Empirical Approach to Evaluate Students Participation in Open Source Software Projects . The IADIS CELDA 2006 conference , Barcelona, Spain, 8-10 December 2006. </li></ul><ul><li>Sowe, S.K., Stamelos . 2006, Involving Software Engineering Students in Free and Open Source Software Projects: Encounters and Experiences. Journal of Information Systems Education , To appear. This work was partially funded through the European Commission, DG Education and Culture, Socrates programme, Minerva action line, project ref: 229405 - CP -1-2006-1- PT - MINERVA – M ( http://www.flosscom.net/ ). </li></ul>
  20. 20. Thanks! Questions? Comments?