Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Computing is Not a Rock Band: Student Understanding of the Computing Disciplines

487 views

Published on

This presentation reports the initial findings of a multi-year study that is surveying major and non-major students’ understanding of the different computing disciplines. This study is based on work originally conducted by Courte and Bishop-Clark from 2009, but which uses a broadened study instrument that provided additional forms of analysis. Data was collected from 199 students from a single institution who were computer science, information systems/information technology and non-major students taking a variety of introductory computing courses. Results show that undergraduate computing students are more likely to rate tasks as being better fits to computer disciplines than are their non-major (NM) peers. Uncertainty among respondents did play a large role in the results and is discussed alongside implications for teaching and further research.

Published in: Education
  • Be the first to comment

Computing is Not a Rock Band: Student Understanding of the Computing Disciplines

  1. 1. Computing StudentUnderstanding oftheComputingDisciplines ! is Not a Rock Band
  2. 2. This presentation reports the initial findings of a multi-year study that is surveying major and non-major students’ understanding of the different computing disciplines. about !
  3. 3. the team Faith-Michael Uzoka Computer Science & Information Systems ! Randy Connolly Computer Science & Information Systems Marc Schroeder Computer Science & Information Systems Namrata Khemka-Dolan Computer Science & Information Systems Janet Miller Counselling
  4. 4. 01Background to this presentation 02The context of our study 03How we did our study 04What our study found. 05What it all means 06Some problems and conclusions RelatedWork Methodology Introduction Results Discussion Limitations andConclusions outline !
  5. 5. Introduction tothestudy
  6. 6. One of the most important achievements in computing education has been the recognition and elaboration of the five different computing disciplines. introduction ! 1Computer Engineering 2Computer Science 3Information Systems 4Information Technology 5Software Engineering
  7. 7. The title of this paper refers to the the fact that the computing disciplines should be understood to be quite unlike the distinct roles in a typical rock band. introduction !
  8. 8. The computing disciplines have considerable overlap between them. ! Despite this overlap, universities have to offer distinct computing degrees that typically do not blend curricula between the different disciplines.
  9. 9. For students, their initial understanding of the different computing disciplines may play a large role in how they decide which (if any) computing program to register in. !
  10. 10. work related
  11. 11. This study is an extension of work by Courte and Bishop-Clark (C&BC) and then validated in a subsequent study by Battig and Shariq. Courte, J. and Bishop-Clark, C. 2009. Do students differentiate between computing disciplines? In Proceedings of the 40th ACM technical symposium on Computer science education (SIGCSE '09). Battig, M. and Shariq, M. 2011. A Validation Study of Student Differentiation Between Computing Disciplines. In Information Systems Education Journal. 9, 5 (October 2011). related work !
  12. 12. In their (C&BC) study, computing and non-computing students were asked to associate job task descriptions with the best disciplinary fit. !
  13. 13. C&BC’s results suggest that students do not always have a clear understanding of disciplinary scopes (especially SE and IT). !
  14. 14. Like the C&BC study, this study examines student knowledge of the five different computing disciplines. methodology !
  15. 15. Unlike the C&BC study, this study tried to capture the overlap between the computing disciplines in the design of its survey. !
  16. 16. In the C&BC study, students were given 15 task descriptions and for each task they had to indicate which of the five disciplines was the best fit for that task. ! Designs hardware to implement communication systems CE CS IS IT SE ⃝ ⃝ ⃝ ⃝ ⃝
  17. 17. The main drawback to the prior studies was that the students had to choose a single discipline for a task … ! which does not capture the possibility of overlap between the disciplines.
  18. 18. To address that drawback, our study allowed the participants to choose how much each task fit with each of the five disciplines. ! X X X X X
  19. 19. Our questionnaire had demographic- related questions and then 31 discipline/task questions. !
  20. 20. ! 1 Designs hardware to implement communication systems 2 Uses new theories to create cutting edge software 3 Builds hardware devices such as iPods 4 Is business oriented 5 Focuses on large-scale systems development 6 Integrates computer hardware and software 7 Troubleshoots and designs practical technical applications 8 Focuses on the theoretical aspects of technology 9 Combines knowledge of business and technology 10 Applies technology to solve practical problems 11 Designs testing procedures for large-scale systems 12 Selects computer systems to improve business processes 13 Applies technical knowledge for product support 14 Utilizes theory to research and design software solutions 15 Manages large scale technological projects Our first 15 questions were the same as the earlier C&BC study:
  21. 21. !plus 15 new tasks added by the authors 16 Develops software systems that are maintainable, reliable, efficient, and satisfy customer requirements 17 Focuses on information, and views technology as a tool for generating, processing and distributing it 18 Utilizes sound engineering practices to create computer applications 19 Provides a support role, within an organization, to help others make the best use of its technical and information resources 20 Uses a wide range of foundational knowledge to adapt to new technologies and ideas 21 Uses technology to give a business a competitive advantage 22 Develops devices that have hardware and software in them 23 Applies mathematical and theoretical knowledge in order to compare and produce computational solutions and choose the best one 25 Understands both technology and business, but with a focus more on the technical side 26 Uses programming skills to create or modify business solutions 27 Develops or maintains web sites 28 Manages a team of software developers 29 Manages a company’s computing department 30 Evaluates and improves the usability (user experience) of computing systems 31 Works with an organization’s data assets
  22. 22. !… and an additional task that is not typically associated with the computing field 24 Focuses exclusively on hardware design, including digital electronics, with little or no involvement in software design
  23. 23. The intent of the study was to find out if relatively-inexperienced students understood the tasks associated with different computing disciplines, prior to enrolment in computing courses/program. !
  24. 24. timeline Questionnaires were provided across ten sections of six introductory computing classes at Mount Royal University in the Fall 2012 semester. Note that there are two computing programs at MRU: a computer science program and a blended IS/IT program. SEPT Of 250 questionnaires that were distributed, 199 questionnaires were properly filled and coded for analysis. JAN Rank ordering analysis along with standard statistical analysis. ! 2012 20132013 MAY
  25. 25. re tus sl
  26. 26. results ! 79% responserate
  27. 27. 25% IS/IT ! 23% CS 52% Non-Major results programofstudy
  28. 28. 59% 17% 12% year 2 year 1 year 3 9% 4% other year 4 results ! yearofstudy
  29. 29. 72% Male results ! gender 28% Female
  30. 30. 81% 11% 5% < 2 Years None 2-5 Years 3% More than 5 results ! computingexperience
  31. 31. Rank order analysis was utilized to determine the students’ ranking of the disciplinary tasks relative to the five computing disciplines. ! A further analysis was carried out to determine the levels of match between students’ task rankings and the disciplinary best fit. results rankorderanalysis
  32. 32. ! examplerankordering Don’t Know % Level of Fit CE CS IS IT SE 15.6% 0 (No Answer) 19.6% 26.1% 25.1% 25.1% 24.1% 1 (No Fit) 2.0% 7.0% 9.5% 9.5% 20.6% 2 1.0% 10.6% 18.6% 15.1% 11.1% 3 7.5% 23.6% 21.1% 16.1% 11.1% 4 20.1% 23.1% 14.1% 20.1% 13.6% 5 (Best Fit) 49.7% 9.5% 11.6% 14.1% 19.6% Mean 3.56 2.39 2.24 2.39 2.28 Median 4.00 3.00 2.00 3.00 2.00 Mode 5 0 0 0 0 Rank 1 3 5 2 4 Question #1 Designs hardware to implement communication systems
  33. 33. ! disciplinematchdistributions Match Level CE CS IS IT SE Very Accurate (5) 4 (100%) 3 (60%) 7 (78%) 5 (50%) 3 (38%) Accurate (4) 0 (0%) 2 (40%) 1 (11%) 2 (20%) 2 (25%) Ok (3) 0 (0%) 0 (0%) 0 (0%) 2 (20%) 2 (25%) Fair (2) 0 (0%) 0 (0%) 1(11%) 1(10%) 1 (12%) Poor (1) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) Total Tasks 4 5 9 10 8 Limitation: not every discipline had the same number of matching questions
  34. 34. Analyses were conducted to determine if there were any clear differences between the three groups of respondents, that is, between IS/IT, CS, and NM (non-major) students. ! Responses to individual tasks were analyzed using one-way ANOVAs. results differencesbetweengroups
  35. 35. Statistically significant differences (p < 0.05) were found between program groups on 19 of the 31 questions … ! Most of these differences occurred between the IS/IT and NM groups.
  36. 36. In general, the IS/IT students were more likely to rank the real-world tasks as better fits to the IS and IT disciplines … !
  37. 37. A significant percentage of respondents either answered “Don’t Know” for a task or didn’t provide a response for a discipline on a task. ! results studentuncertainty
  38. 38. ! results studentuncertainty Don’t Know % Level of Fit CE CS IS IT SE 15.6% 0 (No Answer) 19.6% 26.1% 25.1% 25.1% 24.1% 1 (No Fit) 2.0% 7.0% 9.5% 9.5% 20.6% 2 1.0% 10.6% 18.6% 15.1% 11.1% 3 7.5% 23.6% 21.1% 16.1% 11.1% 4 20.1% 23.1% 14.1% 20.1% 13.6% 5 (Best Fit) 49.7% 9.5% 11.6% 14.1% 19.6% Question #1 Designs hardware to implement communication systems X X X
  39. 39. A significant percentage of respondents either answered “Don’t Know” for a task or didn’t provide a response for a discipline on a task. ! results studentuncertainty Discipline Non-Responses Percentage Don’t Know % CE CS IS IT SE 20.0% 31.4% 30.9% 31.2% 30.9% 31.6%
  40. 40. Non-major students answered DK more frequently, but based on our ANOVA cutoff this difference between groups was not significant. ! Nonetheless, the finding that all our respondents were uncertain with one out over every five tasks is a significant finding.
  41. 41. results ! 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 CE CS IS SE IT IS/IT CS NM disciplineclusterscores Since each disciple was associated with specific tasks, these scores were combined and averaged to create Discipline Cluster Scores.
  42. 42. ! 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 CE CS IS SE IT IS/IT CS NM Due to the high uncertainty percentages discussed earlier, we also compared the cluster scores with the non- responses removed.
  43. 43. The data was subjected to a one-way analysis of variance that revealed only three significant differences between groups. ! The IS/IT students made significantly better matches on three of the cluster scores compared to their non-major peers – for CS, IS and IT tasks.
  44. 44. Since Task 24 was not associated with any of the computer disciplines under study, all Task 24 answers were combined and averaged for each program group. ! resultsnon-computingtaskperformance IS/IT CS Non-Majors Task 24 Average Scores 2.08 2.10 1.89 When analyzed using a one-way analysis of variance, no significant differences between groups were discovered (in task 24 performance). Results showed that Task 24 responses were significantly lower than those for the other tasks.
  45. 45. Discussion
  46. 46. By focusing on students who were taking an introductory computing course, we tried to provide insight into whether students who enroll in computing programs have a clear understanding of the disciplinary outcomes of the respective programs. ! discussion
  47. 47. Based on our rank-order analysis the best matching occurred for tasks that related to CE, followed by IS (78%), CS (60%), IT (50%) and SE (38%). ! These results are reasonably close to those encountered by the C&BC study.
  48. 48. Results showed statistically significant differences between the three program of study groups (IS/IT, CS, and NM) on 19 of the 31 items. !
  49. 49. Non-major students tended to rank tasks as having a lower fit with each discipline. ! IS/IT students tended to rank tasks as having a higher fit with each discipline.
  50. 50. At best this means the IS/IT students were more likely to be correct in their matchings. ! At worse it might suggest a response bias where IS/IT students were more likely to assume that there is a higher fit for all tasks. However, if there had been a response bias in place, we would have seen similar scores associated with Task 24, and this was actually not the case.
  51. 51. Like the earlier C&BC study, our results show that students are not always clear about the disciplinary “fit” of different computing tasks. ! However, by allowing students to specify a degree of disciplinary fit, our study showed that by and large students are able to get their discipline matches close despite being inexperienced with computing.
  52. 52. This could be construed as a more encouraging result than that reported in C&BC. !
  53. 53. Another important result was the lower likelihood that students would correctly identify the IS, SE, and IT tasks. ! also This highlights how important it is for faculty in these fields to better articulate what these fields encompass, and to better communicate this information to prospective and current students alike.
  54. 54. Our data showed that the most confusion about what discipline a task belonged to were those tasks connected to real-world tasks. ! also Uses programming skills to create or modify business solutions Manages large scale technological projects Develops or maintains web sites Evaluates and improves the usability (user experience) of computing systems Given that these larger projects often involve a variety of different skills and abilities, this uncertainty could even be construed as a positive sign.
  55. 55. There is a rain cloud in this sunny picture The very significant percentages of task and discipline uncertainty across all five sub- disciplines does indicate that all three student groups (IS/IT,CS,NM) have large gaps in their knowledge about the disciplines. ! however
  56. 56. ! thus This ignorance was likely masked in the C&BC approach since it did not provide an option for specifying uncertainty. Designs hardware to implement communication systems CE CS IS IT SE ⃝ ⃝ ⃝ ⃝ ⃝ versus
  57. 57. limitations
  58. 58. The main limitation of our study is similar to that of the C&BC study that inspired it: namely, if the task descriptions were too clear or too vague, then this would compromise the statistics and any conclusions drawn from them. ! limitations As well, the five disciplines did not have the same number of tasks for which the discipline was the best fit.
  59. 59. The other key limitation of our study is that we did not have any CE or SE students in our study due to our university not having a CE or SE program. ! limitations This limitation could conceivably be addressed in the future if data was obtained from universities that have a CE and SE programs.
  60. 60. conclusion
  61. 61. Over the last two decades, computing has undergone a reasonable level of differentiation into five sub- disciplines. ! conclusion This has generated some ambiguity about the computing subdisciplines which resides in the minds of students, faculty, and even employers. Other disciplines (such as engineering) also grapple with task/skill understanding by students and educators.
  62. 62. Our study adds to the literature on disciplinary task/skill identification via the enhancement of the C&BC instrument … ! conclusion by identifying an additional 15 skills from the ACM sub-discipline descriptions and by allowing participants to specify a degree of disciplinary fit.
  63. 63. Our results show that students are not always clear about the disciplinary “fit” of different computing tasks … ! But, when students provided an opinion about fit, major and non-major students alike were actually often close to correctly identifying the correct discipline. This result is a new finding and a by-product of our revised survey design.
  64. 64. Our study also showed that IS/IT students had a better task understanding that those enrolled in the computer science program. ! finally

×