Analysis of Student Feedback¶

Analysis of Student Feedback 2019¶

1. 60% of respondents at NIT Calicut felt that their background was sufficient as compared to 31% in 2018. This is due to the following: A few theory lectures were arranged in the beginning of the semester to discuss background material in computer organisation (paging and address translation, interrupts and exceptions and other features of XSM which are significantly different from the MIPS architecture). Moreover, one senior student volunteer who had done the project in the previous was assigned to mentor each student doing the project in the present year. These steps helped to solve the difficulties substantially. We conclude that conducting a few theory lectures on topics in computer organization that are relevant to the project at the beginning of the course can bring about substantial improvements in student performance.
2. There was a significant increase in the percentage of students who reported that the project brought about substantial improvements to their coding skills (from 24% to 46%). On discussions with students to analyze this feedback, the following was observed. We had made substantial improvements to the XSM debugging environment (which is more or less similar to the GDB environment) before the laboratory was run in 2019. We had also insisted that the students be familiar with the debugging environment at an early stage of the project. Many students reported that the project helped them in improving their code debugging skills considerably; and this resulted in many students reporting that the project improved their coding skills in the feedback form.
3. A significant drop in the percentage of students completing Phase 5 (file system implementation) was observed (from 76% in 2018 to 48% in 2019). Upon checking the facts which lead to this drop, we found that a significant percentage among the 84% students who had completed Phase 4 was working on the file system implementation, and the feedback was collected before they could complete the file system implementation. Most of these students had completed subtaintial parts of the file system. Hence, we still expect that 60-75% of the class to successfully complete (the whole or significant parts of) the file system implementation phase when the course is conducted in future as well.

Analysis of Student Feedback 2018¶

1. Almost every respondent reported that they could complete Phase 3 (process management) in both the courses (100% at NIT Calicut and 95% at IIT Palakkad).
2. 42% of the students at IIT Palakkad and 50% of the students at NIT Calicut noted that their background in computer organization was insufficient.
   Interaction with students seem to suggest the following issue. Computer organization typically are based on the MIPS architecture and tend to stress more on the fetch execute cycle, pipelining, cacheing and other architectural design level issues, but lays less stress on page table address translation and the semantics of instructions like push, pop, call, ret and interrupt/exceptions that manipulate the stack. However these ideas need to be understood clearly for doing the project.
3. 53% of the respondents at IIT Palakkad and 88% of the respondents at NIT Calicut noted that 90% of the project can be done completely based on the road map.
4. 48% of the respondents at IIT Palakkad (and 76% at NIT Calicut) completed Phase 5 (process management, interprocess communication and file system implementation). 33% among those who completed Phase 5 at IIT Palakkad (respectively 47% at NIT Calicut) completed all the phases of the project.
   The disparity in the effectiveness of the road map and the performance statistics in the courses at NIT Calicut and IIT Palakkad is due to the following. The course at NIT Calicut was an elective and hence only students who had sufficient interest and background to follow the project roadmap had enrolled for the project (64 of a total of 158 CS students (40%)). At IIT Palakkad, the project was part of a compulsory core course.
5. 88% of the respondents at NIT Calicut and 68% of the respondents at IIT Palakkad reported that doing the lab gave a clear advantage in understanding the theory.
   The difference appears to be due to the difference in the instructional methodology adapted for the OS theory course (core in both the institutions). At NIT Calicut, the theory course had a huge (150+) student enrollment and followed a standard theory approach based on Silberschatz et. al., Operating System Concepts. On the other hand, at IIT Palakkad, the class size was small (30+) and the course was based on the OS kernel design approach based on C. Crowley Operating Systems: A Design-Oriented Approach. As the latter text covers many of the design issues encountered during the project, the enforcement that the laboratory gave to the understanding of theory was less dramatic.

6. Whereas 22% of the students at NIT Calicut who completed Stage 5 or above reported that more than 15 hours of work per week was needed for the project, only 11% of the students at IIT Palakkad who completed Stage 5 or above reported so. 67% of the respondents at IIT Palakkad who had completed Stage 5 or above responded that 10-15 hours/week was sufficient whereas only 44% of the respondents at NIT Calicut reported that 10-15 hours/week was sufficient. These differences appear to be linked to the differences in students' prior exposure to programming in the respective institutions.

   Consequently 87% of the respondents at NIT Calicut noted that the project helped them in improving their coding skills, whereas only 37% of the respondents at IIT Palakkad noted that the project improved their coding skills.