2018 IQAP Review of Bachelor of Computer Engineering Programme Final Assessment Report and Implementation Plan
In accordance with the Royal Military College (RMC) Institutional Quality Assurance Process (IQAP), this final assessment report provides a synthesis of the external evaluation and internal response and assessments of the Bachelor of Computer Engineering programme offered by the Department of Electrical and Computer Engineering.
This report identifies the significant strengths of the programme, together with opportunities for programme improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and, timelines for acting on and monitoring the implementation of those recommendations.
Overview of Programme Review Process:
The IQAP programme review process was conducted in concert with the Canadian Engineering Accreditation Board’s. The Programme Self-Study report, finalized in September 2018, consisted of the CEAB-mandated self-study components plus an IQAP supplement which addressed elements of the IQAP process not covered by the CEAB’s requirements. In combination this included the CEAB graduate attributes and IQAP degree-level expectations for these programmes, an analytical assessment of the programmes, course outlines, programme-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members.
The CEAB Programme Visitor, selected by CEAB and approved by the Dean of Engineering (Dr. Dean McNeill, PEng) acted in the role of external reviewer. He was aided in this by the other members of the CEAB visit team who looked at issues of interest across all of RMC’s undergraduate Engineering programmes and whose observations formed part of the CEAB visit report on the programme. The internal reviewer (Dr Imed Zaguia) was selected by the Dean from a list of possible reviewers. The internal reviewer and CEAB team reviewed the self-study documentation and conducted a site visit to RMC on 4–6 November 2018. In the remainder of this document, the CEAB visit team and the internal reviewer are collectively referred to as the External Review Committee (ERC).
At the College level, interviews were conducted by the CEAB Visit Chair, Vice Chairs, and General Visitors. This included meetings with the Commandant, Principal, Vice-Principal (Academic), Vice-Principal (Research) and Dean of Graduate Studies, Dean of Engineering, Dean of Science, Interim Dean of Social Sciences and Humanities, Associate Dean of Engineering (Accreditation and Programme), Chief Librarian, Comptroller, Director of Cadets, Director of the Success Centre, Registrar, Unit General Safety Officer, and the Heads of the Departments of English, Mathematics and Computer Science, and Physics and Space Science.
At the programme level, interviews were conducted jointly by the CEAB programme visitor and the IQAP internal reviewer. This included meetings with the Dean of Engineering, Associate Dean of Engineering (Accreditation and Programme), Department Head, members of the faculty, the Technical Officer and members of the technical staff, the administrative and clerical staff, teaching assistants, and a selection of students from all years of the programme. The visit also included tours of all undergraduate laboratory facilities and representative classroom spaces.
The CEAB sent RMC their Report of the Visiting Team on 24 December 2018 and their Decision Letter on 27 June 2019. These were provided to the IQAP internal reviewer as input to the ERC report, which was finalized by the internal reviewer. The originally-submitted ERC report appears to have been lost in the cyber-attack that RMC suffered in July of 2020. Dr Zaguia was able to locate a copy in his personal archives, which he provided to the Dean of Engineering on 25 January 2022.
The self-study documentation, CEAB report, CEAB decision letter, and ERC report together provide a thorough analysis of the current situation in the programme. In the following sections, observations are taken from both the CEAB and ERC reports. Recommendations are taken from the ERC report, which is specific to IQAP, and the CEAB decision letter which includes recommendations relevant to both CEAB and IQAP processes.
The Dean of Engineering and the Head of the Department of Electrical and Computer Engineering, after consultation with faculty and staff of the programme, produced this Final Assessment Report in May of 2022. The Faculty and the Department look upon this as an important opportunity to better understand the programme’s context and identify opportunities for improvement.
Significant Strengths and Weaknesses of the Programme:
The CEAB report and ERC identified a number of strengths of the Bachelor of Computer Engineering programme:
- Collegiality of the faculty members is excellent. Members commonly share materials when teaching sections in the same term, and when courses are reassigned to other faculty from year-to-year. Faculty members are engaged and enthusiastic about the courses they are offering.
- Laboratory resources are also very good.
- The technical staff are intimately involved in supporting the laboratory instruction and are held in exceptionally high regard by the students and staff.
- The capstone design course is co-taught with students in the Electrical Engineering program and project teams may be made up of students from both programs. This is a benefit as it allows for projects to span both disciplines while allowing Computer Engineering students to focus on the aspects of projects relevant to their own program. At the same time, students gain exposure to technical issues beyond their own area.
The CEAB report and ERC also identified several areas of concern that influence the status and quality of the programme:
- Efforts need to be made to reduce the number of students admitted who have a low probability of success. Increased significance should be attached to high school academic performance when making admittance decisions.
- A change in [the introductory computing course] from Matlab to Python would provide better preparation for programming in later courses and would be more appropriate for an introductory programming language.
- [Students] expressed the view that their Arts courses were of little use and that completing these courses necessitated a waste of the Engineering students precious time. […] The current attitude of dismissal toward these Arts courses needs to be replaced by a more positive view and an understanding of their importance in order that the students develop the skills these courses can provide.
Summary of the CEAB Findings and ERC Recommendations with the Programme’s and Dean’s Responses
The following findings and recommendations are taken from the CEAB decision letter and the ERC report. In some cases they have been reworded for clarity. Suggestions from the CEAB Visit Report are not included since CEAB identifies them as just that – suggestions from an individual visitor – and does not require a response from the institution. In some cases CEAB suggestions became ERC recommendations; those are addressed.
Responses from the Programme and the Dean are provided in italics below each finding or recommendation with a corresponding implementation plan in the next section.
CEAB Findings that apply to all Programmes
The following CEAB Findings, reported in the CEAB decision letter, are common to all Engineering programmes.
CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two.
In the tracking system that RMC used for Graduate Attributes (GA) up to the 2018 CEAB visit, each GA was mapped onto multiple “indicators” which were learning outcomes in specific courses, or occasionally appearing in multiple related courses. Comments provided in our 2012 CEAB Accreditation, when CEAB’s GA programme was still in its initial phase, suggested that this met CEAB’s intent. Since then, CEAB has provided further direction that the tracking system should have three levels: the twelve CEAB GAs, each mapped onto multiple “indicators”, and each of those mapped to multiple “learning outcomes” which are specific to courses. Indicators and learning outcomes are determined by the institution and programme. RMC’s response to this finding is described in CEAB Common Finding 3, below.
CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed.
This is related to CEAB Common 1. Since each indicator was specific to a single course, or at most a small number of courses, there were very few measurements associated with each indicator. CEAB further expect that each GA be Introduced at some point in the program, then Developed (deepened or enhanced) and then finally Applied. While RMC’s programmes do generally follow this pattern, the structure of RMC’s GA mapping did not make this clear. See the response in CEAB Common 3.
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated.
CEAB mandates a Continual Improvement (CI) process, to which the GAs are one input. Other inputs are expected to come from both internal and external stakeholders. While the RMC Engineering programmes do receive significant input from external stakeholders, this has been neither systematic nor well documented.
Programme response. The Electrical and Computer Engineering programme has been fully involved in the development of the new GA and CI processes and is committed to implementing them in conjunction with the rest of the Faculty.
Dean’s response. In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes, chaired by the Associate Dean (Accreditation and Programme). The committee’s mandate was to recommend changes to Engineering programmes’ GA and CI practices and processes, with the aims of addressing CEAB’s concerns, improving the processes’ value to RMC as an institution, and keeping the administrative burden of the processes as lightweight as possible. The committee’s work was delayed by the advent of the pandemic, then further by the cyber-attack that struck RMC’s computer networks in July of 2020. However, the committee’s recommendations are now being provided as a Faculty-wide directive on GA and CI. The intent is that the new GA and CI processes will be in place for the fall semester of 2022.
CEAB Findings that apply to the Computer Engineering Programme
The following CEAB Findings, reported in the CEAB decision letter, are specific to the Computer Engineering Programme.
CEAB Computer Finding 1. For some attributes, there is only a single indicator assessed. As noted in criteria 3.1.2, while it is present within the curriculum, Equity is not mapped and is therefore not assessed
Programme response. This is related to CEAB Common Finding 2 and will be addressed in the changes to the GA mappings that will occur in the context of the response described under CEAB Common Finding 3. We acknowledge the lack of equity mapping which is an error in the programme’s GA tables.
Dean’s response. I concur with the programme’s response.
ERC Recommendations
The following recommendations appear in the ERC report.
ERC Recommendation 1. A change in [the introductory computing course] from Matlab to Python would provide a better preparation for later programming courses and would more appropriate for a programming language.
Programme response. We concur with this recommendation and have had ongoing discussions with the Department of Mathematics and Computer Science requesting this change for over a decade.
Dean’s response. The introductory computing course is common to all students in Science and Engineering, so a change in the programming language used requires agreement across all programmes in both Faculties. Some other programmes, particularly Mechanical and Aerospace Engineering, claim to rely on Matlab knowledge in second-year courses. This tension has made a change in languages difficult. However students who complete first year at Royal Military College Saint-Jean do take their introductory programming course in Python and appear to have no difficulty with the courses in the Mechanical and Aerospace Engineering programmes. This suggests that a change may be possible.
ERC Recommendation 2. [Students] expressed the view that their Arts courses were of little use and that completing these courses necessitated a waste of the Engineering students precious time. […] The current attitude of dismissal toward these Arts courses needs to be replaced by a more positive view and an understanding of their importance in order that the students develop the skills these courses can provide.
Programme response. There is little the programme can do to change the students’ attitude toward their Arts courses, as these are offered by other Departments.
Dean’s response. The Arts courses taken by Engineering students are part of RMC’s Common Curriculum, which respond to RMC’s status as a university of the Canadian Armed Forces. Recommendation 9 of the Final Report of the Core Curriculum Committee to Faculty Board, Royal Military College, July 2020 was that “Science and Engineering students who successfully complete their degree programme should be awarded a Minor in Arts as part of that degree.” This was implemented as a Minor in Military Arts, available to all Science and Engineering students effective with the graduating class of 2022. The Minor requires no additional work beyond the 9.5 credits in Arts which form an integral component of all Engineering programmes. We believe that this formal recognition will help the students have a more positive view of their Arts courses.
Implementation Plan
| Recommendation | Proposed follow-up and resource implications | Responsibility for leading follow-up | Timeline for addressing recommendation |
|---|---|---|---|
| CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two. | See CEAB Common Finding 3 | ||
| CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed. | See CEAB Common Finding 3 | ||
|
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated. |
In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes. The committee’s recommendations are being provided as a Faculty-wide directive on GA and CI. Implementation of the changes will impose significant additional work on all Engineering Departments as well as the Dean’s office, both during the transition and on an ongoing basis. It may be appropriate to re-create the position of Associate Dean (Accreditation and Programme) on an ongoing basis to provide oversight and support. |
Dean of Engineering |
Directive on GA and CI to be released June 2022. New processes to be implemented starting in September 2022. Review of process outcomes and revision of processes in June 2023. |
| CEAB Computer Finding 1. For some attributes, there is only a single indicator assessed. As noted in criteria 3.1.2, while it is present within the curriculum, Equity is not mapped and is therefore not assessed | See CEAB Common Finding 3. | ||
| ERC Recommendation 1. A change in [the introductory computing course] from Matlab to Python would provide a better preparation for later programming courses and would more appropriate for a programming language. | We will re-initiate discussion with the Department of Mathematics and Computer Science about a change in programming language for the introductory computing course. | Dean of Engineering | Discussion in 2022–2023 with an intent to make the change effective the fall of 2023. |
| ERC Recommendation 2. [Students] expressed the view that their Arts courses were of little use and that completing these courses necessitated a waste of the Engineering students precious time. […] The current attitude of dismissal toward these Arts courses needs to be replaced by a more positive view and an understanding of their importance in order that the students develop the skills these courses can provide. | This has been addressed with the introduction of the Minor in Military Arts, effective with the graduating class of 2022. |
Complete. |
Conclusion
The External Review Committee, consisting of the CEAB visit team and the internal reviewer, provided a thorough and fair review of the Bachelor of Engineering in Computer Engineering programme. Activities are underway to address the ERC report’s recommendations. CEAB-specific recommendations are also being addressed in the context of ongoing CEAB accreditation activities.