2018 IQAP Review of Bachelor of Electrical Engineering Programme Final Assessment Report and Implementation Plan
In accordance with the Royal Military College (RMC) Institutional Quality Assurance Process (IQAP), this final assessment report provides a synthesis of the external evaluation and internal response and assessments of the Bachelor of Electrical Engineering programme offered by the Department of Electrical and Computer Engineering.
This report identifies the significant strengths of the programme, together with opportunities for programme improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and, timelines for acting on and monitoring the implementation of those recommendations.
Overview of Programme Review Process:
The IQAP programme review process was conducted in concert with the Canadian Engineering Accreditation Board’s. The Programme Self-Study report, finalized in September 2018, consisted of the CEAB-mandated self-study components plus an IQAP supplement which addressed elements of the IQAP process not covered by the CEAB’s requirements. In combination this included the CEAB graduate attributes and IQAP degree-level expectations for these programmes, an analytical assessment of the programmes, course outlines, programme-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members.
The CEAB Programme Visitor, selected by CEAB and approved by the Dean of Engineering (Dr Diane Kennedy, PEng) acted in the role of external reviewer. She was aided in this by the other members of the CEAB visit team who looked at issues of interest across all of RMC’s undergraduate Engineering programmes and whose observations formed part of the CEAB visit report on the programme. The internal reviewer (Dr L Sangalli) was selected by the Dean from a list of possible reviewers. The internal reviewer and CEAB team reviewed the self-study documentation and conducted a site visit to RMC on 4–6 November 2018. In the remainder of this document, the CEAB visit team and the internal reviewer are collectively referred to as the External Review Committee (ERC).
At the College level, interviews were conducted by the CEAB Visit Chair, Vice Chairs, and General Visitors. This included meetings with the Commandant, Principal, Vice-Principal (Academic), Vice-Principal (Research) and Dean of Graduate Studies, Dean of Engineering, Dean of Science, Interim Dean of Social Sciences and Humanities, Associate Dean of Engineering (Accreditation and Programme), Chief Librarian, Comptroller, Director of Cadets, Director of the Success Centre, Registrar, Unit General Safety Officer, and the Heads of the Departments of English, Mathematics and Computer Science, and Physics and Space Science.
At the programme level, interviews were conducted jointly by the CEAB programme visitor and the IQAP internal reviewer. This included meetings with the Dean of Engineering, Associate Dean of Engineering (Accreditation and Programme), Department Head, members of the faculty, the Technical Officer and members of the technical staff, the administrative and clerical staff, teaching assistants, and a selection of students from all years of the programme. The visit also included tours of all undergraduate laboratory facilities and representative classroom spaces.
The CEAB sent RMC their Report of the Visiting Team on 24 December 2018 and their Decision Letter on 27 June 2019. These were provided to the IQAP internal reviewer as input to the ERC report, which was finalized by the internal reviewer. The originally-submitted ERC report appears to have been lost in the cyber-attack that RMC suffered in July of 2020. Dr Sangalli was able to locate a draft of the report and their visit notes in their personal archives, which permitted them to recreate the ERC report with all findings. This was provided to the Dean of Engineering on 28 March 2022.
The self-study documentation, CEAB report, CEAB decision letter, and ERC report together provide a thorough analysis of the current situation in the programme. In the following sections, observations are taken from both the CEAB and ERC reports. Recommendations are taken from the ERC report, which is specific to IQAP, and the CEAB decision letter which includes recommendations relevant to both CEAB and IQAP processes.
The Dean of Engineering and the Head of the Department of Electrical and Computer Engineering, after consultation with faculty and staff of the programme, produced this Final Assessment Report in May of 2022. The Faculty and the Department look upon this as an important opportunity to better understand the programme’s context and identify opportunities for improvement.
Significant Strengths and Weaknesses of the Programme:
The CEAB report and ERC identified a number of strengths of the Bachelor of Computer Engineering programme:
- Faculty balance teaching, research, committee work, volunteer work. The general sentiment is that that workload is acceptable.
- Faculty work collegially and willingly share course materials with new instructors and instructors who take over the course when away on sabbatical leave.
- Faculty actively engage with students both in lectures and labs.
- The labs are clean, safe, and well-equipped.
- The technicians do an excellent job of maintaining the labs and supporting the students and faculty.
The CEAB report and ERC also identified one area of concern that influences the status and quality of the programme:
- Students pointed out that the lack of application in their math courses impeded their success in the program. This echoes the sentiments of the faculty who find that mathematics proficiency of the students is lacking.
Summary of the CEAB Findings and ERC Recommendations with the Programme’s and Dean’s Responses
The ERC report does not include any recommendations beyond those from CEAB.
The following findings and recommendations are taken from the CEAB decision letter. In some cases recommendations have been reworded for clarity. Suggestions from the CEAB Visit Report are not included since CEAB identifies them as just that – suggestions from an individual visitor – and does not require a response from the institution.
Responses from the Programme and the Dean are provided in italics below each finding or recommendation with a corresponding implementation plan in the next section.
CEAB Findings that apply to all Programmes
The following CEAB Findings, reported in the CEAB decision letter, are common to all Engineering programmes.
CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two.
In the tracking system that RMC used for Graduate Attributes (GA) up to the 2018 CEAB visit, each GA was mapped onto multiple “indicators” which were learning outcomes in specific courses, or occasionally appearing in multiple related courses. Comments provided in our 2012 CEAB Accreditation, when CEAB’s GA programme was still in its initial phase, suggested that this met CEAB’s intent. Since then, CEAB has provided further direction that the tracking system should have three levels: the twelve CEAB GAs, each mapped onto multiple “indicators”, and each of those mapped to multiple “learning outcomes” which are specific to courses. Indicators and learning outcomes are determined by the institution and programme. RMC’s response to this finding is described in CEAB Common Finding 3, below.
CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed.
This is related to CEAB Common 1. Since each indicator was specific to a single course, or at most a small number of courses, there were very few measurements associated with each indicator. CEAB further expect that each GA be Introduced at some point in the program, then Developed (deepened or enhanced) and then finally Applied. While RMC’s programmes do generally follow this pattern, the structure of RMC’s GA mapping did not make this clear. See the response in CEAB Common 3.
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated.
CEAB mandates a Continual Improvement (CI) process, to which the GAs are one input. Other inputs are expected to come from both internal and external stakeholders. While the RMC Engineering programmes do receive significant input from external stakeholders, this has been neither systematic nor well documented.
Programme response. The Electrical and Computer Engineering programme has been fully involved in the development of the new GA and CI processes and is committed to implementing them in conjunction with the rest of the Faculty.
Dean’s response. In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes, chaired by the Associate Dean (Accreditation and Programme). The committee’s mandate was to recommend changes to Engineering programmes’ GA and CI practices and processes, with the aims of addressing CEAB’s concerns, improving the processes’ value to RMC as an institution, and keeping the administrative burden of the processes as lightweight as possible. The committee’s work was delayed by the advent of the pandemic, then further by the cyber-attack that struck RMC’s computer networks in July of 2020. However, the committee’s recommendations are now being provided as a Faculty-wide directive on GA and CI. The intent is that the new GA and CI processes will be in place for the fall semester of 2022.
CEAB Findings that apply to the Electrical Engineering Programme
The following CEAB Findings, reported in the CEAB decision letter, are specific to the Electrical Engineering Programme.
CEAB Electrical Finding 1. It is not possible to conclude whether graduates do/do not possess the attribute based on the collected data.
Programme response. This is related to CEAB Common Finding 2 and is addressed by the plan described under CEAB Common Finding 3.
Dean’s response. I concur with the Programme response.
ERC Recommendations
The following recommendations appear in the ERC report.
ERC Recommendation 1. Ensuring the professional licensing of the military faculty. This should be achieved by encouraging and supporting military faculty who seek licensure.
Programme response. As of May 2021 the Canadian Armed Forces provides reimbursement of PEng licensing expenses to military faculty members in the Faculty of Engineering at RMC.
Dean’s response. I concur with the Programme response. As a standard practice, the Dean counsels all arriving military faculty on the importance of licensure to the quality and standing of our programme. With the reimbursement of licensing expenses now in place, there is a significantly greater willingness among the military faculty members to begin the licensing process.
Implementation Plan
| Recommendation | Proposed follow-up and resource implications | Responsibility for leading follow-up | Timeline for addressing recommendation |
|---|---|---|---|
| CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two. | See CEAB Common Finding 3 | ||
| CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed. | See CEAB Common Finding 3 | ||
|
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated. |
In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes. The committee’s recommendations are being provided as a Faculty-wide directive on GA and CI. Implementation of the changes will impose significant additional work on all Engineering Departments as well as the Dean’s office, both during the transition and on an ongoing basis. It may be appropriate to re-create the position of Associate Dean (Accreditation and Programme) on an ongoing basis to provide oversight and support. |
Dean of Engineering |
Directive on GA and CI to be released June 2022. New processes to be implemented starting in September 2022. Review of process outcomes and revision of processes in June 2023. |
| CEAB Electrical Finding 1. It is not possible to conclude whether graduates do/do not possess the attribute based on the collected data. | See CEAB Common Finding 3. | ||
|
ERC Recommendation 1. Ensuring the professional licensing of the military faculty |
As of May 2021 the Canadian Armed Forces provides reimbursement of PEng licensing expenses to military faculty members in the Faculty of Engineering at RMC. | Dean of Engineering | Complete |
Conclusion
The External Review Committee, consisting of the CEAB visit team and the internal reviewer, provided a thorough and fair review of the Bachelor of Engineering in Electrical Engineering programme. Activities are underway to address the ERC report’s recommendations. CEAB-specific recommendations are also being addressed in the context of ongoing CEAB accreditation activities.