2018 IQAP Review of Bachelor of Civil Engineering Programme
Final Assessment Report and Implementation Plan
In accordance with the Royal Military College (RMC) Institutional Quality Assurance Process (IQAP), this final assessment report provides a synthesis of the external evaluation and internal response and assessments of the Bachelor of Civil Engineering programme offered by the Department of Civil Engineering.
This report identifies the significant strengths of the programme, together with opportunities for programme improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and, timelines for acting on and monitoring the implementation of those recommendations.
Overview of Programme Review Process:
The IQAP programme review process was conducted in concert with the Canadian Engineering Accreditation Board’s. The Programme Self-Study report, finalized in September 2018, consisted of the CEAB-mandated self-study components plus an IQAP supplement which addressed elements of the IQAP process not covered by the CEAB’s requirements. In combination this included the CEAB graduate attributes and IQAP degree-level expectations for these programmes, an analytical assessment of the programmes, course outlines, programme-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members.
The CEAB Programme Visitor, selected by CEAB and approved by the Dean of Engineering (Ms Emily Cheung, FEC, PEng) acted in the role of external reviewer. She was aided in this by the other members of the CEAB visit team who looked at issues of interest across all of RMC’s undergraduate Engineering programmes and whose observations formed part of the CEAB visit report on the programme. The internal reviewer (Dr Philippe Constantineau) was selected by the Dean from a list of possible reviewers. The internal reviewer and CEAB team reviewed the self-study documentation and conducted a site visit to RMC on 4–6 November 2018. In the remainder of this document, the CEAB visit team and the internal reviewer are collectively referred to as the External Review Committee (ERC).
At the College level, interviews were conducted by the CEAB Visit Chair, Vice Chairs, and General Visitors. This included meetings with the Commandant, Principal, Vice-Principal (Academic), Vice-Principal (Research) and Dean of Graduate Studies, Dean of Engineering, Dean of Science, Interim Dean of Social Sciences and Humanities, Associate Dean of Engineering (Accreditation and Programme), Chief Librarian, Comptroller, Director of Cadets, Director of the Success Centre, Registrar, Unit General Safety Officer, and the Heads of the Departments of English, Mathematics and Computer Science, and Physics and Space Science.
At the programme level, interviews were conducted jointly by the CEAB programme visitor and the IQAP internal reviewer. This included meetings with the Dean of Engineering, Associate Dean of Engineering (Accreditation and Programme), Department Head, members of the faculty, the Technical Officer and members of the technical staff, the administrative and clerical staff, teaching assistants, and a selection of students from all years of the programme. The visit also included tours of all undergraduate laboratory facilities and representative classroom spaces.
The CEAB sent RMC their Report of the Visiting Team on 24 December 2018 and their Decision Letter on 27 June 2019. These were provided to the IQAP internal reviewer as input to the ERC report, which was finalized by the internal reviewer. The ERC report was submitted to the Dean of Engineering on 15 July 2019.
The self-study documentation, CEAB report, CEAB decision letter, and ERC report together provide a thorough analysis of the current situation in the programme. In the following sections, observations are taken from both the CEAB and ERC reports. Recommendations are taken from the ERC report, which is specific to IQAP, and the CEAB decision letter which includes recommendations relevant to both CEAB and IQAP processes.
The Dean of Engineering and the Head of the Department of Civil Engineering, after consultation with faculty and staff of the programme, produced this Final Assessment Report in May of 2022. The Faculty and the Department look upon this as an important opportunity to better understand the programme’s context and identify opportunities for improvement.
Significant Strengths and Weaknesses of the Programme:
The CEAB report and ERC identified a number of strengths of the Bachelor of Civil Engineering programme:
- The programme offers a wide variety of experience, both educational and field experience, for the Civil Engineering students.
- The integration of military, physical, and educational experiences result in diverse graduates.
- Students speak highly of their experience and opportunities at RMC.
- The faculty are very collaborative and appreciate the leadership of the department including the open-door policy.
- The labs are well equipped, well managed and appreciated by all of the students as well as faculty.
- The Survey Field School (CEE/GCF363), Civil Engineering Field School (CEE/GCF393) and Civil Engineering Project (CEE/GCF493) courses are seen as particularly innovative and effective.
The CEAB report and ERC also identified several areas of concern that influence the status and quality of the programme:
- For courses in which the final mark is determined by instructors who are not also the supervisors of the course work, the process of mark submission is not documented and not universally perceived as transparent.
- The procedure of course allocation was not universally perceived to be transparent.
- Allocation of courses with significant engineering design components to instructors with Professional Engineering licenses needs to be more systematic.
- Full-time faculty appears to be sufficient for the number of students, but barely to sustain the program in both languages. The unavailability of French-speaking faculty, complicated a by a lengthy and strenuous hiring process has resulted in the cancellation of some key courses in French, resulting in the disaffection of some French-speaking students for the program.
Summary of the CEAB Findings and ERC Recommendations with the Programme’s and Dean’s Responses
The following findings and recommendations are taken from the CEAB decision letter and the ERC report. In some cases they have been reworded for clarity. Suggestions from the CEAB Visit Report are not included since CEAB identifies them as just that – suggestions from an individual visitor – and does not require a response from the institution. In some cases CEAB suggestions became ERC recommendations; those are addressed.
Responses from the Programme and the Dean are provided in italics below each finding or recommendation with a corresponding implementation plan in the next section.
CEAB Findings that apply to all Programmes
The following CEAB Findings, reported in the CEAB decision letter, are common to all Engineering programmes.
CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two.
In the tracking system that RMC used for Graduate Attributes (GA) up to the 2018 CEAB visit, each GA was mapped onto multiple “indicators” which were learning outcomes in specific courses, or occasionally appearing in multiple related courses. Comments provided in our 2012 CEAB Accreditation, when CEAB’s GA programme was still in its initial phase, suggested that this met CEAB’s intent. Since then, CEAB has provided further direction that the tracking system should have three levels: the twelve CEAB GAs, each mapped onto multiple “indicators”, and each of those mapped to multiple “learning outcomes” which are specific to courses. Indicators and learning outcomes are determined by the institution and programme. RMC’s response to this finding is described in CEAB Common Finding 3, below.
CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed.
This is related to CEAB Common 1. Since each indicator was specific to a single course, or at most a small number of courses, there were very few measurements associated with each indicator. CEAB further expect that each GA be Introduced at some point in the program, then Developed (deepened or enhanced) and then finally Applied. While RMC’s programmes do generally follow this pattern, the structure of RMC’s GA mapping did not make this clear. See the response in CEAB Common 3.
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated.
CEAB mandates a Continual Improvement (CI) process, to which the GAs are one input. Other inputs are expected to come from both internal and external stakeholders. While the RMC Engineering programmes do receive significant input from external stakeholders, this has been neither systematic nor well documented.
Programme response. The Civil Engineering programme has been fully involved in the development of the new GA and CI processes and is committed to implementing them in conjunction with the rest of the Faculty.
Dean’s response. In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes, chaired by the Associate Dean (Accreditation and Programme). The committee’s mandate was to recommend changes to Engineering programmes’ GA and CI practices and processes, with the aims of addressing CEAB’s concerns, improving the processes’ value to RMC as an institution, and keeping the administrative burden of the processes as lightweight as possible. The committee’s work was delayed by the advent of the pandemic, then further by the cyber-attack that struck RMC’s computer networks in July of 2020. However, the committee’s recommendations are now being provided as a Faculty-wide directive on GA and CI. The intent is that the new GA and CI processes will be in place for the fall semester of 2022.
CEAB Findings that apply to the Civil Engineering Programme
The following CEAB Findings, reported in the CEAB decision letter, are specific to the Civil Engineering Programme.
CEAB Civil Finding 1. The actions [taken to improve the programme] were a result of curriculum feedback unrelated to GAs and the CI process. A report was not completed, nor information provided. Results are presented with limited assessment of the results.
Programme response. Curriculum reviews conducted since the CEAB visit have been conducted within the CI process, have made use of information derived from the GAs, and have been appropriately documented. The Department is committed to continuing this under the new GA and CI process explained under CEAB Common Finding 3.
Dean’s response. I concur that the Department has adequately addressed this under the current GA and CI processes, and that this will be addressed further in the context of CEAB Common Finding 3.
CEAB Civil Finding 2. The process of mark submission is not documented and may not be perceived as transparent.
Programme response. As explained in the ERC report, “This concern is related to the mark submission process in courses where the final mark is determined by instructors who are not also the supervisors of some of the course work.” This includes CEE/GCF393 (Civil Engineering Field School) and CEE/GCF493 (Civil Engineering Project), both of which have components of the evaluation performed by instructors who are not the primary supervisors. The Department has since discussed the mark submission process at multiple departmental meetings and documented it for each case. The Department considers this finding resolved.
Dean’s response. This concern was raised by a single faculty member during the CEAB/IQAP visit and does not appear to have been widely shared among the faculty members. Nevertheless, the Department has taken active steps to address and the faculty member who raised the issue has stated they are satisfied with the outcome. I concur that this finding is resolved.
CEAB Civil Finding 3a. The procedure of course allocation to faculty is not documented, nor open or transparent.
Programme response. The process of course allocation to faculty in the Department has been made fully open and transparent in the years following the CEAB visit. The Department considers this finding resolved.
Dean’s response. I have reviewed the course allocation process in the Department and am satisfied that it is now adequately documented, open, and transparent. I concur that this finding is resolved.
CEAB Civil Finding 3b. Appropriate allocation [of Engineering Design] to [licensed Professional Engineers (PEng)] requires additional attention to ensure licensed engineers are teaching and given authority particularly in design related courses.
Programme response. As demonstrated by the program’s 2018 CEAB Specified Academic Unit (specified AU) assessment, the CEAB requirement that a minimum amount of Engineering Design content be taught by licensed PEng has been met. However, the Department acknowledges that there is essentially no surplus. As part of the teaching assignment process, the Department is ensuring that teaching assignments for courses with significant Engineering Design content are directed to instructors holding PEng licenses wherever possible. The Department now analyzes specified AU counts each year in light of teaching assignments, to ensure that that minimum thresholds are at least met and exceeded where possible. This is frequently challenging, particularly when sessional and term instructors must be hired. It is even more challenging for courses taught in French, as the pool of French-speaking Civil Engineers with PhDs and PEng status is vanishingly small in the Kingston area. A further challenge is the annual turnover of the Department’s military faculty members, few of whom arrive at RMC with PEng licenses.
Dean’s response. I concur with the points raised by the Programme. Additionally, as of May 2021 the Canadian Armed Forces provides reimbursement of PEng licensing expenses to military faculty members in the Faculty of Engineering at RMC. We anticipate that this will result in more fully-licensed faculty members organic to the Department, which should relieve some of the pressure on CEAB specified AUs.
CEAB Civil Finding 4. Courses are not fully offered for francophone students.
Programme response. As a matter of policy, all mandatory courses are offered in both French and English, and elective courses are offered in sufficient numbers in each language to ensure that francophone students are able to complete their programme of study entirely in French. In a few cases, francophone students have been required to take elective courses in English, normally due to unexpected last-minute staffing shortfalls. The Department currently has three vacant faculty positions, all of which will be staffed by new hires capable of teaching in French. The hiring process is currently underway. This will improve the Department’s ability to offer all courses in French in the future, including changes in response to unexpected circumstances.
Dean’s response. The inequities occasionally faced by francophone students in this program and elsewhere remain an ongoing concern. This will remain a significant risk to the Civil Engineering programme at least until the new hires are in place.
CEAB Civil Finding 5. Reaction to hiring academic staff and support staff is constrained by the hiring process. Funds are available but the process to hire and retain qualified staff including francophone staff is not efficient.
Programme response. Since the CEAB/IQAP visit, RMC has received permission from National Defence Human Resources to adopt a less cumbersome hiring process for permanent faculty members. Our initial experience has been positive. Civil Engineering has yet to complete a hire through the new process so it is difficult to say how much of an improvement it will be. The hiring process has been improved, but it is still quite constraining (heavy bureaucratic burden) and labour-intensive. The fundamental difficulty of attracting highly qualified francophone faculty and staff to a predominantly anglophone city is an ongoing concern for all programmes at RMC and one that is unlikely to ever be resolved. However, carefully-targeted recruiting under the new hiring process may be of some assistance.
Dean’s response. Other programmes have now completed hires under the new process, which has proven to be significantly more nimble and makes RMC look more like a civilian university, from the perspective of faculty candidates. I consider the finding regarding the inefficient hiring process to have been resolved.
ERC Recommendations
The following recommendations appear in the ERC report. In some cases they echo CEAB findings already addressed.
ERC Recommendation 1. The comprehensive curriculum review planned to be conducted by the Department in 2019 needs to be fully documented and justified with reference to the Graduate Attributes.
Programme response. This is related to CEAB Civil Finding 1. The planned curriculum review was undertaken in 2021 and was both fully documented and justified in accordance with information captured in the Graduate Attributes. This is an ongoing process. Future curriculum reviews will follow a similar approach. This recommendation is considered addressed.
Dean’s response. I have reviewed the now-completed 2021 curriculum review and am satisfied that this recommendation has been addressed.
ERC Recommendation 2. The department needs to use the full range of quality indicators to measure the quality of its program. The absence of an alumni survey was flagged as a deficiency. Other stakeholders, as in the field courses and engineering projects, should also be systematically surveyed.
Response. See the Programme and Dean’s responses to CEAB Common Finding 3.
ERC Recommendation 3. Though the mark submission process for courses in which the final mark is determined by instructors who are not also the supervisors of the course work was clear to all but one faculty member, it needs to be documented for it to be clear to all members, and indeed also to external reviewers who can relate this perceived weakness to a quality criterion of an engineering program.
Response. See the Programme and Dean’s responses to CEAB Civil Finding 2.
ERC Recommendation 4. Until all members of faculty, including the military, have their Professional Engineering Licenses, there will a high degree of risk that the accreditation of the program will be withdrawn, or else that Military Faculty without a license will have to be demoted to “teaching assistants.”
Programme response. This recommendation is factually incorrect. CEAB does not require that all members of the faculty hold the PEng designation and there is not a “risk that the accreditation of the program will be withdrawn” on that basis. There is also no chance that “Military Faculty without a license will have to be demoted to ‘teaching assistants.’” To the extent this recommendation is correct, it is addressed under CEAB Civil Finding 3.b.
Dean’s response. I concur with the programme response and believe this is fully addressed under CEAB Civil Finding 3.b.
ERC Recommendation 5. Knowledge of international relations is a Degree Level Expectation (DLE) for engineering programs, but the course identified by the programme as meeting this learning objective is a basic course on Canadian History. Either the DLE should be removed or a course in International Relations, such as POE/POF116 Introduction to International Relations, should be added to the programme.
Programme response. The Department acknowledges that the current programme does not meet the international relations DLE; however, we do not believe there is space in the programme to provide the suggested course without adversely impacting either CEAB AU requirements or other DLE’s.
Dean’s response. This is an issue for all Engineering programmes and was raised as recommendation 3 of the Final Report of the Core Curriculum Committee to Faculty Board, Royal Military College, July 2020. The recommendation reads “The Committee recommends that the College find a way to provide Engineering students with exposure to international relations.” In response to this, the RMC Senate directed the Dean of Social Sciences and Humanities and the Dean of Engineering to jointly raise an ad hoc committee to investigate. The committee’s final report, released 1 November 2021, explored twelve alternatives but was unable to reach consensus on a viable approach. The committee’s report will be presented to Senate on 8 June 2022 for discussion.
Implementation Plan
| Recommendation | Proposed follow-up and resource implications | Responsibility for leading follow-up | Timeline for addressing recommendation |
|---|---|---|---|
| CEAB Common Finding 1. Learning outcomes and graduate attribute indicators used interchangeably. There appeared to be some confusion between the differences between the two. | See CEAB Common Finding 3 | ||
| CEAB Common Finding 2. Insufficient indicators were used. In some cases, there was only a single measurement for an indicator or there was reliance on a single course and/or the indicator. For some graduate attributes not all indicators were used. Often, Introduced and Applied were identified, but no Developed. | See CEAB Common Finding 3 | ||
|
CEAB Common Finding 3. Stakeholder engagement is limited mostly to internal representation. A broader set of external stakeholders including alumni (still in military and ex-military) should be engaged. Also, stakeholder roles in the improvement process is inadequately demonstrated. |
In response to CEAB Common Findings 1, 2 and 3, as well as certain programme-specific findings, the Dean of Engineering convened a committee with representatives from all programmes. The committee’s recommendations are being provided as a Faculty-wide directive on GA and CI. Implementation of the changes will impose significant additional work on all Engineering Departments as well as the Dean’s office, both during the transition and on an ongoing basis. It may be appropriate to re-create the position of Associate Dean (Accreditation and Programme) on an ongoing basis to provide oversight and support. |
Dean of Engineering |
Directive on GA and CI to be released June 2022. New processes to be implemented starting in September 2022. Review of process outcomes and revision of processes in June 2023. |
| CEAB Civil Finding 1. The actions [taken to improve the programme] were a result of curriculum feedback unrelated to GAs and the CI process. A report was not completed, nor information provided. Results are presented with limited assessment of the results. | See CEAB Common Finding 3 | ||
| CEAB Civil Finding 2. The process of mark submission is not documented and may not be perceived as transparent. | The process of mark submission has been documented and briefed to all faculty members. |
Department Head |
Complete |
| CEAB Civil Finding 3a. The procedure of course allocation to faculty is not documented, nor open or transparent. | The process of course allocation to faculty in the Department has been made fully open and transparent in the years following the CEAB visit. |
Department Head |
Complete |
| CEAB Civil Finding 3b. Appropriate allocation [of Engineering Design] to [licensed Professional Engineers (PEng)] requires additional attention to ensure licensed engineers are teaching and given authority particularly in design related courses | The Department has put processes in place to ensure assignment of courses with Engineering Design content to appropriately-qualified instructors. Military faculty are now able to be reimbursed for PEng license expenses, which will lead to greater availability of PEng in the Department. |
Department Head |
Complete |
| CEAB Civil Finding 4. Courses are not fully offered for francophone students. | Staff three open faculty positions with appropriately-qualified instructors able to teach in French. | Department Head |
Summer 2023 |
| CEAB Civil Finding 5. Reaction to hiring academic staff and support staff is constrained by the hiring process. Funds are available but the process to hire and retain qualified staff including francophone staff is not efficient. | RMC has adopted a less-cumbersome process for hiring permanent faculty members. | Vice Principal Academic | Complete |
| ERC Recommendation 1. The comprehensive curriculum review planned to be conducted by the Department in 2019 needs to be fully documented and justified with reference to the Graduate Attributes. | This recommendation was fully addressed in the 2021 curriculum review. Future curriculum reviews will be made in accordance with the changes identified in CEAB Common Finding 3. | Department Head | Complete |
| ERC Recommendation 2. The department needs to use the full range of quality indicators to measure the quality of its program. The absence of an alumni survey was flagged as a deficiency. Other stakeholders, as in the field courses and engineering projects, should also be systematically surveyed. | See CEAB Common Finding 3 | ||
| ERC Recommendation 3. Though the mark submission process for courses in which the final mark is determined by instructors who are not also the supervisors of the course work was clear to all but one faculty member, it needs to be documented for it to be clear to all members, and indeed also to external reviewers who can relate this perceived weakness to a quality criterion of an engineering program. | See CEAB Civil Finding 2 | ||
| ERC Recommendation 4. Until all members of faculty, including the military, have their Professional Engineering Licenses, there will a high degree of risk that the accreditation of the program will be withdrawn, or else that Military Faculty without a license will have to be demoted to “teaching assistants.” | See CEAB Civil Finding 3.b. | ||
| ERC Recommendation 5. Knowledge of international relations is a Degree Level Expectation (DLE) for engineering programs, but the course identified by the programme as meeting this learning objective is a basic course on Canadian History. Either the DLE should be removed or a course in International Relations, such as POE/POF116 Introduction to International Relations, should be added to the programme. | A College-level committee on International Relations for Engineers was struck to investigate this issue, reporting in November 2021. The report has been provided to Senate for a decision | Dean of Engineering and Dean of Social Sciences and Humanities | A Senate decision is expected not later than Fall 2022, with any implementation to take effect in the Fall of 2023. |
Conclusion
The External Review Committee, consisting of the CEAB visit team and the internal reviewer, provided a thorough and fair review of the Bachelor of Engineering in Civil Engineering programme. Activities are underway to address the ERC report’s recommendations. CEAB-specific recommendations are also being addressed in the context of ongoing CEAB accreditation activities.