2015 IQAP Review of Post Graduate Programs in Chemistry and Chemical Engineering - Final Assessment Report and Implementation Plan (FAR)
In accordance with the Royal Military College (RMC) Institutional Quality Assurance Process (IQAP), this final assessment report provides a synthesis of the external evaluation and internal responses and assessments of the graduate programs in Chemistry and Chemical Engineering offered by the Department of Chemistry and Chemical Engineering. This report identifies the significant strengths of the programs, together with opportunities for program improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in the Final Assessment Report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and timelines for acting on and monitoring the implementation of those recommendations.
Overview of Program Review Process:
The Programme Self-Study report was completed in June, 2015. For the programmes under review it contained the degree-level expectations for these programmes, an analytical assessment of the programmes, course outlines, programme-related data, survey data from the Office of Quality Assurance and appendices with sample examinations and CVs of faculty members.
Two arm’s-length external reviewers (Douglas Evans, Trent University and Jean Koclas, École Polytechnique de Montréal) and an internal reviewer (Xiaohua Wu from RMC of Canada) were selected from a list of possible reviewers and approved by the Dean. They reviewed the self-study documentation and conducted a site visit to RMC on the 19th and 20th of November, 2015. The visit included interviews with the Dean of Graduate Studies and Research, Vice-Principal Academic, Dean of Science, Head of Chemistry and Chemical Engineering Department, Chief Librarian, as well as several civilian and military members of the Chemistry and Chemical Department faculty, and several students in the programmes. In addition, the ERC toured the Library, Slowpoke-2 Reactor, Environmental Sciences Group facilities, and various faculty and shared research laboratories. The ERC subsequently produced a report based on the Self-Study and site visit. The report was circulated to department members and discussed with members of the Chemistry and Chemical Engineering Department faculty.
The reviewers submitted their report in January 2016, in which they describe how the Chemistry and Chemical Engineering Department programmes met the IQAP evaluation criteria and the norms of chemistry and chemical engineering departments elsewhere in Canada.
The Departmental report and the External Review Committee (ERC) report together provide a thorough analysis of the current situation of the Department of Chemistry and Chemical Engineering. All of the actors involved have seized the opportunity to provide useful and important analysis and recommendations about the current Departmental context and the way forward.
Significant Strengths and Weaknesses of the Program:
The ERC identified a number of strengths of the Chemistry and Chemical Engineering Department Programmes:
- “Admission requirements are appropriate to the degrees being offered… the standard requirements for admission, including the grade-point average required for admission are appropriate both to the field(s) and to applicable provincial, national and professional standards.”
- “there is agreement between RMC and the nearby Queen’s University that allows RMC graduate students to take courses at Queen’s, which is positive to the learning experience of the RMC students.”
- “Availability of graduate supervision is good with a faculty complement that is large enough to support a good graduate program.”
Chemistry and Chemical Engineering identifies three primary areas of concern noted by the ERC that influence the status and quality of the programme:
- Separate degree level expectations (DLEs) and learning objectives (LOs) were not clear for the various programmes or degree levels offered.
- Course requirements for the Master’s and Doctoral level programs offered by the Department of Chemistry and Chem Eng are quite high by comparison with other Ontario universities.
- The assessment of students and the effectiveness of measuring learning outcomes for specific courses.
The Department Head, after consultation with faculty and staff in the programme, submitted a response to the Reviewers’ Report in January 2016. The Dean of Graduate Studies and Research in consultation with the Dean of Science and the Head of the Department prepared this Final Assessment Report in January 2022. Specific recommendations are discussed, and follow-up actions and timelines provided.
Summary of the Reviewers’ Comments or Recommendations with the Program’s and Dean’s Responses
The ERC identified the following areas of concern or issues that require attention:
Recommendation 1: The way in which information was presented in the Self Study Report (SSR), suggested that the three master’s level degrees (i.e. MASc, MSc and MEng) had the same objectives and outcomes and could hence be interchangeable.
Departmental Response 1: Graduate Degree Level Expectations (GDLEs) in the areas of i) depth and breadth of knowledge, ii) research and scholarship, iii) application of knowledge, iv) communication skills, v) awareness of the limits of knowledge, and vi) professional capacity are provided for MASc, MEng and MSc degrees. For MEng degrees (which are course-based, rather than thesis-based), several of the criteria analyzed refer specifically to the MEng final project but are referred to as ‘thesis’ in the various tables (page B-1). The presentation of the tables in this manner was meant to consolidate information, but we see how it might appear that the material is interchangeable. There are, in fact, significant differences in the expectations of students in the different degree programs, and the Department will make these differences clearer in subsequent program SSRs.
Dean’s Response: The Department has clarified the differences in the objectives and outcomes of the different Master’s programs. This comment will inform future SSRs.
Recommendation 2: The ERC noted that there were no learning outcomes (LOs) provided for each Master’s and Doctoral level program.
Departmental Response 2: It is true that specific learning outcomes (LOs) are not provided for each GDLE. This was not an oversight, but rather a result of the fact that we do support a number of different degree programs (n=12; Table XVI – page 59) and there would be different LOs for each of these. The Department will develop generic LOs for each degree program (MASc, MEng, and MSc).
Dean’s Response: The Dean concurs with this response; the development of generic LOs for each degree program will be an objective for the next SSR.
Recommendation 3: The reviewers were left with the impression that most of the assessment of performance is left to the supervisor and that, to most faculty members, quality assurance means making sure that the student produces an acceptable quantity and quality of research.
Departmental Response 3: In fact, a number of people beyond the direct supervisor(s) contribute to the assessment of all students in our programs. These include the instructors of the graduate courses taken by the students, the members of the transfer examination committee (in the cases of Masters’ degree students transferring to a Ph.D. program), the members of the comprehensive examination committee (for Ph.D. students), and the members of the thesis examination committee for all graduate students. Further, the examination committees for Ph.D. and Masters’ degree theses at RMC include: i) an external examiner, ii) an examiner that is internal to RMC, but external to our department, iii) an internal departmental examiner that is not the supervisor, as well as iv) the supervisor(s). Given that each student has an examination committee, and that we have a very rigorous thesis examination process, we feel that the quality of our students is assured. However, the Department will develop Program level LOs that will allow for more measurable and visible assurance of quality.
Dean’s Response: The Department’s response provides useful clarification. For their next SSR the Department develop Program-level LOs that will allow for more measurable and visible assurance of quality.
Recommendation 4: The ERC commented that the Self-Study report inferred that there are no standard course requirements for any post-graduate programs that are “aimed at the unique RMC mission”.
Departmental Response 4: The Department acknowledges that this section of the program brief is not entirely clear. In limiting compulsory courses, students are allowed flexibility to select courses most suited to their needs, and this is most often in support of the successful completion of their thesis research. There are, in fact, compulsory courses at both the Master’s and Doctoral level for all students in the program. These courses (CC504 for Master’s and CC604 for PhDs) are seminar courses in which the student is expected to effectively communicate the background of their field of study to all interested members of the department in a ~30 minute presentation and answer a range of questions from the audience. A complementary written paper of ~30 pages, prepared in the style of a submission to a refereed journal in the field of study, must accompany the seminar. In addition, certain courses are considered compulsory within the different degree programs. The Department will clarify the compulsory courses both across degree programs (i.e. CC504 or CC604) and within different fields in subsequent briefs.
Dean’s Response: The Department’s clarification is satisfactory. The Department will clarify the compulsory courses both across degree programs and within different fields in their next SSR.
Recommendation 5: The ERC commented that it was not clear how a student comes to be registered for a particular degree over another.
Departmental Response 5: The distinction between a MASc and an MSc program is determined by the nature of the thesis work. In almost all cases, a student registered in a MASc program would have an undergraduate degree in engineering. However, this is not a hard requirement and if the nature of the thesis is entirely ‘engineering-based’, a MASc may be granted to a student coming in without an undergraduate degree in engineering. Historically, the Master’s level degree given to engineers was called ‘MEng’, but this was changed to ‘MASc’ when our graduate programs were accredited under OCGS. At that time, it was pointed out that the ‘MEng’ applied only to course-based Masters’ degree programs (i.e. those without a thesis requirement).
Dean’s Response: The Department’s clarification is satisfactory. The Dean recommends including a similar clarification in the next SSR.
Recommendation 6: It was noted that for both Master’s and Doctoral level programs, the course requirements are quite high by comparison with other Ontario universities. Particularly for the Ph.D. degree the requirement of eight (8) courses, 50% of which were taken at RMC were seen to be rather onerous, particularly for students who have taken their Master’s degree outside of RMC.
Departmental Response 6: Given the number of different disciplines and degree programs in our department, it is not surprising that there is considerable disparity amongst department members on the topic of ideal number of graduate courses required. This topic will be reviewed again and efforts will be made to bring our programs in line with those of other Ontario universities. It should be noted that the 50% course requirement taken at RMC, is much less demanding that that at other civilian universities such as at Queen’s for which the minimum requirement is 75%. Further, the agreement existing between RMC and Queen’s University for recognizing PG courses has greatly reduced the number of PG courses taken at RMC solely for fulfilling the minimum number of courses required and is positive to the learning experience of RMC students.
The situation raised by the reviewers about several highly specialized graduate courses with low student enrolment reflects a compromise between maximizing faculty resources, and taking advantage of the knowledge, background and skills of the faculty to better serve the needs of the students. In most cases (if not all), a given faculty member will agree to offer a graduate course even if the enrolment is as low as only one student. Faculty members are often willing to take on higher-than-normal teaching loads to enhance the student’s success and improve the Department’s graduate programs. Based on this recommendation we will be standing up a committee to examine the number of courses that all of our graduate students will take, with topics that will be beneficial to several fields: stats, communications, synthetic methods/characterization, etc.
Dean’s Response: The Department’s response is satisfactory. The stated plan to analyze offerings and introduce efficiencies represents excellent shepherding of resources.
Recommendation 7: The ERC commented that there were a significant number of courses with one or two students enrolled suggesting that the material being offered in these courses is highly specialized, i.e. the courses do not appeal or are not deemed useful to most students.
Departmental Response 7: The situation raised by the reviewers about several highly specialized graduate courses with low student enrolment reflects a compromise between maximizing faculty resources, and taking advantage of the knowledge, background and skills of the faculty to better serve the needs of the students. In most cases (if not all), a given faculty member will agree to offer a graduate course even if the enrolment is as low as only one student. Faculty members are often willing to take on higher-than-normal teaching loads to enhance the student’s success and improve the Department’s graduate programs.
Dean’s Response: The situation and strategies described by the Department are a natural consequence of the numerically small population of graduate students at RMC relative to larger Provincial universities. Faculty should be applauded for their efforts aimed at ensuring the success of their students.
Recommendation 8: No specific indications of significant innovation or creativity were presented in the Self-Study report provided or during the site visit.
Departmental Response 8: The Department agrees that they did not highlight enough of this in the SSR and will do so in the next SSR. The Department notes that graduate students are being encouraged (and are succeeding!) in publishing in leading high-quality journals, with high Impact Factors (IF) in their respective fields. We also need to highlight awards and recognition obtained by our students, some winning the GG Gold Medal, some with CGS and NSERC scholarships, and some with prizes from conferences. In addition, members of the Department recognize the importance of publishing in high-quality journals and pursuing innovative research and often use their research funding to send graduate students to conferences, encouraging them to present their research findings. The Department will endeavour to make this point clearer in future reviews.
Dean’s Response: The Department’s response is satisfactory. The Department will aim to better elaborate these innovative and creative aspects of their programs in the next SSR.
Recommendation 9: The external review committee was not provided with any information on the assessments used in specific courses or on their effectiveness in measuring achievement of learning outcomes.
Departmental Response 9: Assessment criteria are available for all courses taught in the Department as part of their syllabus. We regret that they were not made available to the reviewers during their site visit. The SSR for the next IQAP review will be much more detailed, in part due to the new requirements from the QC, and will include mapping of course requirements and assessments to DLEs, PLO, etc.
Dean’s Response: The Department’s response is satisfactory. Details of assessment/evaluation methodologies should be provided in their next SSR.
Recommendation 10: The external review committee computed several faculty indicator values from data provided in the brief, noting that the faculty’s CVs were not standardized, and this made the faculty assessment difficult to carry out. The ERC considered the average publications per year to be adequate but somewhat low.
Departmental Response 10: The Department is unclear as to how the review committee came up with the average annual numbers of publications as they are much lower than the numbers obtained from the CV’s of the faculty members. Based on our calculations using the provided CVs in the Program Brief, we calculate the average publications per year between 2009 and 2015 for Full, Associate and Assistant Professors to be 12, 5.3, and 10.5, respectively. Further, it should be noted that only single year data were provided by one Full professor and one Assistant professor, and hence both of these averages appear lower than they actually are. Such data will be compiled correctly in upcoming SSRs. With respect to the h-index of citations, we note that less than 40% of our faculty has set up a profile on on-line tools such as Google Scholar where such an index is calculated. Hence, the h-index numbers available to the reviewers would not be complete. In addition, much of the output by our professors would not be captured in the h-index metric. There is a lot of work being done with DND, other government agencies, and large external industries, where peer-reviewed reports become the major deliverable, as opposed to peer-reviewed journals. The Department acknowledges that the CVs provided are not in standard format, and that incomplete data were available to calculate meaningful Faculty Indicators. The Department will correct these items.
Dean’s Response: The Dean supports the standardization of CVs on the understanding that QA will review the CV template in consultation with the program.
Recommendation 11: The issue of difficulty in accessing on-line electronic journals within the RMC library system was raised.
Departmental Response 11: On-line electronic journal acquisition is an important issue for RMC, and efforts continue to be made to access the Scholars Portal. In the meantime, the department will continue to work with the Chief Librarian to identify resource shortfalls, while using external resources as needed to meet short-term needs.
Dean’s Response: The Department’s response correctly describes the existing situation and continuing efforts to ensure access to appropriate electronic journals. The program will need to continue to communicate its needs to the Chief Librarian while making use of all available resources.
Recommendation 12: The student and faculty surveys were noted to be of limited usefulness.
Departmental Response 12: The Department acknowledges that the current surveys are of limited usefulness and will refine them, and/or find other ways to obtain this information (e.g. open forums with graduate students).
Dean’s Response: The Department, working with QA will discuss how best to improve this process for the next cyclical program review (CPR) in 2023. The recent BSc Chemistry CPR (2022) established some best practices that will inform these improvements.
Recommendation 13: The Ammunitions Program was singled out as holding significant promise for enhancement, yet the sustainability of this program was unclear to the reviewers.
Departmental Response 13: The Ammunitions Program certainly does hold promise for enhancement. The Department is pleased to report that a new faculty member, Dr. Shem Lau-Chapdelaine, was successfully recruited and began working at RMC in November 2020. There are now three dedicated professors for this program, with a future hiring process planned for Winter 2023. We also have Dr Danny Page as the coordinator of the program, with teaching relief, and an experienced military faculty fully dedicated to the program since 2021.
Dean’s Response: The department’s response, highlighting recent and forthcoming indeterminate hires, baselined research funding, and additional faculty resources allocated, strongly supports the sustainability of the program.
Recommendation 14: It was noted that there are very small numbers of students in some of the Masters-level program, and that this could be an issue if students in these various “programs” are sufficiently disparate that they do not share a common language or academic background.
Departmental Response 14: The specialized nature of different programs in our department inevitably leads to some very small groups. We endeavour to keep students in these circumstances interacting with others through the seminar program. However, it should be noted that there are also large research groups within our department with excellent interactions occurring on a regular basis. For example, the Environmental Sciences Group headed by Dr. Kela Weber and Dr. Barb Zeeb’s Phytoremediation group typically both have multiple graduate students at any given time (10 and 4 currently, respectively). The Nuclear Science faculty (Dr. Emily Corcoran and Dr. Paul Chan) are another such example (4 and 2 current graduate students). The respective students from these groups have frequent interactions because they are in similar fields. We note that, from a student’s point of view, small class sizes are often perceived as an asset for academic programs rather than a drawback.
Dean’s Response: The department’s response is satisfactory.
Recommendation 15: Concern was raised with regards to lack of wireless internet access.
Departmental Response 15: The issue of wireless internet access has been addressed in that WiFi is now largely available throughout RMC.
Dean’s Response: The issue of wireless internet access has been addressed.
Recommendation 16: In discussion with both faculty and administrators, governance structure of the graduate programs was not clear to the ERC (i.e. how decisions are made concerning curriculum review, development of program regulations, etc.).
Departmental Response 16: The program will improve how the governance structure is conveyed in it’s next cyclical program review in 2023. The department recognizes the importance of clearly articulating these to internal faculty/staff as well as to external reviewers.
Dean’s Response: Following department-level discussions, graduate program changes are brought to the college Graduate Studies Committee (GSC) for consideration and, if recommended by GSC, progress to the Faculty Council for approval. RMC’s Office of Quality Assurance works with these committees and reports all major modifications to programs to the Quality Council of Ontario.
Recommendation 17: Concerns were expressed with regards to the nature of the MEng degree.
Departmental Response: At this time, the only students graduating with a MEng degree from the Department of Chemistry and Chemical Engineering are the ammunition engineers. They follow a prescribed plan of eight interrelated engineering courses; four subject matter courses in the fall term, and four courses of engineering application in the winter term. In the SSR of the next IQAP Review process, we will make a clear distinction between the different programs and provide more information regarding the nature of each.
Dean’s Response: The department’s response is satisfactory. The SSR for the next IQAP review should seek to make a clear distinction between the different Master’s programs and provide more information regarding the nature of each.
Implementation Plan
| Recommendation | Proposed Follow-up and Resource Implications | Responsibility for Leading Follow-up | Timeline for Addressing Recommendation |
|---|---|---|---|
| 1. Separate objectives and outcomes be provided for the three masters’ level degrees (i.e. MASc, MSc and MEng). |
Provide separate outcomes and objectives. Work with Teaching and Learning Support to develop these for each program |
Dept Head/IQAP coordinator | End of 2022/2023 AY |
| 2. Learning outcomes (LOs) be provided for each Master’s and Doctoral level program. | Provide LOs for each Master’s and Doctoral level program. Work with QA and TLS to develop these during 2022/2023 AY | Dept Head/IQAP coordinator | End of 2022/2023 AY |
| 3. Assessment of student performance be made clearer to ensure it provides a more measurable and visible indicator of quality. | Develop Program Level LOs that will allow for more measurable and visible assurance of quality | Dept Head/IQAP coordinator | See # 2 |
| 4. The Self-Study report include a list standard course requirements for any post-graduate programs that are “aimed at the unique RMC mission” |
The compulsory courses are in place. They will be presented more clearly in the next cyclical review. |
Dept Head/IQAP coordinator | Completed. Change noted for next review in 2023 |
| 5. Clarification as to how a student comes to be registered for a particular degree over another. | Clarify how a student comes to be registered in a particular Master’s degree program in the next SSR. | Dept Head/IQAP coordinator | Completed. Change noted for next review in 2023. |
| 6. Review course requirements for Master’s and Doctoral level programmes to ensure they align with other Ontario universities. | Review committee will review the total number of courses required for each graduate program. | Dept Head/Graduate Studies coordinator | This will be included in the scope of the next cyclical review, which will commence Oct 2022. |
| 7. The ERC commented that there were a significant number of courses with one or two students enrolled suggesting that the material being offered in these courses is highly specialized | In our next cyclical review, we will clarify that the relatively small population of graduate students coupled with the desire of the department to offer graduate programs in several fields naturally results (1) a requirement for some highly specialized courses, and (2) low registration in some of those courses. | Dept Head/IQAP coordinator | Completed |
| 8. Provide indications of significant innovation or creativity in the Self-Study report. |
Elaborate innovative and creative aspects of their programs in the next SSR. |
Dept Head/IQAP coordinator | Completed. Change noted for next review in 2023 |
| 9. The SSR should provide information on the assessments used in specific courses and their effectiveness in measuring achievement of learning outcomes. | Details of assessment/evaluation methodologies will be provided in their next SSR. TLSC has developed a module to assist with this, program to review. | Dept Head/IQAP coordinator | To be reviewed Jan 2023. |
| 10. Faculties CVs be standardized to facilitate faculty assessment. | The Program and the Office of QA will review which template should be used for CV’s for the 2023 CPR |
Dept Head/IQAP coordinator Director QA |
Fall 2022 |
| 11. On-line electronic journals be more accessible to students. | The program submitted subscription requests in 2021 and will continuing efforts to ensure access to appropriate electronic journals. |
RMC Chief Librarian |
Reviewed annually, updated as a part of ongoing monitoring annually due June 15, 2023. |
| 12. Student surveys were found to be of limited usefulness and in future, the effort of conducting the surveys could be used in preparation of other more useful information. |
Refine surveys, and/or find other ways to obtain the required information (e.g. open forums with graduate students). |
Dept Head/IQAP coordinator | Survey questionnaires will be reviewed in February 2023 |
| 13. Review the sustainability of the Ammunition program. | No action required – the department’s response strongly supports program sustainability. | Completed | |
| 14. Small numbers of students in the various graduate programs could be an issue. | No action required – small numbers of students are seen as beneficial. | Completed | |
| 15. Improve access to wireless internet | No action required – wireless internet is available across campus (effective 2020-2021) | Completed | |
| 16. Governance structure of graduate programs is unclear | Program to incorporate feedback in next SSR in consultation with QA. |
Program IQAP team Director of QA |
Clarification to be included in next Self-Study report (2023). |
| 17. Concerns with nature of MEng degree | See # 1 above |
Dept Head DGS |
End of 2022/2023 AY |
Conclusion
The ERC Report is largely positive. It confirms that the Chemistry and Chemical Engineering Department is delivering academically rigorous programmes with an excellent spectrum of courses to its students, despite the resourcing challenges experienced by a small department attempting to run a multi-faceted graduate program. Overall, the Master’s and PhD programmes are of very good quality and compare well with those of similar programmes in Ontario, especially in smaller universities. The Dean of Graduate Studies, in consultation with the Dean of Science and the Head of the Chemistry and Chemical Engineering Department, is responsible for monitoring the Implementation Plan.