FAR - 2016 IQAP Review of War Studies Programmes

Compliant with the Royal Military College (RMC) Institutional Quality Assurance Process (IQAP), this Final Assessment Report (FAR) provides a synthesis of the external evaluation and internal response and assessments of the multidisciplinary graduate programmes in War Studies offered by the Faculty of Arts. This report identifies the significant strengths of the programmes, together with opportunities for programme improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.

This report includes an Implementation Plan that identifies:

• Who will be responsible for approving the recommendations set out in the Final Assessment Report;
• Who will be responsible for providing any resources entailed by those recommendations;
• Any changes in organization, policy, or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and
• Timelines for acting on and monitoring the implementation of those recommendations.

Overview of Programme Review Process:

The Programme Self-Study Report was completed in the summer of 2014.  For the programmes under review, the MA and Ph.D. in War Studies contained the degree level expectations for these programmes, an analytical assessment of the programmes, course outlines, programme-related data, survey data from the Office of Quality Assurance, and appendices with sample examinations and CVs of faculty members.

Two arm’s-length external reviewers (Dr. David Haglund, Queen’s University and Dr. Michael Neiberg, U.S. Army War College) were selected from a list of possible reviewers and approved by the Deans’ Council. They reviewed the self-study documentation and conducted a site visit to RMC on 15 and 16 January 2015.  The visit included interviews with the Dean of Arts, Associate Vice-Principal, Research, Chair of War Studies, Associate Chair of War Studies, Head of the History Department, Head of the Political Science Department, as well as several members of the War Studies faculty, two of whom were former chairs of the programme in addition to several graduate students in the programmes.  The ERC subsequently produced a report based on the Self-Study and site visit.  The report was circulated to department members and discussed with members of the War Studies faculty.

During the writing of this report, an anomaly in the process was detected and an arm’s-length internal reviewer (Dr. Don Mcgaughey, RMC, Department of Electrical and Computer Engineering) was engaged to shadow the visit described above and offer additional comments and recommendations.  The ERC was amended to reflect his observations.

The reviewers submitted their report in March 2015 (and amended it in May 2016). In their report, they find that the War Studies Programme requirements are clear and that the outcomes are identified.  The ERC identified strengths in leadership, diverse and committed faculty, and a solid national and international reputation.  Despite having suffered, along with other programmes at RMC, the impacts of WFA and budgetary contraction, the programme was assessed as being well-positioned to grow and strengthen.

The English and French streams of the programmes were not found to be equivalent but functional and compliant with the mandate for graduate programmes at RMC.  The programmes were found to be very strong and competitive relative to other similar Canadian university programmes and were noted to be, perhaps, slightly more demanding.

Issues with online journal access were noted by some students and were relayed by the ERC as an ongoing concern.

Significant Strengths and Areas of Concern of the Programme:

The ERC identified several strengths of the War Studies Programmes:

• Overall, the programme retains a “solid national and international reputation” (16). This is because several of the faculty have research profiles recognized at the national and international levels.
• Admission requirements are consistent with graduate programmes at other Ontario universities, and the academic expectations of students enrolled in the programme are in some cases more rigorous. This is a partial reflection of changes implemented by War Studies faculty and leadership since the last programme review.
• The ERC Report identified teaching and student mentoring as particular strengths of the programme, noting that the faculty, “manifest a strong and obvious commitment to student mentoring.” (12) Students who met with the ERC praised the faculty, the quality of supervision, residential seminars, and the core course, WS 500/600.

The ERC identified several areas of concern for the War Studies Programmes:

• The need to better track the accomplishment of learning outcomes that relate to specific GDLEs.
• The need for new faculty, especially to broaden its areas of coverage;
• The need for more faculty members who can teach in French;
• The need for better IT strategy and blended learning platforms;
• The need for a full-time Programme Representative;
• The need to involve senior faculty from the Department of Political Science and other relevant departments in long-term challenges confronting the programme; and
• The need for more research support.

The Programme Chair, after consultation with faculty and staff in the programme, submitted a response to the Reviewers’ Report in May 2015 and amended it in November 2016 to consider the internal reviewer’s comments.  The Dean of Graduate Studies in consultation with the Dean of Arts prepared this Final Assessment Report.  Specific recommendations are discussed, and follow-up actions and timelines are provided.

Summary of the Reviewers’ Recommendations with the Dean’s Responses

The ERC identified some areas of concern or issues that require attention.  These issues are discussed in the order that they appear in the ERC Report.  The consolidated response of the Program and Dean of Graduate Studies is shown in italics following each ERC comment:

1. First, funding cuts have curtailed the ability of the War Studies programme to offer courses outside of Kingston.
   
   To continue to meet the needs of our distance students, the programme has begun to offer seminars via video conference software platforms. 
    
2. The programme lacks hardware and software and technical support for this new method of course delivery.
   
   Based on concerns expressed by War Studies faculty regarding the inadequacy of existing hardware and software, the War Studies programme is now using desktop video conferencing system “Big Blue Button” that IT integrated into the Moodle Learning Management System (LMS).  This video conferencing solution in conjunction with Moodle has been used since 2016 with good results.
    
3. The ERC raised concerns that the identified GDLEs do not appear to drive admission requirements, curriculum content, modes of delivery, the basis of the evaluation of student performance, and commitment of resources.  It further recommended that the programme keep a separate evaluation of each of the LOs/GDLEs.
   
   The programme recognizes that its GDLE/LO matrix is in evolution with this being the first appraisal under the new IQAP Framework.  The process of continual improvement on the use of GDLEs and LOs to drive programme development will continue.  The programme concurs with the intent to track LOs and GDLE accomplishments separately.   The program will contact the Dean of Engineering in order to evaluate the attribute-tracking system used in that faculty.  In Fall 2021, the Chair served on the IQAP review committee, and is working on developing Program Objectives and Program Learning Outcomes in preparation for the next review cycle. 
    
4. The ERC report repeatedly draws attention to the lack of a full-time Programme Representative dedicated to the War Studies programme and located at Cavalry House.
   
   As of 30 June 2020, the Division of Continuing Studies (DCS) was dissolved, which permitted the reallocation of personnel within DCS to other positions.  The result was the allocation of two part-time support staff to War Studies.  They are being cross-trained to ensure redundancy should a staffing gap occur, but are being employed as a Program Support Coordinator and as a Student Support Coordinator.  The former, 50% of whose time is allocated to provide administrative support to a Canada Research Chair, is focused on infrastructure, finances, the website, and support to instructors.  The latter, 50% of whose time is allocated to support the Masters’ of Defence Studies offered by the Canadian Forces College in Toronto, is focused on admissions and student records.  This arrangement has proven tenable to date due to cooperation between stakeholders and the alignment of workflows.
    
5. The ERC noted tension among War Studies faculty “about whether the programme is too much based in History and not enough in related disciplines like Politics and Economics.”
   
   The programme is addressing this issue by incorporating more courses taught by faculty in the Political Science, English, French Studies, Defence Studies, and Military Psychology and Leadership Departments.  From the Fall 2016 term to the Winter 2022 term, excluding directed studies and special topics, the distribution of credits by Department appears as follows:

Academc Year	HIS	POL	MPL	ENG	FRE	DDS	Sessional
16-17	14	11	0	2	1	0	0
17-18	18	14	6	0	2	3	7
18-19	24	8	0	0	2	2	0
19-20	14	9	6	2	2	2	0
20-21	20	10	2	2	0	0	0
21-22	14	7	0	0	0	2	0

6. The ERC noted the entry standards for the WS Programme were higher than the minimum required by the SGS and suggested rationalization of the standards to enhance enrolment|
   
   The Programme has carried out an extensive evaluation of the conditions leading to reduced enrolment and devised a strategic plan which it feels will reinvigorate the program and grow the student base.  No plans are in place to reduce the admission standards, but the Admission Committee recommended reform to the process of referee recommendations that was supported by the War Studies Committee.
    
7. The ERC noted that students have noticed a disparity between the extensive range of courses listed in the RMC graduate calendar and the limited range of courses offered on an annual basis.
   
   After consulting with the current War Studies faculty, we are removing courses from the graduate calendar that have not been offered in recent years, or that we no longer have the resources to offer.
    
8. The ERC noted the detrimental impact of the recent retirements of senior faculty. The report states: “The programme has lost a real wealth of talent recently, including several world-class scholars.” It emphasizes the importance of hiring replacement faculty to ensure the long-term viability and reputation of the programme.
   
   This is a serious problem, to which the programme is responding in several ways. First, the War Studies programme has input into the hiring decisions made by academic departments at RMC, even if the number of new faculty hires in the foreseeable future will be relatively small. We will attempt, whenever possible, to articulate our needs to departments hiring new faculty. We had the opportunity to do this in 2015 and 2018, as the History Department successfully conducted faculty searches for international historians who have the potential to contribute to the War Studies programme.  Secondly, the programme is also adapting to the reduction in senior faculty by developing the teaching and supervision capabilities of more junior members of faculty. Faculty members who have taken on increasing responsibilities in recent years include Dr. Howard Coombs (History), and Dr. Christian Breede (Political Science). Third, the designation of a Tier-1 Canada Research Chair in War Studies (Dr. Doug Delaney) will help retain a key member of the faculty. It has also helped attract a Social Sciences and Humanities Research Council (SSHRC) Post-doctoral Fellow from 2014-16, another for 2016-2018, and again in 2018-2020.   The pandemic interrupted the presence of a post-doctoral fellow in 2020-2022. Finally, we continue to hire sessional instructors with specific expertise to fill gaps in the curriculum.
    
9. The report notes the impact of funding cuts on the ability of faculty to conduct research and present it to academic audiences. As the report states: “Several members of the department are nationally and internationally recognized scholars, publishing in first-rate academic venues, directing journals, and winning academic prizes. Heavy teaching loads, limited funding for travel, and bureaucratic obstacles to travel threaten to put this record of considerable accomplishment at risk. War studies faculty need to be present at the major conferences in their field and they must have adequate funding for research.”
   
   There are three concerns identified in the report.  The first is the heavy teaching loads.  As of 2021, the College created a committee examining workload (including teaching) across the university.  The results of their deliberations are not yet known. The second is limited funding for travel.  This situation since 2016 has improved somewhat with Short-Term Research Needs funding, the Canadian Defence Academy Research Program, and additional funding provided to restart research activities in the wake of the pandemic.  Since 2016, some of the bureaucratic obstacles to travel have been removed, largely through the delegation of authorities to authorize travel and also through the streamlining of some of the procedures. 
    
10. The need for a greater range of course offerings in French to create a form of linguistic balance.
    
    This problem exists in symbiosis with the relatively small number of students who undertake studies in French.  Given the current emphasis on ensuring sound resource stewardship, priority is being given to attempting to attract more students who wish to study in French, be they Francophones or bilingual students.  In addition, bilingual students are reminded that their choices are much greater as they can draw on offerings from both sets. 

Implementation Plan

Recommendation	Proposed Follow-up	Responsibility for Leading Follow-up	Timeline for Addressing Recommendation
1. First, funding cuts have curtailed the ability of the War Studies programme to offer courses outside of Kingston.	Finalize installation of new hardware.  Complete software evaluation and secure adequate licences for offerings.	CIO	Completed.  All courses are now being delivered as Virtual Classroom (synchronous with remote students) or Distance Learning (asynchronous)
2. The need for a better IT strategy and blended learning platform (Address lack of blended learning hardware and software.  Complete trial of BBB and move to implementation phase).	Finalize installation of new hardware.  Complete software evaluation and secure adequate licences for offerings.	CIO	Completed.  The learning platform, Moodle, contains an integrated virtual classroom (Big Blue Button) and instructors use Moodle for the distribution of material and either Big Blue Button or Moodle for a virtual classroom.
3. The ERC raised concerns that GDLEs are not connected to admissions, curriculum, or student performance.	Align governance mechanism (the War Studies Committee) with the GDLEs to inform policy development and implementation on admissions, curriculum, and performance.	Chair	Ongoing.  Based on the new IQAP process, GDLE will inform drafting of PO, PLO and CLO.
4. The need for a full-time program representative.	Addressed with the transition to DGS structure.  WS now has 50% of a CR-04 doing admin support and 50% of a CR-04 shared with MDS for program support.	DGS	Completed
5. The need for new staff, especially to broaden its areas of coverage.	Encourage departments to take WS needs into account when staffing positions and to seek to get new faculty to teach or supervise WS students.	Chair	Ongoing – this is a function of when supporting departments seek to engage new faculty as well as identification of the opportunities for teaching and supervision.
6. The ERC suggested rationalization of the standards after noting the entry standards for the WS Programme were higher than the minimum required by the SGS.	No plans were in place to reduce the standards, but the Admissions Committee recommended reform to the process of referee recommendations that were supported by the WS Committee.	Chair	Completed.  Ph.D. applicants require 3 referees.  MA students should have two.  Exceptions to this are to be explained in the applicant’s statement of purpose.
7. Dealing with the WFA, especially in terms of the program’s course offerings (Address imbalance between a range of courses in calendar and offerings).	Assess the graduate studies calendar and remove all courses no longer being offered or planned to be offered in the foreseeable future.	Dean of SSH	Ongoing – the current Chair is reviewing how to split the two term courses while also reducing overlap between the current set of courses to develop an effective plan to reform the offerings.
8. The need to involve senior faculty from Politics and other relevant departments in long-term challenges confronting the program.	Engage faculty to broaden those actively supervising in the programme.	Chair of the programme	Ongoing.  This is being done through the War Studies Committee, which meets twice a year.
9. The need for more research support.	Normalize funding support for sponsored students and seek additional funding opportunities.	Dean of SSH	Ongoing.  While the funding issues and bureaucratic obstacles have been reduced, the pandemic also suppressed the demand for travel.
10. The need for more staff members who can teach in French.	Develop French language capacity within RMC to attract the student population to justify offerings. Integrate into programme strategic plan.	Chair	Completed.  There are sufficient professors to cover the French-language offerings in the program on a recurring basis.  The number of francophone students is increasing.

Conclusion:

The ERC Report is largely positive.  It confirms that War Studies is delivering an academically rigorous programme to its students and that its standards meet or exceed those of similar programmes in Ontario.  However, the ERC also identified some areas requiring improvement and some dangers to the future health of the programme. RMC is already taking steps to address the issues raised, such as the balance of course offerings, and the accuracy of the course listings in the RMC graduate calendar. We have introduced new hardware and software to support the programme’s video conference courses and developing the capabilities of faculty who are interested in contributing to the programme.  RMC has re-organized the administrative support for the programme in the form of a post-graduate programme representative with cleaner reporting lines.  Since the initial site-visit, travel policies have been simplified.  Work related to the hiring of new faculty with expertise relevant to the programme, and the provision of additional funds to support conference and research travel are currently in progress.

The Dean of Graduate Studies, in consultation with the Dean of Social Sciences and Humanities and the Chair of the War Studies programme, is responsible for monitoring the Implementation Plan.