PART 15: FACULTY ASSESSMENT POLICY

Starting point:

- This is a core task of the Educational Quality Control Unit and the Study Programme Committee and an important element in the quality assurance of the programme: how are the intended competencies actually tested?

- There is a need for an overview (how and what do we test?). It is initially about the form (how do you communicate this to the students, criteria). In addition, the content will also be viewed in the context of horizontal and / or vertical coherence of the programme (learning path).

- Emphasise good practices. Often not yet visible enough => more reflection than control. The faculty wants to think along about the assessment policy, not control it.

- Some concrete issues have highlighted this topic even more: low pass rates in some subjects (for example, in higher years), discrepancy in points in the assessment of papers (research paper / master's dissertation), lack of transparency for students (for example, the assessment procedure of oral exams), multiple choice and higher cut-off score, lack of insight into local academic culture for guest and new lecturers.

- An umbrella faculty assessment committee and an assessment committee per study programme are established to monitor and coordinate these matters. Lecturers will be asked to provide information about their assessment policy (including exam questions). Students will also be interviewed more in depth about the assessment practice

This policy documents consists of:

1. Brief background about the UGent assessment policy
2. A proposal for the Faculty Assessment Concept
3. A proposal for establishing the Faculty Assessment Committee + working method
4. A proposal for establishing Assessment Committees per study programme + working method

Brief background about the UGent assessment policy

UGent assessment policy = coherent whole of measures and provisions to monitor and promote assessment quality. UGent opts for an assessment model consisting of three phases (aiming, measuring and guaranteeing) and three levels of involvement (lecturers, study programmes and the university). In a first phase, a vision on qualitative testing is made explicit. This is the AIMING phase. Subsequently, in a second phase, the accomplishment of the proposed quality objectives is verified (MEASURING phase). Finally, in a third phase, the assessment quality is guaranteed (GUARANTEEING phase) in agreements and regulations, in the organisation and planning of the evaluations and by supporting lecturers and study programmes.

UGent assessment concept: Educational translation of 'Dare to think' in the field of assessment. The UGent ‘Measuring Integrated Knowledge Competencies' assessment concept indicates what our institution wants to emphasise in further assessment development. With this policy we also want to characterise the individuality of assessment at Ghent University more educationally.

More concretely:

1. Valid assessment is based on intended competencies
2. Integration also tests knowledge
3. Authentic assessment in various contexts promotes transfer
4. Discipline-crossing assessment encourages critical sense
5. Feedback stimulates creativity
6. Action and interaction require the highest reliability
7. Transparency installs trust in assessment
8. Self-reflection is the basis for lifelong learning
9. Varied assessment practice respects diversity

Assessment vision of the faculty of Political and Social Sciences, based on UGent's assessment concept

Quality assessment control takes place at different levels, namely within the study programme itself by the assessment committee and the study program committee and outside the study program by the examination committee and the education quality control unit.

Faculty assessment concept

Valid assessment is based on intended competencies

One of the most crucial quality assessment criteria is validity: Evaluation has to be meaningful, usable and valid to make a statement about the achievement of the predetermined competencies by students. In 2013-14 the faculty rewrote the educational competencies that we aim at among students. These written competencies form the basis for well-considered choices regarding evaluation. The evaluation form always has to be suitable for measuring the intended competencies at a certain moment. The Faculty Assessment Committee will conduct an annual evaluation based on our competence matrices per study programme: are the evaluation methods used in relation to the intended learning outcomes (exams and papers) for the Bachelor's and Master's degree programmes? In addition to this global overview of the assessment methods for the full Bachelor and Master programmes, an analysis is also made of the assessment methods for the individual learning outcome blocks.

Great diversity of evaluation forms from an educationally innovative perspective.

Our programmes use classical evaluation methods (written and oral exams) to verify whether students have acquired sufficient academic knowledge. However, we want to focus even more on various innovative evaluation forms with which higher, broader and / or more complex competencies are evaluated. In our assessment policy we wish to emphasise this more strongly by mapping out existing good practices in this area and disseminating them more widely in our training programmes. To this end, we also want to make sure that these educationally innovative evaluation forms (such as peer assessment, papers, etc.) are implemented and applied in practice according to the highest quality standards and in a transparent manner.

The education quality cell also provides certain tools for this (such as assessment matrices, information afternoons, etc.)

Feedback stimulates creativity

The faculty wants to focus on high-quality feedback! In particular interim or non-periodic evaluations with immediate feedback are one of the most powerful stimuli for learning. These encourage students to regularly focus on their studies, spread over the entire duration of a course unit. The faculty wants to maximise the potential of efficient feedback possibilities and focuses on automated or non-automated feedback, individual or collective feedback, peer feedback, etc. Organising classical feedback more collectively and repetitively is an option (you can reuse classical feedback, because the same remarks resurface). It is important to coordinate employees in this aspect. To guarantee the reliability of the assessment and the quality of the feedback, the use of clear evaluation criteria is crucial.

Transparency and communication

Transparency and communication of assessment is essential. The faculty assessment committee, therefore, wants to provide guidelines in this regard. Internal communication among lecturers of our faculty about their assessment is crucial in order to arrive at a high-quality assessment policy. We want to guarantee this by:

• Annual checks of the course sheets by the faculty assessment committee; these describe the evaluation forms, moments and conditions for passing a course in a clear terminology. Criteria are not only necessary for the sake of transparency, but also serve as a guide for students to learn about certain things (for example, to clarify what a high level is). Give students the right tools!
• Pointing out the importance of the following aspects to the lecturers:
  • Explaining a course sheet in the first lecture
  • Including a course sheet (for example at the front of the course)
  • Mentioning papers and the like that count towards grades on the sheet + providing partial results to the student via personal communication (see also new tool in Minerva, gradebook)
  • Providing example questions
  • Drawing up a kind of information sheet about the design of the course that is given during the first lecture; when and where, learning material, lesson preparation, exam, communication (for example, via Minerva), lesson plan and / or reading list
  • Personal feedback after the exam. Efforts can be made to provide systematic feedback for the high-stake subjects (for example, the master's dissertation, research paper, papers, etc.)

Self-reflection contributes to personal development in the context of lifelong learning

Although assessment usually serves as an assessment tool, evaluation can also be used as a means of learning (= at individual level, self-reflection). With a view to lifelong knowledge development, it is important that students reflect during their UGent period on their own learning process. Students can do this through the educational evaluations, the ombudsperson and through his/her representatives on the study program committees and Educational Quality Control Unit.

Good planning and distribution during examinations as part of high-quality and fair testing

It is ensured that that assessment is well programmed. We strive for fixed exam schedules for all years (exceptions, of course, are made in the event of programme changes, unavailable auditoria, public holidays, etc.). Students and lecturers are consulted. The use of Centauro should make things more clear for the students

Assessment committees as a sustainable quality assurance.

Within the faculty, faculty and programme-specific assessment committees are established, with the following objectives:

1. Ensuring consistent quality assurance of assessment form in the study programme that fits within the overall vision of assessing the study program
2. Stimulating validity, reliability and transparency of assessment (interim and final evaluations) and clear communication / feedback to the lecturers and students of the study program.
3. Achieving a coherent assessment practice within each study programme in which all proposed training competencies are achieved and tested in a valid manner.

Faculty assessment Committee: Composition & method

Founded in the academic year 2013/14, as a sub-committee of the Educational Quality Control Unit

1. Who?
   Director of studies (chairs the committee), Chair Study Programme Committee, ombudsperson, 1 assistant academic staff, 1 member of the Educational Quality Control Unit, 1 member of education innovation, 1 student • by invitation from someone of the Department of Educational Policy, curriculum manager / FSA (regarding exam schedules)
2. Frequency: annually
3. Quality assurance check by Faculty Assessment Committee:
   • The faculty assessment committee prepares template exam questions (with, among other things, name, study programme, year, session, examiner, duration of exam, material usable during exam, multiple choice (with our without negative marking, negative marking type, student name, designated grade space to give points per question) + feedback)
   • The faculty assessment committee makes a quality checklist for lecturers: list on the basis of which the lecturer can check during and after the preparation of the exam questions whether the various aspects of the quality assurance of the exam questions have been met (think of the three-fold validity, reliability and transparency + items related to exam form such as multiple choice, open questions, open book, etc.).
   • The faculty assessment committee sets up a screening matrix that will be used by the programme-specific Assessment Committees
   • Annual evaluation on the basis of our competence matrixes per study programme: are the proposed programme competencies sufficiently achieved and adequately assessed?
   • The faculty assessment committee also acts as a first-line assessment committee (operation specific to the programme) for the common courses from 1e Bachelor
   • Quality control Master's dissertation (every four years) by the faculty assessment committee (for example, an assessment matrix)
   • General problems such as pass rates
   • Exam scheduling (publication deadlines)
   • The Assessment Committees (per study programme) report where necessary to the Faculty Assessment Committee about their operation. On the basis of all reports, a global advice report is drawn up and fed back to study programme committees (listing good & bad practices in order to make sure the future assessment policy is evidence-based.

Assessment Committees per study programme: composition & operation

1. Founded in the academic year 2013/14, in the hands of the Educational Quality Control Unit (????)
2. Who?
   • Chair study programme Committee (chair), 2 additional professorial staff, 1 member from the assistant academic staff, 1 member from the education quality control unit + 1 student will be involved in the operation of the programme-specific assessment committees. • On invitation, other employees (possibly external) can attend the meetings.
3. Quality assurance control by programme-specific assessment Commissions:

   A. For exams of individual course units

   • Annual checking the course sheets (section evaluation + final score calculation) See faculty assessment concept (point 4 transparency and communication)
   • • Screening courses
      Which courses?
      New courses
      If educational evaluations show that there are problems or at the request of students
      If pass rates show extreme peaks or troughs (outliers)
      When?
      For new courses/new lecturers: Lesson week 13 (1th + 2the semester) (reason: proactive supervision)
      For other courses (week after the end of the exam period)
      What?
      Final exam questions (questions themselves, who drafts them / intervision about drafting exam questions / do the questions represent the entire course), etc.)
      Exam organisation (How much time? Who marks exams? ...)
      Assessment criteria, correction key (if applicable)
      A screening matrix is used that is prepared by programme-specific assessment committee:
      Evaluation form and organisation
      Assessment vision: how do we test learning outcomes, and what is the link with the study programme competency? Crucial components are:
     o Objective (subject competence), interpretation of the relationship of the subject competence with the study programme competence
     o Grading method: list criteria and link those criteria to grade levels
     o Formulating a link between the learning outcomes and grade levels (overview table - see assessment matrix for the Master's dissertation, research paper)
      Assessment communication (ECTS, syllabus, feedback, ...)
      Coordination (instruction from staff regarding assessment, peer review when making exams)
      Actions and conclusions
      Grade distribution: lecturers receive their grade distribution for the relevant course. This provides a statistical representation of the exam grades calculated per academic year (OASIS).
      Interested parties can also use the 'assessment matrix' (education tip: http://onderwijstips.ugent.be/tips/stel-een-toetsmatrijs-op-voor-een-valide-beoordeli/ When preparing an exam, it is important that the various objectives and / or components of the course are represented in a balanced manner. The use of an assessment matrix is primarily intended for evaluations in which representativeness is aimed for, but it is certainly also useful to get a well-considered view of a 'regular' exam.
      The programme assessment committee discusses the various files and also links them to pass rates. Subject to technical feasibility, the coherence between exam results is also checked (which does not necessarily mean that poor coherence with exam results from other courses is problematic)
      Feedback study programme assessment committee to lecturer (possible suggestions)
   • Recommendations for study programme committee
      Prepare global advice on the basis of all submitted files.
      Urgency and more specific advice on problem subjects

   B. Research paper and papers (at least every four years)

   • research paper: for example, creating an assessment matrix by analogy with assessment matrix of the master's dissertation.
   • • papers:
      distribution of papers within the model trajectory over the Bachelor and Master years (with a view to the construction of various standard learning tracks.)
      continuous assessment criteria

Pro Memoria

Study programme committee: members of assessment committee are often members of the study programme committee, things that are reported there (for example, by students) can be fed back to the assessment committee.

Examination Board: ensuring completeness and accuracy of, among other things, the grades. Problems can also be identified here.

Educational quality control unit:

• educational evaluations, assessment is also reviewed here (educational quality control unit can identify good practices, as well as problem subjects)
• can provide score distribution tables (can be obtained from OASIS) (statistical representation of the exam grades calculated per academic year- runs over a period of 5 years)
• pass marks per course

Activities & instruments for study programme committees and / or assessment committees to monitor assessment quality

Implemented in the faculty of Political and Social Sciences

Optional activities:

In addition to the above basic activities, it is advisable to work on certain aspects of evaluation depending on the programme or faculty-specific priorities / accents or occasional problems or issues.

Below you can find a non-exhaustive list of examples of such optional actions of which we have already achieved quite a lot:

Appendix

Screening matrix