For the first time this spring, candidates taking Exam 9 were given an Integrative Question (IQ), which tests multiple areas of the syllabus. An IQ, which is composed of several interrelated subparts, tests at a higher Bloom’s level and is worth more points than a traditional exam question (approximately 10-15% of the total points available). For the Spring 2017 sitting of Exam 9, the IQ was worth 7.5 of the total 60 points possible and was composed of 5 subparts. In addition, the Fall 2017 sitting of Exam 8 will feature one IQ (read here for a CAS announcement with further information). We spoke with Syllabus and Examination Committee Chair William Wilder to learn more about the grading process for this new type of question.
Why is the CAS moving towards Integrative Questions?
The main reason for the addition of IQs is job relevance. IQs synthesize various topics and reflect the fact that actuaries must often combine principles from multiple areas when completing their work. As such, IQs emphasize “synthesizing solutions.” This also allows examiners to ensure that candidates understand the material on a holistic level.
Although the Syllabus & Exam Committee is completely supportive of this move, it might interest candidates to know that the introduction of more synthesized and job-relevant questions comes at the direction of the CAS board. An action paper was submitted to the board that recommended using IQs as a way to test candidates on multiple subject areas and to make questions more job-relevant. The Syllabus and Examination Committee was then asked to implement this request on exams. The implementation of IQs will be gradual and began with Exam 9.
How did exam writers approach writing IQs for the first time?
The writers for the IQ on Exam 9 are experienced question writers. They were asked to write potential IQs and were dedicated only to writing these questions. Though IQs are new, exam questions that synthesize multiple topics have been written before. Exam 5 usually has one longer question that combines reserving and pricing principles and is similar to an IQ; both types of questions test at a higher Bloom’s level and ask candidates to synthesize what they have learned from multiple areas of the syllabus. The Exam 9 team borrowed and expanded upon these best practices in writing and planning for grading the IQ this spring. They also met to discuss how the question should be structured, including what kind of data to include, how many subparts it should have, and how interrelated it would be.
The team of writers eventually came up with several questions, one of which was selected to appear on the Spring 2017 sitting for Exam 9, while other questions were used as sample questions that candidates can use to prepare for the IQ. These sample questions “give candidates a glimpse about how writers thought about the process.”
Assigning points (including partial credit) for IQs posed a new challenge to graders. How did you prepare volunteers to grade these questions?
Experienced graders were asked to grade the IQs on Exam 9. As usual, graders worked in pairs and had to “agree to a very tight degree” on the amount of points to award or deduct for a question. The IQ graders collaborated and made sure that each pair graded consistently. As always, the grading process was approached in a structured way to ensure accuracy and fairness to candidates. For a typical question, a single grading pair is responsible for the whole question. However, due to the extensive nature of the IQ, pairs were assigned one or more subparts of the IQ. The pairs collaborated to make sure that they graded the subparts consistently and deducted the same number of points for the same types of mistakes. After ensuring the fairness of the grading process, the Exam Committee also assessed how candidates performed on the question in comparison to the question writers’ expectations.