4 Selected Response Items

While not the sole assessment method, TBL relies heavily on selected response item (commonly called multiple-choice questions or MCQ) quizzes in the Readiness Assurance Tests (RAT) and in the group application activities. A well-developed MCQ is an efficient and reliable way to generate valid evidence that reflects conclusions about student learning. As with everything discussed in this manual, selected response items demand a deliberate and evidence-based approach. The process to develop quality MCQs can be exhaustive to ensure that each item reliably reflects the intended construct at an appropriate cognitive level. A low stakes quiz may merely involve several hours of instructor time to develop questions, colleagues to review them and a review of the questions for quality and validity after administration. Meanwhile, a high-stakes licensing exam generally involves a panel of expert developers, a review by subject matter experts, a field testing phase of the item, hundreds of hours and thousands of dollars to develop a test bank with an adequate quantity of items (Downing & Haladyna, 2006; Guidelines for the Development of Multiple-Choice Questions, 2010; Williams, 2020).

It is beyond the scope of this manual to adequately prepare instructors to develop high-quality selected response items, but because MCQs are so prevalent in TBL, an overview of the process will be described.

The Anatomy of a MCQ

A MCQ is built according to a consistent framework:

(adapted from Gierl, n.d.)

Identifying the Construct

As with the development of a TBL course and TBL learning modules, the development of quality MCQs begins at the end by determining what is to be measured by a selected response test and each item or question that make up the test. A MCQ test should evaluate an overall construct, while individual items (questions) should target a specific concept that is a component of the larger construct. Selected response items do this by forcing students to make a choice (Downing & Haladyna, 2006; Gierl, n.d.; Sibley & Ostafichuk, 2015; Sibley & Roberson, 2016).

Not only does an instructor need to identify the specific concept that an item will test, a target cognitive level needs to be identified according to a model such as Bloom’s taxonomy. Varying cognitive levels are desired depending on the activity it is designed for.

• RAT items will typically focus on remembering, understanding and light application. These questions will often ask students to perform tasks such as: identify, distinguish, classify and organize. Questions will usually begin with: “What is…?” and “Why does…?”
• Group application activities should push students into higher level application, analysis and evaluation. Questions designed for the application activities will typically contain verbs such as: solve, compare, categorize, organize and design. Questions will often contain a superlative in their wording, such as: “What is the most…?” or “Which is the best…?” in order to force a specific choice. Students will be required to construct a rationale for their choice in order to adequately answer and defend their decision (Roberson & Franchini, 2014; Sibley & Ostafichuk, 2015; Sibley & Roberson, 2016; Williams, 2020).

Guiding Principles for Writing Selected Response Items

The following considerations should be taken into account when developing new MCQs or revising existing ones:

• Items should represent a specific and important concept or topic
• Each item should pose a clear question that students could answer without looking at the options
• Avoid negatively worded stems or options
• All options should be homogenous in terms of wording, grammar, length and content
• Avoid “all of the above” and “none of the above” options
• Distractors should all be plausible and none should be obvious as a distractor (all options could be correct)

(Gierl, n.d.; Williams, 2020)