MULTIPLE CHOICE QUESTIONS Design and Evaluation

1
MULTIPLE CHOICE QUESTIONS
DESIGN AND EVALUATION
Helen Huntley
Senior English Language Fellow
Ministry of Education and Training
Hanoi, Vietnam
2
Workshop Goals

To review the advantages and
disadvantages of multiple choice tests

To explore tips and techniques for
writing successful multiple choice items

To evaluate and correct multiple choice
questions and answers
3
Objective Testing
“Tests in which the learning material is
divided into small units, each of which
can be assessed by means of a test
item with a single correct answer that
can be specified in advance.”
Reed, J. (2000). Assessing vocabulary. Cambridge University
Press
4
Advantages of Multiple Choice
Tests


good discrimination among levels of ability

high reliability

valid indicators of overall language ability

highly practical: convenient to administer

well-established procedures to analyze items

writing ability does not affect performance

cost-effective format - easy scoring and
grading

familiarity of students with MCQs
5
Disadvantages of Multiple
Choice Tests

tendency to treat words as independent linguistic
units

difficulty in construction

field-testing, analysis, refinement required

25% chance of guessing correctly

items may test knowledge of distractors


wrong answer may result from lack of understanding
of syntax (English)

permits only limited sampling of total knowledge

tests only recognition knowledge

restricts what can be tested

washback may be harmful

cheating may be facilitated
6
Terminology

Every multiple choice item has a stem,
which presents a stimulus and several
options or alternatives to choose
from.

One of these options, the key, is the
correct response. The others serve as
distractors
7
Match the Test Items to the
Blueprint and Task Specifications

Addresses curriculum objectives in terms of
performance (can do)


Outline of the test

Skills to be included

Item types and tasks

Topics to be covered

Elicitation and response formats for items

Number of items in each section

Time allocation

Plan for scoring
8
Using Item Indices to Accept,
Discard, or Revise Items

Item facility: the extent to which an item is
easy or difficult for the proposed group of test
takers

Item discrimination: the extent to which
an item differentiates between high- and low-
ability test-takers

Distractor efficiency: the extent to which
the distractors ‘lure’ a sufficient number of

test takers
9
Multiple Choice Tips I

Each test item should be designed to measure a specific
objective

The question or task should be clear, testing only one
concept

The answer should involve interpretation of the material,
not just background knowledge

A context should be provided for the items

The items should not show an insensitive or unfair bias

The number of response options should be the same in
each question

Ambiguity in the answers must be avoided; the intended
answer should clearly be the only correct one.

The options should all be similar in length and level of
difficulty
10
Multiple Choice Tips II

Items at different levels of difficulty should be
included: start with the easiest first


Both the stem and options should be stated as
simply and directly as possible

Writing illogical or nonsensical distractors should be
avoided

Extraneous clues should not influence the answer

The correct answers should appear in all positions (A,
B, C, D)

Words should not be repeated in the answers

A negative ‘stem’ should be avoided The questions
should not be dependent on previous items

Appropriate punctuation should be used
11
Helpful Resources

ESL dictionaries

ESL online dictionaries

/>
Thesaurus

Online thesaurus


Grammar reference texts

High school textbooks

Flesh-Kincaid Readability Grade (MSW)
12
Sample MC Questions
Evaluate the following multiple choice
items.

What is the objective of each question?

Identify problem areas (see Multiple
Choice Tips)

Correct/improve problem areas
13
MCQ#1
Excuse me, do you know _____?
a. where is the post office
b. where the post office is
c. where post office is
14
MCQ#2

An architect is a person who does not
_____.
a. design automobiles
b. design buildings
c. design houses

d. design offices
15
MCQ#3
My sister _____ as an air hostess.
a. working
b. works
c. she works
d. job
16
MCQ#4
I usually participate in badminton
competitions _____ November.
a. from
b. on
c. at
d. to
17
MCQ#5

Tuesday and Wednesday come before
Thursday but after _____.
a. Sunday
b. Saturday
c. Monday
d. Friday
e. none of the above
18
MCQ#6
The answer to the last question is the
_____ day of the week.

a. first
b. forth
c. three
d. fifth
19
MCQ#7
Hilary lives in New York City. _____ works
in the hotel business.
a. her
b. he
c. him
d. she
20
MCQ#8
I’m thirsty. I think I’ll drink an _____.
a. orange
b. knife
c. apple cider
d. umbrella
21
MCQ#9
While Susan prepared the ____ for lunch,
her mother watched television.
a. kitchen
b. children
c. chicken
d. cartoon
22
MCQ#10
Can you type _____ on a computer?

a. typing
b. to type
c. typed
d. type
23
MCQ#11
Although the plan has some
disadvantages, they are outweighed
by the _____.
a. advantages
b. negatives
c. drawbacks
d. problem
24
MCQ#12
Roberto is lazy. He always takes a siesta
_____ the afternoon.
a. in
b. during
c. at
d. throughout
25
MCQ#13
My eyesight has really been deteriorating
lately. I wonder if I need glasses. I
think I’d better go to the _____ to
have my eyes checked.
a. pediatrician
b. dermatologist
c. optometrist

d. audiologist