CHAPTER 6METHODS OF MEASURING BEHAVIOR

CHAPTER 6
METHODS OF MEASURING BEHAVIOR

Nguyen Tien Dzung
Hanoi University of Science and Technology
Email:
Website:


Chapter Overview
1.
2.
3.
4.
5.

Deciding on a Method
Tests and Their Development
Types of Tests
Observational Techniques
Questionnaires

© 2012 NTD

Research Methods - Chapter 6

2


1. Deciding on a Method to Measure Behavior:
Some Important First Considerations

● Is the tool you propose to use reliable and valid?
● Base your choice of research tools on how you have
asked the research question

© 2012 NTD

Research Methods - Chapter 6

3


2. Tests and Their Development
● A test should measure the nature and extent of
individual differences
● A good test differentiates people based on true
scores

© 2012 NTD

Research Methods - Chapter 6

4


Why Use Tests?
●
●
●
●
●

Help determine outcomes of experiments
Can be used to diagnose strengths and weaknesses
Assist in placing individuals in appropriate programs
Assist in selecting applicants
Used to evaluate a program’s effectiveness

© 2012 NTD

Research Methods - Chapter 6

5


3. Types Of Tests: Overview
Types of Tests

What It Does

Achievement Test

Assesses an individual’s knowledge of a
specific area

Attitude Test

Assesses an individual’s feelings about
an issue

Personality Test

Assesses stable individual behavior
patterns

© 2012 NTD

Research Methods - Chapter 6

6


Achievement Test Sources
● Standardized
● Commercially prepared for wide use
● Scoring instructions included

● Researcher-made
● Designed by user for specific purpose
● Scoring instructions specific to test

© 2012 NTD

Research Methods - Chapter 6

7


Achievement Tests Referencing
What Comparison Do Tests Make?
● Norm-referenced—Individual’s scores are

compared to the scores of other individuals.
● Criterion-referenced—Individual’s scores are
compared to defined performance standards

© 2012 NTD

Research Methods - Chapter 6

8


Multiple-choice Achievement Items
Anatomy of a Multiple-Choice Item
12. Intelligence tests that are given to
preschool children

a. favor middle-class children

STEM
Clearly written
CORRECT
ANSWER

b. have questionable construct validity
c. are based on motor skills
d. are no fun at all

DISTRACTERS
Should be plausible (b & c), not easily ruled out (d)

© 2012 NTD

Research Methods - Chapter 6

9


To Use Or Not To Use?
● Advantages
●
●
●
●
●
●

● Disadvantages

Versatile
Easy to score
Simple to take
Poor writers not penalized
Good items used again
Good distracters are
diagnostic
● Hard to fake correct answer

© 2012 NTD

● Limit student’s options

● No opportunity to practice
writing
● Some people don’t do well
on them
● Limits content to be
assessed
● Items must be well written

Research Methods - Chapter 6

10


Item Analysis: How To Tell If Your
Items Work
● Questions should discriminate those who know the
material from those who don’t
● Item analysis provides two measures of a
question’s ability to discriminate
● Difficulty index
● Discrimination index

© 2012 NTD

Research Methods - Chapter 6

11


Computing Indices

First Steps
1. Rank scores from highest to lowest
2. Choose top 27% of scores for ―high‖ group
3. Choose bottom 27% of scores for ―low‖ group

© 2012 NTD

Research Methods - Chapter 6

12


Computing Indices
12. Intelligence tests that are given to preschool children
a. favor middle-class children
b. have questionable construct validity
c. are based on motor skills
d. are no fun at all
Item Alternative

A

B

C

D

Total

High Group
(n = 41)

23

12

4

2

41

Low Group
(n = 41)

11

9

15

6

41

Total

34

21

19

8

82

© 2012 NTD

Research Methods - Chapter 6

13


Computing Indices
Difficulty index
● Proportion who
answered item correctly

Discrimination index
● Proportion in high group who
answered correctly – proportion
in low group who answered
correctly

NCh  NCl
D
T

NCh  NCl
d
0.5T

NCh

= number of people in high group answering correctly

NCl

= number of people in low group answering correctly

T

= total number of people in high and low groups

© 2012 NTD

Research Methods - Chapter 6

14


Discrimination Level

Relationship Between Item Discrimination
and Difficulty
Perfect
+1.00
Discrimination

When:
Difficult,
Easy,
½ gets item
Good discrimination
Good discrimination
right,
½ gets item
wrong
0
&
50%
½ right is in
0%
100%
upper half,
Difficult,
Easy,
½ wrong is in
Poor discrimination
Poor discrimination
lower half

-1.00
Difficulty Level
© 2012 NTD

Research Methods - Chapter 6

15

Attitude Tests : Thurstone’s Scale
Item

Agree

No Strong
Feeling

Disagree

_____

_____

_____

_____

_____

_____

The dining room should serve
gourmet food.

_____

_____

_____

My parents don’t appreciate how
smart I am.

_____

_____

_____

My professors don’t appreciate
how smart I am, either.

_____

_____

_____

The day before Thanksgiving
should be a holiday.
Final exams should be elective.

© 2012 NTD

Research Methods - Chapter 6

16

Thurstone Scales
Method of Equal Appearing Intervals
● Statements indicating an attitude toward a topic are
written
● Judges rank the statements from least favorable to
most favorable
● Statements receiving consistent ratings are given
the average score
● A set of statements covering the entire range of
attitudes is selected

© 2012 NTD

Research Methods - Chapter 6

17


Thurstone Scales
Administration
● Respondents check items with which they agree
● Well-formed attitudes indicated by consistently checking
either high or low items
● Poorly-formed or inconsistent attitudes indicated by
inconsistent pattern or by checking off many neutral
items

© 2012 NTD

Research Methods - Chapter 6

18


Likert Scales
● Statements indicating an attitude toward a topic are
written
● Items with clearly positive or negative attitudes are
selected
● Statements are listed with a space for respondent
to indicate degree of agreement

© 2012 NTD

Research Methods - Chapter 6

19


A Likert Scale
Directions: Indicate to what extent you agree or disagree with the statements listed below by
circling one of the following:
SA means that you strongly agree with the statement
A means that you agree with the statement
U means that you are undecided about the statement
D means that you disagree with the statement
SD means that you strongly disagree with the statement

Item

Rating

Government has no business funding child care
programs.

SD

D

U

A

SA

Child care should be supported by federal, state,
and local tax dollars.

SD

D

U

A

SA

© 2012 NTD

Research Methods - Chapter 6

20


Scoring Likert Responses
Method Of Summated Ratings
Item

Rating

Government has no business funding
child care programs.

SD

D

U

A

SA

Child care activities are supported by
federal, state, and local tax dollars.

SD

D

U

A

SA

● Items are weighted
● Weights of unfavorable items are reversed
● Average score is computed

© 2012 NTD

Research Methods - Chapter 6

21


Personality Tests
● Projective tests
● Present respondent with ambiguous stimulus

● Structured tests
● Questions are objective
● Example: true-false, multiple choice, yes-no

© 2012 NTD

Research Methods - Chapter 6

22


4. Observational Techniques
● Researcher observes and records
● Does not interfere with behavior

© 2012 NTD

Research Methods - Chapter 6

23


Techniques For Recording Behavior
Technique

How it Works

Example

Duration
recording

The researcher records the
length of time that a behavior
occurs.

How much time is spent in verbal
interaction between two children?

Frequency
recording

The researcher records the
number of times a behavior
occurs.

How often are questions asked?

Interval
recording

The researcher observes a
subject for a fixed amount of
time.

Within a 60-second period, how
many times do members of the
group talk to another person?

Continuous
recording

The researcher records
everything that happens.

During a 1-hour period, all the

behavior of a 6-year-old boy is
recorded.

© 2012 NTD

Research Methods - Chapter 6

24


Observational Techniques? Be Careful!
● Pitfalls to Avoid
●
●
●
●

© 2012 NTD

Observer effects
Observer bias
Fatigue
Changing definitions

Research Methods - Chapter 6

25