(EBOOK) methods in educational research from theory to practice (2nd edition)

SECOND EDITION

METHODS IN

EDUCATIONAL
RESEARCH
From Theory to Practice

MARGUERITE G. LODICO • DEAN T. SPAULDING • KATHERINE H. VOEGTLE



Methods in
Educational
Research



Methods in
Educational
Research
From Theory to Practice
S ECO N D E DITIO N

MARGUERITE G. LODICO
DEAN T. SPAULDING
KATHERINE H. VOEGTLE


Copyright © 2010 by John Wiley & Sons, Inc. All rights reserved.
Published by Jossey-Bass

A Wiley Imprint
989 Market Street, San Francisco, CA 94103-1741—www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or
by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted
under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright
Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or
on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the
Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax
201-748-6008, or online at www.wiley.com/go/permissions.
Readers should be aware that Internet Web sites offered as citations and/or sources for further information
may have changed or disappeared between the time this was written and when it is read.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or completeness
of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for
a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult
with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit
or any other commercial damages, including but not limited to special, incidental, consequential, or other
damages.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call
our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax
317-572-4002.
Jossey-Bass also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic books.
Library of Congress Cataloging-in-Publication Data
Lodico, Marguerite G.
Methods in educational research : from theory to practice/Marguerite G. Lodico, Dean T. Spaulding,
Katherine H. Voegtle.—2nd ed.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-470-43680-6 (pbk.)

1. Education—Research—Methodology. I. Spaulding, Dean T. II. Voegtle, Katherine
H. III. Title.
LB1028.L586 2010
370.72—dc22
2009051930
Printed in the United States of America
SECOND EDITION

PB Printing

10 9 8 7 6 5 4 3 2 1


CONTENTS

Boxes, Exhibits, Figures, and Tables ix
Preface xix
Acknowledgments xxiii
The Authors xxv
1

Introduction to Educational Research
Chapter Objectives

1

1
2

Educational Accountability and Educational Research

Conducting Educational Research 9
Philosophical Frameworks for Educational Research
Research Ethics
2

16

Types of Educational Research Designs
and Related Major Concepts 23
Chapter Objectives

23

Types of Approaches Used in Educational Research
3

13

Descriptive Statistics
Chapter Objectives

45

45

Characteristics of Data

46

Summarizing Data Using Descriptive Statistics 48

24


CONTENTS

VI

4

Educational Measurement: Archival Data
and Standardized Instruments 79
Chapter Objectives 79
Measurement in Education

80

Evaluating the Quality of Standardized Instruments:
Reliability and Validity 93
Issues in Finding and Using Standardized Instruments 101
5

Qualitative Measures and Procedures 111
Chapter Objectives

111

Characteristics of Qualitative Measurement 112
Sampling in Qualitative Research 134
6

Qualitative Research
Chapter Objectives
Research Vignette

141

141
142

Understanding Qualitative Research

142

Steps in Designing Qualitative Research 160
Evaluating Narrative Inquiry and Phenomenological
Research 169
7

Organization and Analysis of Qualitative Data 179
Chapter Objectives

179

Analysis of Qualitative Data

180

Steps in Analyzing Qualitative Data 180
8

Descriptive Survey Research: Quantitative Research
Chapter Objectives
Research Vignette

197
198

Characteristics of Descriptive Survey Research

198

Steps in Conducting Descriptive Survey Research
Evaluating Descriptive Survey Research
9

Experimental Research
Chapter Objectives
Research Vignette

227
228

227

222

201

197

CONTENTS

VII

Understanding Experimental Research

228

Steps in Planning and Conducting Experimental Research

230

Threats to Experimental Validity 241
Single-Subject Research Designs 253
10

Nonexperimental Approaches: Causal-Comparative
and Correlational Research 265
Chapter Objectives
Research Vignette

265
266

Causal-Comparative Research

266

Correlational Research 271
Multiple Regression Studies 288
11

Inferential Statistics
Chapter Objectives

293

293

Beyond Descriptive Statistics: Inferential Statistics 294
Steps in Analyzing Data Using Inferential Tests 302
Designs with More than One Independent
or Dependent Variable 306
12

Action Research

311

Chapter Objectives 311
Research Vignette

312

Understanding Action Research

313

Steps in Conducting Action Research 321
Data Sources for Action Research
Evaluation of Action Research
13

330

346

Program Evaluation in Education 361
Chapter Objectives
Research Vignette

361
362

What Is Program Evaluation? 363
Types, Approaches, and Models of Program Evaluation 371
Steps in Designing Program Evaluation 375
Program Evaluation Data and Critiquing Evaluation Reports 376
Training and Careers in Program Evaluation 378


CONTENTS

VIII

14

Identifying and Researching a Topic 383

Chapter Objectives
Getting Started

383

384

Identify a Research Topic

384

Refine Your Topic as You Search
Search the Literature

387

391

Identify and Summarize Key Information from Articles 404
When Do I Have Enough? 406
15

The Research Proposal 409
Chapter Objectives

409

Preparing a Research Proposal
References

427

Appendix A

435

Appendix B

469

Index

499

410


BOXES, EXHIBITS,
F I G U R E S , A N D TA B L E S

Boxes
1.1

Educational Reform and the No Child Left
Behind Act 2

1.2

Action Research Example

4.1

What Is the Real Graduation Rate for U.S.
Students? 82

4.2

High-Stakes State-Mandated Tests

4.3

Web Sites with Information on
Standardized Tests 106

7.1

Sample Interview Transcript from a Narrative Inquiry
Study 184

7.2

Thick Description of a Latino Youth 188

7.3

Contrasting Perspectives in a Case Study

8.1

Ethical Issues in Descriptive Survey Research 207

8.2

Samples of Ideal and Realistic Populations 213

9.1

Internal and External Validity in Action!
What Works in the Lab Might Not Work in the
Classroom 242

9.2

The Four Rules of Single-Subject Research 260

6

86

190


BOXES, EXHIBITS, FIGURES, AND TABLES

X

10.1

Ethical Issues in Correlational Prediction Studies 279

10.2

Inferences About Causation

12.1

Ethical Issues in Action Research

12.2

Action Research Journals

12.3

Action Research Centers or Institutes

13.1

Train Them and They Will Come. . . . Well, Maybe:
The Role of the Program Evaluator to Deliver Bad
News 370

13.2

Examples of Evaluation Objective and Benchmark

14.1

Sample Topics for Research Proposals

14.2

Useful Web Sites for Finding Information and Sources
for Reviewing the Literature 394

287
328

356
357

371

387

Exhibits
3.1

Sample Pearson r Correlation

5.1

Sample Observational Protocol and Recording
Sheet 117

5.2

Sample Field Notes Using an Observational
Protocol 120

5.3

Sample Interview Protocol for Feasibility Study 126

8.1

Student Perception Survey

8.2

Sample Demographics Form

8.3

Student Perceptions of the Impact of School
Uniforms on the School Community 210

8.4

Database for 10 Students from Alysia’s Study
on School Uniforms 220

73

205
208

11.1

Type I and Type II Errors

11.2

A 2 ؋ 2 Factorial Design 306

11.3

A 2 ؋ 2 Design Showing Mean Scores of Boys and Girls
on a Math Achievement Test 307

302


BOXES, EXHIBITS, FIGURES, AND TABLES

XI

12.1

Sample of Two Educators Using Critical Reflection to
Define a Research Problem 324

12.2

Questions for Planning an Action Research Study

12.3

Classroom Log for Special Education Teacher

12.4

Structured Student Log

12.5

Field Notes from Action Research Study 334

12.6

Checklist to Record Student Writing Strategies

12.7

Checklist with Open-Ended Categories 337

12.8

Teacher Survey on Use of Technology 340

12.9

Grading Rubric for Student Photography Project

327

331

333

336

343

12.10 Sample Planning Table for an Action Research Study 345
12.11 Summary of Frequency of Responses to Technology
Survey 355
13.1

Math Outcomes for State Annual Report

14.1

Distinguishing Among Types of Research: Experiential
Report Versus Empirical Study 393

14.2

Article Summary Sheet for Primary,
Research-Based Sources 404

15.1

Writing Tips: Title Page 413

15.2

Writing Tips: Introduction

15.3

Writing Tips: Statement of Purpose

15.4

Writing Tips: The Review of Literature 417

15.5

Writing Tips: Hypotheses or Subquestions

15.6

Writing Tips: Method Section

414

Figures
3.1

Sample Frequency Polygon

3.2

Sample Histogram

51

378

51

423

415

419


BOXES, EXHIBITS, FIGURES, AND TABLES

XII

3.3

Examples of Positively Skewed and Negatively Skewed
Distributions 52

3.4

Grades in Distance Education Classes 52

3.5

Distributions of Test Scores for Two School Districts 57

3.6

Normal Curve

3.7

“Box-and-Whisker” Graphs

3.8

Relationship Between High-Jump and Long-Jump
Scores 67

3.9

Scatterplot of Children’s Weight and Amount of
Physical Exercise 68

63
65

3.10

Scatterplot with Prediction Line 69

3.11

Scatterplot Representations of Correlation Coefficients
of Different Sizes 70

3.12

Scatterplot of Relationship Between Achievement Test
Scores and Test Anxiety 71

3.13

Relationship Between Achievement and Self-Esteem 74

4.1

Normal Curve with Test Scores 90

4.2

Overview of the Process in Developing Standardized
Instruments 94

7.1

Steps Involved in Analyzing Qualitative Data 193

9.1

Random Selection and Random Assignment

9.2

Number of Words Spoken by a Child with Autism
During Baseline Period 254

9.3

Sample A-B-A Design

9.4

A-B-A-B Design

9.5

Sample Multiple-Baseline Design

9.6

Group Experimental and Single-Subject
Research Add to Professional Knowledge and
Decision Making 261

10.1

256

257
259

Television Watching and Body Weight 272

234


BOXES, EXHIBITS, FIGURES, AND TABLES

XIII

10.2

Predicted Correlations Among Four Variables 275

10.3

Scatterplot Showing Restriction of Range in Amount of
Physical Exercise 282

10.4

Correlation Matrix for Study of Bullying 283

10.5

Example of Intervening Variable 286

11.1

A Distribution of 100 Scores 297

11.2

Normal Curve

11.3

Graph Showing Interaction of Gender and
Computer Use 308

12.1

Action Research Process

12.2

Rating Scale Using Nonverbal Responses 338

12.3

Photographs as Data Sources in Action Research

12.4

Use of Concept Map to Depict Relationships Among
Themes 354

12.5

Use of a Histogram to Display Results from a Parent
Survey 356

13.1

The Summative and Formative Process in Program
Evaluation 366

13.2

Overview of a Logic Model 373

14.1

Ways to Generate a Research Topic

14.2

Development of a Research Question 390

15.1

How Research Proposals Aid the Development of
Research 412

298

322

342

385

Tables
1.1

Sample Value-Added Data for Two School Districts 9

1.2

Qualitative and Quantitative Approaches: The Scientific
Process 12

1.3

Frameworks and Assumptions Underlying Educational
Research 17


BOXES, EXHIBITS, FIGURES, AND TABLES

XIV

3.1

Scales of Measurement

46

3.2

Frequency Table of Number of Books Read by 30
Students 50

3.3

Sample Grouped Frequency Table

3.4

Calculation of Mean from a Frequency Table 54

3.5

Means and Standard Deviations for the Number of
Words Written 64

3.6

Sample Table Showing Relationship Between Two
Variables 66

3.7

Student Math and ELA Scores 76

3.8

Correlations Among Five Variables 77

5.1

Characteristics of Qualitative Measurement 112

5.2

Advantages and Disadvantages of Surveys Versus
Interviews 122

5.3

Types of Interview Questions and Probes 128

5.4

Summary of Purposeful Sampling Strategies 135

6.1

Types of Narrative Inquiry

6.2

Theoretical Assumptions Underlying Narrative
Inquiry 148

6.3

Theoretical Assumptions Underlying
Phenomenological Research 150

6.4

Types of Ethnographic Studies

6.5

Steps in Conducting Qualitative Research 166

6.6

Criteria for Evaluating Qualitative Research
Studies 174

7.1

Common Code Categories and Examples
of Code Names from a Life Story of a Latino Youth

50

145

153

7.2

Report Formats for Presenting Findings from
Qualitative Research 194

8.1

Types of Descriptive Survey Designs 202

183


BOXES, EXHIBITS, FIGURES, AND TABLES

XV

8.2

Simple Random Selection

215

8.3

Criteria for Evaluating Descriptive Survey Research

9.1

Experimental Research

9.2

Hypotheses in Experimental Research 233

9.3

Factorial Design

9.4

Math Scores from Johanna’s Research Study 240

9.5

Threats to Internal Validity 244

9.6

Threats to External Validity 245

222

229

237

10.1

Common Statistical Tests for Examining Relationships
Between Variables 280

10.2

Practical Interpretations of Correlation
Coefficients 284

10.3

Multiple Regression Analysis for Study of Reading 289

11.1

Commonly Used Inferential Statistical Tests 304

12.1

Action Research Pioneers

12.2

Benefits of Action Research to Practitioners and the
Field of Education 317

12.3

A Triangulation Matrix

12.4

Questions for Identifying Patterns in an Action
Research Study 352

13.1

Components of a Logic Model

13.2

Example of an Evaluation Matrix 376

13.3

Data from School Records for Fifth-Grade Students
in Math 377

14.1

Commonly Used Databases in Education 397

14.2

General and Academically Focused Search Engines
and Their Web Sites 401

15.1

Sample Design 422

316

351

374



In loving memory of our colleagues Huey Bogan and Mark Ylvisaker. Huey inspired his students in
Teacher Education to be reflective practitioners who continually strive to improve their practice.
Mark exemplified the integration of research and clinical practice in his numerous
publications and inspired his students in Communication Sciences and
Disorders to become scholar practitioners. We miss them both dearly.



PREFACE

Four years ago we wrote the first edition of Methods in Educational Research: From
Theory to Practice with many expectations. As educational psychologists, we wanted
to write a textbook from the fundamental perspective of how one learns in
general and, more specifically, how one learns through conducting research.
In addition, we wanted to create a book that would pertain to techniques and
instructional practices underlying good teaching as well as to teach people about
educational research. We wanted to pay close attention to the metacognitive processes associated with learning about research and developing and becoming an
active participant in the educational research community.
We believe the purpose of this second edition is still to assist students, primarily graduate students, who are practitioners in education or related fields (administration, school psychology, or school counseling) to use educational research so
that they can become more effective educators. Specifically, the purpose of this
book is to help students develop a broad and deep understanding of research
methodologies that can be used to analyze and improve their practices. Overall,
we believe that we accomplished much of what we set out to do with the first
edition of this book; however, in using it for the past three years, we realized that
some areas needed to be expanded. This second edition expands on areas previously discussed as well as brings you several new chapters. We hope you enjoy the
additions and wish you the best in your educational research pursuits.
In chapter 1 , we have updated information on No Child Left Behind
(NCLB) and school reform as well as the knowledge and research skills needed


XX

PREFACE

by educators in the 21st century. Chapter 2 has been expanded to
provide more in-depth coverage of key concepts that we believe students need
to be able to read and understand the research on a particular topic or issue.
Chapter 3 focuses specifically on descriptive statistics and provides students
with detailed examples of basic statistical computations and how results are displayed. Chapter 4 has an expanded section on archival data and descriptive

statistics has been moved to chapter 3 . Chapters 5 through 7 have been
expanded, focusing on the different quantitative and qualitative types of
research, with specific data exercises embedded in each chapter to give students
a realistic framework for the types of data and possible analyses they may use
when conducting research. We have embedded sampling strategies for
descriptive survey research in chapter 7 . In chapter 8 , we expanded types
of qualitative research to those most relevant to practitioners. Chapters nine
through eleven are similar to those in the original edition. Perhaps the biggest change we have made to our book is chapter 12, which focuses entirely
on action research. The chapter includes a data activity to give students a
sense of the action research process. While the action research chapter is
placed late in the book, there are no concepts in it that preclude coverage
earlier in the semester. Chapter 13 , program evaluation, features a section
on logic modeling. Last, chapters 14 and 15 focus on generating ideas and
researchable topics and preparing the research proposal. We moved these
chapters to the end of the book to give instructors more flexibility in when
they are assigned.
The book includes special features designed to assist the teaching and learning processes:
❒ Research vignettes illustrating research that is tied to practice and used to
make decisions about educational practices open each chapter on research
approaches and are discussed throughout these chapters.
❒ The book includes extensive discussion of research issues and concepts
relevant to the accountability movement and using data to make decisions in
educational settings.
❒ Developmental processes involved in researching and writing a research
proposal are emphasized.
❒ Research proposals using both an action and a descriptive survey approach
are included in appendices, because we feel these approaches are useful to
practitioners. The appendices also include criteria for evaluating proposals
using these approaches.
❒ Key concepts students should know are set in bold letters in each

chapter.


PREFACE

XXI

❒ Suggested readings are provided at the end of each chapter to extend the
discussion of general issues raised in the chapter and provide citations for
sample studies that illustrate the type of research discussed.
❒ Discussion questions or activities are provided to stimulate thinking about
the issues raised in the chapter or encourage students to apply the concepts
presented.
❒ We did not include sample studies in the book; however, each chapter discussing a specific research approach (for example, descriptive survey, action,
experimental) includes a list of studies that may be used for class discussion
or assignments. Since many studies are available as full-text documents, we
decided to decrease our carbon footprint by encouraging students to access
these online.



PT
R GT M
WEON T S
A C KCNHOAW
LED

Any textbook on educational research owes a debt to the numerous people who
have built the rich and varied literature in this field. In some sense, this book
grew out of the conversations and relationships we have enjoyed with colleagues

over many years, especially at meetings of the American Educational Research
Association and the American Evaluation Association. Although we cannot
name all of these persons, we certainly could not have begun to think about
this book without the stimulation of many people in these vibrant educational
communities.
However, many people closer to home also made this book possible. The
College of Saint Rose and especially our dean, Margaret M. Kirwin, provided
substantial support for our work by granting us sabbatical leaves and making
available to us capable graduate assistants. The revisions of the book were
informed by feedback from faculty members who were using it and from students in our classes. Members of our department consistently encouraged us in
our writing, and our department chair, Richard Brody, always managed to get
people to cover courses as needed each semester. Our colleagues who are practicing educational researchers—James Allen, Aviva Bower, Donna Burns, David
DeBonis, Ron Dugan, Margaret McLane, Heta-Maria Miller, Travis Plowman,
and Ismael Ramos—each contributed his or her own special expertise and pedagogical ideas to the book.
Moira DeSanta, our graduate assistant, carefully read and edited chapters from our book. We are also certainly grateful to the staff at Jossey-Bass,