The world of the Counselor An introduction to the counseling profession 5e chapter 13

Research & Evaluation

1






“The inquiry of truth, which is the love-making, or wooing of
it; the knowledge of truth, which is the presence of it; and the
belief of truth, which is the enjoying of it; is the sovereign
good of human nature.” (Sir Francis Bacon, 1997/1597 , p. 6)
(from p. 430 of book)
Purposes:
 Inquiry of truth
 Development of new paradigms
 Without research, knowledge is stagnant
 Validates what we are doing
 In a sense, we are all “practitioner—scientists”

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

2






Literature Review

 Examination of prior research
 Use electronic data bases, such as
▪ ERIC
▪ PsycINFO
Statement of the Problem
 Places research in its historical context
 Discusses why issue at hand is important
 Points you in direction of developing research questions,
statements, and/or hypotheses
 See Box 13.1, p. 432 for an example

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

3




Two types:
 Quantitative
▪ Assumes objective reality which science can examine
▪ Probability that certain behaviors, values, or bleiefes
either cause or are related to other bheaivors values or
beliefs
 Qualitative
▪ Multiple ways of viewing knowledge
▪ Make sense of the world by immersing on in the
research situation
 See Table 13.1, p. 435 to describe differences

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

4


EXPERIMENTAL



Manipulating treatment to
show causation
Types
 True
 Quasi
 One Shot Case Study
and Group Pretest-PostTest Design

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

NONEXPERIMENTAL



Looking at relationships
among variables
Types
 Correlation
 Survey
 Ex Post Facto (Causal
Comparative)

5




True Experimental Research
 Crème de la crème of research
 Independent and dependent variables
 Random assignment
 Causation
▪ See Figure 13.1, p. 435

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

6




Quasi-Experimental
 Manipulating independent variable
 Looking for causation
 No random assignment
 Use of intact groups
 Nature of quasi-experimental research leads to less
credibility (more threats to internal validity) (see Box 13.2,
p. 437)
 See Box 13.3, p. 437

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

7


 One Shot Case Study and One Group Pretest/Post-Test

Design
▪ No random assignment
▪ There is manipulation of independent variable
▪ Example: “ABA” design
▪ Take Baseline measurements then offer treatment and
then take baseline measurements again
▪ Do repeatedly
▪ See Figure 13.2, p. 438

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

8




Correlational Research
 Studies relationship between variables using correlation coefficient
(positive correlations and negative correlations)
 Two types:
▪ Bivariate: Studies the relationship between two variables
▪ Simple: Here and now (see Figure 13.3, p. 439)
▪ Predictive: Now and the future (e.g.: GRE’s and GPA in college)

▪ Multivariate: The relationship among more than two variables
▪ Multiple regression: (e.g., relationship among empathy, being
nonjudgmental, building an alliance, and success in counseling)
▪ Many types: Discriminant analysis, factor analysis, differential
analysis, canonical correlational, and path analysis (last two are
considered “causal”)

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

9




Survey Research
 Uses questionnaires or interviews
 Purpose
▪ To gather information from a targeted population
▪ To gain information about the values, behaviors,
demographics, and opinions of a population
▪ See Steps: p. 441
▪ Does not tell us underlying reasons why
▪ Not causal
▪ Uses descriptive statistics

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

10



Ex Post Facto (Causal-Comparative)
 Use of intact groups
 Cannot manipulate independent variable
 Convenient, but hard to decide the reasons for differences
that may be found
 Example: Comparison of NCE students from CACREPaccredited programs with those from non-CACREP
accredited programs
▪ Even if you find differences, what are the reasons for
them???? (see some possible reasons, top p. 442)

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

11









Naturalistic-phenomenological philosophy
Many ways to interpret reality
Observe, describe, and interpret phenomena within
social context
Relies on “case study” method—focusing upon and
deeply probes and analyzes events or phenomena

A number of methods that allow “themes” to emerge
Types
 Grounded theory
 Phenomenolgoical Approach
 Ethnographic Research
 Historical Research

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

12




Grounded Theory
 Developed in 1960s by Glaser and Strass
 Describes a process (not a moment in time)
 E.g., Question: “How do counselors develop a theoretical
orientation toward their careers”
▪ Interview counselors with designated list of questions
(see questions, p. 443)—other questions might emerge

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

13




Grounded Theory (cont’d)

 Steps:
▪ Preparing: Reflecting on biases and preparing info collection
▪ Data collection: Gather information based on your process
(e.g., focus groups)
▪ Note taking: Keeping notes so you don’t have to rely on
memory
▪ Coding: Identifying common themes until you get
“saturation”
▪ Writing: Sort out themes into major categories and write up
findings

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

14




Phenomenological Designs
 Similar to grounded theory but less interested in
development of a “theory”
 E.g., Want to know about possible PTSD experienced by
those who escaped the Twin Towers during 9/11
▪ Purposeful sample
▪ Meet with these individuals
▪ Questions to give me “rich” and “thicker” descriptions
▪ Bracket my biases
▪ Data collection similar to Grounded Theory

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

15




Ethnographic Research [Description (graphy) of human cultures (ethno)]
▪ Margaret Mead was one of the first to do this
▪ Understand events through meanings that people make
▪ Steps:
1. Identify the group to be studied
2. Conduct a literature review
3. Decide on what method to immerse oneself in the culture
4. Make a plan for data collection
5. Plan, usually includes:
 Observation (see Box 13.4, p. 446)
 Ethnographic interviews
 Collection of Documents and Artifacts

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

16




Historical Research
 Purpose: To describe and analyze conditions and events
from the past in an effort to answer a research question
 Relies on systematic collection of information

 Generally, try to use primary sources (not secondary
sources):
▪ Oral histories
▪ Documents
▪ Relics

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

17







Descriptive statistics: Measures of central tendency, variability,
and relationship. Often used with survey research
Inferential statistics: Measures whether differences or strengths
of the relationships between groups are likely to be the result
of chance (e.g., t tests, ANOVA, MANOVA, significance of
correlation coefficient, chi square)
 Generally, experimental and ex post facto research uses
inferential statistics while correlational research examines
strength of relationships between variables
Effect size: Practical significance of one’s findings
 Example: male counselors more likely than female
counselors to try to persuade their clients not to have an
abortion, but vast majority of male and female counselors
would not do this despite a significant difference being found

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

18







Relies on inductive analysis: themes and categories emerge
from data
Often, coding is used: breaks down large amounts of data
into smaller parts that hold meaning for the researcher
Use multiple sources
Find themes
 See data collection methods and emergence of themes in:
 Box 13.5, p. 449
 Box 13.6, p. 450

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

19





“The degree to which scientific explanations of phenomena

match reality”
Two types of validity (Internal and External)
 Internal:
▪ The degree to which extraneous variables have been
accounted for and are not involved in rival hypotheses
▪ Discussed earlier in Box 13.2, p. 437
▪ Quantitative research tends to control for this more
than qualitative research

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

20




External validity
 How generalizable are the results?
 Tight controls of quantitative studies sometimes makes it
hard to generalize
 In qualitative research, external validity is the ability of the
researcher to describe the research in ways that will be
helpful to other researchers with other populations

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

21


Is validity valid in qualitative research?

Often, words like “credible and trustworthy” are used instead of
validity. Based on ability to accurately record information and
analyze results.
 Show credibility and trustworthiness by:
 Prolonged and persistent gathering of info
 Triangulation: multiple methods
 Bracketing biases
 Using an “informant” or “participant observer”
 Have an “outside auditor”
 Conduct member checks
 Other methods?
 See Box 13.8



© 2007 Thomson Brooks/Cole, a division of Thomson Learning

22










Abstract
Review of the Literature

Research Hypothesis
Methodology
Results
Discussion, Implications, Conclusions
References (APA)

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

23




The purpose of evaluation
 Has program achieved its goals and objectives
 Has program shown its value



Two types of evaluation
 Formative (Process) Evaluation
 Summative (Outcome) Evaluation

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

24




Formative or Process Evaluation
 Measures ongoing effectiveness
 Often Informal
 Methods:
▪ Ask for verbal feedback
▪ Have participants write down reactions
▪ Complete rating forms

© 2007 Thomson Brooks/Cole, a division of Thomson Learning

25