Tài liệu Conducting Statewide Customer Satisfaction Surveys pdf

Conducting Statewide
Conducting Statewide
Customer Satisfaction
Customer Satisfaction
Surveys
Surveys
Brad Sickles
U.S. Department of Labor
Employment and Training
Administration

Measuring Customer
Measuring Customer
Satisfaction
Satisfaction
T he
T he
Measures
Measures

Participant Customer Satisfaction
Participant Customer Satisfaction

Employer Customer Satisfaction
Employer Customer Satisfaction
Customer ratings on each of the three core
Customer ratings on each of the three core
questions regarding satisfaction are
questions regarding satisfaction are
weighted to account for regional

weighted to account for regional
differences and reported on a 1 to 100
differences and reported on a 1 to 100
scale. Computation of the ACSI is
scale. Computation of the ACSI is
discussed in the WIA reporting guidance.
discussed in the WIA reporting guidance.
The American Customer Satisfaction
The American Customer Satisfaction
Index (ACSI) is used to report statewide
Index (ACSI) is used to report statewide
results to the Department.
results to the Department.

The Core ACSI Questions
The Core ACSI Questions
1. Utilizing a scale of 1 to 10 where “1” means “Very
1. Utilizing a scale of 1 to 10 where “1” means “Very
Dissatisfied” and “10” means “Very Satisfied” what is your
Dissatisfied” and “10” means “Very Satisfied” what is your
overall satisfaction with the services provided from
overall satisfaction with the services provided from
________?
________?
2. Considering all of the expectations you may have had
2. Considering all of the expectations you may have had
about the services, to what extent have the services met
about the services, to what extent have the services met
your expectations? “1” now means “Falls Short of Your
your expectations? “1” now means “Falls Short of Your

Expectations” and “10” means “Exceeds Your
Expectations” and “10” means “Exceeds Your
Expectations.”
Expectations.”
3. Now think of the ideal program for people in your
3. Now think of the ideal program for people in your
circumstances. How well do you think the services you
circumstances. How well do you think the services you
received compare with the ideal set of services? “1” now
received compare with the ideal set of services? “1” now
means “Not very close to the Ideal” and “10" means “Very
means “Not very close to the Ideal” and “10" means “Very
Close to the Ideal.”
Close to the Ideal.”

Participant Customer
Participant Customer
Satisfaction
Satisfaction
T he
T he
Measures
Measures
The weighted average of participant ratings on
The weighted average of participant ratings on
each of the three questions regarding overall
each of the three questions regarding overall
satisfaction reported on a 1 to 100 scale.
satisfaction reported on a 1 to 100 scale.


Who:
Who:
Participant with a case closure
Participant with a case closure

When:
When:
After case closure and no later than
After case closure and no later than
60 days after date of case closure
60 days after date of case closure

How:
How:
Telephone survey
Telephone survey

Required completions:
Required completions:
500 statewide
500 statewide

Required response rate:
Required response rate:
70%
70%

Employer Customer Satisfaction
Employer Customer Satisfaction
T he

T he
Measures
Measures
The weighted average of employer ratings on
The weighted average of employer ratings on
each of the three questions regarding overall
each of the three questions regarding overall
satisfaction reported on a 1 to 100 scale.
satisfaction reported on a 1 to 100 scale.

Who:
Who:
Employers who receive a substantial
Employers who receive a substantial
service.
service.

When:
When:
1to 60 days after date of the
1to 60 days after date of the
completion of a service. For employers who
completion of a service. For employers who
listed a job order where no referrals were made,
listed a job order where no referrals were made,
contact should occur 30 to 60 days after a job
contact should occur 30 to 60 days after a job
order was listed.
order was listed.


How:
How:
Telephone survey
Telephone survey

Required completions:
Required completions:
500 statewide
500 statewide

Required response rate:
Required response rate:
70%
70%


Capturing Customer Satisfaction
Capturing Customer Satisfaction
Results to Assess Local WIBs
Results to Assess Local WIBs

States may use the ACSI questions and approach to measure
States may use the ACSI questions and approach to measure
satisfaction at the local level
satisfaction at the local level

States may also develop their own customer satisfaction
States may also develop their own customer satisfaction
instruments and approach for assessing local WIBs
instruments and approach for assessing local WIBs


Approaches must be applied
Approaches must be applied
uniformly and consistently to all
uniformly and consistently to all
WIBs
WIBs

WIA reporting guidance outlines
WIA reporting guidance outlines
parameters for collecting customer satisfaction results
parameters for collecting customer satisfaction results


Response rates: 70%.
Response rates: 70%.

Response levels below the specified minimums
Response levels below the specified minimums
will
will


invalidate the results.
invalidate the results.

States are required to determine the appropriate sample
States are required to determine the appropriate sample
sizes. States are no longer required to draw minimum
sizes. States are no longer required to draw minimum

sample sizes of 1,000.
sample sizes of 1,000.

Respondents
Respondents
must
must
be told responding to the survey is
be told responding to the survey is
voluntary and the information will be kept confidential.
voluntary and the information will be kept confidential.

States
States
must
must
create a process for creating an up-to-date
create a process for creating an up-to-date
customer list, capturing information on each customer’s
customer list, capturing information on each customer’s
address and telephone number.
address and telephone number.


Surveys to Obtain State Level
Surveys to Obtain State Level
Results
Results



A survey is c
A survey is c
omplete
omplete
when valid answers are provided
when valid answers are provided
by the respondents for each of the core ACSI questions.
by the respondents for each of the core ACSI questions.
Valid answers are responses 1 through 10.
Valid answers are responses 1 through 10.

The calculation of the ACSI score includes only the
The calculation of the ACSI score includes only the
results from
results from
complete
complete
surveys.
surveys.

The introductory statements for both the participant
The introductory statements for both the participant
survey and the employer survey were revised to better
survey and the employer survey were revised to better
set the context for the interviews.
set the context for the interviews.
Surveys to Obtain State Level
Surveys to Obtain State Level
Results
Results


Local program staff must inform the customer at the
Local program staff must inform the customer at the
time of participation about the importance of satisfying
time of participation about the importance of satisfying
customers and the possibility of being contacted for
customers and the possibility of being contacted for
information on his or her experience with the services.
information on his or her experience with the services.

The interview should be limited to 15 minutes or less.
The interview should be limited to 15 minutes or less.

A minimum of five follow-up attempts is required,
A minimum of five follow-up attempts is required,
involving various times of the day before closing the
involving various times of the day before closing the
record.
record.
Surveys to Obtain State Level
Surveys to Obtain State Level
Results
Results


Local programs should collect alternate contact
Local programs should collect alternate contact
information.
information.


A letter in advance of the survey should be sent out
A letter in advance of the survey should be sent out
informing the customer that he or she can expect to be
informing the customer that he or she can expect to be
contacted about his or her satisfaction with the services.
contacted about his or her satisfaction with the services.


Surveys to Obtain State Level
Surveys to Obtain State Level
Results
Results

Creating Questions to
Creating Questions to
Supplement the ACSI
Supplement the ACSI

Customer service
Customer service
is measured by standards set by you
is measured by standards set by you
or your agency.
or your agency.

Customer satisfaction
Customer satisfaction
is measured by the
is measured by the

customer’s standards for the
customer’s standards for the
services, regardless of whether they
services, regardless of whether they
make sense.
make sense.
Customer Satisfaction Not the Same as
Customer Satisfaction Not the Same as
Customer Service
Customer Service

Drivers of Satisfaction
Drivers of Satisfaction
•
Convenience
Convenience
•
Accessibility
Accessibility
•
Ease of use
Ease of use
•
Timeliness
Timeliness
•
Safety
Safety
•
Reliability

Reliability
•
Accuracy
Accuracy
•
Thoroughness
Thoroughness
•
Fairness
Fairness
•
Appropriateness
Appropriateness
•
Attractiveness and cleanliness
Attractiveness and cleanliness
•
Courteous
Courteous
•
Professional
Professional
•
Attentive
Attentive
•
Friendly
Friendly
•
Helpful

Helpful
•
Knowledgeable
Knowledgeable
•
Prompt
Prompt
•
Informative
Informative
•
Honest
Honest
•
Candid
Candid
Service Quality
Service Quality
Measures
Measures
Customer Service
Customer Service
Characteristic
Characteristic

Characteristics of Firms With
Characteristics of Firms With
High Customer Satisfaction
High Customer Satisfaction


Customers define quality
Customers define quality

Variety of services
Variety of services

Customization of services
Customization of services

Convenience of services
Convenience of services

Timeliness of services
Timeliness of services

Continually identify factors that influence satisfaction
Continually identify factors that influence satisfaction

Continually identify expectations and set customer
Continually identify expectations and set customer
service standards well above these expectations (to
service standards well above these expectations (to
“delight” customers)
“delight” customers)

Writing Effective Questions
Writing Effective Questions

Remember your survey's purpose: to provide
Remember your survey's purpose: to provide

you feedback on the level of satisfaction.
you feedback on the level of satisfaction.

If you have doubts about a question, discard it.
If you have doubts about a question, discard it.

State your questions simply.
State your questions simply.

Stay focused when writing questions; avoid vague areas.
Stay focused when writing questions; avoid vague areas.

If a question can be misinterpreted by the respondent, chances are
If a question can be misinterpreted by the respondent, chances are
that it will be.
that it will be.

Include only one subject per question.
Include only one subject per question.

Avoid questions that lead respondents.
Avoid questions that lead respondents.

Consider optional ways to ask questions that deal with sensitive
Consider optional ways to ask questions that deal with sensitive
areas.
areas.

Writing Effective Questions
Writing Effective Questions


Create questionnaire items to determine the key drivers
Create questionnaire items to determine the key drivers
of satisfaction…
of satisfaction…

Constructing Effective
Constructing Effective
Response Items
Response Items

Answer options need to be mutually
Answer options need to be mutually
exclusive and exhaustive.
exclusive and exhaustive.

Keep open‑ended questions to a
Keep open‑ended questions to a
minimum.
minimum.

Respondents interpret time-oriented
Respondents interpret time-oriented
responses differently, such as “never,”
responses differently, such as “never,”
“sometimes,” and “always.”
“sometimes,” and “always.”

Consider a "don't know" responses when you create answer
Consider a "don't know" responses when you create answer

sets.
sets.

Create meaningful scales for rating services.
Create meaningful scales for rating services.


The first series of questions should be easy for the
The first series of questions should be easy for the
respondent to understand and should capture his or her
respondent to understand and should capture his or her
attention and interest.
attention and interest.

Start with general questions and gradually shift to more
Start with general questions and gradually shift to more
specific questions.
specific questions.

Group questions in logical sections and
Group questions in logical sections and
sequence sections or questions in a well thought
sequence sections or questions in a well thought
out order.
out order.

Introduce each new section with a
Introduce each new section with a
summary statement so that participants
summary statement so that participants

have an opportunity to adjust to the
have an opportunity to adjust to the
new set of questions
new set of questions
.
.
Ordering Questions
Ordering Questions

Ordering Questions
Ordering Questions

Position questions about personal or sensitive issues toward the
Position questions about personal or sensitive issues toward the
end of the questionnaire.
end of the questionnaire.

The order of similar items on a list can bias results. Randomly or
The order of similar items on a list can bias results. Randomly or
alphabetically order items and indicate in the instructions how
alphabetically order items and indicate in the instructions how
they are ordered, reducing the likelihood that respondents will see
they are ordered, reducing the likelihood that respondents will see
the first items as most important of the group.
the first items as most important of the group.

Put demographic questions at the end of the survey form.
Put demographic questions at the end of the survey form.

Seek to minimize the number of times the a “skip rule” is

Seek to minimize the number of times the a “skip rule” is
followed as confusion can result with too many skips or when
followed as confusion can result with too many skips or when
skips become overly complicated.
skips become overly complicated.

Resources
Resources

Fowler, F. J., Jr.
Fowler, F. J., Jr.
Survey Research Methods (2nd ed.)
Survey Research Methods (2nd ed.)
. Thousand Oaks, CA: Sage
. Thousand Oaks, CA: Sage
Publications, Inc., 1993.
Publications, Inc., 1993.

Hayes, Bob E.
Hayes, Bob E.
Measuring Customer Satisfaction: Development and Use of
Measuring Customer Satisfaction: Development and Use of
Questionnaires
Questionnaires
. Milwaukee: ASQC Quality Press,1993.
. Milwaukee: ASQC Quality Press,1993.

Payne, S. L.
Payne, S. L.
The Art of Asking Questions

The Art of Asking Questions
. Princeton, NJ: Princeton University Press,
. Princeton, NJ: Princeton University Press,
1951.
1951.

Schuman, H and Presser, S.
Schuman, H and Presser, S.
Questions and Answers in Attitude Surveys: Experiments
Questions and Answers in Attitude Surveys: Experiments
in Question Form, Wording, and Context
in Question Form, Wording, and Context
. New York: Academic Press,1981.
. New York: Academic Press,1981.

Simply Better!
Simply Better!
The Voice of the Customer, Second Edition
The Voice of the Customer, Second Edition
. Washington: U.S.
. Washington: U.S.
Department of Labor, Employment and Training Administration, 1998.
Department of Labor, Employment and Training Administration, 1998.

Sudman, S. and Bradburn, N.
Sudman, S. and Bradburn, N.
Asking Questions: A Practical Guide to Questionnaire
Asking Questions: A Practical Guide to Questionnaire
Design
Design

. San Francisco, CA: Jossey‑Bass Inc., 1982
. San Francisco, CA: Jossey‑Bass Inc., 1982

Vavra, T. G.
Vavra, T. G.
Improving Your Measurement of Customer Satisfaction
Improving Your Measurement of Customer Satisfaction
. Milwaukee:
. Milwaukee:
ASQC Quality Press, 1997.
ASQC Quality Press, 1997.