Development of indicators on consumer satisfaction and Pilot survey

Project for the European Commission
Directorate General on Health and Consumer Protection





Development of indicators
on consumer satisfaction
and Pilot survey


Contract n° B5-1000/03/000382


FINAL REPORT



1 February, 2005

© Communautés européennes, 2005



This report was produced by a joint team INRA and Deloitte & Touche for DG Health &
Consumer Protection and represents the views of the contractor or author. These views
have not been adopted or in any way approved by the Commission and should not be re-
lied upon as a statement of the Commission's or DG Health & Consumer Protection's
views. The European Commission does not guarantee the accuracy of the data included in
this report, nor does it accept responsibility for any use made thereof.




Table of Contents
1.

Introduction 1

1.1

Background to this assignment _________________________ 1

1.1.1

The EU consumer policy _____________________________________________ 1

1.1.2

Policy-makers need information _______________________________________ 1

1.2

Scope of the assignment ______________________________ 2

1.2.1

Main objectives and expected outcomes_________________________________ 2

1.2.2

Policy-making needs ________________________________________________ 2

1.2.3

Geographical coverage_______________________________________________ 3

1.2.4

Sector coverage ____________________________________________________ 3

1.2.5

Additional requirements______________________________________________ 4

1.3

Methodology adopted_________________________________ 4

1.3.1

Overview _________________________________________________________ 4

1.3.2

Clarification of objectives and requirements ______________________________ 5

1.3.3

Desk research______________________________________________________ 5

1.3.4

Organisation of focus group discussions _________________________________ 6

1.3.5

Model development _________________________________________________ 6

1.3.6

Questionnaire design ________________________________________________ 7

1.3.7

Conducting a Pilot survey ____________________________________________ 7

1.3.8

Statistical analysis of the Pilot survey data _______________________________ 8

1.3.9

Development of final guidelines________________________________________ 8

2.

The measurement of consumer satisfaction 9

2.1

Introduction ________________________________________ 9

2.2

Concepts and definitions ______________________________ 9

2.2.1

Consumers ________________________________________________________ 9

2.2.2

Consumer satisfaction ______________________________________________ 10

2.2.3

Manifest and latent variables_________________________________________ 11

2.2.4

Indicators and indexes______________________________________________ 14

2.3

The development of national customer satisfaction indexes __ 15

2.3.1

General concept and origin __________________________________________ 15

2.3.2

The American Customer Satisfaction Index and related approaches __________ 15

2.3.3

The European Consumer Satisfaction Index _____________________________ 17

2.4

Development of a provisional model ____________________ 18

2.4.1

Conceptual model _________________________________________________ 18

2.4.2

Overview of the variables for the provisional model_______________________ 20

2.4.3

Main differences from other models ___________________________________ 22

2.5

Discussion of the variables included in the provisional model _ 22

2.5.1

(Perceived) Quality_________________________________________________ 22

2.5.2

(Perceived) Pricing _________________________________________________ 24

2.5.3

Image ___________________________________________________________ 25

2.5.4

Market and personal factors _________________________________________ 26

2.5.5

Consumer expectations _____________________________________________ 27

2.5.6

Complaint behaviour _______________________________________________ 30

2.5.7

Alternatives and substitutes__________________________________________ 31

2.5.8

Trust and commitment______________________________________________ 31

2.5.9

Overall consumer satisfaction ________________________________________ 33



3.

The Pilot survey 34

3.1

Questionnaire______________________________________ 34

3.1.1

Design of the questionnaire__________________________________________ 34

3.1.2

Structure of the questionnaire________________________________________ 34

3.1.3

Contents of the questionnaire ________________________________________ 35

3.1.4

Rating scales _____________________________________________________ 41

3.2

Sampling plan _____________________________________ 41

3.2.1

Objective ________________________________________________________ 41

3.2.2

Sample __________________________________________________________ 41

3.3

Fieldwork and timing ________________________________ 42

4.

The analysis and validation of the model 43

4.1

Consistency of the proposed model - Reliability and manifest
variables analysis ___________________________________ 43

4.1.1

The method: Cronbach’s alpha _______________________________________ 43

4.1.2

Results __________________________________________________________ 44

4.2

Direct versus indirect measurement of expectations ________ 46

4.2.1

Disconfirmation theory______________________________________________ 46

4.2.2

Indirect measurement ______________________________________________ 46

4.2.3

Results: (dis)confirmation versus overall satisfaction ______________________ 47

4.3

Data Modelling Process ______________________________ 48

4.3.1

The method: SEM _________________________________________________ 48

4.3.2

Why SEM? _______________________________________________________ 50

4.3.3

The modelling process ______________________________________________ 50

4.3.4

Results __________________________________________________________ 52

4.3.5

A consumer satisfaction index ________________________________________ 56

4.3.6

Conclusions ______________________________________________________ 57

5.

Consumer Satisfaction Survey – Guidelines for implementation 58

5.1

Context and objectives_______________________________ 58

5.2

Consumer satisfaction survey - Technical parameters _______ 58

5.2.1

Geographical coverage and target population____________________________ 58

5.2.2

Interview technique ________________________________________________ 59

5.2.3

Sample __________________________________________________________ 61

5.2.4

Final questionnaire_________________________________________________ 62

5.2.5

Research plan_____________________________________________________ 63

Research plan 1 – CATI long questionnaire___________________________________ 63

Research plan 2 – CATI short questionnaire __________________________________ 63

Research plan 3 – Face-to-face Ad hoc ______________________________________ 63

5.3

Consumer satisfaction survey – Indicators _______________ 64

5.3.1

Standard analyses and reports _______________________________________ 64

5.3.2

Analyses based on the model ________________________________________ 69

5.3.3

Other considerations _______________________________________________ 71

Annex1. Selected bibliography and sources used ______________ 72

Annex 2. Overview of the results from the focus group discussions 75

Annex 3. Questionnaire survey used for the Pilot survey ________ 97


Development of consumer satisfaction indicators & Pilot survey 1

1. Introduction
1.1

Background to this assignment
1.1.1 The EU consumer policy
In its ‘
Consumer Policy Strategy 2002-2006’
1
, the European Commission states that the de-
velopment of a consumer policy at European Union (EU) level is the essential corollary of the
progressive establishment of the internal market. Indeed, the free circulation of goods and
services requires the adoption of common, or at least convergent, rules to ensure both suffi-
cient protection of consumer interests, and the elimination of regulatory obstacles and com-
petitive distortions.

The EU consumer policy includes the following areas of concern:

o A number of safety, economic and legal issues relevant to consumers in the internal
market place, and consumer information and education. Products and services placed
on the internal market should be safe. Consumers should receive adequate and rele-
vant information in order to make appropriate choices. Consumers should also be pro-
tected from abusive practices.

o A coherent and common environment that will ensure that consumers are confident in
shopping across borders throughout the EU. Consumers should have comparable op-
portunities to benefit fully from the potential of the internal market in terms of greater
choice, lower prices, and the access to and affordability of essential services.

o Ensuring that internal market rules and practices promote consumer confidence in
cross-border transactions. Barriers to cross-border trade should therefore be over-
come.

1.1.2 Policy-makers need information
In its strategy paper, the Commission underlined the need for consumer policy to be backed
by relevant information and data, in order to adjust policies and set the appropriate priori-
ties. It stated that a more comprehensive, systematic and continuous effort is needed to de-
velop a suitable knowledge base as an essential tool for policy-makers. The development of
indicators on consumer satisfaction is mentioned explicitly as one of the key actions needed
to expand this knowledge base and to improve the quality of consumer policies.

The Commission, through its Directorate General on Health and Consumer Protection (DG
SANCO), has already undertaken preliminary initiatives to gauge the degree of ‘satisfaction’
of consumers. These initiatives include the EUROBAROMETER and focus groups.

Now the time has come to develop a more rigorous policy tool, in the form of a set of con-
sumer satisfaction indicators. To achieve this objective, as a first step the Commission
launched a call for tenders for the development of a European set of indicators for consumer
satisfaction and for the launch of a Pilot survey. The assignment was awarded to a joint
team of INRA Europe and Deloitte. This report presents the final results of this assignment
2
.


1
COM (2002) 208 of 7 May 2002
2
Preliminary research results and a more comprehensive discussion of the components of consumer
satisfaction have already been reported to the Commission within the first and second interim reports.

Only a synthesis of these elements is reported in this final report.
Development of consumer satisfaction indicators & Pilot survey 2


1.2

Scope of the assignment
1.2.1 Main objectives and expected outcomes
The first objective of the assignment was to develop a methodology for the construction of
consumer satisfaction indicators in the European Union. This methodology should be practi-
cal and have a sound scientific basis, reflecting recent research insights into consumer satis-
faction and its measurement.

The second objective was to develop and conduct a Pilot survey, based on the proposed
methodology. The purpose of this Pilot survey was to test the methodology and its underly-
ing modelling, and to propose a preliminary set of indicators.

Both objectives were interlinked. The analysis of the Pilot survey’s outcomes was expected to
indicate some adaptations to the methodology developed in the first stage.

The report was also intended to include an appropriate survey framework (i.e. question-
naires, population and sampling, survey methods, etc.), a proposal for the statistical meth-
ods to be used, and methods for calculating and visualising the consumer satisfaction indica-
tors.

1.2.2 Policy-making needs
There are many ways of designing indicators. Within this assignment, a key criterion was
that the indicators should be designed in such a way that they are helpful in European con-
sumer policy-making.

More specifically, the set of consumer satisfaction indicators to be developed was meant to
enable the European Commission to:
• understand how consumers perceive certain markets, what their main requirements
are, and how key service areas meet their expectations
• identify priorities for improvements, i.e. the areas where improvements will produce
the greatest gain in consumer satisfaction
• benchmark performance amongst EU countries within particular sectors
• benchmark sectors’ performance within a specific country or at the EU level
• set goals for improvement and monitor progress.

Ideally, the indicators should become a reference tool for European policy-makers, allowing
them to gauge overall consumer satisfaction levels and to measure the specific elements that
determine satisfaction levels in individual areas. The indicators should also provide signals of
whether the internal market is functioning properly and whether corrective regulatory or en-
forcement measures need to be taken.

Development of consumer satisfaction indicators & Pilot survey 3


1.2.3 Geographical coverage
The original specification within the tender (issued for the first time in 2002) was targeted at
the former composition of the EU, covering the 15 countries that were then members. The
Pilot survey initially foreseen in the proposal of the contractors covered these countries.

During the assignment the Commission and the contractors agreed to modify the original
work programme, to ensure that the proposed consumer satisfaction indicators would be ap-
plicable to the new EU members (e.g. to avoid ignoring factors that are of particular impor-
tance in these countries). This led to some changes in the initial research approach and in
the sample of countries used in the Pilot survey.

1.2.4 Sector coverage
Some existing tools and surveys in certain countries, such as national customer satisfaction
indexes (see next chapter), produce ‘overall’ indicators of consumer satisfaction, relating to
the economy as a whole.

This was not the purpose of this assignment. Rather, the Commission was interested in the
construction of indicators that covered a relatively small number of sectors
in a rigorous and
in-depth way. The sectors to be included within the scope of this assignment were:

• Utilities (gas, water, electricity)
3

• Postal services
• Mobile telephone
• Fixed telephone
• Urban transport (within town/cities: tram, bus, underground, rail/RER)
• Extra-urban transport (between town/cities: rail, bus)
• Air transport
• Retail banking (retail financial services for individual consumers)
• Insurance.

Extension to other (and possibly all) sectors of the economy could be envisaged at a later
stage, but was not part of the assignment. Care has been taken, nevertheless, that such
possible extensions at a later stage would not be jeopardised.

In summary, the indicator set should allow meaningful comparisons of how consumers feel
(a) across sectors in one Member State; (b) in one sector across Member States; (c) over
time.

3
Following the outcomes of the focus groups, it was decided in common agreement with the Commis-
sion, that the sector Utilities (gas, water, electricity) was to be represented by one of its components
in the Pilot Survey. This was necessary in order to respect the 20 minutes’ duration of the interview.
The electricity sector was selected for the pilot survey. However, the model that was developed re-
mains applicable to all three sectors.
Development of consumer satisfaction indicators & Pilot survey 4


1.2.5 Additional requirements
When developing the tools and methodology, the following considerations were also kept in
mind:

o The methodology should be powerful as well as practical and easy to use at a reason-
able cost. This means that some trade-offs have to be made.
• A ‘perfect’ model for measuring consumer satisfaction would take into ac-
count all possible factors for a given sector. Such a model may be complete,
but it also risks becoming irrelevant from a practical point of view. Indeed,
the questionnaire surveys that are required to feed data into a complex
model might be so cumbersome and time-consuming that it is no longer
feasible to undertake the survey.
• A ‘perfect’ model for one sector will be different from that for another sec-
tor. This means that comparability will be lost. Such a model could also
quickly become outdated, as new changes in the sector would require con-
tinuous adaptation. Therefore, a less perfect model may be preferable if it
allows greater comparability between sectors and is more robust against
changes over time.
There is thus no point in merely constructing a methodology for consumer satisfaction
measurement that is ‘perfect’, if it would not be economically feasible to undertake the

surveys.

o Consumer satisfaction is not something static, but evolves dynamically with the envi-
ronment. For instance, satisfaction is strongly related to expectations, and in a period
of economic downturn (or in regions that are declining), people tend to lower their ex-
pectations. Such considerations should also be built into the survey methodology, to
ensure that the results remain reliable and useful over time. The model should allow
adaptation at a later stage, if this is necessary.


1.3

Methodology adopted
1.3.1 Overview
INRA and Deloitte have undertaken a range of activities and applied several methodologies,
in order to complete this assignment.

The key steps in the assignment were:
o Clarification of objectives and requirements
o Desk research
o Organisation of focus group discussions
o Model development
o Questionnaire design
o Conducting a Pilot survey
o Statistical analysis of the survey data
o Development of final guidelines.

On several occasions, inputs were obtained from experts in the field of customer satisfaction
and statistical analyses, to ensure the validity of the proposals and findings.
Development of consumer satisfaction indicators & Pilot survey 5

1.3.2 Clarification of objectives and requirements
During the early stages of the assignment, the objectives and scope of the assignment were
clearly identified.

The following elements were discussed with the Commission:
o Purpose of the assignment
o Type and nature of the indicators required
o Need for comparability with other initiatives, such as ACSI or EPSI
o ‘Consumer’ concept (as compared with ‘customer’)
o Geographical coverage
o Sector coverage
o Characteristics of the Pilot survey.

This clarification of requirements mainly took place at the beginning of the assignment.
Some elements were discussed following the submission of the interim reports.

1.3.3 Desk research
A major desk research effort was undertaken. The focus was on two main areas:
o Existing (national) models and tools for consumer satisfaction indicators
o Research results in relation to consumer behaviour and satisfaction measurement.

The existing national consumer satisfaction indices and their underlying methodologies and
assumptions were analysed in depth
4
. This analysis allowed us to identify most of the factors
that were likely to be related to consumer satisfaction, as well the way these factors have
typically been aggregated into so-called ‘latent’ variables such as ‘quality’.

At the same time, a literature survey was conducted on consumer satisfaction and its meas-
urement. The state of the art and recent research results were reviewed. This was necessary
to fully understand the current methodologies for consumer satisfaction measurement, and
to be able to develop a robust and scientifically valid methodology that could incorporate the
Commission’s requirements. The full list of documents analysed is included in Annex 1 of this
report.

Thanks to the desk research activities, valuable insight was gained about past and current
theories on consumer satisfaction measurement and the construction of indicators.

The desk research was mainly conducted in the period January–March 2004; it continued
throughout the assignment.


4
A full description of several of these indicators was provided as an annex to the first interim report.
Development of consumer satisfaction indicators & Pilot survey 6


1.3.4 Organisation of focus group discussions
In the first phase of the assignment, eleven focus group discussions were organised in a
number of different cities across Europe, including three in new EU countries
5
. The focus
groups consisted of consumers that were representative of the population by sex, age and
socio-cultural background. The focus groups were led by a psychologist or psychologically
trained moderator, who made sure that all topics were discussed, that all participants had a
chance to express their thoughts and that views were widely exchanged.

The main purpose of these focus groups discussions was to get a clear picture of the most

relevant factors for consumer satisfaction/dissatisfaction. This picture served as a comple-
ment, and partially a validation, of the results of our desk research.

The specific objectives of the focus groups were:
o To develop a good understanding of what ‘consumer satisfaction’ and ‘consumer dis-
satisfaction’ really mean for European consumers (in relation to the sectors that were
selected for the Pilot survey)
o To identify the most relevant drivers of consumer satisfaction, both generally and in
each of the sectors concerned
o To assist in the development of a model for consumer satisfaction measurement, a
model that could be applicable across sectors and countries.

The group dynamic of the focus groups yielded a rich inventory of opinions and attitudes.
The focus group outcomes helped to enrich and validate all the key manifest variables that
had been identified throughout the desk research
6
.

The focus group discussions took place in the period March–April 2004.

1.3.5 Model development
Based on the results of the desk research and the focus groups, we developed a ‘provisional’
and fairly comprehensive model for consumer satisfaction measurement. The model is pre-
sented in the next chapter.

This model incorporates a long list of factors that are related to consumer satisfaction (e.g.
reliability, availability of staff, price transparency, respect of confidentiality, trust and so on),
including some specific requirements of the Commission concerning its consumer policy (e.g.
cross-border purchasing).

These factors, or ‘manifest variables’
7
, were grouped into higher level ‘constructs’ – the so-
called ‘latent variables’ (e.g. quality, price, image, market and personal factors), between
which cause and effect relationships are assumed to exist (e.g. ‘price’ affects ‘image’). Sur-
veys undertaken on the basis of such a model allow the measurement of consumer satisfac-
tion and also explain the main factors which contribute to it.


5
In Germany, France, and the UK, two focus group sessions were organised at different locations.
One focus group discussion was organised in Italy, Spain, Poland, Hungary and the Czech Republic.
6
Detailed information about the outcomes of the focus group discussions were provided in the first
interim report and its annexes.
7
The definition of “manifest” and “latent” variables will be discussed in the next chapter.
Development of consumer satisfaction indicators & Pilot survey 7


This model incorporated several characteristics that are also found in national models for
consumer satisfaction measurement. However, it also had distinct features. New elements
and relationships were added. It was also designed in such a way as to allow a cross-sector
coverage, yet can take into account specific characteristics per sector.

The model development was an interactive process, carried out in the period March–May
2004.

The model developed at that stage was a ‘provisional’ one, since it had to be validated and
refined following the Pilot survey. A number of modifications were indeed carried out in No-

vember 2004, following a detailed data-analysis.

1.3.6 Questionnaire design
The provisional model formed the basis for the design of the questionnaire survey.
Essentially, the questionnaire started from the lists of manifest and latent variables that had
been defined in the model. For each manifest variable, an appropriate survey question was
constructed. All questions were to be rated on a scale from 1 to 10. Where needed, ques-
tions were adapted to take into account the specificity of the sector concerned.

In addition, survey questions were added in order to screen respondents, identify service
suppliers, and to capture the respondents’ profile. Questions in relation to overall satisfaction
for the service were also added.

The questionnaire design took place in the period June–July 2004. It was discussed with the
Commission and approved before the survey was launched.

1.3.7 Conducting a Pilot survey
The main objective of the Pilot survey was to obtain reliable data that could be used to vali-
date the model and finalise the methodology.

In agreement with the Commission, a sampling plan for the Pilot survey was established.
The original plan, which included 2,400 surveys in 15 countries, was modified into one
whereby 3,600 consumers would be surveyed in eight countries: Germany, the UK, Italy,
Belgium, Portugal, Sweden, Poland and the Czech Republic. This allowed for a sample size of
450 interviews per country. For each interviewee, questions were asked in relation to two
sectors.

This approach greatly increased the statistical validity of the results, while providing a good
representativeness of Europe by region (North-South and West-East) and by country size.

The Pilot survey in the eight countries took place in the period August-September 2004.
Development of consumer satisfaction indicators & Pilot survey 8


1.3.8 Statistical analysis of the Pilot survey data
Based on the results of the Pilot survey, we conducted a detailed statistical analysis. The aim
was to define a consumer satisfaction model, ideally common to the 11 sectors covered by
the project.

The main statistical analyses conducted in this phase included:
o Consistency tests on the initial model (Cronbach's alpha method)
o Validation of the indirect measurement of expectations
o Data modelling (Structural Equation Modelling – SEM)

The statistical analysis of the survey data was conducted in September and October 2004.

1.3.9 Development of final guidelines
The objective of this final part was to draw guidelines for the implementation of a (continu-
ous) Consumer Satisfaction Survey and the production of relevant indicators. The final part
of this report - based on the model developed - includes:
o Technical specifications: geographical coverage, target group, sample, interview
technique, data collection and questionnaire
o Analysis and reporting: standard and model-based.

The finalisation of the methodology and the preparation of this final report were carried out
in the period September–November 2004.

Development of consumer satisfaction indicators & Pilot survey 9

2. The measurement of consumer satisfaction
2.1

Introduction
This chapter discusses the elements and key concepts of the proposed methodology and the
underlying model for the construction of consumer satisfaction indicators.

In section 2.2 we introduce and define a number of terms and concepts. This is important in
view of the interpretation of the results, since there is no universally agreed definition or in-
terpretation of many of the concepts discussed in this report.

Section 2.3 presents the characteristics of a number of existing models and methodologies
for customer satisfaction measurement. Since this subject has been extensively discussed
and documented in the first interim report, this final report summarises the key findings in
this regard.

We then propose in section 2.4 a new model for consumer satisfaction measurement and its
components. This model has been the basis for the Pilot survey. It was later slightly modi-
fied, in view of the outcomes of this Pilot survey (see chapter 4).

The final section of this chapter discusses the components of the model.

We would like to stress that there are different ways and methods of measuring consumer
satisfaction, depending on the context and purpose of the exercise. The existing methodolo-
gies differ from each other in one or more aspects and most of them continue to evolve.

The methodology that we propose in this report can therefore not be considered as the ‘de-
finitive’ answer to the on-going discussion on consumer satisfaction measurement. We be-
lieve, nevertheless, that it is a highly relevant approach that takes into account the specific
nature of the scope of the assignment (sectors and countries) and other requirements and

expectations of the Commission – which are somewhat different from the assumptions and
requirements that underlie other models and approaches.


2.2

Concepts and definitions
2.2.1 Consumers
The concepts of ‘consumers’ and ‘customers’ are related. In some languages, hardly any dis-
tinction exists between these two terms. Even in English the terms are often used inter-
changeably. For instance, the term ‘customer satisfaction’ is widely used, but a closer look
would reveal that in most cases a more precise term would be ‘consumer satisfaction’.

Some experts make the following distinction between a consumer and a customer:
o The consumer is the one who uses a product or service,
o Whereas a customer pays for the product/service, but may not be the consumer.

Simply stated: the consumer is the ‘user’; the customer is the ‘buyer’.

Another dividing line is that organisations (e.g. companies) can be customers, whilst con-
sumer as a concept is reserved for individuals. Consumer in this project means “any natural
person who buys a product for purposes that do not fall within the sphere of his/her com-
mercial or professional activity” (see Art. 2, e), directive (98/6/EC). By this, we clearly mean
that we address the end-consumer (B2C) and not any business intermediaries (B2B).
Development of consumer satisfaction indicators & Pilot survey 10


These differences may be important when modelling a consumer satisfaction indicator. In-
deed, satisfaction requires experience and use of a product or service. Individuals who pay
for a product or service but do not use it do not assess the product or service in the same

way as consumers.

In this report, we have chosen to use the term ‘consumer’ systematically, unless reference is
made to other documents or existing indicators. It should be borne in mind, however, that
the model and methodology developed also allow for analysis of the satisfaction of ‘custom-
ers’, at least when they are end-users and not any kind of business intermediary.


2.2.2 Consumer satisfaction
A widely accepted definition of ‘satisfaction’ is:

‘Satisfaction is the consumer’s fulfilment response. It is a judgement that a product or ser-
vice feature, or the product of service itself, provided (or is providing) a pleasurable level of
consumption-related fulfilment, including levels of under-or-over fulfilment.’

(Oliver, 1997)

In less ‘technical’ terms: satisfaction is the consumer’s assessment of a product or service in
terms of the extent to which that product or service has met his/her needs or expectations.
Failure to meet needs and expectations is assumed to result in dissatisfaction with the prod-
uct or service
8
.

Depending on the context, the meaning of ‘consumer satisfaction’ may differ:

o Consumer satisfaction may relate to a particular feature or characteristic of a product
or service (e.g. the accuracy of the information provided by a bank), or alternatively it
may relate to the product/service as a whole.

In general, it is the satisfaction about the product/service as a whole that merits atten-
tion, since this satisfaction influences the consumers’ future buying and consuming be-
haviour. Yet it is also important to understand the factors that contribute to (dis)satis-
faction. Often, dissatisfaction about one particular feature of a service (e.g. the un-
friendliness of staff) leads to dissatisfaction about the service as a whole, even if the
satisfaction about the other features is high.

o Another conceptual distinction within measures of consumer satisfaction is whether
they relate to satisfaction about a single ‘service encounter’ (e.g. your most recent ex-
perience with a bank) or rather reflect ‘cumulative satisfaction’ (e.g. in relation to all
your experiences with banks). It is possible to be quite satisfied about one particular
transaction in the post office, but be dissatisfied about the quality of the postal services
overall (or vice-versa).

Most existing national customer satisfaction indexes are based on the measurement of
cumulative satisfaction. This implies that different elements are part of the consumer’s
assessment, from their pre-purchase intentions up to their loyalty towards a particular
provider.


8
We must add a nuance when it comes to ‘commodity’ services such as urban transport. Consumers
may have no explicit or specific expectations for such services, making it hard to define satisfaction in
relation to prior expectations.
Development of consumer satisfaction indicators & Pilot survey 11


o Another potentially important distinction is between consumer satisfaction about a
particular supplier, and consumer satisfaction about the sector as a whole. For in-
stance, one may be quite satisfied with the offerings and performance level of a par-

ticular mobile phone operator, but in general feel uncomfortable with the pricing and
marketing strategy as a whole.

Most national customer satisfaction indexes measure the satisfaction level for the sec-
tor as a whole. Yet they do this by calculating a weighted average for the satisfaction
about individual suppliers.

For the development of the proposed methodology, we have made the following choices in
this regard:

o We measure both consumer satisfaction about services overall, as well as about the
services’ particular characteristics: quality aspects, pricing issues, image, etc. This dou-
ble approach makes it possible to identify the key factors that contribute to increasing
or decreasing overall satisfaction.

o We measure the cumulative satisfaction relating to the last 12 months
9
– in as far as
this makes sense (sometimes there is only one service encounter over that period).

o Survey respondents are asked about their experience with their main provider of ser-
vices. This is then aggregated to present an assessment for the sector as a whole.
There is, however, no calculation of the satisfaction about the services of a particular
supplier.

2.2.3 Manifest and latent variables
Measuring consumer satisfaction is, in principle, relatively straightforward: a representative
sample of consumers is asked to what extent they are satisfied or dissatisfied about a par-
ticular service. In general, such satisfaction is measured on ordinal scales (e.g. not at all sat-
isfied – not fully satisfied – more or less satisfied – quite satisfied – very satisfied) or simple

numerical scales (e.g. from 0 to 4 or from 1 to 10).

Often it is also useful to know why consumers are (dis)satisfied, i.e. to understand which
factors or criteria contribute positively or negatively to the measured level of satisfaction. Un-
derstanding what makes a consumer happy or unhappy allows suppliers and policy-makers
to implement appropriate action.

Those who are interested in measuring consumer satisfaction are usually also interested in
its implications. For instance, will a consumer continue to use the same supplier or will
he/she switch to an alternative? Currently, the prediction of consumer behaviour – in particu-
lar their loyalty towards the supplier – is what most suppliers are interested in, rather than
the satisfaction itself.

Identifying and measuring such factors is, however, not straightforward. Indeed, there are
many factors that affect and/or are related to consumer satisfaction. For a typical service, as
many as thirty or so of these factors may have to be considered. Moreover, these factors
differ between products, services and sectors; and individuals may rank the relative impor-
tance of these factors in different ways.


9
Respondents that have not used the service over the last 12 months are excluded from the survey.
Development of consumer satisfaction indicators & Pilot survey 12


Whilst these problems can be overcome when focusing on a particular environment (e.g. a
specific product or service for a specific target group), some trade-offs have to be made
when designing a tool that allows valid comparisons across user groups, countries and sec-
tors. This is the case in this assignment and it was also the challenge faced by those in
charge of designing the national customer satisfaction indicators that already exist.

One way of approaching this issue is by creating a ‘hierarchy’ of factors:

o First of all, there are the measurable ‘factors’ or ‘criteria’ related to consumer satisfac-
tion (e.g. prompt service, adequate opening hours, competitive pricing, etc). These are
called the manifest variables. Each manifest variable corresponds to a question in
the survey to the consumers. Many of these manifest variables are similar across sec-
tors, but some are specific. Even if they are identical, their importance may vary be-
tween sectors.

o Secondly, a number of related manifest variables have been aggregated into ‘latent
variables’
10
. A latent variable reflects a relatively complex dimension of consumer
satisfaction that cannot be measured directly. Examples of such latent variables are
quality, price and image.

This approach is consistent with methodologies used elsewhere. It has several advantages.
Firstly, at the level of the latent variables, a model can be constructed that is identical across
sectors and countries. Thus, it becomes meaningful to make comparisons between countries
and sectors.

Secondly, when using latent variables, the model can remain stable over a considerable pe-
riod of time, hence increasing comparability of results. Specific survey questions and mani-
fest variables may change over time and across sectors, but the model at the level of latent
variables remains unaffected.

Lastly, provided the latent variables have been chosen appropriately and have been labelled
in a meaningful way, the use of latent variables makes it possible to explain the key ele-
ments affecting consumer satisfaction, without undue reference to a long list of very specific

characteristics.

Since some latent variables – in particular quality – consist of quite a considerable number of
manifest variables, during the model design it appeared useful to create an intermediate
level between manifest and latent variables, called ‘drivers’. Thus, a driver groups a few
strongly related manifest variables or survey questions. The ‘drivers’ category was therefore
added to facilitate the reading of the list of manifest variables and to categorise them into
sub-groups. The model construct and its related methodology are only based on the mani-
fest and latent variables.

Please note that, given the absence of consensus on the definitions and terms, the terminol-
ogy adopted here – manifest variables/drivers/latent variables – is not necessarily the one
used by all the authors. We have opted for a terminology and definitions that are most rele-
vant for our approach and which are acceptable to all experts. The following table provides
definitions and alternative terms also found in the literature.


10
The aggregation of manifest variables into latent variables was based on assumptions about logical
relationships between them, derived from desk research and focus groups. The results of the Pilot
survey have to a very large extent confirmed the logic of the proposed groupings.
Development of consumer satisfaction indicators & Pilot survey 13



Terminology
adopted in this re-
port
Definitions adopted
in this report

Terms also found in the litera-
ture with an identical or similar
meaning
Manifest variables
A manifest variables
corresponds to a question
in the survey to the con-
sumers
Measurement variables
Drivers
Attributes
Characteristics
Criteria
Drivers
A driver groups a few
strongly related manifest
variables or survey ques-
tions

(This intermediate level is not used
elsewhere – national models or lit-
erature)
Latent variables
A latent variable reflects a
relatively complex
dimension of consumer
satisfaction that cannot be
measured directly
Variables
Latent factors

From a practical point of view, it is also useful to qualify the latent variables (and the under-
lying manifest variables) according to their explanatory power. They can be categorised into
three groups:

o ‘Explanatory variables’: these are the factors that directly influence consumer satisfac-
tion. In the literature, theses explanatory variables are also referred to as ‘antece-
dents’, ‘causes of satisfaction’ or ‘input factors’.

o ‘Resultant variables’: these are the measures of overall consumer satisfaction. In gen-
eral, there are only a very few such manifest variables.

o ‘Consequent variables’: these represent the consequences of the consumer (dis)satis-
faction, and express concretely the outcomes from the consumer satisfaction/dissat-
isfaction. Other terms that can be found are ‘consequences’ or ‘outcomes of satisfac-
tion’.

In summary:

Terminology adopted
in this report
Definitions adopted
in this report
Terms also found in the lit-
erature with an identical or
similar meaning
Explanatory variables
Factors that directly influ-
ence the consumer satis-

faction
Antecedents
Causes of satisfaction
Input factors
Resultant variables
Measures of overall con-
sumer satisfaction
Overall results
Outputs
Output factors
Consequent variables
Consequences of the con-
sumer (dis)satisfaction
Consequences
Outcomes of satisfaction

Development of consumer satisfaction indicators & Pilot survey 14


2.2.4 Indicators and indexes
Indexes (or ‘indices’) are constructs that try to capture a complex situation into a single fig-
ure. Well-known examples are the consumer price index, the inflation rate, the consumer
confidence index, the stock market index, etc. Such an index is calculated by aggregating
weighted averages of results or prices of a large number of different goods, services, etc. In
most cases, an index covers most of the products and services of a given economy, and is
presented as a single number or ratio (a value on a scale of measurement).

Most existing national customer satisfaction measurement systems
11
produce such an index

for consumer satisfaction. The index is often available at several levels, which are then ag-
gregated to produce an overall, nationwide and cross-sector index. For instance, the Ameri-
can Customer Satisfaction Index (ACSI) (see section 2.3.2.) includes four levels:
o A national customer satisfaction index
o Indices for seven broad areas of the economy
o Indices for 39 economic sectors
o Indices for over 200 major companies and government agencies within these sectors.

An ‘indicator’ is also a statistical figure capturing a given situation. The term ‘indicator’ refers
to a quantitative measure that provides a simple and reliable basis for assessing a feature
(e.g. consumer satisfaction or an aspect of it) and measuring the related performance, proc-
esses and outcomes over time. A useful indicator allows comparisons over time, across sec-
tors and countries. An indicator could be derived from a single quantitative measure, or
rather be based on the measures of several manifest variables. An indicator may even be
related to a single customer.

Again, we found no agreed and universally accepted definition of both terms – and their dif-
ference – in the literature.

Based on the needs of the Commission, we consider that the term ‘indicators’ is most appro-
priate for what is required. Firstly, this assignment covers only a limited number of sectors –
so it is not even possible to create an overall index at European or national level. Secondly,
the need is not so much to have a single figure, but rather to produce a range of measures
that capture different aspects of consumer satisfaction. Lastly, using the term indicators is
helpful for distinguishing this methodology from the typical ‘index’-approaches that are more
industry-oriented and economic-driven.


11
See next section.

Development of consumer satisfaction indicators & Pilot survey 15


2.3

The development of national customer satisfaction indexes
12

2.3.1 General concept and origin
The last fifteen years have witnessed the emergence of a number of national customer satis-
faction ‘indexes’ or ‘barometers’. A customer satisfaction index (CSI) is meant to be a na-
tionwide gauge of how adequately companies (and other organisations) satisfy consumers.

There are many similarities between the existing CSIs and their underlying methodologies.
The basic structure of the CSI models has been developed over a number of years. The
models and methodological developments are based on scientific advances in the under-
standing of consumer behaviour, customer satisfaction measurement and product/service
quality.

A key feature underlying all approaches is that they are based on a ‘model’. This model con-
sists of a number of latent variables (such as ‘quality’ or ‘image’) and the cause and effect
relationships between them. Each of these latent variables includes several manifest vari-
ables that act as concrete proxies for the latent variable. Consumer satisfaction is the latent
variable that is at the centre of the model; it is encased within a system of variables relating
to causes and effects. A good model should be capable of predicting a pattern of relation-
ships and effects

In 1989 the Swedish Customer Satisfaction Barometer (SCSB) was the first truly national
customer satisfaction index for domestically purchased and consumed products and services.
Originally, it contained two primary explanatory variables of satisfaction, namely the per-

ceptions of a customer’s recent performance experience with a product or service, and the
customer expectations regarding that performance.

Since then, other models and methodologies have been developed, taking their inspiration
from the SCSB and its successors. In general, the models have become more complex and
the number of manifest variables has increased over time.

2.3.2 The American Customer Satisfaction Index and related approaches
The American Customer Satisfaction Index
(ACSI) was introduced in 1994. Like its Swedish
predecessor, it is a uniform and independent measure of household consumption experience.
ACSI produces a customer satisfaction index based on measures from seven broad economic
areas, 39 industrial sectors and more than 200 companies and public agencies.

The ACSI survey is funded partly by corporate subscribers, who receive industry benchmark-
ing date and company-specific information.

The ACSI model is a set of causal equations that link the latent variables of customer expec-
tations, perceived quality and perceived value to customer satisfaction. In turn, satisfaction
is linked to consequences as defined by customer complaints and customer loyalty. When
simplified, this model may be presented as follows.



12
This section only presents some of the key findings about the existing national indexes. More infor-
mation about these approaches can be found in the first interim report and its annexes.
Development of consumer satisfaction indicators & Pilot survey 16

The ACSI approach includes a number of features that can also be found in other similar na-
tional indexes:
o It is based on an econometric model with measures of an index of satisfaction and
measures of related indices for latent variables that are general enough to be compa-
rable across companies, industries and sectors.
o Customer satisfaction itself is measured as a latent variable using several manifest
variables (questions).
o Customer satisfaction is embedded in a system of cause-and-effect relationships. This
serves to validate the index.
o Finally, a primary objective is to estimate the effect of ACSI on customer loyalty, a
construct of universal importance for future business performance.

This model is an evolution of the original Swedish Barometer. It started from the same
framework, but became more complex in several regards. It rapidly evolved to include two
distinct types of perceived quality – namely the product quality and the service quality, this
distinction being used solely for the manufactured products.

The ACSI model became the basis of the frameworks used in customer satisfaction initiatives
undertaken in New Zealand, Taiwan, Austria, Norway, Hong Kong and elsewhere.

The Norwegian development is worthwhile mentioning. The Norwegian Customer Satisfac-
tion Barometer was created in 1996. It is almost identical to ACSI, except for the introduc-
tion of a new manifest variable ‘corporate image’ and its relationships with customer satisfac-
tion and customer loyalty. The argument for this addition was that people relate corporate
image to organisation-related associations.
Customer
satisfaction
Customer

expectations

Loyalty
Customer
complaints
Perceived
value
Perceived

quality

Consumer
satisfaction
Consumer

expectations

Loyalty
Consumer
complaints
Perceived
value
Perceived

quality

Development of consumer satisfaction indicators & Pilot survey 17


2.3.3 The European Consumer Satisfaction Index

The ECSI initiative (European Consumer Satisfaction Index
) is another variation of the ACSI
model. The so-called ‘European Customer Satisfaction Programme’ was launched by a num-
ber of private and non-profit European organisations, complemented by some national plat-
forms. Three objectives were pursued with this initiative:
o Provide companies, public services, consumers, investors, regulators and policy-makers
with an annual customer satisfaction index and analysis of companies and public ser-
vices in Europe
o Provide companies and public services surveyed with the means to analyse the percep-
tions of their customers (causes and effects) and to compare them with the percep-
tions of customers of other companies and public services at different levels (sector,
country, Europe, USA, East Asia)
o Introduce the European Consumer Satisfaction Index as a recognised economic macro
indicator measuring the performance of the national and European economy.

The experience started in 1997, with a European feasibility study for coordinated national
CSIs. The first result was a model for CSI measurement, based on the Swedish and Ameri-
can model. The pilot phase study took place in 1999 and covered 11 countries (100,000 in-
terviews conducted). The survey conducted in 2003 covered nine countries, five common
industries and required 90,000 interviews. In 2003, nine countries were involved in the en-
deavour (Denmark, Finland, Greece, Iceland, Ireland, Norway, Portugal, North-West Russia
and Sweden) and five industries were covered (retail banking, fixed-line telecoms, mobile
phones, insurance and supermarkets).

ECSI has been modelled in a way that is similar to ACSI. Yet there are some differences:

o The split between ‘product quality’ (hardware) and ‘service quality’ (software) has been
generalised. Quality is related to the consumer’s quality experience with a service, and
refers both to:
• its technical components (product quality or hardware): security/safety, informa-

tion, reliability, confidentiality, etc.
• the quality of the associated functional services (service quality or software), such
as opening hours, friendliness of personnel, consumer dialogue, complaint han-
dling, etc.

o The latent variable ‘customer loyalty’ has been specified in a different way. It includes
likelihood of retention, the likelihood of recommending the company or brand, and the
likelihood of an increase in the amount of customers purchasing the product.

o The variable ‘customer complaints’ was not taken into account.

o The variable ‘corporate image’, included in the Norwegian index, has become a latent
variable, with effects on customer expectations, satisfaction and loyalty.
Development of consumer satisfaction indicators & Pilot survey 18


Graphically, this can be displayed as follows:



It should be noted that, unlike the American and other national approaches, there is quite
some variation in the implementation of the measurement system – in terms of the coverage
of the model, the characteristics of the population surveyed, the size of the sample, the spe-
cific questions included, and so on. This variation is noticeable both between countries and
even within countries for different years.


2.4

Development of a provisional model

2.4.1 Conceptual model
Based on the requirements of the Commission, the results of the extensive desk research
and the organisation of the focus groups, we have gradually developed a fairly comprehen-
sive model for the measurement of consumer satisfaction. In this model, satisfaction is
treated as an overall evaluation of the consumption experiences (in general with several pro-
viders). It is comprehensive because all factors that may be related to consumer satisfaction
within the sectors concerned have been incorporated.

The model includes nine latent variables, each of which consists of several manifest vari-
ables. Based on the research undertaken, assumptions were made about the main cause-
and-effect relationships between the latent variables.
Customer
satisfaction
Image

Customer

expectations

Loyalty
Perceived

quality

Perceived
value
Soft features

Hard features

Consumer
satisfaction
Image

Consumer

expectations

Loyalty
Perceived

quality

Perceived
value
Soft features

Hard features

Development of consumer satisfaction indicators & Pilot survey 19


The following graph illustrates the model proposed:


The latent variables on the left-hand side of the model – Market / Image / Expectations /
Quality / Pricing (and the underlying manifest variables) – are the explanatory variables that
contribute to explaining satisfaction. The latent variables on the right-hand side – Complaint
behaviour / Alternatives / Trust – are the consequent variables.

The review of literature suggests that there could potentially be links between all the latent
variables listed. However, we have only drawn the paths (cause-effect relationships) which
were assumed to be the most probable and relevant ones given the scope of this project. As
will be demonstrated later in this report, the results of the Pilot survey have proven that
most of these assumptions were correct (see chapter 4).

Two important remarks need to be made at this stage:

o From a strictly scientific point of view, it is not possible to create a unique model for
consumer satisfaction measurement that will perfectly fit all sectors –not even at the
level of latent variables.

o The model that was created served for the design of the questionnaire for the Pilot
survey. It was only a provisional model, since an analysis of the survey responses was
needed to confirm the appropriateness of the grouping of manifest variables into latent
variables, and the existence of the proposed cause-and- effect relationships. Again this
has to a large extent been the case (see chapter 4).
Market &
personal
factors
Image
Trust &
commitment
Alternatives &
substitutes
Complaints
Perceived
quality
Perceived
pricing

Expectations
Overall
satisfaction
Market &
personal
factors
Image
Trust &
commitment
Alternatives &
substitutes
Complaints
Perceived
quality
Perceived
pricing
Expectations
Overall
satisfaction
Development of consumer satisfaction indicators & Pilot survey 20


2.4.2 Overview of the variables for the provisional model
The following table lists the latent variables, the drivers
13
and the manifest variables of the
provisional model. Each manifest variable corresponded with a question in the Pilot survey.

In this provisional model, the number of manifest variables exceeded 50. This high number
was achieved because absolute care was taken not to overlook any specific factor that might

contribute to or be related to consumer satisfaction.

Type
Latent
variables
Drivers Manifest variables
14

Reliability
Correct delivery of the service in due time
High quality of the technical service
Problems occurring with the service
Comfort of transport
(for transport sectors)

Information
Complete and regular information provision
Security/Safety
Provision of a safe and secure service
Confidentiality
Respect of privacy and discretion when dealing with delicate
problems
Quality
Consumer services
Availability of staff to contact (any time)
Ease of buying new services
Wide network of connections
(for transport sectors)

Professional, helpful and friendly staff

Maintenance and modernisation of infrastructure
Nearby agencies or point of sale
Friendliness with the customers
Flexible attitude and easy to deal with
Customer-minded. Putting the customer first
Prompt and adequate reaction to problems and complaints
Helpfulness in resolving problems
Satisfactory solving of problems
Affordability
Affordability
Special attractive offers
Pricing
Reasonable/competitive prices
Profitability
(banking/insurance)

Cost of taxes
Explanatory
variables
Price
Transparency
Clear and understandable tariffs and invoices
Easy payment process
Correct invoices

13
Drivers mentioned in
italics
are drivers with only one manifest variable or survey question. In other
words, the driver is identical to the manifest variable.

14
Not all manifest variables/survey questions are applicable to all sectors. The wording and content of
the survey questions may also differ somewhat between sectors.
Development of consumer satisfaction indicators & Pilot survey 21


Image
Overall corporate
image
Overall reputation. Positive image
Belief that one is being fairly treated
Unique image
Familiar with the supplier and what they do
Popularity
Technologically advanced and ability to innovate
Respecting the environment
Confidence
Confidence in continuity of service provision
Competitive structure
Sufficient competition
Ease of changing from one supplier to another
Differences between suppliers in the market
Cross-border
opportunities
Possibilities for cross-border purchase
Accessibility
Availability of services everywhere for everyone
Knowledge of the services of the supplier
Explanatory
variables

Market and
personal
factors
Personal factors
Care about the services to deal with
Tendency to stick with the same suppliers
Concern about always getting the best deal
Preference to work with a national supplier
Belief in the advantages of a liberalised market


Overall
satisfaction
Overall satisfaction based on total experience
Resultant
variables
Consumer
satisfaction
Meeting expectations
Extent to which experience meets expectations


Negative appraisals
Speaking negatively about the supplier
Complaint
behaviour
Complaints
Forwarding complaints to the provider
Forwarding complaints to third parties

Alternatives
and substi-
tutes
Move to an alternative
or substitute

Abandoning service
Turning to another supplier
Positive appraisal
Recommending to friends or colleagues
Positive out-
come/stability
Continuing to use this type of service
Continuing to use the same supplier
Consequent
variables
Trust and
commitment
Tolerance
Sensitivity to price increase or decrease

These variables will be discussed in section 2.5.