Customer satisfaction report

EUROCONTROL

Customer Satisfaction
Report

IANS

A Centre of Excellence in ATM Training

Report


Copyright Notice
© 2006 European Organisation for the Safety of Air Navigation (EUROCONTROL). All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise.
Any other use is subject to prior written consent by EUROCONTROL. Request shall be addressed to:
Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupéry, L-1432 Kirchberg/Luxembourg.

The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you want
and to make your stay in the Institute as enjoyable as possible. All Institute personnel are there to
ensure that your stay at the Institute is successful. However, if you do have a complaint (or a
compliment) please tell us. If you are not satisfied with the service we provide or you would like to
propose an improvement then please fill out the form at />or contact directly.

ii

© Copyright 2006 EUROCONTROL


Table of Contents

INTRODUCTION..................................................................................................................................... 4
ATM TRAINING ...................................................................................................................................... 5
ATC TRAINING .................................................................................................................................... 12
E-LEARNING ........................................................................................................................................ 13
STAKEHOLDER FEEDBACK.............................................................................................................. 14
COMPLAINTS ...................................................................................................................................... 15
CONCLUSION ...................................................................................................................................... 16

Printed on 30/11/2004

Produced by the Business and Administration Unit

3


Customer Satisfaction Report

Introduction
This is the 7th report that the Institute devotes to the
assessment of the level of satisfaction of its
customers. The study on which this report is based
has been conducted in the framework of the quality
improvement programme implemented at the
Institute since end of 1996.

•

Measuring customer satisfaction on a regular basis
is one of the key elements of the quality strategy
implemented by the Institute since 1997.

•

Our objective is to understand exactly how our
customers perceive the Institute and, based on their
level of satisfaction, to improve our services and
products.
This report proposes a summary and an analysis of
the data collected in 2005.

The customer segments
assessed in 2005
IANS applies a policy to measure the satisfaction of
its customers with all its products and services.
These include:
•
•
•
•
•
•

The ATM course participants
The ATC course participants
The E-Learning Students
Training Sponsors
Consultancy Bodies.
User Groups

•

A Student Evaluation Form filled in by the
students at the end of a course. This evaluation
form is used for both the ATM course
participants and ATC course participants.
An E-Learning Evaluation Form filled in by a
student upon completion of an E-Learning
Module.
A generic Stakeholder Feedback Form filled in
by various stakeholders. The form is available
at
/>
Customer Visits
IANS visits a sample of its customers on a yearly
basis. The customer visits serve as a main source
of feedback on existing products and a source of
information for improvements and new products.
The customer visits include visits to Air Navigation
Service Providers and civil and military State
Authorities.

Complaint Procedure and
Complaint Form
IANS uses an anonymous form, available on its
web-site for the collection of complaints related to
its products, services and staff. A reference to the
complaint procedure and associated form is made
in all products and services. The complaint form is
available
at

/>
Assessment Tools
The assessment is based on a set of assessment
tools and measurement methods. During 2005
further work has been done to increase the integrity
of the data collection.

Evaluation Forms
To measure the satisfaction of its customers with
the products and services IANS uses the following
evaluation forms:

4

© Copyright 2006 EUROCONTROL


ATM Training
The Satisfaction of Students with the ATM Training Products is measured through the use of a Course Evaluation
Form.

Measurement
Method
At the end of each
(classroom) course,
students are presented
with an evaluation form
containing a number of
criteria to be assessed.
Each criterion can be

given a result between 1
and 4 meaning
unacceptable (1),
unsatisfactory (2), good
(3) an excellent (4).
The complete set of
criteria are grouped in
five main categories:
•
•
•
•
•

Training Content
Instructional Team
Documentation
Equipment
Administration

The first three categories
are directly related to the
course. The last two
categories are directly
related to the supporting
environment and
organisation.

Figure 1 Student Evaluation Form

Printed on 30/11/2004

Produced by the Business and Administration Unit

5


Customer Satisfaction Report

From Forms to
Results
We map the marks of the
students on a scale from
0 (all students mark
Unacceptable) to +100
(all students mark
Excellent). When all
students mark Good
then the result is 75 and
when students mark, on
average, between Good
and Excellent then the
result will be around 85.
Likewise, if students fill
in 'all-Uncacceptable'
then the result will be 0.
If students fill in 'all-UnSatisfactory' then the
result will be 25.

Measuring Satisfaction

Marks from Students

Unsatisfactory

Satisfactory

Good

Excellent

1

2

3

4

0

25

75

100

All 3

All 4

Result

All 1

All 2

©Copyright 2003 EUROCONTROL

1

Figure 2 How we measure satisfaction

IANS aims to meet our objective of being a Centre
of Excellence in ATM Training. For us this means
we aim to achieve results on any criterion between
75 and 100. As an example: the overall average (all
courses, all criteria, 3090 forms) for IANS in 2005 is
87.5 – compared to 85 in 2004. The highest result
on a single criterion is 95 (Subject Knowledge of the
Instructor). The lowest result remains pre-course
information (76.5 – compared to 70 in 2004).

6

Results
The results presented below are extracted from the
automated student satisfaction evaluation system.
Results can be presented per category, per training
domain and per criterion. We have selected the
most significant results to be included in this report.

© Copyright 2006 EUROCONTROL


Size of the
Population

3500

This graph shows the
number of evaluation
forms received and
processed by IANS. In
2005 the Institute
continued to optimise the
processing of the forms.

3000
2500
2000

This led to another
significant increase of
the number of processed
forms in 2005.

1500
1000
500
0

Number of Forms

2002

2003

2004

2005

1937

1915

2624

3090

Figure 3 Number of Evaluation Forms

Composition of
the Population
Students indicate on the
evaluation form which
type of staff they are.

1400
1200
1000

This chart shows that a
majority of Students at
IANS in 2005 are
Operational and
Technical Staff.
These results are
consistent with the
results of the previous
years.

1298

977
2004
824

2005
800

693
600

600
345

400
192

223

200
0
Administrative

Managerial

Operational

Technical

Figure 4 Composition of the population (number of students)

Printed on 30/11/2004

Produced by the Business and Administration Unit

7


Customer Satisfaction Report

Satisfaction
100

This chart shows the
satisfaction of the
students with the ATM
courses. The result is
obtained by averaging

the results obtained on
every criterion in the
categories Training
Content, Instructional
Team and
Documentation.

95
90
85
80
75
70

A result of 87 indicates
that, on average,
students mark criteria
between Good and
Excellent.

65
60
55
50

2002

2003

2004

2005

83

84,5

86

87,5

Course Results

Figure 5 ATM Course Results

The increase in
satisfaction has been
achieved through
detailed analysis of the
results of each course
and introducing
corrective action
wherever required.

Segmented View
100

This chart presents the
results per measurement
Category. These are

Training Content,
Instructional Team and
Documentation.

95
90
85
80
75

IANS has improved the
results on each of these
three categories during
the last four years.

70
65
60
55
50

2002

2003

2004

2005

81,5

83

84,5

86,5

Instructor

88

89

91

92

Documentation

80

80,5

83,5

85

Content

Figure 6 ATM Course Results: per Category

8

© Copyright 2006 EUROCONTROL

IANS will continue to
analyse the results and
make improvements
where possible and
practicable.


‘Best’ Results
100

Customers often praise
IANS for its staff and
excellent infrastructure.

90

This informal feedback is
confirmed by the course
evaluation results. The
criteria ‘Instructor
Subject Knowledge’ and
‘Classroom Equipment’
have consistently
received the highest
marks from the students

during the last three
years.

80
70
60
50

IANS will continue to
invest in the competence
of its instructors by
continuation training and
competency assessment
and it will maintain the
building at its high
standard.

2002

2003

2004

2005

Instructor Subject
Knowledge

92,5

93

94,5

95,5

Classroom
Equipment

88,5

89

90,5

91

Figure 7 Criteria with ‘Best’ Results

Areas for
Improvement

100

IANS continuously looks
for opportunities to
improve. Criteria with the
less positive results are
the Pre-Course
Information and the

Course Duration.

90
80
70
60
50
Pre Course
Information
Course Duration
Canteen

2002

2003

2004

2005

61,5

64

70,5

78

71

74

76

76

73,5

72,5

75

77,5

Figure 8 Criteria with less positive Results

In 2004 IANS has put in
place an action plan to
improve the Pre-Course
information and
significant progress has
been made in this
domain in 2004 and
2005.
The optimum duration for
each course will be
reviewed on a courseby-course basis.

In 2005 IANS changed
the layout of the coffee

area and installed new
equipment in the kitchen
to improve the flow of
students.

Printed on 30/11/2004

Produced by the Business and Administration Unit

9


Customer Satisfaction Report

Focus on Safety
IANS is heavily involved
in the development and
delivery of Safety
Courses to support
Safety Regulation and
Safety Management.
In 2005 the number of
courses delivered in the
Safety domain increased
from 36 to 50 and the
number of participants
increased from 614 to
844.
We are pleased to note
that the results in this

relatively new area for
IANS equal the average
obtained in other, more
mature, training
domains.

100
95
90
85
80
75
70
65
60
55
50

2003

2004

2005

Safety

86

88,5

90

Overall

84,5

86

87,5

Figure 9 Safety Domain Course Results

It is a clear indication
that the rigorous
development approach
at IANS leads to
products that meet
customer expectation.

Measuring
Loyalty

100%
90%

In 2004 we introduced a
new question on the
evaluation form: ‘Would
you recommend the
course to a colleague’.

80%
70%
60%
50%
40%
30%
20%
10%
0%

Yes

No

Blank

2004

91%

1%

8%

2005

92%

1%

7%

Figure 10 Would recommend course to a colleague

10

© Copyright 2006 EUROCONTROL

In 2005, like in 2004,
more than 90% of the
students would
recommend the course
that they attended to a
colleague.