An evaluation of the effectiveness of the materials on english for tour guides for the 3rd year students of foreign languages and tourism department at sao do industrial college and suggestions for adaptation

VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
DEPARTMENT OF POST-GRADUATE STUDIES


NGO THI MY BINH

AN EVALUATION OF THE EFFECTIVENESS OF THE MATERIALS ON
ENGLISH FOR TOUR GUIDES FOR THE 3 RD YEAR STUDENTS OF
FOREIGN LANGUAGES AND TOURISM DEPARTMENT AT SAO DO
INDUSTRIAL COLLEGE AND SUGGESTIONS FOR ADAPTATION

Đánh giá hiệu quả của giáo trình Tiếng Anh chuyên ngành H-ớng dẫn
Du lịch cho sinh viên năm thứ 3 khoa Du Lịch Ngoại ngữ tr-ờng Cao
đẳng Công nghiệp Sao Đỏ và gợi ý cho việc hiệu chỉnh

M.A. Minor Thesis

Field: English teaching methodology
Code: 60 14 10

Hai Duong 2010


VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
DEPARTMENT OF POST-GRADUATE STUDIES


NGO THI MY BINH

AN EVALUATION OF THE EFFECTIVENESS OF THE MATERIALS ON
ENGLISH FOR TOUR GUIDES FOR THE 3 RD YEAR STUDENTS OF
FOREIGN LANGUAGES AND TOURISM DEPARTMENT AT SAO DO
INDUSTRIAL COLLEGE AND SUGGESTIONS FOR ADAPTATION

Đánh giá hiệu quả của giáo trình Tiếng Anh chuyên ngành H-ớng dẫn
Du lịch cho sinh viên năm thứ 3 khoa Du Lịch Ngoại ngữ tr-ờng Cao
đẳng Công nghiệp Sao Đỏ và gợi ý cho việc hiệu chỉnh

M.A. Minor Thesis

Field: English teaching methodology
Code: 60 14 10
Supervisor: MA. Nguyen Thi Thom Thom

Hai Duong 2010


ii

TABLE OF CONTENTS
Contents
Candidate’s statement

Page
i

Table of contents

ii

Acknowledgements

iv

List of figures, tables and charts

v

List of abbreviation

vi

Abstract

vii

PART A: INTRODUCTION

1

I.

Rationale and aims of the study

1

II. Scope of the study

2

III. Methods of the study

2

IV. Research questions

3

V. Statement of the problem

4

VI. Design of the study

4

PART B: DEVELOPMENT

5

CHAPTER I: LITERATURE REVIEW
1.1. Teaching Materials in Language Teaching and Learning

5
5

1.1.1. The Roles of Teaching Materials

5

1.1.2. Types of Materials

6

1.2 Materials Evaluation

7

1.2.1. Reasons for Materials Evaluation

7

1.2.2. Definitions of Materials Evaluation

7

1.2.3. Types of Materials Evaluation

8

1.2.4. Materials Evaluators

9

1.2.5. Sources of the Data

10

1.2.6. Models for Materials Evaluation

11

1.2.7. Criteria for Materials Evaluation

13

1.3. Previous Research on Materials Evaluation

15

1.4. Materials adaptation

16

1.4.1. Reason for adapting

16

1.4.2. Adapting techniques

16

1.5. Summary
CHAPTER II: EVALUATION ON THE EFFECTIVENESS OF THE
MATERIALS “ENGLISH FOR TOUR GUIDES”- PART IV

18
19

iii

2.1. Method of data collection and procedures
2.1.1. Survey questionnaires

19
19

2.1.1.1. Participants of the survey

19

2.1.1.2. Description of the survey questionnaires

20

2.1.2. Formal interview to teachers, students and class observations

20

2.2. Findings and Discussion

21

2.2.1 Teacher’s evaluation

21

2.2.1.1. The evaluation of the teachers in terms of audience, aims, content,

methodology, and other criteria

21

2.2.1.2. The evaluation of the teachers in terms of speaking lessons

24

2.2.1.3. The suggestions of the teachers in terms of speaking lessons

24

2.2.2. Students’ evaluation

25

2.2.2.1. The suitability of the course to the audience

25

2.2.2.2 The suitability of the objective’s course in terms of skills

27

2.2.2.3 The suitability of the course’s content in terms of the speaking lessons

28

2.2.2.4. The suitability of the speaking lessons in terms of methodology.

29

2.2.2.5. Students’ suggestion for the speaking lessons

30

2.3. Evaluation findings
CHAPTER III: SUGGESTIONS FOR THE TEXTBOOK’S ADAPTATION
3.1. Suggestions for the adaptation of speaking lessons
3.1.1 Suggestions for the adaptation of speaking lesson in Unit 2: Hai Duong’s
attractions
3.1.2. Suggestions for the adaptation of speaking lesson in Unit 3: Hanoi’s
attractions
3.1.3. Suggestions for the adaptation of speaking lesson in Unit 6: National parks
in Vietnam
3.1.4. Suggestions for the adaptation of speaking lesson in Unit 8: Museums in
Vietnam
3.1.5. Suggestions for the adaptation of speaking lesson in Unit 10: Ha Long Bay
3.2 Evaluation of the adaptation

31
34
34
34
35
37
38
39
40

PART C: CONCLUSION

41

REFERENCES

43

APPENDIX 1

I

APPENDIX 2

IV

APPENDIX 3

VII

APPENDIX 4

VIII


v

LIST OF FIGURES AND TABLES

Page

Figures
Figure 1

The Materials Evaluation Model of Hutchinson and Waters (1993:98)

13

Figure 2

Preparing a presentation

36
Page

Tables
Table 2.1

Students' results of learning tourism English in the previous term

25

Table 2.2

Students' attitudes towards an “English for Tour guides” course

26

Table 2.3

Students' expectations from an “English for Tour guides” course

26

Table 2.4

Students’ opinion on skill’s importance

27

Table 2.5

Students’ opinions on their skills need to improve in the course

27

Table 2.6

Students’ opinion on the suitability of the speaking lessons

28

Table 2.7

Students’ opinion on the methodology’s suitability of the speaking
30
lessons

Table 2.8

Students’ suggestion for the speaking lessons

30


vi

LIST OF ABBREVIATIONS

SIC

Sao Do Industrial College

FLTD

Foreign Languages and Tourism Department

ETP

English for Tourism Purpose

EFT

English for Tour guides

ESP

English for specific purposes

ELT

English language teaching


1

PART A: INTRODUCTION
I. The rationale and aims of the study
There are three reasons why I choose to carry out the research called “An Evaluation of the
Effectiveness of the Materials on “English for Tour guides” for the 3rd year students of
Foreign Languages and Tourism Department at Sao Do Industrial College and
suggestions for adaptation”.
The first reason is in the importance of a material or a text book in the process of learning
and teaching foreign languages.
ELT materials in general and textbooks in particular serve as tool and tutor, guidebook and
gauge. The American Association for Supervision and Curriculum Development asserted
that teachers throughout the world use texts to guide their instruction, so textbooks greatly
influence how content is delivered. Schmidt, McKnight, and Raizen (1997) identified
textbooks as playing an important role in making the leap from intensions and plans to
classroom activities, by making content available, organizing it, and setting out learning
tasks in a form designed to be appealing to students. Therefore, they may be considered
“the visible heart of any ELT program” Sheldon (1988:237). Especially in Vietnamese
context, textbooks are regarded as a staple in almost every EFL class.
With the growth of computer-assisted language learning, the role of this material may be
changing, but it is unlikely that it will ever disappear. Therefore, it is necessary for teachers
to be well-equipped with the assessment tools to evaluate materials to ensure that students
are using the highest quality texts possible and that their language learning experience is
enhanced, not hindered, by the book used in their classrooms. However, there has been
surprisingly little investigation done in terms of how and why materials are selected by
teachers. Moreover, most of them have not been adequately trained in the selection process

and may be unaware of how to judge which texts best suit their situation. That research gap
has propelled me to choose materials evaluation as the theme for this master thesis.
The second reason for this option lies in the fact that in recent years there have emerged
diverse opinions of both the learners and teachers on the actual effectiveness of the
materials “English for Tour guides”- an in-house materials used in teaching ESP for the
third year students at FLTD at SIC in Hai Duong. Arguments have encompassed both the
potentials and the limitations of the materials for guiding students through the learning


2

process as well as the needs and preferences of the teachers and learners who are using it.
Other issues that have lately arisen include the textbook content, the methodology validity,
the role of textbooks in innovation, the authenticity of materials in terms of their
representation of language and the appropriateness of some speaking lessons. It is
essential, therefore, that we should establish and apply a wide variety of relevant and
contextually appropriate criteria for the evaluation of the textbook to assess its
appropriateness against the learning purposes.
The last but not least, being a teacher of foreign languages and having taught the course of
“English for Tour guides” at SIC more than two years, I have found out that there are some
problems with the materials of the course which are needed to be adapted. With the hope
of improving the quality and minus the limitation of the materials to benefit both the
learners and teachers, I conduct this research to discuss and describe the evaluation process
that is to be undertaken at the FLTD at SIC of the textbook “English for Tour guides” used
in this particular learning environment and thus lead to further thought on improving the
quality of the materials by suggestions for the textbook’s adaptation especially in speaking
lessons. These are also the aims of the study.
II. Scope of the study
The study is about the evaluation of effectiveness of the in-house materials “English for
Tour guides” - Part IV – Tourism destinations in Vietnam. Its curriculum consists of 120

periods for the third year Tour guide students of FLTD at SIC. In the scope of this study,
the evaluation of the effectiveness of the materials only focuses on evaluation of the
suitability of the materials’ content and methodology to the course’s requirement and
students’ need. It bases on the evaluation of teachers and students at the school. From
these, I give out suggestions for adaptation especially in speaking lessons to improve
effectiveness and learning motivation for Tour guide students and better serve for teaching
job this subject at SIC.
III. Methods of the study
To achieve the above aim, the study employs qualitative and quantitative research method.
The research methods employed were document analysis, survey questionnaire, informal
interviews and class observation. Of these methods, document analysis was one of the
main instruments with a thorough objective analysis of the textbook under evaluation.


3

Survey questionnaire also plays an important role in the study. According to James H Mc
Millan & Sally Schumacher (1993) “in survey research, the investigator selects a sample of
subjects and administers a questionnaire or conducts interviews to collect data. Surveys are
used frequently in educational research to describe attitudes, beliefs, opinions and other
types of information”. In this study, survey questionnaire was used to collect the data that
relate to the teachers' and students' perspectives and attitudes towards the textbook they
have just finished. Informal interviews were employed to clarify information collected in
the survey. Lastly, class observation was used to get student’s responses to the speaking
lessons after adaptation. These are effective way of collecting data for this study.
The purposes of this evaluation research were to evaluate the effectiveness of the in-house
materials “English for Tour guides” in terms of its objectives, content and methodology
especially the content of the speaking lessons in the materials. This summative evaluation
was conducted right after the teachers and students have just finished the textbook to
determine whether the material has worked well. The researcher employed a method which

was a combination of document analysis, survey questionnaires, informal interviews and
class observation. The researcher's belief was that a combination of different methods to
collect data could provide more reliable and valid information for the evaluation.
IV. Research questions
The study is to find out the answers to the following research questions:
1. How do the materials “English for Tour guides” used at SIC meet the requirements of
the course and meet the demands of students in terms of objectives, content, and
methodology especially the content of the speaking lessons?
2. What suggestions should be made to contribute to the improvement of the speaking
lessons in the materials?
To answer these research questions, the researchers employed the following evaluation
procedures:
 The involvement of the researcher as the evaluator who analyzed the materials under
evaluation by conducting survey questionnaires for teachers and students to get their
opinions about the textbook they have used and carrying out informal interviews with
teachers and students to clarify issues which are raised in the surveys.


4

 The findings are expected to provide convincing evidences for the contribution to the
improvement of the textbooks for future use.
V. Statement of the problem
SIC is a famous college in Hai Duong province located in Chi Linh District. The school is
40 years of foundation. It has more than 500 teachers and 15.000 students. It is known all
over country with electrical and mechanical field. FLTD is a young one- only 5 years old.
The number of students is only about 300 students. The students related to this study are 80
students of the third year. The program of English for Tourism consists of 300 periods,
divided into 4 different parts. In the first Year, they learn knowledge related to hotel’s
work. In the second year they learnt about the tourists on holiday and common knowledge

related to Vietnam. Helping Students to become good tour guides in the future forms the
basis of the 3rd year course, with students examining topics as tourism destinations in
Vietnam from the North to the South and so on.
The program of part IV composes of 4 main parts: Reading is to provide students with
knowledge of famous tourism destinations in Vietnam; Speaking is to provide students
with sentence forms, situations happening when dealing with foreign visitors and train
student’s ability of making presentation on these destinations. Listening part is to train
student’s listening skill; writing part focuses on guiding and training students to write
about topics related to tourism. Language focus helps students to pronounce the words and
intonation correctly and revise some important languages. This course is a basic part
providing necessary knowledge for tourism students and at the same time practice
effectively 4 skills listening, speaking, reading, writing, develop students’ communicative
ability completely.
VI. Design of the study
The study consists of 3 parts: Part I: Introduction composes of rationale and aims, scope,
method of the study, research questions, statement of the problem and design of the study.
Part II: Main content consists of 3 chapters: Chapter I: Literature review deals with
Textbook, course book, and materials, Materials Evaluation, Materials adaptation; Chapter
II: is about evaluation on the materials’ effectiveness and Chapter III: Suggestions for the
materials adaptation
Part III: Conclusion presents a review of the study, implication, limitation and the
recommendations for the improvement of the materials.


5

PART B: DEVELOPMENT
CHAPTER I: LITERATURE REVIEW
This chapter aims to provide a theoretical base to develop an operational framework for
materials evaluation. The first issue addressed in this chapter relates to teaching materials

with regards to their role in language teaching and types of materials. The second part
presents the review of literature concerning major issues in materials evaluation: reasons
for materials evaluation, definitions of materials evaluation, who carries out materials
evaluation, who provides data for materials evaluation, models for evaluation, and criteria
for evaluation. The third part discusses some of the previous research on materials
evaluation in different contexts that the author referred. This chapter also includes some
theoretical discussions about materials adaptation to help forming the recommendations
for materials improvement.
1.1. Teaching Materials in Language Teaching and Learning
Materials are used in all language teaching and play an important role in the process of
language teaching and learning. The roles of teaching materials in language teaching will
be discussed in more details in the next part of this section.
1.1.1. The Roles of Teaching Materials
Crucial roles that teaching materials play in language teaching are indicated by different
authors in the literature. Richards and Rodgers (cited in Nunan, 1991) view instructional
materials as detailed specifications of content, and guidance to teachers on both the intensity
of coverage and the amount of attention demanded by particular content or pedagogical
tasks. Richards and Rodgers' perspective is supported and clarified by Richards (2001),
who points out that teaching materials can serve as the basis for much of the language input
that the learners receive and as the source for much of the language practice that occurs in
the classroom. Richards further explains that materials provide basis for the content of the
lesson, the balance of skills taught, and the kind of language practice students take part in. In
addition, good teaching materials are of great help to inexperienced teachers or poorly
trained teachers (Nunan, 1991; Richards, 2001). They can serve as "a form of teacher
training" (Richard, 2001: 251) and teachers can get ideas on how to plan and teach the
lesson from the materials.
Obviously, teaching materials are a key and crucial component in any language teaching


6

contexts, the teacher needs to take account of the roles as well as the requirements of
teaching materials when designing or selecting materials for his/ her own teaching situation
to facilitate the most learning from the students.
1.1.2. Types of Materials
The teaching materials that a teacher uses can be a textbook, institutionally prepared
materials or his / her own prepared materials (Richards, 2001). The former is referred to as
published textbooks and the latter in-house materials by Robinson (1991). Which one to
choose, published textbooks or in-house materials is what specialists in the field of English
language teaching (ELT) weigh up arguments? Robinson (1991: 58) suggests that no other
medium is as easy to use as a textbook. More important, "a textbook is complete not just in
the physical sense but in the sense that the whole term's or year's course is available to the
students at once". However, Ewer & Boys (cited in Robinson, 1991) present a strong
attack on published textbooks that some really fundamental factors of textbooks such as
the validity of the linguistic contents, the accuracy of the explanations and examples given,
and the number and coverage of the exercises provided have been ignored because the
writers just focus on approach and methodology. Moreover, as Swales (cited in Robinson,
1991: 57) indicates published textbooks are "less self-sufficient in practice materials and in
coverage of skill areas" so these textbooks need to be supplemented by some other
materials.
Similarly, there are also arguments for and against the use of in-house materials. On the one
hand, as Robinson (1991) suggests, there are three advantages of in-house materials. Firstly,
in-house materials are likely to be more specific and appropriate than published materials and
to have greater face validity in terms of the language dealt with and the contexts it is
presented in. Secondly, in-house materials may be more flexible than published textbooks.
Finally, the writers of in-house materials can make sure of the suitability of methodology
for the intended learners. On the other hand, Robinson (1991) points out making in-house
materials are time-consuming and expensive. What is worse is that many locally produced
materials "show a striking resemblance to the published materials that have been rejected"
(Swales, cited in Robinson, 1991: 58).

As stated above, there are both advantages and disadvantages in using published materials
as well as in-house materials, so which one to use must depend on the purposes of the


7

course as well as the facilities and constraints of a particular situation. Based on the
literature definitions of types of materials, it can be said that the textbook under evaluation
in this research is necessary to judge the appropriateness to the intended students.
1.2. Materials Evaluation
Decision on which textbook to use or whether the materials in use are suitable or not is
obviously of great importance in the process of learning and teaching of language. And this
decision can be achieved only by the means of a comprehensive evaluation.
The next part of this chapter is the review of reasons for materials evaluation, definitions, types
of materials evaluation, materials evaluators, source of the data, models for materials evaluation
and criteria for materials evaluation.
1.2.1. Reasons for Materials Evaluation
There are two main reasons for carrying out materials evaluation. Firstly, there may be a
need to choose among the materials available the most suitable ones to use for a particular
situation. Secondly, there can be a need for materials evaluation to determine whether the
materials which has been chosen works for that situation after it has been used for a period
of time (Ellis, 1997).
An evaluation of teaching materials helps to identify particular strengths and weaknesses of
the materials in use. Thus it will help to determine whether the materials are effective and
efficient. As a result, the decision whether to use the materials again or not and what should
be changed to the material can be made for better use in the future.
In short, the findings of materials evaluation will provide input for responsible people to
evaluate the effectiveness of the materials.
1.2.2. Definitions of Materials Evaluation
There are many ways of defining evaluation. According to Brown (1995) evaluation is

defined as "the systematic collection and analysis of all relevant information necessary to
promote the improvement of a curriculum and assess its effectiveness within the context of
the particular institution involved". This definition requires that information be gathered
and analyzed in a systematic manner and that only relevant information should be
included, that is other information can be ignored.


8

Nunan (1988) suggests that evaluation is "a process not a final product" that means it takes
place at any time of the material design. The first and foremost emphasis of evaluation is to
determine whether the goals and objectives of a language program are being attained.
According to Hutchinson & Waters (1993) evaluation is really a matter of judging the fitness
of something for a particular purpose. "Given a certain need, and in the light of the resources
available, which out of number of possibilities can represent the best solution? There is no
absolute good or bad - only degrees of fitness for the required purpose".
From the above definitions, it can be inferred that materials evaluation involves the
determination of what needs to be evaluated, the objectives and requirements for the
materials, and the judgments of the value of the materials being evaluated in relation to the
objectives and requirements determined.
1.2.3. Types of Materials Evaluation
There are three types of evaluation according to the reasons for materials evaluation:
preliminary evaluation, formative evaluation and summative evaluation.
Preliminary Evaluation: Preliminary evaluation takes place before a course commences with
the aims of choosing the materials that will be relevant and appropriate for a particular group
of learners. This also has the purpose of identifying which aspects of the published materials
need to be adapted to suit the purposes of the evaluators (Robinson, 1991; Ellis, cited in
Tomlinson, 1993). This can be done by ''determining a set of criteria which are used to reach a
decision regarding which book to adopt and how it needs to be adapted" (Ellis, cited in
Tomlinson, 1993: 220).

Formative Evaluation: Formative evaluation is normally carried out while the course is on-going
and the findings are used to make modifications to the materials when they are being used
(Dudley - Evans, 1998)
Summative Evaluation
Summative evaluation is referred to as evaluation as retrospective evaluation (Ellis, 1997).
It takes place at the end or sometimes after the course has been implemented and very
valuable for "durable course" (Dudley - Evans, 1998). According to Richards (2001), a
summative materials evaluation is conducted to determine whether the materials have


9

worked well. The findings will normally lead to the decision whether to repeat or use the
materials again or not: which tasks and activities are suitable and which are not; and which
aspects of the materials need to be changed and how to modify them to make the materials
more effective and efficient for future use (Ellis, 1997). In other words, a summative
evaluation is carried out to determine the worth and value of teaching materials after they
have been used. This kind of evaluation therefore can serve as a means of "testing" the
validity of a before - program evaluation and it can point out "ways in which the predictive
instruments can be improved for future use" (Ellis, 1997: 37).
In short, whatever the types of evaluation, preliminary, formative, or summative are, it is
very important that the evaluator identifies clearly the reasons for their evaluation so that it
can be beneficial for the use of the materials (Robinson, 1991).
1.2.4. Materials Evaluators
A great deal of published work on materials evaluation (Dudley - Evans & St. John, 1998;
Tomlinson, 1998; Robinson, 1991; Richards, 2001) reveals that materials evaluation can be
carried out by outsiders (those who are not involved in the program, for example,
consultants, inspectors, and administrators) or insiders (e.g. teachers, students, course or
materials designers); each method has its own advantage as well as disadvantage.
1.2.4.1. Materials Evaluation by Outsiders

According to a number of researchers, outsiders will bring "fresh" and "objective"
perspectives into the evaluation, which is the advantage of evaluation carried out by
outsiders. However, there may be some disadvantages: outsiders as evaluators will have
their own views on language teaching methods. They may not understand the teaching and
learning situation in which the evaluation is being carried out. So, as Robinson (1991)
suggests, they may be unsympathetic and it may take more time for them to get to know
the local situation such as learners' needs, facilities as well as constraints. Consequently, it
may be hard for them to make exact judgments and realistic recommendations; or may be
what they find has already been found (Dudley - Evans & St. John, 1998).
1.2.4.2. Materials Evaluation by Insiders
Materials Evaluation by insiders has some advantages especially if the overall purpose of
evaluation is for "development" and thus for better use in the future. As they have already


10

involved in the program, they will understand the teaching and learning situation well
(Tomlinson, 1998). Tomlinson’s view is strongly supported by Holliday (cited in Dudley Evans &St. John, 1998) and Richards (2001). According to Holliday (1998:296), the
evaluator's understanding of "cultural and political factors" of the institution in which the
evaluation takes place would be very important for the evaluator to make exact judgments
and realistic recommendations. Richards (2001) holds the view that the involvement of the
insider plays a very important part in the success of evaluation because "as a consequence,
they will have a greater degree of commitment to acting on its results". Richards also points
out insiders, especially teachers, can 'monitor' when the materials are being used, they
therefore can determine if and to what extent the materials 'work' for their purposes and thus
they can make modifications to improve the effectiveness of the materials. However, the
insiders "may be too close and involved" (Dudley - Evans and St. John, 1998: 131) so the
evaluation may be influenced by their teaching experience and their subjective points of
views.
In short, who will carry out the evaluation outsiders or insiders should be determined

according to the purposes of the evaluation and to the facilities as well as the constraints of the
situation in which the evaluation takes place.
1.2.5. Sources of the Data
Another important issue in materials evaluation is to determine the sources from which the
data for evaluation will be collected. According to Lynch, (1996) evaluation data can come
from a variety of sources such as students, instructors, administrators, evaluators, and others
who interact with the program. Among those sources, the learners are considered the key
one, especially in the summative evaluation (Richards, 2001). According to Richards, the
learners can provide 'evidence' of what they have learned from the course and the
materials, and whether it is relevant to their needs. They can also provide data for
evaluation on the way the program or materials was taught.
Another important source of data for evaluation is the teachers who are much interested in
the suitability of the materials (Richards, 2001), they will report their experiences with the
materials and what they think about them. Furthermore, they can be helpful in providing
practical suggestions for improvement (Robinson, 1991).


11

In addition to the student and teacher sources, records and documents used can also
provide useful information for evaluation. This source includes planning documents,
statements of needs, aims and objectives of the course, teachers' lesson records, teachers and
students' journal and so on (Robinson, 1991). The program documents can supply evaluators
with basic information about the program. They can also help suggest evaluation questions
to be pursued and later, be used to match against "subsequent developments" (Robinson,
1991: 71). Lynch (1996) points out that students and teachers' journals can be used as a
rich source of information from the participants' points of view. So they should be
encouraged to keep daily or weekly journals of their "experiences and perceptions", which
are then used as a source of information for the evaluation.
As stated above, data for evaluation can come from a variety of sources. However, to have

reliable evaluation, it is advisable that information about the same point should be obtained
from different sources (Robinson, 1991). Especially, also according to Robinson, the
teachers and the learners should be involved in the evaluation because they have taken part
in designing or selecting and using the materials. This idea taken from his own judgment
that learners can give information of how effective the materials are for their work whereas
teachers can be helpful in providing practical suggestions for improvement. However,
Robinson (1991) also warns that the teachers are likely to feel that the objective of the
evaluation is to "put up" them, and very often it's assumed that "any resultant evaluation"
document will be negatively critical" (Robinson, 1991: 69). He then advises that it's
necessary for the evaluator to inform the purposes and methodology plans of the evaluation
to people who will provide information for the evaluation, especially the teachers who have
used the materials.
1.2.6. Models for Materials Evaluation
There are different models and criteria for materials evaluation suggested by different
authors in the literature. The popular models are the ones suggested by Hutchinson and
Waters (1993), McDonough and Shaw (1993) and Ellis (1997).
1.2.6.1. Evaluation Model by Ellis (A Micro - Evaluation)
A micro - evaluation is the evaluation of one particular teaching task which the evaluator
has a special interest in (Ellis, 1997). In this model Ellis suggests the practice of a detailed


12

empirical evaluation and focuses on evaluation at the task level with reference to its actual
teaching-learning context. The aim in this model is to identify the match between task
planned and task in use. He also suggests that dimensions focused on macro - evaluation
(e.g. approach, purpose, focus, scope, evaluators, timing, and types of information) can also
be applicable to micro evaluation. This type of evaluation can only be conducted when the
materials are being used in the classroom. The steps involved in the evaluation model
suggested by Ellis (1997) are: Choosing a task to follow; Describing the task with

specification of input, procedures, language activities, and outcome; Planning the evaluation
with reference to the dimensions above; Collecting information before, while and after the
task was used, and what and how the task was performed; Analysis of the information
collected; Findings of the evaluation and making recommendations for future teaching and
Writing the report.
It could be said that a micro evaluation may be more manageable than a macro-evaluation,
but it is time-consuming.
1.2.6.2. Evaluation Model by McDonough and Shaw (A Combination of Macro- and
Micro-Evaluation)
McDonough and Shaw (1993) suggest a three-stage evaluation model, namely external
evaluation, internal evaluation and overall evaluation:
- External evaluation (Macro-Evaluation) is the examination of the claims made for the
whole materials by the author or publisher. They can be in the forms of claims made on the
cover of the book, the instructions, and the table of content. The evaluation at this stage helps
to identify whether the material is potentially appropriate; if the external evaluation shows
that the materials are potentially appropriate then the internal stage starts; if the findings
show that the materials are inappropriate, the evaluation will be finished at the external
stage.
- The internal stage (Micro-Evaluation) requires an in-depth look at two or more units to
examine whether the claims made by the author are the one found in the internal evaluation.
- The overall evaluation of the materials is to determine the suitability of the materials by
considering a number of factors such as the usability factor, the generalizability factor,
adaptability factor and flexibility factor.


13

1.2.6.3. Evaluation Model by Hutchinson and Waters
Hutchinson and Water (1997)' model is a macro - evaluation which as defined by Ellis
(1997: 37) as the "overall assessment of whether an entire set of materials has worked" for

a particular situation. In their model, Hutchinson and Waters (1993: 96) define material
evaluation is a "matter of judging the fitness of something for a particular purpose". In this
view, the evaluation is divided into 4 steps (see Figure 1).
Define criteria
On what bases will you judge materials?
Which criteria will be more important?

Subjective analysis
What realizations of the criteria do you
want in your course?

Objective analysis
How does the material being evaluated
realize the criteria?

Matching
How far does the material match your needs?

Figure 1: The Materials Evaluation Model of Hutchinson and Waters (1993:98)
As can be seen in Figure 1, this model presents a logical procedure for material evaluation.
To examine whether the materials are suitable for a certain group of learners or not, there
must be certain criteria against

which our judgments or evaluation are based. It is also a

must to determine the objectives or the requirements for the materials because we cannot
measure the success of a particular activity or a whole set of materials if there is no clear
objective for it.
In summary, although these three models vary in the processes and the nature of the
purposes, they all serve to evaluate whether the set of the materials is appropriate to a

certain situation or group of learners. An evaluator must be based on the purposes of the
evaluation, time available, facilities as well as" constraints of the context in which the
evaluation takes place to decide which model to follow. In the case of this study,
Evaluation Model by Hutchinson and Waters is considered to be the most suitable one and
will be chosen by the writer.


14

1.2.7. Criteria for Materials Evaluation
Criteria are what evaluators use to "reach a decision regarding what needs to be evaluated"
(Tomlinson, 1998: 220) which are the bases upon which evaluators depend on when
making judgments. Defining criteria for evaluation is one of the crucial issues evaluators
must take into account before any evaluation can take place. Criteria for materials
evaluation depend what is being evaluated and why they need to be evaluated (Dudley Evans and St. John, 1998). Researchers (Sheldon, 1988; Wallace, 1998; Hutchinson &
Waters, 1993; Ellis & Johnson, 1994) have suggested certain sets of criteria of which the
two most popular are the set suggested by Sheldon (1988) and the one suggested by
Hutchinson and Waters (1993).
1.2.7.1. Criteria Defined by Sheldon
Sheldon (1988) suggests a wide range of criteria that can be used to evaluate almost all
aspects of materials. The criteria suggested by Sheldon include rationale, availability, user
definition, layout/graphic, accessibility, linkage, selection/ grading, physical characteristics,
appropriateness, authenticity, sufficiency, cultural bias, educational validity, stimulus/
practice/ revision, flexibility, guidance, and overall value for money.
12.7.2. Criteria Defined by Hutchinson and Waters
Hutchinson and Waters (1993) in their checklist for materials evaluation suggested 5
criteria for evaluation as follows:
Audience: The first criterion in materials evaluation in the audience of the materials with
regards to different information such as ages; sex; study or professional fields; status with
respect to professional fields; knowledge of English, of areas of work or study and of the

world; educational background, interest and so on.
Aims: Like any language course, which has its own aims and objectives, the evaluation of the
course needs to be carried out to find out if the material can satisfy those aims and
objectives of the course.
Content: Content is one of the most important elements of the materials. Therefore it is
necessary to measure if the content of the materials is suitable to the content prescribed in
terms of language description, language points, macro - skills and their proportion, and


15

micro -skills. The evaluation also needs to find out if the text - types of the materials are
suitable to the requirements of the course. In general it is very important to take into
consideration the learners' subject - matters in terms of level of knowledge, types of topics,
and how these topics are presented. Finally, the evaluation needs to find out if the way the
content is organized and sequenced within a unit and throughout the course is appropriate.
Methodology: Many aspects in terms of materials methodology need to be evaluated. First of
all, the evaluation has to indicate whether the theories of learning on which the course is
based are in line with the ones of the materials under evaluation. Secondly, evaluators need to
measure whether the learners' expectations about learning are the ones the materials are
intended to. Thirdly, it is necessary to determine whether the kinds of exercises/ tasks
included in the materials are suited to course requirements. Another aspect to be evaluated
is the teaching and learning techniques: Whether the teaching - learning techniques that
can be used with the materials are those required by the course and suitable to the learners.
Furthermore, the evaluation also needs to discover if teaching aids available for use are
those required by the materials. In addition, the evaluation needs to find out whether the
materials supply the teachers with necessary guidance and support for teaching the course.
Finally, it is necessary to determine if the materials are flexible so that they can be used in
different orders to suit the teaching contexts.
Other Criteria: In addition to the criteria mentioned above, some other criteria are also

suggested such as the price and availability of the materials.
From the two sets of criteria by Sheldon (1988) and Hutchinson and Waters (1993)
presented above, it is remarkable that almost all aspects of teaching materials can be
evaluated when using the set of criteria suggested by Sheldon (1988). This, however,
would require a lot of time and effort, and even hard to apply because to evaluate
everything at the same time is unrealistic (Dudley - Evans & St. John, 1998). In reality,
criteria defined by Hutchinson and Water appear to be more manageable and suitable to
condition in Vietnam. That is also the reason why Criteria Defined by Hutchinson and Waters
are chosen for this study.


16

1.3 Previous Research on Materials Evaluation
There have been a certain number of studies carried out by researchers to evaluate published
or in-house teaching materials, each of which may make use of different approaches to
materials evaluation, they all, however, demonstrate how materials evaluation provides
feedback for improvement and modification. The major purpose of these studies is to evaluate
the effectiveness of the materials in use or its suitability to the course aims. However, there
are no evaluation related to tourism field, especially there have not been any studies mention
to evaluation of the in-house materials “English for Tour guides” used for the third years
students at Foreign Languages and Tourism Department at SIC.
1.4. Materials Adaptation
1.4.1. Reasons for Adapting
Once the materials have been evaluated, potential problem areas can be identified: What the
materials offer can not be exactly what our learners' need; The materials methodology may
not match our own; Our general aims may not match the aims of the materials; The aims of
a particular lesson/unit in the materials may not match our lesson - by - lesson aims. We
will have to prioritize and select. We may need to supplement the materials.
There are 5 main ways of modifying materials: Adding, including expanding and

extending; Deleting, including subtracting and abridging; Modifying, including
restructuring and rewriting; Simplifying and Reordering.
1.4.2. Adapting Techniques
1.4.2.1. Adding
The notion of addition is that materials are supplemented by putting more into them, while taking
into account the practical effect on time allocation. First, we can certainly add in this quantitative
way by the technique of extending "This means that the techniques are being applied within the
methodological framework of the original materials: in other words, the model is not itself
changed" (McDonough and Shaw, 1993:89). We can do this in the following situation: A second
reading passage parallel to the one provided is helpful in reinforcing the key linguistic features tenses, sentence structures, vocabulary, cohesive devices - of the first text. Second, more farreaching perspective on addition of materials can be termed expanding. This kind of addition is
not just extension of an existing aspect of content. They go further than this by bringing about a


17

qualitative as well as a quantitative change... This can be thought of as a change in the overall
system' (McDonough and Shaw, 1993:90).
1.4.2.2. Deleting or Omitting
Deletion is clearly the opposite process to that of addition. As we saw in the previous
section that materials can be added both quantitatively (extending) and qualitatively
(expanding), the same point applies when a decision is taken to omit materials. The most
straightforward aspect of reducing the length of materials is subtracting.
Addition and deletion often work together. Material may be taken out and then replaced with
something else. The methodological change is greater when, for example, grammar practice
is substituted after the omission of an inappropriate communicative function, or when a
reading text is replaced by a listening passage.
1.4.2.3. Modifying
Modifying can be sub-divided under two related headings. The first of these is rewriting,
when some of the linguistic content needs modification, the second is restructuring, which
applies to classroom management.

- Rewriting may relate activities more closely to learners' own background and interest,
introduce model of authentic language, or set most purposeful problem-solving tasks
where the answers are not always known before the teacher asks the question.
- Restructuring: For many teachers who are required to follow a course book rather
strictly, changes in the structuring of the class are sometimes the only kind of adaptation
that is realistically possible. For example, the materials may contain role-play activities
for groups of certain size. The logistic of managing a large class (especially if they all
have same LI) are complex from many points of view, and it will probably be necessary
to assign one role to a number of pupils at the same time. Obviously the converse where the class is too small for the total number of roles available - is also possible if
perhaps less likely.
1.4.2.4. Simplifying
The technique of simplification is a type of modification, namely a "rewriting" activity. The
elements of a language which can be simplified are: the instructions and explanations that


18

accompany exercises and activities, and even the visual layout of materials so that it becomes
easier to see how different part fit together. However, texts, most often reading passages are
applied this technique. Usually, the emphasis has been on changing various sentences - bound
elements to match the text more closely to the proficiency level of a particular group of
learners. Simplification could be in the following forms: Sentence structure; Lexical content,
and grammatical structures.
Simplification has a number of further implications. Firstly, once linguistic items are
changed, stylistics will be affected, and therefore the meaning of intention of the original
text is changed. Secondly, simplification of content is required when the complexity of the
subject-matter is regarded as being too advanced. Thirdly, simplification can refer to the
ways in which the content is presented: we may decide not to make any changes to the
original text, but instead to lead the learners through it in a number of graded stages.
1.4.2.5. Re-ordering

This procedure refers to the possibility of putting the parts of a course book in a different
order. This may mean adjusting the sequence of presentation within a unit, or taking units
in a different sequence from that originally intended.
1.5. Summary
Chapter one has presented the major roles that materials play in the process of teaching and
learning as well as important issues of materials evaluation that need to be taken into
account when carrying out materials evaluation. The discussion implies that issues such as
why to evaluate, what to evaluate, how to evaluate, who the participants of evaluation are
and other issues of materials evaluation should be decided according to the purposes of the
evaluation and the situation in which the evaluation takes place. After studying all the above
issues carefully, the writer decided to choose the evaluation model and evaluation criteria of
Hutchinson and Waters which are considered to be the most suitable one for the context of
the study. The evaluation of the effectiveness of the material will be carried out basing on
the analysis the suitability of the material “English for Tour guides” part IV- Tourism
destinations in Vietnam in terms of the aims, content and methodology. The last section of
this chapter deals with theoretical views on adapting techniques which will be used in the
adaptation of the materials.


19

CHAPTER II: EVALUATION ON THE EFFECTIVENESS OF THE MATERIALS
“ENGLISH FOR TOUR GUIDES”- PART IV
In this chapter, I wish to show the attitudes and the expectations of the teachers and third year
students in Sao Do Industrial College towards the effectiveness of the materials “English for
Tour guides” part IV- Tourism destinations in Vietnam. The first section in the chapter
reveals the method of data collection and procedures. The second deals with the finding and
discussion of the data collections, primarily with the survey questionnaires and interview.
The evaluation of the materials will be based on the evaluation model and criteria of
Hutchinson and Waters. And the main purpose of this chapter is to find out the answer for the

first research question whether the material “English for Tour guides” used at SIC meet the
requirements of the course and meet the demands of students in terms of objectives,
content, and methodology especially the content of the speaking lessons.
2.1. METHOD OF DATA COLLECTION AND PROCEDURES
Survey questionnaires for the third year students and for teachers of English in my institution
are considered to play an important part in the data collection process of my study. Equally
important, informal interview to both teachers and students and class observations are at the
same time involved in the process.
2.1.1. Survey questionnaires
2.1.1.1. Participants of the survey
Giving our 6 survey questionnaires for teachers and 9 ones for students were thoroughly
completed by 4 teachers of EFT and by 80 third year students in SIC right after the course
finished. Necessarily, some characteristics of the Tourism English course, of the surveyed
students and the teachers should be briefly introduced.
As a matter of fact, in SIC, students are scheduled to study English in six terms, two of which
are for Basic English with the in-house Basic English materials and the last four terms spent
on EFT with 4 materials: English for Tour guides part I, II, III and IV.
The surveyed students are of class 03 HDDL 2 and 03 HDDL 2. There are 80 students. Those
students are of pre-experience (low-experience) at the very early of their career and the
course for them here is English for Tour guides focusing on tourism destinations in
Vietnam. There is a strong possibility that they are basically different from those of jobexperiences in view of the expectations. As for the surveyed teachers, they have not been