An evaluation of the textbook English for graduate students of social sciences and humanities at USSH, VNU= Đánh giá giáo trình tiếng Anh chuyên ngành dành cho

VIET NAM NATIONAL UNIVERSITY, HA NOI
UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES
FACULTY OF POST – GRADUATE STUDIES
*****************




TRAN THI THU HUONG


AN EVALUATION OF THE TEXTBOOK ENGLISH FOR
GRADUATE STUDENTS OF SOCIAL SCIENCES AND
HUMANITIES AT USSH, VNU

(Đánh giá giáo trình Tiếng Anh chuyên ngành dành cho học viên
cao học ngành xã hội nhân văn, trường Đại học Khoa học xã hội và
Nhân văn, Đại học Quốc gia Hà Nội)


M.A Combined Programme Thesis
English methodology
Major code: 60 14 10




Hanoi - 2011

VIET NAM NATIONAL UNIVERSITY, HA NOI
UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES

FACULTY OF POST – GRADUATE STUDIES
*****************




TRẦ N THỊ THU HƢƠNG

AN EVALUATION OF THE TEXTBOOK ENGLISH FOR
GRADUATE STUDENTS OF SOCIAL SCIENCES AND
HUMANITIES AT USSH, VNU

(Đánh giá giáo trình Tiếng Anh chuyên ngành dành cho học viên
cao học ngành xã hội nhân văn, trường Đại học Khoa học xã hội và
Nhân văn, Đại học Quốc gia Hà Nội)

M.A Combined Programme Thesis
English methodology
Major code: 60 14 11

Supervisor: Nguyễ n Thú y Hƣơng, M.A



Hanoi - 2011

5


TABLE OF CONTENTS

Chapter I: Introduction 10
1.1Background to the study 10
1.2 Aims of the study 13
1.3 Research questions 13
1.4 Scope of the study 14
1.5. Design of the study 14
Chapter II: Literature review 16
2.1 Materials in language teaching and learning 16
2.1.1 Roles of teaching material EFL/ESL classes 16
2.1.2 Types of materials 18
2.2 ESP and ESP materials 20
2.2.1 A brief overview of ESP 20
2.2.2 The role of ESP materials 23
2.3 Materials evaluation 25
2.3.1 Definitions of materials evaluation 25
2.3.2 Techniques of materials evaluation 26
2.3.3 Criteria for materials evaluation 28
2.3.4. Types of materials evaluation 33
2. 3. 5 Models for material evaluation 34
2. 4. Material adaptation 38
2.4.1 Reasons for adaptation 38
2.4.2 Techniques for adaptation 39
2.5 Previous studies on materials evaluation 41
Summary 43
Chapter III: Methodology 45

6
3.1. Research questions 45
3.2 Data collection instruments 45

3.2.1 Document analysis 45
3.2.2 Questionnaires 47
Summary 50
Chapter IV: RESULTS AND DISCUSSION 51
4.1 Results of data analysis 51
4.1.1 Document analysis 51
4.1.2 Data collected from questionnaires 57
4.2. Major findings 71
4.2.1 The suitability of the ESP material to the aims 71
4.2.2 The suitability of the ESP material to the content of the material 71
4.2.3 The suitability of methodology 73
CHAPTER V: CONCLUSION AND RECOMMENDATIONS 75
5.1 The strengths and weaknesses of the material 75
5.1.1 Strengths 75
5.1.2 Weaknesses 76
5.2 Recommendations 76
5.2.1 To the content 77
5.2.2 To the methodology 78
5.3 Limitations 79
5.4 Suggestion for further study 79
5.5 Conclusion 79
REFERENCES………………………………………………………………….74






7
List of abbreviations

CFL College of Foreign language
VNU Vietnam national university, Hanoi
USSH University of Social Sciences and Humanities
ESP English for special purposes
ESL English as second language
ELT English Language Teaching
EFL English as a Foreign Language
TESOL Teaching English to Speakers of Other Languages



8
Lists of Figures and Tables

Figure 1: Materials Evaluation Model of McDonough and Shaw (1993, p.3)
Figure 2: Material Evaluation process (Hutchinson and Waters, 1993, p.98)
Figure 3: Teachers‟ experience of teaching graduate students at USSH.
Figure 4: Number of ESP classes of each teacher
Figure 5: Students‟ working experience in their specialist field
Figure 6: Student‟s purposes of taking ESP course
Figure 7: Opinions of the ESP teachers and ex-students on the aims of the material
Figure 8: Students‟ opinions on the difficulty level of the language items in the
material
Figure 9: Teachers‟ opinions on the difficulty level of the language items in the
material
Figure 10: Opinions of the ESP teachers and ex-students on the specialist vocabulary
Figure 11 : Opinions of the ESP teachers and ex-students on the topics of reading
texts
Figure 12 : Opinions of the ESP teachers and ex-students on grammar structures

Figure 13 : Opinions of the ESP teachers and ex-students on the frequency of
different types of class-arrangement


9

Table 1 : The Selected criteria checklist for materials evaluation by
Hutchinson and Waters (1993) and Cunningsworth (1984)
Table 2 : The frequency of specialist topics students read
Table 3 : Description of language focus in the material
Table 4 : The frequency of skills covered in the material
















10
Chapter I
Introduction

1.1Background to the study
1.1.1 An overview of English teaching and learning for graduate students at
USSH, VNU
The University of Social Sciences and Humanities (USSH) is a prestigious
educational institution where thousands of students of social sciences and humanities at
different levels, ranging from undergraduate to graduate ones are being trained. They
are trained to work in different fields of social sciences, including Journalism, Tourism,
History, Sociology, etc. After graduation, a large number of learners will need English
for their jobs. Therefore, English is taught as a compulsory subject, aiming at providing
learners sufficient knowledge and skills. During the whole course of graduate study,
English accounts for 7 credits, equal to 105 class hours. According to the syllabus,
there are two main stages: General English (GE) and English for Specific Purposes
(ESP). In stage 1 comprising the first 4 credits, students are to achieve Basic English
level C with general English knowledge and communication skills. 45 periods of
English for Specific Purposes are expected to be covered in stage 2. However, for
years, there was no ESP material for use, so during this stage, teachers and learners had
to use the last 6 units in New Headway Intermediate by Liz and John Soars (1998)
instead.
1.1.2 The graduate learners
To become a graduate student at the university, one has to be in the top group in
entrance written exam of English and a specialized subject of their field; their results of
English test scores must be from 50 out of 100 points at the minimum. It means that
their English should be at pre-intermediate level. Thus, graduates are quite good at
grammar, vocabulary and have fairly good writing skill. Moreover, before taking ESP

11
course, the graduates have already finished the GE stage with 60 class hours and at the
end of this stage they are assumed to have achieved the intermediate level of
proficiency. In addition, they pay much attention to ESP because they are aware of the

importance of English language in general and ESP in particular to their future job and
study. However, most of them have very limited access to documents in English, even
in their field of work.
1.1.3 The teaching staff
The Department of Foreign languages at USSH, VNU, has now become a unit
under ULIS, VNU since 2009. It has a large teaching staff of English, both full-time
and part-time teachers and all of them have to pass a serious recruitment exam which
involves both English knowledge and skills and teaching skill, which helps to select
qualified teaching staff.
1.1.4 The ESP material
Graduates of social sciences and humanities are from different fields: Literature,
Journalism, Psychology, International studies, Tourism, etc. whose demands for
terminologies and specialized English expressions are varied. Practically, providing a
helpful ESP program that can meet the demands of different sub-groups of learners of
social sciences and humanities is a great challenge. To be more precise, with one
textbook, not only learners of Tourism but also other groups of students like
Journalism or International studies can get useful knowledge and skills.
As has been mentioned previously, for many years, the English teachers at
USSH used the 6 last units in New Headway Intermediate by Liz & John Soars
together with teaching materials designed or selected by themselves for the 45 periods
for ESP. However, the demand of a specially designed ESP material for an ESP
training course of social sciences and humanities was increasing, which forced the
authorities of USSH to find a solution, and English for graduate students of social
sciences and humanities, compiled by the Dean of the Department of Foreign

12
language, Dr. Lam Quang Dong, who is the most experienced teacher with more than
10 years teaching English at USSH, was the result. This material has been officially
used as the core teaching material for students of social sciences and humanities at
USSH since 2008.

This material contains 3 topic areas: Population and economic issues,
Superstitions and customs, and Religions. Each unit is organized in such sections as
follows: Language focus, Reading, Listening, which in some lessons is replaced by
Writing. Language focus section deals with certain grammatical points in which
learners are required to do rewriting exercises, gap-fill and correcting mistakes
exercises or translating sentences from English into Vietnamese and vice versa. In the
reading section, students read a passage on some topics related to their studying fields,
like What is in a name?, Black cats and broken mirrors, Customs, and so on. Learners
are provided tasks to gain some new terminologies, cultural and social knowledge and
they also have much chance to practice reading and translating skills. For Listening,
teachers in charge of each class have to select their own listening passages to teach
their students. In the writing section, the author provides different kinds of exercises
varying from gap-fill to matching, writing stories or essays and describing graphs or
charts, etc. in order for students to build sentences and extend them into paragraphs and
essays. The material was highly appreciated by the administrators, teachers and
students at USSH because it seemed to meet learners‟ needs.
1.1.5 Arising problem
However, after two semesters using the new course-book, there have been a lot
of reflections from both teachers and learners on the appropriateness of the material
and the learning outcomes. Some reflections were positive but some were extremely
negative. Therefore, an investigation into the appropriateness of this textbook to

13
learners‟ needs would be of great importance, as this study would provide information
for the author to improve it so that students‟ language levels and teaching aims are met.
These above mentioned facts inspired me to conduct this study “An evaluation
of English for graduate students of social sciences and humanities at USSH, VNU”.
Hopefully, the results of this study will be of much help to the teachers of graduate
students at USSH in achieving the objectives of the ESP program in the coming years.
1.2 Aims of the study

The paper is aimed at evaluating the textbook English for graduate students of
social sciences and humanities in terms of aims, content, methodology based on users‟
responses toward its appropriateness to the course requirements. Based on the findings,
recommendations on how to improve it, and thereby, improve the effectiveness of ESP
teaching and learning at USSH shall be given.
And the specific aims are:
 to evaluate the appropriateness of the material to the course requirements in
terms of aims, content, and methodology. Then areas for improvement of this
textbook will be identified.
 finally, to raise some recommendations to improve this textbook in order to
meet the learning demands of learners.
 In addition, theoretically the improved textbook of social sciences and
humanities, which is rather rare, can be an effective and helpful referential
material for teachers and learners.
1.3 Research question
To achieve the above mentioned aims, the following research question was
posed:

14
- To what extent does the ESP material English for graduate students of social
sciences and humanities at USSH, VNU meet the course requirements in
terms of aims, content, and methodology?
1.4 Scope of the study
Hutchinson and Waters (1993), Brown (1995) and many other language
evaluation experts agreed that evaluating a teaching material is a complicated process
based on different criteria such as audience, content, physical appearance, cultural bias,
needs and so on. However, because of time limitation, the researcher just focuses on
the criteria stated in the requirements of the ESP course for graduates of social sciences
and humanities at USSH, VNU, including aims, content and methodology and time
allocation.

1.5. Design of the study
The paper is organized into five chapters:
1. Chapter I is the introduction that presents such basic information as rationale,
aims, methods, research questions, and design of the study.
2. Chapter II presents a review of related literature that provides introduction to
textbook and textbook evaluation but the focus is on the importance of the
appropriateness of the textbook and learning targets. Some understandings about
criteria of a good textbook will also be proposed.
3. Chapter III consists of rationale for the research method applied, description of
three groups of subjects, data elicitation tools and data collection procedures.
4. Chapter IV, the main section of the study, presents the data analysis and
findings concerning students‟ language levels in an ESP course, the present
situation of using English for graduate students of social sciences and
humanities at USSH in order to lay bare its strengths and weaknesses. Based on

15
these, an evaluation of the appropriateness of this textbook to learning targets
was done.
5. Chapter V proposes some recommendations from the perspectives of teaching
staff and students in terms on how to improve this textbook so that learners‟
needs were met. In addition, the limitations of this study are also presented.


16
Chapter II
Literature review
2.1 Materials in language teaching and learning
2.1.1 Roles of teaching material EFL/ESL classes
The relationship between materials and other components in language teaching
such as students, teachers, teaching method and evaluation have been controversial for

many years. Since the end of 1970s, there has been a movement to make learners rather
than teachers the center of language learning. According to this teaching approach,
learners become the targets of teaching and learning process. Therefore, materials,
curriculum, teaching method and evaluation should be designed for learners and their
needs. For the teaching and learning English to be successful, besides teachers,
learners, curriculum, method and evaluation, materials play a decisive part in language
teaching and learning. The importance of teaching and learning materials can not be
denied and so far, there have been a great number of discussions on different
perspectives on the role of materials in relation to other elements. Teaching materials
are considered the essential components in most language courses because they help
students and teachers know teaching and learning content. Discussing the importance
of English language teaching materials to learners, Allwright (1981) indicates that
materials help students know what to learn. In addition, he says, they are resource for
ideas, activities for instruction and they give teachers rationale for what to do. He also
emphasizes that materials control learning and teaching. It is undoubted that materials
are of importance in teachers‟ language instruction. Both experienced and
inexperienced teachers rely on the materials. O‟Neill (1982) supports Wright‟s ideas
that materials help learning and teaching. It is true that in many cases, teachers and
students rely heavily on materials and materials determine the components and

17
methods of learning. They control the content, method and procedures of learning.
Students learn what is presented in the textbook and the way the textbook presents
materials in the way the teacher teaches it. Therefore, the materials become the center
of instruction and one of the most important influences on what goes on in the class. In
this view, Tomlinson (1998, p.20) confirms “materials facilitate then curriculum,
teaching method and evaluation in language learning, increase the learners‟ knowledge
and experience of the language”. He considers materials as the basic for much of the
language input that the learners receive and as the sources for much of the language
practice that occurs in the classroom.

Dudley-Evans and St John (1998) also indicate the four functions of teaching
materials. The first function is that materials are authentic source of language, which
means that materials need to present real language as it is used in real situation and be
suitable to a wide range of learners‟ needs. The second one is that materials are a
source of learning support. To enhance learning, according to the writers, the materials
need involve students in thinking about and using language; the activities in the
materials need to “stimulate cognitive, not mechanical process”; and the learners also
need “a sense of progression” (p.171). The third one is that materials are a source for
stimulation and motivation. To fulfill this function, materials need to be “challenging
yet achievable; to offer new ideas and information whilst being grounded in the
learner‟s experience and knowledge” (Dudley-Evans and St John, 1998, p.172). These
authors also explain that the input must contain concept and knowledge that are
familiar to the learners, but it must also offer something new, a reason for learners to
communicate and to get involved. The last one is that materials are a source of
reference, which is understood as that materials need to be “complete, well-laid out and
self-explanatory” (Dudley-Evans and St John, 1998, p.172). In other words, materials

18
should provide explanations; examples and practice activities that have answers and
discussion keys do that learners can use them for their self-study.
Obviously, good teaching materials are of great help to both inexperienced and
experienced teachers, especially the inexperienced or poorly trained teachers because
they also provide models for teachers to follow in developing their own materials
(Nunan, 1998). Therefore, the first thing the teachers need to do when preparing his
course in his own situation is to write or to select the appropriate materials.
In USSH, English for post graduates of social sciences and humanities was
designed by Lam Quang Dong, Dean of the Department of foreign language. Due to
the lack of time and the unavailability of various resources, the material was compiled
from various texts and most of the exercises were designed by the author. After being
used for two years, it is necessary to evaluate the material to see if it were appropriate

or not.

2.1.2 Types of materials
The following section is devoted to a brief account of different types of materials
and the advantages and disadvantages of using these types of materials. According to
Robinson (1991), the first type is published textbooks and the second type is in-house
materials.
 Textbooks
The term “textbook” refers to course book, of which the teacher and usually each
student has a copy, is principle to be followed systematically as the basic for a
language course. There are some arguments for and against using a textbook.
According to Ur (1996), textbooks have many advantages as follows:
A textbook provide a clear framework: teachers and learners know where they are
going and what is coming next. In many places, the textbook serves as a syllabus.
Besides, the textbook provides texts and learning tasks, which are likely to be of an

19
appropriate level for most of the students. It is also the cheapest way of providing
learning material for each learner (p.184).
However, not everything in the textbook is wonderful (Hammer, 1991). According
to Hammer and Ur, though the textbooks are well planned, they can be inappropriate
for teachers and learners. The counter-arguments for using textbooks are also listed by
Ur (1996) as follows:
- Inadequacy: Every class in fact every learner has their own learning needs: no
one textbook can possibly supply these satisfactorily.
- Irrelevance: lack of interest: the topics dealt with in the textbook may not
necessarily be relevant or interesting for your class.
- Limitation: a textbook is confining: its set structure and sequence may inhibit a
teacher‟s initiative and creativity, and lead to boredom and lack of motivation
on the part of the learners.

- Homogeneity: textbooks have their own rationale, chosen teaching, and learning
approach. They do not usually cater for the variety of levels of ability and
knowledge, or learning styles and strategies that exist in most classes.
- Over-easiness: teachers find it too easy to follow the textbook uncritically
instead of using their initiative; they may find themselves functioning merely as
mediators of its content instead of as teachers in their own right.
Therefore, once using a textbook, teachers must consider these counter-arguments.
 In-house materials
In-house materials or institutionally prepared materials are also the teaching
materials that teachers use (Richards, 2001). One of the most importance factors that
in-house materials have is that they are always designed for particular group of learners
so they correspond to learners‟ needs and match the aims and objectives of the course.

20
According to Robinson (1991), using in-house materials, has many advantages as
follows:
“First, in-house materials are likely to be more specific and appropriate than
published materials and to have greater face validity in terms of the language
dealt with and the contexts it is presented in. second, in-house materials may be
more flexible than published textbooks. Last, the writer of in-house materials
can make sure of the suitability of methodology for the intended learners.
However, in-house materials take time to write and their production is relatively
expensive”. (p.58)
In short, both textbooks and in-house materials have their own advantages and
disadvantages. Which one should be used depends on the purposes of the course as
well as particular situation. Which category does the material under investigation fall
into? In this particular situation, English for graduate students of social sciences and
humanities belongs to in-house material section.
2.2 ESP and ESP materials
2.2.1 A brief overview of ESP

English for Specific purposes (ESP) has become an important part of English
language learning. People want to learn English, not for the pleasure or prestige of
knowing the language, but because English is the key to the international currencies of
technology and commerce. Knowledge of a foreign language has been generally
regarded as a sign of a broad education. English created a new generation of learners
who knew specifically why they were learning a language such as doctors who needed
to keep up with developments in their field, etc.
From the early 1960‟s, the development of English for Specific Purposes has
been reflected in the increasing number of ESP courses offered for overseas students in
English speaking countries. Since then, the teaching of English for Specific Purposes
(ESP) has been a lively stimulating part of English language teaching (ELT). It has

21
been generally acknowledged that, while the remaining a part of ELT, ESP developed
its own procedures, such as needs analysis, its own materials and its own teaching
methodology (Dudley-Evans and St. John, 1998). The emphasis in the definition of
ESP has been on how ESP teaching develops procedures appropriate to learners whose
main purpose is learning English for a specific aim/use other than just learning the
language system. That purpose may be educational, or may be professional, and ESP
seeks its justification on how well it prepares learners to fulfill the purposes required of
them. ESP has been defined by many researchers and writers. Anthony (1997) notes
that there has been considerable recent debate about what ESP means despite the fact
that it is an approach that has been widely used over the last four decades. Mackay and
Mountford (1978) indicated that the term ESP is generally used to refer to the teaching
of English for a clear utilitarian purpose. El-Minyawi (1984) pinpointed that ESP
courses are based solidly upon the need to express the facts and ideas of some special
subjects after which the student should be able to read the specialized subjects
confidently and speak about them fluently. The definition of Strevens (1988) makes a
distinction between four absolute characteristics and two variable characteristics
including:

Absolute characteristics:
- ESP is designed to meet specific needs of the learners
- ESP is related in content (that is in its theme and topic) to particular disciplines,
occupation and activities.
- ESP is centered on language appropriate to those activities in syntax, lexis
discourse
- ESP in contrast with “General English”
Variable characteristics:
- ESP may be restricted to the learning skills to be learned.
- ESP may not be taught according to any pre-ordained methodology

22
At the 1997 Japanese Conference on ESP, Dudley-Evans offered a modified
definition. The revised definition by Dudley-Evans and St. John postulates with
absolute and variable characteristics as follow:
+ Absolute characteristics:
- ESP is defined to meet specific needs of the learner
- ESP makes use of the underlying methodology and activities of the discipline it
serves
- ESP is centered on the language (grammar, lexis), skills, discourse and genres
appropriate to these activities
+ Variable characteristics
- ESP may be related to or designed for specific disciplines
- ESP may use, in specific teaching situations, a different methodology from that
of general English
- ESP if likely to be designed for adult learners, either at a tertiary level institution
or in a professional work situation, It could, however, be for learners at
secondary school level
- ESP is generally designed for intermediate or advanced students
- Most ESP courses assume some basic knowledge of the language system but it

can be used with beginners (Dudley Evans & St. John, 1998, p.4-5)
In my point of view, the best definition for ESP is the one by Hutchison and Waters
(1987), they defined it as an approach to language teaching in which all decisions as to
content and method are based on the learner‟s reason for learning. They added also that
the term ESP implies that it is English which is somehow peculiar to the range of
principles and procedures that define that particular profession. Besides, ESP does not
involve a particular kind of language, teaching material or methodology.
Regarding types of ESP, Carter (1983) identifies three types of ESP:

23
- English as a restricted language (for example, the language used by air traffic
controllers or by waiters)
- English for Academic and occupational purposes
- English with specific topics
The language used by air traffic controllers or by waiters are examples of
English as a restricted language.
The second type of ESP identified by Carter (1983) is English for Academic and
Occupational Purposes. In the “Tree of ELT” (Hutchison and Waters, 1987), ESP is
broken down into three branches: English for Science and Technology (EST), English
for Business and Economics (EBE), and English for Social Studies (ESS). Each of
these subject areas is further divided into two branches: English for Academic
Purposes (EAP) and English for Occupational Purposes (EOP). An example of EOP for
the ESS branch is “English for Social Sciences and Humanities”. In other words, the
studied material English for graduate students of social sciences and humanities is an
EOP branch of ESS.
The third and final type of ESP identified by Carter (1983) is English with
specific topics. Carter notes that it is only here where emphasis shifts from purpose to
topic. This type of ESP is uniquely concerned with anticipated future English needs of,
for example, scientists requiring English for postgraduate reading studies, attending
conferences or working in foreign institutions.

In short, it is clear that ESP plays very important role in the English language
teaching and learning. Therefore, it is extremely necessary to pay much attention to the
teaching and learning of ESP.
2.2.2 The role of ESP materials
It is undeniable that materials are a key component in most language courses
and teaching materials is vital in the process of successful language teaching and
learning. Materials are also the center of instruction and one of the most important

24
influences on what goes on in the classroom. Many authors indicate this crucial role
that teaching materials play in language teaching. Brown (1995, p.139) defines
“materials as any systematic description of the techniques and exercises to be used in
the classroom teaching”. Allwright (1981) also accepts that materials should teach
students to learn, that they should be resource books for ideas and activities for
instruction/learning, and that they should give teachers rationales for what they do.
Similarly, according to Cunningsworth (1995), teaching materials are sources of ideas
and stimulation for classroom activities, which provide learners with practice in
communicative interaction. Besides, teaching materials can function as a reference
source for learners on grammar, vocabulary and pronunciation. Moreover, teaching
materials can be considered as a syllabus reflecting learning objectives that have
already been determined and a support for less experienced teachers who have yet to
gain confidence. At this point, materials are vital resources because they stimulate and
develop students‟ linguistics skills. This role is even more important and essential in
the ESP context as in this point, ESP materials can be the primary source of exposing
learners to the language they need to learn.
In short, teaching materials have great efforts on the success of any language
teaching program, especially the teaching and learning of ESP. The teacher needs to
take into account the roles as well as the requirements of teaching materials when
designing, selecting or changing material for his/her own teaching situation to make
them suitable to the learners and the aims of the course. In the particular situation at

USSH, VNU, ESP material of social sciences and humanities plays a key role in
providing both teachers and learners sources in their teaching and learning process.
Without it, they will get a large number of difficulties in finding materials, identifying
what to teach and learn and they likely have to spend a lot of time and efforts during
their ESP course.


25
2.3 Materials evaluation
2.3.1 Definitions of materials evaluation
Until now, there have been a number of materials evaluations of different author
in ELT. According to Tomlinson (1998, p.3), materials evaluation is defined as “a
systematic appraisal of the value of materials in relation to their objectives and to the
objectives of learners using them.” (Ellis, quoted in Figen, 2002, available at
./home/wwwsbe/thabs/2002/EDS.htm)
Hutchinson and Waters (1993, p96) also give definition of materials evaluation,
as “Evaluation is a matter of judging the fitness of something for a particular purposes.
Given a certain need, and in light of the resources available, which is out of a number
of possibilities can represent the best solution?” In other words “Evaluation is basically
a matching process: matching needs to available solutions.” Hutchison and Waters
(1993, p.96) also give definition of material evaluation as “Evaluation is a matter of
judging the fitness of something for a particular purposes. Given a certain need, and in
the light of the resources available, which is out of a number of possibilities can
represent the best solution?” In other words “Evaluation is basically a matching
process: matching needs to available solutions.” (Hutchinson and Waters (1993, p. 97).
Dubin and Olshtain (1986, p.3) support Hutchinson and Waters by stating “evaluation
means the same as testing and that while students are being tested evaluation is taking
place in terms of judging the fitness of something for a particular purposes.”
In conclusion, from the general definition of evaluation and the definition of
specific area of materials evaluation, it can be inferred that materials evaluation involve

the determination of what needs to be evaluated, the objectives and requirements of the
materials, and the judgments of the value of the materials being evaluated in relation to
the objective and requirements determined.



26
2.3.2 Techniques of materials evaluation
As mentioned in section 2.3.4, there are three types of evaluation and each of
them has their own method to collect data (Robinson, 1991). The importance is that
researchers should identify the suitable methods in the study. A lot of technique used to
evaluate materials attract much attention and have been discussed by many researchers:
Dudley-Evan and St. John (1998). According Hutchinson & Waters (1993), Lynch
(1996), Richards (2001), Robinson (1991), and according to Hutchinson & Water
(1993, p.153), the useful evaluation techniques area questionnaires, interviews, test
results, discussion and other informal means.
Dudley-Evan & St John (1998, p.132) stated that in evaluating and ESP course,
one or more of the data collection methods can be used: checklist and questionnaires,
accession, discussion and record keeping.
The different methods used for evaluation have been mentioned but they all
have their possible advantages and limitations as follows.
Questionnaires are the most popular methods of data collection. They can be
used to elicit students‟ and teachers‟ comments on a wide range of issues. The
advantage is that they can be given to a large group of respondents at the same time
and they are not expensive (Wallace, 1998). However, questionnaires need to be
carefully designed to avoid respondents‟ misinterpretation of questions.
Interviews, as Robinson (1991) stated, are also a valuable method for
evaluation. They have advantages of in-depth information on specific questions but the
problem with this type of instruments is that it is very time-consuming though they can
provide the opportunity for more extended exploration of the issues than

questionnaires.

27
Observation can focus on any observable aspect of the lesson and can provide
an objective eye to identify things that may not be very clear to the teacher. However,
it is a specialized work and requires careful preparation.
Document analysis refers to any documents that are involved in evocation, such
as: curriculum description, policy statement, newspapers, and program brochures,
correspondence and so on (Lynch, 1986). According to Robinson (1991), document
analysis can be records. He points out that records can be reports of lessons taught,
statements of needs, aims and objectives of the course. It is also trencher‟s checklist
used to evaluate the lesson taught. In brief, according to him, records can be any
documents that are related to the course valuation.
The numerous authors have discussed some other types of instruments used
(including checklists, discussion and tests) to collect data. However, each stage
requires suitable instruments. Robinson (1991) suggests that questionnaires, interviews
and tests should be used for formative evaluation but summative evaluation needs the
use of questionnaires, checklists and tests.
To sum up there has been a large number of different methods to collect data.
The researcher should identify what types of these instruments to gee used effectively
in their study. For example, with the aims of evaluating the effectiveness of the
textbook used for her students. Vo (2004) used document analysis, interviews and
questionnaires to collect data. Nguyen (2004) applied the materials analysis in addition
to questionnaires to gather information from both teachers and students. They all have
common techniques, using questionnaires to collect data because of the big numbers of
participants in their research. Also, from their studies, questionnaires were suggested to
be the best method for date collection in their particular teaching contexts.
According to Hutchinson & Water (1993) and Robinson (1991), employing
instruments in an evaluation project will depend on what suits the particular situation;