An evaluation of american headway 1 for first year junior college students in electric power university submitted in partial fulfilment of requirements for the degree of master of arts in tesol

M IN IS TR Y OF EDUCATIO N A N D TR A IN IN G

HANOI UNIVERSITY

NG UYEN TH I BICH NGOC

AN EVALUATION OF AMERICAN HEADWAY 1
FOR FIRST YEAR JUNIOR COLLEGE STUDENTS
IN ELECTRIC POWER UNIVERSITY

t

TIMJNG И М
ĩHÚNO TIN THƯ VIỀN
NN-VH Nưứ c ỉiGOẲl

Т4ГѴ

SUBMITTED IN PARTIAL FULFILMENT OF REQUIREMENTS FOR
THE DEGREE OF MASTER OF ARTS IN TESOL

SUPERVISOR: Dr. L A M QUANG DONG

Hanoi
October 2008


ACKNOW LEDGEM ENTS
I have had considerable help from a number o f people to accomplish this work, and I
would like to thank them here.

I, first and foremost, owe a great debt to my thesis supervisor. Dr. Lam Quang Dong,
for his immense encouragement.

W ithout the valuable instructions, comments,

criticisms, questions, and corrections which I received along the way fk)m him. this
thesis would not exist.
I would like to give my sincere gratitude to Dr. Nguyen Van Dai and M.Ed Nguyen
Thai Ha - the leaders and the whole staff members o f the Department o f Post-Graduate
Studies at Hanoi U niversity fo r their useful lectures and assistance, valuable comments,
and suggestions.
I acknowledge the excellent assistance o f Dr. Pham Xuan Khang- Head o f Fundamental
Scientific Faculty, m y colleagues，and students in Electric Power University fo r their
support, assistance，and cooperation in performing this work.
I would be remiss i f I d id n 't express my special thanks to my parents, my younger sister
whose encouragement and care were accompanying me during the long journey to the
university. O f course, nothing o f this would have been possible i f it had not been for the
love and support from my husband who has helped me in so many ways to ease the
burden o f the work. Needless to say, their patience and support have kept me going this
far and their love w ill keep me going further.


ABSTRACT
This study adopts both quantitative and qualitative approaches to evaluate the textbook

American Headway 1 for the first year junior college students in Electric Power University.
The study attempts to find out how much the textbook fits with the course aims and the
students' needs on the content and methodology.
Data were gathered by means o f existing information and questionnaires. The existing
information data were collected by only one subject- the researcher. The questionnaires

data were obtained by a total o f 200 students from 5 classes and 7 teachers o f English.
The results indicate that the textbook was basically suited to the course objectives and the
students7 needs. Nevertheless, the textbook

failed

to provide the

students with

pronunciation. Moreover, it did not completely satisfy the students^ needs on its content
(some difficult grammatical structures, difficult and abundant vocabulary, the difficulties o f
writing, listening, and speaking skill), its methodology (some difficult kinds o f exercises,
the organization o f pair and group activities, some aids, necessary supplementary missing
materials), and its time requirement.
Based on the findings, a number o f recommendations were made to overcome its
weaknesses by using the adding, replacing, and modifying technique as well as to enhance
English teaching and learning for the first year junior college students in Electric Power
University.


LIST OF TABLES AND FIGURES
Table 1

The Factors that need examining in the materials evaluationprocess.

Table 2

The detailed information o f the student questionnaires.

29

Table 3

The detailed information o f the teachers.

31

Table 4

The detailed information o f the topics in the textbook.

44

Figure 1

Evaluation Model by Hutchinson & Waters (1987).

11

Figure 2

Evaluation Model by McDonough & Shaw (1993).

13

Figure 3

Evaluation Model by Ellis (1998).

14

Figure 4

Materials adaptation framework by McDonough & Shaw (1993).

23

Figure 5

The students，
judgments about the aims o f the textbook.

38

Figure 6

The teachers’ judgments about the aims o f the textbook.

39

Figure 7

13

The students^ judgments about the Presentation & Practice o f the

45

language points & language skills.

Figure 8

The teachers9 judgments about the Presentation & Practice o f the

45

language points & language skills.
Figure 9

The students’ judgments about the difficult level o f the language

46

points & language skills given in the textbook.
Figure 10

The teachers’ judgments about the difficult level o f the language

46

points & language skills given in the textbook.
Figure 11The students9judgments about the variety, attractiveness o f the text-

47

types & the familiarity, interest o f the topics in the textbook.
Figure 12The teachers5judgments about the variety, attractiveness o f the text-

47

types & the familiarity, interest o f the topics in the textbook.
Figure 13

The students’ judgments about the teaching-learning techniques in the

51

textbook.
Figure 14

The teachers’ judgments about the teaching-learning techniques in the

51

textbook.
Figure 15

The students’ some overall judgments about the textbook.

58

Figure 16

The teachers’ some overall judgments about the textbook.

57


LIST OF ABBREVIATIONS
EPU

Electric Power University

GE

General English

ESP

English for Specific

M .A

Master o f Arts

B.A

Bachelor o f Arts

c.v

Curriculum Vitae

TESOL

Teaching English to Speakers o f Other Languages

AHI

American Headway 1

Purposes


TABLE OF CONTENTS
ACKNOWLEDGEMENTS............................................................................................................i
ABSTRACT..................................................................................................................................... ii
LIST OF TABLES AND FIGURES.......................................................................................... iii
LIST OF ABBREVIATIONS..................................................................................................... iv
TABLE OF CONTENTS..............................................................................................................V
CHAPTER I: INTRODUCTION................................................................................................ 1
1.1. Background to the Study and Statement o f the Problem....................................................1
1.1.1. Overview o f Electric Power University........................................................................... 1
1.1.2. Statement o f the problem.................................................................................................2
1.2. Aims o f the study.............................................................................................................. 3
1.3. Research questions.............................................................................................................. 3
1.4. Limitations o f the study......................................................................................................4
1.5. Scope o f the study............................................................................................................... 4
1.6. Organization o f the study....................................................................................................5

CHAPTER II: LITERATURE REVIEW............................................................................... 6
2.1. Some basic concepts in the study....................................................................................... 6
2.1.1. Evaluation........................................................................................................................ 6
2.1.2. Needs................................................................................................................................ 6
2.1.3. Learning styles................................................................................................................. 7
2.1.4. Textbooks......................................................................................................................... 7
2.2. Materials Evaluation...........................................................................................................7
2.2.1 The role o f materials......................................................................................................... 7
2.2.2. Types o f materials evaluation......................................................................................... 8
2.2.3. Guidelines for evaluation................................................................................................ 9

2.2.3.1. Guidelines for evaluation by Cunningsworth (1995).................................................. 9

2.23.2. Guidelines for evaluation by Grant (1987).............................................................. 10
2.2.4. Types o f materials evaluators..................................................................................... 10
2.2.5. Evaluation models for materials evaluation................................................................11
2.2.5.1. Evaluation model by Hutchinson and Waters............................................................ 11


2.2.5.2. Evaluation model by McDonough & Shaw................................................................ 12
2.2.5.3. Evaluation model by E llis ............................................................................................14
2.2.6. Tools for gathering information....................................................................................15
2.3.

Previous studies............................................................................................................. 18

2.4. Adapting materials.............................................................................................................19
2.4.1. Reasons for adapting materials...................................................................................... 19
2.4.2. Techniques for adapting materials................................................................................. 21

CHAPTER III: METHODOLOGY......................................................................................... 25
3.1. Data collection tools.......................................................................................................... 25
3.1.1. The statement o f the existing inform ation.....................................................................25
3.1.2 Questionnaires................................................................................................................. 26
3.1.2.1. The statements o f the questionnaires........................................................................ 26
3.1.2.2. Question formats........................................................................................................ 27
3.1.2.3. Student questionnaires............................................................................................... 28
3.1.2.4. Teacher questionnaires...............................................................................................29
3.2. Subjects..............................................................................................................................30
3.2.1. The researcher as a subject.............................................................................................30
3.2.2. The teachers................................................................................................................... 30

3.2.3. The students....................................................................................................................31
3.3. Data collection procedures...............................................................................................31
3.3.1. Existing information collection procedures...................................................................31
3.3.1.1. Literature review..........................................................................................................31
3.3.1.2. The aims o f the course................................................................................................32
3.3.1.3. Textbook..................................................................................................................... 32
3.3.2. Teacher questionnaire collection procedures................................................................ 32
3.3.3. Student questionnaire collection procedures................................................................. 33

CHAPTER IV: RESULTS AND DISCUSSION................................................................... 34
4.1. The results......................................................................................................................... 34
4.1.1. The fitness o f the textbook to the aims o f the course....................................................34
4.1.1.1. Statements o f aim s...................................................................................................... 34
4.1.1.2. Textbook analysis results............................................................................................34

vi


4.1.1.3. Questionnaire results...................................................................................................38
4.1.1.4. Matching and discussion.............................................................................................39
4.1.2. The fitness o f the textbook content to the students’ needs.......................................... 40
4.1.2.1. Textbook analysis results........................................................................................... 40
4.1.2.2. Questionnaire results.................................................................................................. 44
4.1.2.3. Matching and discussion............................................................................................ 48
4.1.3. The fitness o f the textbook methodology to the students’ learning styles and
expectations.............................................................................................................................. 49
4.1.3.1. Textbook analysis results........................................................................................... 49
4.1.3.2. Questionnaire results................................................................................................... 51
4.1.3.3. Matching and discussion.............................................................................................52
4.1.4. Suggestions for the textbook improvement.................................................................53

4.1.4.1. Teacher questionnaire results..................................................................................... 54
4.1.4.2. Student questionnaire results..................................................................................... 55
4.2.

Major findings.............................................................................................................. 56

4.2.1. Strong points.................................................................................................................. 56
4.2.2. Weak points.................................................................................................................... 56

CHAPTER 5: RECOMMENDATIONS AND CONCLUSION........................................ 59
5.1. Recommendations............................................................................................................. 59
5.1.1. Areas o f unfitness...........................................................................................................59
5.1.1.1. A im s............................................................................................................................ 59
5.1.1.2. Content........................................................................................................................ 59
5.1.1.3. Methodology...............................................................................................................60
5.1.1.4. Time allocated to lessons............................................................................................61
5.1.2. Techniques for adaptation........................................................................................... 61
5.1.2.1. Adding..........................................................................................................................61
5.1.2.2. M odifying.................................................................................................................... 66
5.1.2.3. Replacing.................................................................................................................... 69
5.1.3. A sample unit for adaptation....................................................................................... 69
5.1.3.1 Rationale for the adaptation........................................................................................ 69
5.1.3.2. Suggested adaptation................................................................................................... 69


5.2. Suggestions for further study.............................................................................................71
5.3. Conclusion.......................................................................................................................... 71

REFERENCES............................................................................................................................... 74
APPENDIX 1 .................................................................................................................................. 80

APPENDIX 2 .................................................................................................................................. 84
APPENDIX 3 .................................................................................................................................. 92
APPENDIX 4 ................................................................................................................................ 100
APPENDIX 5 ................................................................................................................................ 104
APPENDIX 6 ................................................................................................................................ 105
APPENDIX 7 ................................................................................................................................ 113


CH APTER I: IN TR O D U C T IO N
1.1. Background to the Study and Statement o f the Problem
1.1.1. Overview o f Electric Power University
The School o f Electricity was established in 1966. Thanks to the development o f the
country, the increasing requirement o f human resources in the field o f Electricity, the
School o f Electricity has also been developing. It was upgraded into Electric Power College
in 2001 and then Electric Power University (EPU) in 2006. Nowadays, EPU still maintains
to train at three levels: junior college, college, and university in eight specialties: Electric
Power System, Information Technology, Thermal Power Generation, Hydro Power,
Automation Technology, Energy Management, Electric Mechanics, Electronic Mechanics,
and Electronic Telecommunication.
Up to now, EPU has not had an English Faculty yet. Therefore, English teaching which is a
subject in Fundamental Science Faculty is undertaken by twelve female teachers. More than
half o f them finished Master o f Arts (M .A) courses with English teaching experiences from
3 to 25 years.
The student body has been increasing to the current number o f more than 2,000 because o f
the high demand for human resources in the field o f Electricity. Their backgrounds o f
English are varied since some had learnt English or other foreign languages at school, but
those from mountainous areas either have never been exposed to a foreign language or have
very low level o f English.
English has been taught as a compulsory subject in EPU since 1996. Students study
General English (GE) in two terms (150 periods) at three levels (university, college and

junior college) and continue to learn English for Specific Purposes (ESP) in one term (60
periods) at two levels (university and college). English textbooks currently applied in EPU
are diverse, including Lifelines, American Headway, Headway，and New Headway. Many
meetings have been taking place in order to discuss the selection o f textbooks, to strengthen
the quality o f English teaching and learning in EPU. The textbooks, yet, remain

1


problematic, especially for junior college students. In 2004, the system o f modules was
designed and used. After two years, this system o f modules was replaced with the adoption
o f American Headway Լ 2 by Soars, L. & J. (2001). However, the examination results
have not considerably been improved. The students still have a lot o f difficulties in
exploring the textbook.
1.1.2. Statement o f the problem
Fostering the quality o f teaching and learning is always a burning problem o f each school.
There are many factors which directly affect the training results, for instance, goals,
materials, teaching, testing, equipments, and the students՜ attitude, etc. However, in the
limited scope o f a thesis and with the expectation o f contributing partly to enhancing the
quality o f English teaching and learning in EPU, the researcher made a decision to
investigate, study a textbook- a kind o f material since it is one o f the important factors
determining the quality o f teaching and learning. The textbook examined here is American

Headway 1 (A H I) which is currently applied for the first-year junior college students in
EPU. In addition, there are some other reasons which urged me to conduct this study as
follows:
In fact, English teaching and learning for the first-year junior college students in EPU still
suffers from a lot o f restrictions. This is shown by the results o f the term-end examinations
which have been low in recent years. More than 50% o f the students usually get the English
marks below average and certainly these students have to retake the English course. A lot

o f students may reluctantly learn English to face with the examinations. Furthermore, many
teachers o f English have controversial views on the application o f A H I currently used for
the junior-college students in EPU. As a teacher who has directly been teaching English in
EPU, I may understand the most clearly about the existing problems.
It is likely that A H I is a quite nice textbook both in content and form. The information is
update as it was published in 2001. Nonetheless, it is not a perfect one, especially for the
jun io r college students. Perhaps, it has some weaknesses, for example, some difficult
grammatical structures, insufficient pronunciation practice, and so forth. Consequently,

2


most o f the students often make mistakes o f those grammatical structures or have wrong
pronunciation o f some very basic words.
So far, there have been just a small number o f studies about the materials evaluation, for
instance, Nguyen Thi Hai Ha (2004)，Nguyen Thi Thanh Van (2004)，Tran Thi Thuy Nga
(2005)，Hoang Hoa Lien (2005)，and Pham Thi Thanh Huong (2006) examined ESP
materials; Le Thi Ngoe Diep (2006) and Vu Thi Yen Nga (2004) evaluated General English
materials (Powerbase，Lifelines); Akin & Guceri (2001) and Ngo Thi Thu Huong (2007)
assessed in-house materials, and so forth. Nevertheless, there has not been a thesis about
the evaluation o f A H I by Soars, L. & J. yet.
From the above reasons, the researcher decided to investigate, study, and adapt A H I
currently used for the first-year junior college students in EPU. Hopefully, the study w ill be
o f some significance to the researcher, the teachers, the first year junior college students,
and the authorities in EPU as the findings o f the study w ill supply them with scientific
evidence to either enhance the quality o f the textbook or to abandon it altogether.
Moreover, this is also an opportunity for the researcher to learn to analyze a study, evaluate,
and report about an issue before completing her Master o f Arts (M .A) course at Hanoi
University.
1.2. Aims o f the study

The study aims to find out how much A H I fits with the course aims and the students’ needs
in terms o f the content and methodology. The unsatisfactory areas o f the textbook are, then,
adapted to improve its effectiveness for future courses.
1.3. Research questions
According to Cunningsworth (1995)，there are many criteria for textbook evaluation, for
example, aims and approaches, design and organization, language content, skills, topic,
methodology, teacher’ s book, and practical considerations. Due to various constraints and
the limitations o f this study, only a number among these criteria can be evaluated in order
to find the answers to the following research questions:

3


L How much do the aims o f A H I f it with the aims o f EPU's English program ?
2. How much does the content o f A H I satisfy the students ’ needs?
3. How responsive to the students ’ learning styles and expectations is the methodology
provided in A H I?
In order to answer the specific questions above, the researcher first reviews the literature on
materials evaluation. Based on the theories in the literature, the researcher analyses the
course aims and the textbook, and then carries out a survey to explore the teachers9 and
judgments about the textbook. Finally, the researcher recommends the textbook
adaptation.
1.4. Lim itations o f the study
In this thesis, three factors might affect the outcomes o f the evaluation research as follows:
First, the evaluators were all the insiders. Thus, their subjective points o f views may
influence the textbook evaluation.
Second, the suggested adaptations are just what the researcher thinks they should be, based
on her specific teaching context in EPU. These suggestions, therefore, may be subjective
and unsuitable for different students in a different teaching context.
Finally, the researcher only applied three criteria and two data collection tools in order to

evaluate the fitness o f the textbook to the aims o f the course and the students’ needs.
1.5. Scope o f the study
A strong consensus has built up worldwide in recent decades in support o f a large number
o f evaluative criteria suggested by some writers such as Cunningsworth (1984), Hutchinson
& Waters (1987), McDonough & Shaw (1993)，Brown (1995)，Littlejohn (1998)，and Ur
(1991). However, due to the time constraints, the researcher o f the study is only interested
in Hutchinson & Waters (1987)’criteria in the textbook evaluation as follows:
♦ Aims o f the textbook.

4


♦ Content o f the textbook (language points, language skills, text-types, and topics).
♦ Methodology o f the textbook (kinds o f activities/exercises, teaching-learning techniques,
aids, flexibility, and language guidance).
It is also noted that some sub-criteria chosen for the study are modified so that they are
better suited to the researcher’ s particular teaching-learning context.
1.6. Organization o f the study
The study is organized into five main chapters as follows:
Chapter 1 discusses the background to the study and statement o f the problem, the aims, the
research questions, the limitations, the scope, and the organization o f the study.
Chapter 2 presents a review o f literature, concentrates on the issues relating to materials
evaluation.
Chapter 3 describes the data collection instruments, the subjects as well as the data
collection procedures.
Chapter 4 discusses the findings o f the study, points out the strengths and weaknesses o f the
textbook.
Chapter 5 presents solutions to improve the textbook, suggests for further study, and
concludes the study.

5


C H APTER II :

L IT E R A T U R E R E V IE W

2.1. Some basic concepts in the study
The title o f the thesis and the research questions result in the demand for exploring some
key concepts which are o f great importance before going more deeply into the study.
2.1.1. Evaluation
There are a lot o f definitions o f materials evaluation, however, the one o f Hutchinson and
Waters (1987) is likely to be the clearest because it provides a comprehensive
understanding o f the evaluation process. They regard evaluation as a matter o f judging the
fitness o f something for a particular purpose.
2.1.2. Needs
In deed, the term needs is not as straightforward as it might appear. When we read or hear
it, many words and expressions are triggered in our minds: uWants-Desires-DernandsExpectation-Motivations-Lacks-Constraints֊Requirements^^ (Brindley 1984: 28，cited in

Richards 2001; 54). More specifically and practically, Robinson (1991; 7) considers needs
as what the students themselves would like to gain from the language course. Needs can be
classified into different types, for example, target needs and learning needs (Hutchinson &
Waters 1987)，situation needs and language needs (Brown 1995)，etc. Yet, in brief, the term
“ needs” is very broad one referring to the language demand o f the particular learners and
local learning situation.
Furthermore, five needs experienced by beginner students should be considered when
analyzing the students’ needs. Those five needs are: the need to communicate effectively,
the need to be familiar with the language systems, the need for challenge, the need to take
on more responsibility for their own learning, and the need for cross-cultural awareness
(Abbs & Freebaim 1990，cited in Cunningsworth 1995: 9 ᄀ).

6


2.1.3. Learning styles
According to Richards, Platt & Weber...Richards et a l (1992: 61), a learning style can be
considered the particular way in which a learner tries to learn something. In second or
foreign language learning, different learners may prefer different solutions to learning
problems. For example, some learners may want explanations for grammatical rules; others
may not need explanations. Some learners may feel w riting down words or sentences helps
them to remember the words or sentences. Others may find they remember things better i f
they are associated with pictures. Obviously, a learner may have more than one learning
styles, a teacher may also have different teaching styles, and a textbook may have various
teaching-learning techniques. The problem is to have the unification among the learners,
the teachers, and the textbook (Grant 1987: 11).
2.1.4. Textbooks
Materials may be printed (books, worksheets, etc), non-printed (cassette or audio materials,
videos, etc), or both printed and non-printed ones (self-access or the Internet ones)
(Richards 2001; Brown 1995). A textbook, therefore, can be viewed as a kind o f materials.
According to Richards (2001: 254-255), a textbook not only provides a structure and a
syllabus for a program, a variety o f learning resources, effective language models and input
but also helps standardize instruction or maintain quality, etc. However, a textbook may not
reflect students’ needs. As a result, the impact o f a textbook on a program ，teachers, and
learners has to be carefully assessed.
2.2. M aterials Evaluation
2.2.1 The role o f materials
Teaching materials are also o f great importance in language classrooms and have
undergone many fluctuations and dramatic shifts over the years. There are a lot o f different
views on their roles. Yet, materials may have a great influence mainly on learners, teachers,
and contexts.

7


For the learners, materials are regarded as sources for their language input, language
practice, communicative interaction, useful learning aid, motivation, stimulation，and
increase o f knowledge (Richards 2001; Cunningsworth 1995; Dudley-Evans & St. John
1998; Tomlinson 1998; and Grant 1987).
For the teachers, materials are considered forms o f teacher training- they provide ideas on
how to plan and teach lessons as well as formats that teachers can use, supports for less
experienced ones who have yet to gain in confidence, and supports for saving them an
extraordinary amount o f time (Richards 2001: 251; Cunningsworth 1995: 7).
For the contexts, materials can function as syllabuses. These syllabuses reflect learning
objectives that have already been determined, identify what and the order in which it should
be taught/ learned，and indicate what methods should be used (Cunningsworth 1995: 7; Ur
1991: 184; and Grant 1987: 8). In addition, materials are valid, useful and labor-saving
tools (Grant 1987，W illiam 1983，Ansary & Babaii 2002). As a result，A H I can be
considered a syllabus in the case that there is not an English syllabus for the first-year
junior college students in EPU.
Nobody can deny the importance o f materials in an English course. However, teachers
should keep away from much reliance on them which should be viewed as better servants
than masters (Cunningsworth 1995: 15) because “ the perfect textbook does not exist.”
(Grant 1987).
2.2.2. Types o f materials evaluation
Robinson (1991: 59) proposes three types o f materials evaluation: preliminary, formative,
and summative. Preliminary evaluation takes place before a course begins and involves
selecting the most appropriate from the publications that are available. Formative
evaluation is conducted while the course is ongoing so that modifications can be made to
the materials whereas summative evaluation takes place at the end o f a course and
addresses the question o f whether the materials have been effective.

Cunningsworth (1995: 14) introduces three types o f materials evaluation: pre-use, in-use,
and post-use which respectively take place before a course book is used, during its use and

8


after use. According to Cunningsworth, pre-use is the most difficult kind o f evaluation
whilst post-use aims to identify strengths and weaknesses for future decisions.
Richards (2001: 288-291) refers to three types: illuminative, formative, and summative.
Illuminative evaluation seeks to find out how different aspects o f the program work or are
being

implemented.

Formative

evaluation

focuses

on

ongoing

development

and

improvement o f the program. Summative evaluation seeks to make decisions about the
worth or value o f different aspects o f the curriculum and takes place after a program has

been implemented.
Although the different terms are used to demonstrate the types o f materials evaluation, they
are basically similar. First, preliminary, pre-use, and illuminative evaluations mean a
process to seek for good materials. Next, formative and in-use evaluations interpret how
materials are being exploited. Certainly, summative and post-use evaluations are used to
determine how effectively materials have been used.
2.2.3. Guidelines fo r evaluation
2.2.3.1. Guidelines fo r evaluation by Cunningsworth (1995)
Cunningsworth (1995:

15-17) offers four guidelines in approaching any materials

evaluation exercise as follows:
♦ Course books should correspond to learners’ needs. They should match the aims and
objectives o f the language-learning program.
♦ Course books should reflect the uses (present or future) which learners w ill make o f the
language. Select course books which w ill help to equip students to use language effectively
for their own purposes.
♦ Course books should take account o f students’ needs as learners and should facilitate their
learning processes, without dogmatically imposing a rigid 'method*.
♦ Course books should have a clear role as a support for learning. Like teachers, they
mediate between the target language and the learner.

9


2.2.3.2. Guidelines fo r evaluation by G rant (1987)
Grant (1987: 118-121) suggests the three stages o f evaluation: initial, detailed, and in-use.
♦ Initial evaluation involves the ‘ C ATALYST, test applied in the classroom. The eight
letters in the word including с (Communicative) - A (Aims) - T (Teachability) - A

(Available Add-ons) - L (Level) - Y (Your impression) - s (Student interest) ֊ T (Tried and
Tested) represent the eight criteria by which the decision is made whether a textbook is
suitable for the classroom.
♦ Detailed evaluation is carried out to decide how far the course book suits the student, the
teacher, and the syllabus. Grant also provides a three ֊֊ part questionnaire which should be
applied to one o f the textbook, i f possible, to two different textbooks and then should be
discussed with colleagues.
♦ In-use evaluation deals with the constant evaluation by using the questionnaire supplied
or modified and one can ensure that the teacher is the master, and not the slave, o f the
textbook!

These guidelines much help the researcher to have a comprehensive thinking o f materials
evaluation during the evaluation process.
2.2.4. Types o f materials evaluators
There are two types o f materials evaluators: insiders and outsiders (Tomlinson 1998;
Dudley-Evans & St. John 1998; Robinson 1991; and Richards 2001).
The outsiders are the ones who are not involved in the program, for example, consultants,
inspectors, and administrators. They may provide an objective approach to their materials
evaluation and can supplement the teachers’ perceptions o f what happens in a course with
independent observation and opinions (Robinson 1991: 68; Richards 2001: 291). However,
as no observer is truly objective, they w ill have their own views on language teaching
methods, administrative procedures, etc. It w ill take them much time to be aware o f the
local needs and constraints, too.

10


The insiders are the course designers, teachers, and students. Obviously, these insiders can
provide the most valid information in the evaluation process because they can understand
the learning and teaching situation and learners’ needs than any others (Tomlinson 1998;

Richards 2001; and Robinson 1991).

2.2.5. Evaluation Models for materials evaluation
2.2.5.1. Evaluation Model by Hutchinson & Waters
Hutchinson & Waters (1987) provide a framework for materials evaluation (see Figure 1).

Figure 1: Evaluation Model by Hutchinson and Waters (1987: 96)

11


As shown in Figure 1，evaluators should follow four major steps: setting out the criteria;
carrying out the subjective analysis; implementing the objective analysis; and matching the
findings. Three options may happen after matching as follows: 0 = does not match the
desired feature; 1 = partly matches the desired feature; and 2 = closely matches the desired
feature. When the materials partially mismatch the desired features in some criteria, the
shortcomings should be adapted.
Furthermore, the authors also supply four basic criteria (see Appendix 1).
The first criterion, audience^ involves the analysis o f the students’ personal information (the
students’ age, sex, knowledge o f English, interest and their educational backgrounds).
The second criterion deals with the aims o f the course and the materials.
The next criterion, content, helps evaluators to have an overview o f the content treatment o f
the materials. When analyzing the content subjectively and objectively, researchers should
focus on some sub-criteria (language description, language points, language skills, texttypes, topics, content organization, and sequence).

Methodology, the last criterion, concerns the theories o f learning, the learners’ attitudes to
learning English, types o f tasks/exercises, teaching-learning techniques, aids, language
guidance and flexibility o f the materials.
In addition to the main criteria above, Hutchinson & Waters (1987) also refer to some other
criteria, for instance, the price and the availability o f the materials.

2.2.5.2. Evaluation Model by McDonough & Shaw
McDonough & Shaw (1993) offer a three-stage evaluation model (see Figure 2). As
indicated in Figure 2，the first stage, the external evaluation, concentrates on the claims
made for the materials by the author/ publisher. Next, the internal evaluation is concerned
with analyzing the extent to which the factors in the External evaluation actually match the
internal consistency and organization o f the materials as stated by the author/ publisher.
The final stage, the overall evaluation, deals with the factors that help any evaluator to
assess the suitability o f the materials for specified groups or individuals. Also, this stage

12


aims to enable the evaluator to decide the extent to which the materials have realized their
stated objectives.

Figure 2: Evaluation Model by McDonough & Shaw (1993: 75)
Also, according to this model, the following factors need to be examined in each stage:
The External evaluation
丁he intended audience
The pronciency level
The context and presentation
o f language items
Whether the materials are to
be core or supplementary
The role and availability o f a
teacher9s book
The inclusion o f a
vocabulary list/index
The table o f contents

The use o f visuals and
presentation
The cultural specificity o f
the materials
The provision o f
audio/video material
The inclusion o f tests

The In te rn a l evaluation
The treatment and
presentation o f the skills
The sequencing and grading
o f the materials
The type o f reading，
listening, speaking and
writing materials The
appropriateness o f tests and
exercises,
The self-study provision
The teacher-leamer
6balance’ in use o f the
materials

The overall evaluation
The usability fa cto r, the
extent to which the materials
can be integrated into a
particular syllabus as 'core ，
o r 4supplementary9
The generalizability fa cto r:

the usefulness o f some parts
o f the materials
The adaptability fa cto r: the
possibility o f materials
modification such as
adding/extracting or using
several parts o f the materials
in another context.
The fle x ib ility fa cto r: the
flexibility o f sequencing and
grading; the flexibility o f the
materials themselves to be
integrated easily into various
types o f syllabus

Table 1: The Factors that need examining in the materials evaluation process

13


In the materials evaluation process，McDonough & Shaw note that it is unnecessary to
examine both external and internal evaluation simultaneously. After completing the first
stage, the evaluator can decide the materials’ appropriateness to adopt or select them. I f the
evaluation shows the materials to be potentially appropriate and need to be more inspected,
the internal evaluation w ill be conducted. I f not, it is advisable to exit at the first stage and
evaluate other materials i f necessary. Even after the second stage, evaluators still have the
option not to select the materials i f they so wish. However, this is often avoided i f a
detailed inspection o f the materials is undertaken. Also, the success or failure o f the
materials can be determined only after those materials are used in the classroom with real
learners.

2.2.5.3. Evaluation Model by Ellis

Figure 3: Evaluation Model by Ellis (1998: 228)
Ellis (1998) argues that any evaluations are likely to be the results o f a whole series o f
micro-evaluations carried out on a day-by-day and lesson-by-lesson basis. Therefore, he

14


suggests a model that can be best carried out as a series o f micro-evaluations (see Figure 3).
This modal aims to identify the match between task planned and task in use. Some
dimensions used for macro evaluation such as approach, purpose, focus, scope, evaluator,
and types o f information can be applied in a micro evaluation process.
According to the framework, evaluating a task consists o f five main steps: describing the
task, planning the evaluation, collecting information, analyzing information, reaching
conclusions, and making recommendations for future teaching.
In summary, E llis’ model may be more manageable and specific than the ones o f
Hutchinson & Waters, McDonough & Shaw when evaluating a task is implemented.
However, E llis’ model may be time-consuming and require a lot o f efforts i f there is a
comprehensive demand for materials evaluation. McDonough & Shaw’ s model may save
time i f the materials can be seen inappropriately at the first stage. However, i f the first
evaluation shows that the materials may be potentially suitable, and need to be more
inspected, then a second evaluation needs to be carried out. Perhaps, what, first,
distinguishes Hutchinson & Waters9 model from other models is the clear-cut and logical
description o f the stages. Second, the proposed criteria are presented more simply and

systematically which can help evaluators visualize what must be done to carry out materials
evaluation. Third, the guidelines offer a straightforward analytical matching process to
evaluators. As a result, the researcher decided to choose Hutchinson & Waters9 model with

some modifications to follow in the thesis.
2.2.6. Tools fo r gathering inform ation
Robinson (1991) suggests seven basic procedures for data collection in materials
evaluation:

questionnaires,

checklists,

interviews,

observation,

discussion,

existing

information (records), and assessment.
♦ Questionnaires
Questionnaires are any written instruments that present respondents with a series o f
questions or statements to which they are to react either by writing out their answers or
selecting from among existing answers (Brown 2001: 6). They are the most widely used

15