An evaluation of the coursebook “English for Pharmaceutical course” for the second - year students at Văn Hiến medical and pharmaceutical school.

i

VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POST- GRADUATE STUDIES
===========

  ===========

NGUYỄN THỊ THẢO

AN EVALUATION OF THE COURSEBOOK
“ENGLISH FOR PHARMACEUTICAL COURSE”
FOR THE SECOND - YEAR STUDENTS AT VĂN HIẾN MEDICAL
AND PHARMACEUTICAL SCHOOL
(Đánh giá giáo trình “English for pharmaceutical course” dành cho sinh
viên năm thứ hai Trường Trung cấp Y, Dược Văn Hiến)

M.A. PROGRAM PROGRAMME THESIS

FIELD: ENGLISH LANGUAGE TEACHING METHODOLOGY
CODE: 60 14 10

HA NOI- 2013


ii

VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POST - GRADUATE STUDIES

===========

  ===========

NGUYỄN THỊ THẢO

AN EVALUATION OF THE COURSEBOOK
“ENGLISH FOR PHARMACEUTICAL COURSE”
FOR THE SECOND - YEAR STUDENTS AT VĂN HIẾN MEDICAL
AND PHARMACEUTICAL SCHOOL
(Đánh giá giáo trình “English for pharmaceutical course” dành cho sinh
viên năm thứ hai Trường Trung cấp Y, Dược Văn Hiến)

M.A. PROGRAM PROGRAMME THESIS

FIELD: ENGLISH LANGUAGE TEACHING METHODOLOGY
CODE: 60 14 10
SUPERVISOR: Dr. NGÔ HỮU HOÀNG

HA NOI - 2013


iv

ABBREVIATIONS

BA -

Bachelor of Arts

ELT - English Language Teaching
ESP - English for Specific Purposes
HUT - Hanoi University of Technology


v

LIST OF FIGURES AND TABLES
Figures
Figure 1: Materials Evaluation Process of Hutchinson and Waters (1987)
Figure 2: Materials Evaluation Model of McDonough and Shaw (1993)

Tables
Table 1:

Types of information in the teachers and students‟ questionnaires

Table 2A: The teachers‟ opinions about the aims of the materials ( Q1 - 6)
Table 2B: The students‟ opinions about the aims of the materials ( Q1 - 6)
Table 3A: Students‟ opinions on the skills of the material ( Q7 - 11)
Table 3B: Teachers‟ opinions on the skills of the material ( Q7 - 11)
Table 4A: Students‟ opinions on the language points and text types of the material
( Q12 - 16)
Table 4B: Teachers‟ opinions on the language points and text types of the material
( Q12 - 16)
Table 5A: Students‟ opinions on the topics of the material ( Q17 - 20)
Table 5B: Teachers‟ opinions on the topics of the material ( Q17 - 20)
Table 6A: Students‟ opinions on the tasks/exercises of the material (Q20 - 25)
Table 6B: Teachers‟ opinions on the tasks/exercises of the material (Q20 - 25)
Table 7A: Students‟ opinions on the teaching - learning techniques of the material

( Q25 - 30)
Table 7B: Students‟ opinions on the teaching - learning techniques of the material
( Q25 - 30)


vi

TABLE OF CONTENTS

Page
PART A: INTRODUCTION ......................................................................... 1
1.Background and Rationale of the study ...................................................... 1
2. Aims of the study ......................................................................................... 2
3. Significance of the study .............................................................................. 3
4. Scope of the study ........................................................................................ 3
5. Organization of the study ............................................................................ 3
PART B: DEVELOPMENT…………………………………………………….5
CHAPTER 1: LITERATURE REVIEW AND THEREOTICAL
BACKGROUND .............................................................................................. 5
1.1. Previous studies…………………………………………………………..…5
1.2. Materials in Language Teaching and Learning ...................................... 6
1.2.1. Roles of Materials ................................................................................ 6
1.2. 2. Types of Materials .............................................................................. 7
1.2.2.1. Textbooks …………………………………………………………..8
1.2.2.2. In - house materials…………………………………………………8
1.2.2.3. ESP materials……………………………………………………….9
1.3. Materials Evaluation ...................................................................... …….11
1.3.1. Definitions of Materials Evaluation.................................................... 11
1.3.2. Reasons of Materials Evaluation ........................................................ 12
1.3.3. Types of Materials Evaluation ........................................................... 13

1.3.4. Principles of Materials Evaluation ..................................................... 14
1.3.5. Models of Materials Evaluation ......................................................... 14
1.3.5.1. Evaluation by Hutchinson and Waters (1987) …………………...14
1.3.5.2. Evaluation by McDonough and Shaw (1993)………………….....16
1.3.5.3. Evaluation by Ellis ( 1997)………………...…………………....…17


vii

1.3.6. Criteria for Materials Evaluation…….. ........................................ …….18
1.4. Research instruments in materials evaluation……………..……..….......19
1.4.1. Document analysis………………………………………………......…..19
1.4.2. Questionnaires…………………………………………………..…....….19
1.4.3. Interviews………………………………………………………….…..…20
1.4.4. Records…………………………………..……………………..…….….20
1.4.5. Assessment…………………………………………………..……….….21
1.4.6. Checklists………………………….…………………………………......21
1.5. Summary ..............................................................................................…21
CHAPTER 2: METHODOLOGY .................................................................22
2.1. Research questions restated……….……………….…………..………...22
2.2. Data collection instruments………………….………………………..….22
2.2.1. Document analysis…………………………………………….…….....23
2.2.2. Questionnaires …………………………………………………..…..…24
2.3. Subjects………………………………………………..…………………...25
2.3.1. The teachers…………………………………………………………….25
2.3.2. The students…………………………………………………………....26
2.4. Data collection procedures………………………………………………..26
2.5. Summary……………………………………………………….…………...27
CHAPTER 3: DATA ANALYSIS AND DISCUSSION .............................. 28
3.1. The results from the evaluation of aims………………………………..…28

3.1.1. Aims of the course………………………………………………..…..…28
3.1.2. Results from the analysis of material aims and questionnaires……..…28
3.2. The results from the evaluation of the content……………………….….30
3.2.1. ESP material analysis………………………………………………..….30
3.2.2 Teachers and students‟ opinion on the content of the material……..….32
3.2.2.1. Skills…………………………………………………………..…..32


viii

3.2.2.2. Language points and text types………………………………..….33
3.2.2.3. Topics…………………………………………………………..….35
3.3. Results from the evaluation of the methodology……………...…………..36
3.3.1. Methodology implied in the material in terms of tasks/ exercises ...……36
3.3.2. Methodology implied in the material in terms of teaching - learning
techniques …………………………….……………………………………….38
3.4. Summary……………………………………………..……………….…….39
CHAPTER 4: RECOMMENDATIONS AND CONCLUSION….……………..….40

4.1. Recommendations for the material improvement………………………..40
4.2. Limitations of the study and suggestions for further study……………...42
4.3. Conclusions of the study…………………………………………………….42
References ………………………………………………………………………I
Appendixes ............................................................................................. …… V
Appendix

1:

Hutchinson

and

Waters‟s

criteria

checklist

for

materials

evaluation…………………………………………………………………………V
Appendix 2: Questionnaire for students ....................................................….XIV
Appendix 3: Questionnaire for teachers .......................................................... XX
Appendix 4: A sample unit ....................................................................... XXVII


1

PART A
INTRODUCTION
In part A, I will introduce rationale, aims, significance, scope and
organization of the study.
1. Rationale of the study.
English has been the main language of international communication, which
has become an integral part of most modern professions. Along with this trend,
branch of English Language Teaching (ELT) – English for Specific Purposes (ESP)
has been focused on more. Harding (2007: 6) puts it: “in ESP – English for Specific
Purposes – the purpose for learning the language is paramount and relates directly

to what the learner needs to do in their vocation or job”.
ESP material for pharmaceutical students at Van Hien School is designed to
help students get effective knowledge related to their job in the future. This material
has been introduced to students since 2009. It has been officially used as the core
material for the students of Pharmacy at Van Hien School. English is a compulsory
subject at this school. English course for pharmaceutical students is divided into two
stages. The first stage consists 60 class hours of general English with the
coursebook Headway Elementary written by Liz and John Soars (2000). And in the
second part (the third semester), they have 30 class hours of ESP with English for
Pharmacy. It was designed by two first teachers but they moved to other school.
This material contains 25 units. Each unit includes one reading text and
pronunciation of new words. The reading text introduces the theme of unit related to
pharmacy. The second one is the list of new words with pronunciation.
Teachers and students feel dissatisfactions with this material because of
many reasons. Until now, no research on the course book evaluation has been
carried out to examine how well the material being in use match the course
requirements in terms of aims, content and methodology. And based on those
reasons, the researcher decides to evaluate this material named “English for
pharmaceutical course” with the hope that it will help to improve the material‟s


2

quality and get better learning efficiency as well as stimulate students‟ interest in
language learning to meet the students‟ needs and the course requirements.
2. Aims of the study
This study is to evaluate the ESP material “English for pharmaceutical
course” to determine how the material satisfies the requirements in terms of aims,
content and methodology and what the material effects on students. From the
evaluation, some suggestions for using the material will be shown to change and

improve the effectiveness of the material for the better use.
This study aims at answering the following questions:
1. What effects does the material “English for pharmaceutical course” have on
students in terms of aims, content and methodology?
2. What are students and teachers‟ attitudes towards this material?
3. Significance of the study
The findings of the study are expected to be useful not only to the researcher,
the English teachers who are teaching English but also to the second year
pharmaceutical students at Van Hien Intermediate School. Besides, the evaluation
results will show the suitability and unsuitability of the ESP material in use, so that
the researcher will make some suggestions to improve the material to be better use
in the nearest time.
4. Scope of the study
In textbook and evaluation, many criteria need to be examined like layout,
cultural bias, audience, aims, content, methodology, and so on. However, because
of limited time, the researcher bases on the Hutchinson and Waters‟ (1987) criteria
for evaluation with focus on the three following criteria: aims of the material,
content of the material (in term of language points covered, skills, text types, and
topics), and methodology ( in terms of types of tasks and exercises, and teaching learning techniques). And the researcher only investigates the attitude of the second
year pharmaceutical students and English teachers at Van Hien Intermediate School
based on their feedback.


3

5. Organization of the study
The study is organized into two parts as follows:
Part A discusses about rationale, aims, significance, scope and organization
of the study.
Part B, development, the main part in this study, consists of four chapters.

Chapter 1 presents a review of literature of this study and theoretical
background about materials and materials evaluation. Previous studies are presented
first. This chapter is concentrated on the issues related to ELT material evaluation
including the roles and types of materials in language. Then, it presents major issues
in materials evaluation such as: definitions, reasons, types, principles, models as
well as criteria of materials evaluation. The chapter ends with research instruments
in materials evaluation.
Chapter 2 describes the choices of methodology adopted in the study, the
data collection instruments, the subjects as well as the data collection procedures
Chapter 3 presents the results of the study and the discussion of those results,
data analysis and the findings of the study, pointing out the strengths and
weaknesses of the material.
Chapter 4 shows recommendations of the study, the limitations of the study,
suggestions for further study and conclusion of the study.


4

PART B
DEVELOPMENT
CHAPTER 1
LITERATURE REVIEW AND THEORETICAL BACKGROUND

Chapter 1 aims to provide the background information about materials and
materials evaluation. The first part reviews previous studies. In the second one,
some main points of materials in ELT will be presented. The third part is about the
specific points about materials evaluation such as: definitions, reasons, types,
principles, models, criteria of materials evaluation. The chapter ends with research
instruments in materials evaluation.
1.1. Previous studies

Until now, there have been many studies carried out to evaluate materials in
general and ESP material in particular, such as Litz (2001), Akin and Guceri (
2001), Nguyễn Thị Phương Dung ( 2002), Võ Thị Anh Đào ( 2003), Nguyễn Thị
Bích Liên ( 2004), Trần Thị Thúy Nga ( 2005).
The first example is the study of Nguyễn Thị Phương Dung (2002). In her
own study, she recommends criteria for the adaptation of the reading material for
the fourth - year students of Electronics and Telecommunications at Hanoi
University of Technology after conducting a questionnaire survey for teachers and
students, using Hutchinson and Waters‟ (1987) framework for materials evaluation.
Four criteria are focused on that study: the audience, the aims, the content and the
methodology. The subjects of the study are 80 students and 11 teachers of HUT.
The questionnaires are well - organized and able to help her to collect necessary
information. She concludes that the existing material basically meet the
requirements of the course. However, the material has some weaknesses which
should be modified and added.


5

Võ Thị Anh Đào ( 2003) also chooses the criteria proposed by Hutchinson
and Waters ( 1987) to evaluate ESP materials for students of Husbandry and
veterinary Science at Tay Nguyen University in terms of language level, aims,
content and methodology. The subjects of the study are 40 second year students and
four teachers. The methods used in the study are document analysis, questionnaires
for students and interview for teachers. The study is well presented. After collecting
information from the document analysis and the survey questionnaires, she
concludes that the material is difficult for students in terms of grammatical structure
and lexical items. However her thesis reveals some weaknesses. And it should be
implemented some parts.
From the above mentioned studies, it can be concluded that the major

purpose of conducting an evaluation of materials is to evaluate the effectiveness of
the material in use and its suitability to the course requirements. It is also found that
the criteria proposed by Hutchinson and Waters (1987) are chosen in almost studies.
Document analysis, questionnaire and interview seem to be popular methods to
collect data for materials evaluation.

1.2. Materials in English Language Teaching and Learning
Roles of materials, Types of materials will be shown in this section.
1.2.1. Roles of Materials
Materials play an important role and a key in any language teaching and
learning. Richard (2001: 251) states that:
The teaching materials can serve as basis for much of language input that the
learners receive and as the source for much of the language practice that
occurs in the classroom.
It means that materials are “basis for the content of the lesson, balance of skills
taught, and the kind of language practice students take part in” (p. 251). Brown
(1995:139) defines materials as “any systematic description of the techniques and
exercises to be used in classroom teaching”. It emphasizes that teaching materials


6

are crucial component in the language classroom and there is hardly any class
without teaching materials.
Dudley - Evans and St. John (1998) point out four functions of teaching
materials: a source of language, a learning support, a source of motivation and
stimulation, and a source for reference. As a source of language, materials need to
present in real language as it is used in real situations and they must be suitable to a
wide range of needs of the students. As a learning support, materials need to be
reliable, that is to say, the materials need “to work, to be consistent and to have

some recognizable patterns”. (p. 171). As a source of stimulation and motivation,
materials need to be “challenging yet achievable; to offer new ideas and information
whilst being grounded in the learners‟ experience and knowledge (p. 172). As a
source of reference, materials help the learners to make “efficient use of the
resources in order to facilitate self- discovery (Tomlinson, 1998:11). For this
function, Dudley - Evans and St. John (1998:172) shows that materials need to be
“completed, well laid out and self - explanatory”.
The roles of ESP materials are shown more clearly. According to Hutchinson
(1987:107), the language learning process is facilitated by “providing a path
through complex mass of the language to be learnt”. As Prabhu (1994: 94)
emphasize: “The fact that materials need to be used as sources rather than as preconstructed courses should not be regarded as a weakness of task-based teaching; it
can in fact be a strength for any form of teaching”.
ESP materials are used in almost Vietnamese Universities and Colleges. In
Van Hien School, ESP material is the main component which provides the content
of the lesson, the skills and the activities in the class. Because they believe that
materials “represent the visible heart of any ELT program” (Sheldon, 1988:238).
1.2.2. Types of materials
In the broad sense of the concept, „materials‟ as defined by Tomlinson
(1998) is “anything which is used to help to teach language learners.” It can be in
the form of a textbook, a workbook, a cassette, a CD-Rom, a video, a photocopied


7

handout, a newspaper, a paragraph written on a white board: anything that presents
or informs about the language being learned. Materials of these kinds can obviously
be exploited effectively for language learning. Because of the limited scope of the
study, only three types of materials concerned with materials evaluation, especially
in ESP are presented. They are textbooks, in-house materials, ESP materials.
1.2.2.1. Textbooks

Tomlinson (1998:9) states that:
A textbook provides the core material for a course. It aims to provide as
much as possible in one book and is designed so that it could serve as the
only book, which the learners necessarily use during a course. Such a book
usually includes work on grammar, vocabulary, pronunciation, functions and
the skills of reading, writing, listening and speaking.

Textbooks are a key component in most language programs. In some
situations they serve as the basis for much of the language input learners receive
and the language practice that occurs in the classroom. They may provide the basis
for the content of the lessons, the balance of skills taught and the kinds of language
practice the students take part in.
In other situations, the textbook may serve primarily to supplement the
teacher's instruction. For learners, the textbook may provide the major source of
contact they have with the language apart from input provided by the teacher. In the
case of inexperienced teachers, textbooks may also serve as a form of teacher
training - they provide ideas on how to plan and teach lessons as well as formats
that teachers can use.
Much of the language teaching that occurs throughout the world today could
not take place without the extensive use of commercial textbooks. Learning how to
use and adapt textbooks is hence an important part of a teacher's professional
knowledge.
1.2.2.2. In - house materials


8

In-house materials will provide students with activities that suit the specific
needs of their future or current jobs or, as Sheerin (1989: 25) points out: “In-house
produced material is extremely valuable as it is inevitably more precisely geared to

the needs of students than published material”. When determining what tasks to
include in ESP materials, teachers should thus above all strive to select and/or
design tasks that will simulate the learner‟s real-life business situations as closely as
possible.
When designing in- house materials, teachers can rely on some general
considerations that can also be used when deciding on the appropriate textbook.
According to Haycraft (1987: 127), some of the most important considerations are
as follows:
-

The length of the course and the target audience of the course;

-

The appropriate structural grading: students should be taught what they need
to know “in the right order with the right priorities”;

-

The vocabulary should be useful and in current use;

-

The appropriate idiomatic English;

-

The materials should be “visually alive” and “well presented”.

1.2.2.3. ESP materials

Materials selection, adaptation, or writing is an important area in ESP
teaching, representing a practical result of effective course development and
providing students with materials that will equip them with the knowledge they will
need in their future business life.
One of the most important issues regarding ESP materials selection and/or
writing is whether the materials selected should be solely or primarily subject
specific and what the most appropriate ratio of general materials to subject-specific
materials is. General materials focus on one‟s general ability to communicate more
effectively, while subject-specific materials focus on a particular job or industry
(Ellis and Johnson, 1994).


9

According to Prabhu (1994: 94), another important issue regarding materials
is that they should be used as sources: “The fact that materials need to be used as
sources rather than as pre-constructed courses should not be regarded as a weakness
of task-based teaching; it can in fact be a strength for any form of teaching”.
To sum up, after analyzing learner needs and setting objectives for the
course, the ESP teacher has to select materials that will help the students achieve the
course objectives (Ellis and Johnson, 1994). These materials should also relate
closely to the learners‟ specific skills and content needs, which is an important
precondition for full exploitation of the materials as well as the learners‟ motivation
1.3. Materials Evaluation
Materials evaluation is an important part of materials selection as well as the
materials development process. In both cases, evaluation is primarily “concerned
with relative merit. There is no absolute good or bad – only degrees of fitness for
the required purpose” (Hutchinson and Waters, 1994: 96). An evaluation of printed
ESP materials will thus above all serve to locate the materials that will best suit the
learners‟ needs with regard to their future or current work area. When no suitable

printed materials are found, the evaluation of existing materials can serve as a
springboard for development of in-house produced materials. In-house materials
themselves should also be evaluated in order to provide the basis for their revision
with a view to improving their quality and their suitability to the target learners‟
needs.
1.3.1. Definition of Material Evaluation
Many definitions of evaluation exist. Most definitions refer to program
evaluation; some refer to project or policy evaluation. Some definitions use one
term to refer to all types of evaluation, for example including policy and program
evaluation under the umbrella title of policy evaluation. This section provides a
number of examples of evaluation definitions.
Evaluation is often defined as an activity that judges worth. For example,
evaluation is:


10

 …the determination of merit, worth, or significance… (Scriven 2007)
 a course of action used to assess the value or worth of a program. (Farell et
al. 2002)
Some definitions include the notion of improvement. For example:
 [Evaluation is] a set of research questions and methods geared to reviewing
processes, activities and strategies for the purpose of improving them in
order to achieve better results. (Kahan & Goodstadt 2005)
 The real purpose of an evaluation is not just to find out what happened, but
to use the information to make the project better. (Community Tool Box
undated)
From those definitions of evaluation, it can be inferred that materials
evaluation involves the activity of the worth judge, the notion of improvement.
1.3.2. Reasons of materials evaluation

This section will discuss about some reasons of materials evaluation.
Sheldon (1988) has suggested several reasons for textbook evaluation. He states that
the selection of a textbook is indicator of an educational decision in which there is
considerable professional, financial, and even political investment. Through
evaluation, teachers will become familiar with the content of available
textbooks and recognize the weaknesses and strengths of each.
Tomlinson (1999a: 11) lists the following reasons for evaluating materials:
1. To select a textbook for a course
2. To select materials to supplement a course book
3. To select materials from different sources in an eclectic manner.
4. As a basis for adaptation of materials in order to make them more suitable
for a particular course.
5. As a basis for improving materials (trialling or piloting materials)
6. In order to edit materials produced by others.
7. In order to review proposed materials for a publisher.
8. In order to review published materials for a journal


11

9. In order to help teachers or trainee teachers develop their understanding of
methodology and/or materials writing.
Evaluation is considered as the confirmation of practice if the practice is
good. In the other case when there is any problem with the practice. Evaluation is
for the consideration of ways to innovate or change the practice for a better one.
The material at Văn Hiến Intermediate School, “English for pharmaceutical
course”, is used every school year. It has both advantages and disadvantages. So it
is necessary to evaluate this material, which becomes the main purpose for carrying
out this research of the author.
1.3.3. Types of evaluation

With the different researchers, they will have different ways to categorize the
types of evaluation.
Tomlinson (1998: 10) divides materials evaluation into three stages: pre-use, in
-use and post-use evaluation while Robinson (1991:59) classifies materials
evaluation into three types: preliminary, formative and summative. Although
different terms are used to indicate types of materials evaluation, these terms are
basically similar:
 Pre-use (preliminary) evaluation: The material is evaluated before it is
taught, and it is the most common type of evaluation. But even a meticulous
pre-use evaluation can be unreliable, as there may not be adequate
information about the impact of the material in the classroom.
 In-use (formative) evaluation: This is done while using the material in the
classroom but it is seldom done in a methodical way. If the whilst-evaluation
is done with planning and focus it can generate useful data.
 Post-use (summative) evaluation: Post - use evaluation takes place at the end
of the course or when the course is finished. Therefore, it does not affect the
course. The purpose of Post-use evaluation is to assess a mature project‟s


12

success in reaching its stated goals and makes improvement in subsequent
versions of the course or materials.
Each type has its own characteristic. So choosing a suitable type depends on the
aims of the study. This study is carried out after finishing the course of the second
year pharmaceutical students so it belongs to post - use evaluation.
1.3.4. Principles for materials evaluation
Cunningworths (1995) indicates four general principles based on good language
teaching practice to help teacher to evaluate materials. The first principle is that
teaching materials should match the aims and objectives of the language learning

program. The second one is that teaching materials should correspond to students‟
needs. Next, teaching materials should be selected in order to help students to use
language effectively for their own purposes. And the last principle is that teaching
materials should mediate between the target language and the learners. They should
have a clear role as a support for learning.
1.3.5. Models of materials evaluation
There are a lot of models for materials evaluation. However, the most
commonly adopted models are suggested by Hutchinson & Waters (1987),
McDonough & Shaw (1993), and Ellis (1997).
1.3.5.1. Evaluation by Hutchinson and Waters (1987)
Hutchinson and Waters (1987) divides the evaluation process into 4 stages
(Figure 1): defining criteria, subjective analysis, objective analysis, and matching.
They further add that to make the best choice, different parts involved in the course
have to be considered: teachers, students, and sponsors. This is a perfect macroevaluation.


13

DEFINE CRITERIA
On what bases will you
judge materials?
Which criteria will be
more important?

SUBJECTIVE ANALYSIS

OBJECTIVE ANALYSIS

What realization of the criteria
do you want in your course?

How does the material being
evaluated realize the criteria?

MATCHING
How far does the material
match your needs?

Figure 1: Materials Evaluation Process of Hutchinson and Waters (1987:98)
 The first step is to determine criteria for evaluation. In this step, the different
evaluators have different purposes for evaluation so the selection of evaluation has
to base on what the materials will be judged and which criteria the evaluators
choose for their evaluation.
 The second step, the subjective analysis, is the analysis of the course in terms of
materials requirements under the defined criteria so the evaluator should describe
in detail for the learners and the course. According to them, subjective analysis
should not be considered as a fixed set of requirements. The evaluator should use


14

the materials evaluation process as a means of questioning and developing his
own ideas as to what is required.
 The third step is to determine the objective analysis, that is, analyses the materials
in terms of the selected criteria.
 The last one is the matching process, which finds out how far the material matches
the course requirements.
Hutchinson and Waters (1987) also suggest their own checklist ( appendix 1)
of criteria for objective and subjective analysis. The purpose of the checklist is for
evaluating teaching material by focusing on big issues like audience, the aims, the

content, and the methodology.
1.3.5.2. Evaluation by McDonough and Shaw (1993)
McDonough and Shaw suggests a model for textbook evaluation which
involves three stages: external evaluation, internal evaluation, and overall
evaluation (figure 2). This is a combination of macro- and micro- evaluation.

Macro- evaluation

inappropriate/ potentially appropriate

(External)
EXIT
Micro- evaluation

inappropriate/ appropriate

adopts/ select

(Internal)

EXIT
Figure 2: Materials Evaluation Model of McDonough and Shaw (1993:75)
 Firstly, external stage (macro - evaluation) that examines the organization of
materials stated by the author or the publisher including claims made on the cover
page and information in introduction and table of contents. This kind of evaluation
gives information about the intended audience, the proficiency level, the context
of use, presentation and organization of materials, and authors' opinion about


15

language and methodology, use of audio-visual materials, vocabulary list and
index, cultural aspects, tests and exercises included in the book.
 Secondly, internal stage ( Micro - evaluation) in which the following factors
are examined: a) the presentation of the skills, b) the grading and sequence of
the materials, c) authenticity or artificiality of the listening materials,

d)

authenticity or artificiality of the speaking materials, e) appropriateness of
tests and materials, and f) appropriateness of the materials for different
learning styles and claims made by the authors for self-study.
 The last stage is overall evaluation in which usability, adaptability, and flexibility
factors are examined.
1.3.5.3. Evaluation by Ellis ( 1997)
Ellis (1997) suggests the practice of detailed evaluation and focuses on
evaluation at the task level with reference to its teaching learning context. The aim
in this model is to identify the match between task planned and task in use. It can be
conducted when the materials are being used in the classroom. Ellis introduces the
following steps:
1. Choosing a task to follow;
2. Describing the task with specification of input, procedures, language
activities, and outcomes;
3. Planning the evaluation with reference to the dimensions above;
4. Collecting information before, while and after the task was used, and what
and how the task was performed;
5. Analyzing the information collected;
6. Reaching conclusions relating to what has been discovered, and making
recommendations for the future teaching;
7. Writing the report.

In summary, these models are very useful in the process and nature of the
purpose. So the evaluators have to base on the purposes of evaluation, time
available, facilitates and the context in which the evaluation takes place to decide


16

which model will follow. In this study, a material evaluation is carried out to
identify the suitability of the ESP material, English for pharmaceutical course, used
for Pharmaceutical students to the course requirements. The model designed by
Hutchinson and Waters (1987) is used in this study because it is the clearest and the
most suitable to the specific context at Văn Hiến Intermediate School.
1.3.6. Criteria for materials evaluation
According to Tomlinson (1998:220), criteria are what evaluators use to
“reach a decision regarding what need to be evaluated”. Criteria for materials
evaluation depend on what is being evaluated and why they need to be evaluated.
There are numerous criteria proposed by different authors.
William (1983) suggests seven criteria, each of which has the following aspects:
 General criteria: give introductory guidance on the presentation of language
items and skills.
 Speech criteria: suggest aids for the teaching of pronunciation: e.g. phonetic
system
 Grammar criteria: offer meaningful situations and a variety of techniques
for teaching structural units.
 Vocabulary criteria: distinguish the different purposes and skills involved in
the teaching of vocabulary.
 Reading criteria: provide guidance on the initial presentation of passages for
reading comprehension.
 Writing criteria: demonstrate the various devices for controlling and guiding
content and expression in composition exercises.

 Technical criteria: contain appropriate pictures, diagrams, tables, etc…
According to Hutchinson and Waters (1987), there are four main criteria for
materials evaluation any evaluator should concern: the audience, the aims, the
content, and the methodology.


17

 The audience of the materials: the evaluator should obtain information about
and from learners to find out whether the materials are suitable to the
students‟ age, knowledge of English, interest and so on.
 Aims of the materials: the evaluator has to check if the materials match the
aims and objectives of the course.
 Content of the materials: the evaluator has to check whether the materials‟
language points, macro-skills/ micro-skills, and topics suit the learners‟
needs.
 Methodology of the materials: the evaluator has to find out if techniques,
aids, guidance provided in the materials satisfy the learners and the teachers
of the course.
In conclusion, from these criteria presented above, I decide to choose the criteria
suggested by Hutchinson and Waters (1987) to this research because they are the
most suitable with specific context at Văn Hiến Intermediate School. These criteria
do not only help the evaluator with a logical insight of the material but also create
the flexibility in the conduction of a detailed material analysis.
1.4. Research instruments in materials evaluation
Brown (1995), Hutchinson and Waters (1994), Robinson (1991) claim that
most materials are evaluated using one or more of the following instruments:
document analysis, questionnaires, interview, records, assessment and checklists.
1.4.1. Document analysis
Document analysis is an important source which the data are collected.

Document analysis is the use of content analysis and other techniques to analyze
and summarize printed materials and existing information.
Nunan (1993) states that this instrument is used to collect qualitative data
that concerned with qualities and non-numerical characteristics. In evaluation, the
analysis of existing documents can provide useful information. One of the biggest
advantages of document analysis is that in some situations, a document represents a
reflection of reality, so it is useful for determining value, interest or public attitudes.
1.4.2. Questionnaires


18

Questionnaire is one of the most common tools to collect data. Brown (1989)
indicates that “Questionnaires are any written instruments that present respondents
with a series of questions or statements to which they are to react either by writing
out their answers or selecting from among existing answers”. The best advantage of
this instrument is that it provides an easy quick form of data collection. Therefore,
data can be collected on a large scale and reach a geographically dispersed sample
of a population.
Moreover, it is economical because it involves only duplication or mailed
expenses (Creswell, 2005). Besides advantages, questionnaire also has
disadvantages. Creswell (2005:361) claims that “individuals may lack any personal
investment in the study and decide not to return the instrument”. Besides, the
researchers do not have a means for explaining questions so respondents may
misinterpret items in the questionnaires ( Creswell, 2005).
Questionnaires items can be relatively open ended and closed ended. An
open item is one in which the respondent can decide what to write and how to say
(Nunan, 1992). Open ended questions include items where the actual question is not
followed by response option for the respondent to choose from but rather by some
blank space to fill. A closed item is one in which the range of possible responses is

determined by the researcher. Closed ended question is one in which the respondent
is provided ready made response options to choose from by encircling or ticking or
by putting an x in the appropriate slot/box.
1.4.3. Interviews
According to Creswell (2005:360), interview is “a form, on which the
researcher asks questions from an interview guide, listens for answers or observes
behavior, and records responds on the survey”.
Depending on the evaluation goals and the specific evaluation context, the
evaluators can make use of the following three types of qualitative interview
format: the structured, semi-structured, and unstructured (Patton cited in Lynch,
1996). Interview may be face - to - face, voice - to - voice, or screen - to - screen.
But it is time - consuming.
1.4.4. Records