An evaluation on the validity of end term english tests used for year 11 students at a vietnamese public high school in the north east area of vietnam

VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGE AND INTERNATIONAL STUDIES
FACULTY OF POST-GRADUATE STUDIES

*******************

VŨ MAI DUYÊN

THE DEVELOPMENT AND EVALUATION OF UTHENTIC
SUPPLEMENTARY MATERIALS FOR FIRST-YEAR STUDENTS IN
GENERAL ENGLISH COURSE AT THÁI BÌNH UNIVERSITY

PHÁT TRIỂN VÀ ĐÁNH GIÁ TÀI LIỆU CHÂN THỰC BỔ TRỢ CHO
SINH VIÊN NĂM THỨ NHẤT TRONG KHÓA HỌC TIẾNG ANH CƠ
BẢN TẠI TRƯỜNG ĐẠI HỌC THÁI BÌNH

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology
Code:

60140111

Hanoi, 2015


VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGE AND INTERNATIONAL STUDIES
FACULTY OF POST-GRADUATE STUDIES

*******************

VŨ MAI DUYÊN

THE DEVELOPMENT AND EVALUATION OF UTHENTIC
SUPPLEMENTARY MATERIALS FOR FIRST-YEAR STUDENTS IN
GENERAL ENGLISH COURSE AT THÁI BÌNH UNIVERSITY

PHÁT TRIỂN VÀ ĐÁNH GIÁ TÀI LIỆU CHÂN THỰC BỔ TRỢ CHO
SINH VIÊN NĂM THỨ NHẤT TRONG KHÓA HỌC TIẾNG ANH CƠ
BẢN TẠI TRƯỜNG ĐẠI HỌC THÁI BÌNH

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology
Code:

60140111

Supervisor:

Dương Thu Mai, PhD

Hanoi, 2015


DECLARATION
I certify that I am the sole author of this thesis. All the material in this study
which is not my own work has been identified and acknowledged. This thesis
has not been submitted for assessment in any other university or institution
wholly or partially. I also accept all the requirements of ULIS relating to the

retention and use of M.A graduation thesis deposited in the library.

Vũ Mai Duyên

i


ACKNOWLEDGEMENTS
I owe my deepest gratitude to my supportive supervisor, Duong Thu Mai,
Ph.D, for her whole-hearted assistance, encouragement as well as the profound
guidance she gave me while I was doing my research.
I would like to take this opportunity to express my thanks to all my teachers
and lecturers in Faculty of Post-graduate Studies, University of Languages and
International Studies, Vietnam National University, Hanoi for their valuable
instruction and assistance throughout the realization of this thesis.
I am heartily thankful to the administrators, my colleagues, and first-year
students at Thái Bình University for their willingness to participate in the research.
Without their help, this project could not be fulfilled.
Last but not least, I am indebted to my beloved family and friends who have
unfailingly inspired me to complete this study.
Hanoi, September 2014

ii


ABSTRACT
This study attempts to develop and evaluate a set of authentic supplementary
materials

for

students

in

a

General

English

course

at

Thái

Bình

University.Approaches, principled frameworksand selection criteria are identified to
serve as basic guideline for the process of developing materials. The material
evaluation proceeds using survey questionnaire and interview. Quantitatively and
qualitatively analytical results revealed that the developed materials appeared to
bring about positive enhancement in students’ learning English.Bothstudents and
teachers demonstrated highly supportive judgments toward the materials’
effectiveness in nearly all aspects but not much in writing skill. Man-Whitney U test
result indicatedsignificant relations between students’ evaluations and their English
proficiency. Based on the findings, the thesis provides pedagogic implications and
suggestions for further research which would be of considerable interest to English
language teachers, learners as well as material developers and evaluators.

iii


LISTOF ABBREVIATIONS
ASM

set of Authentic Supplementary Materials

C/F

Credit / Fair (students)

CLT

Communicative Language Teaching

ELT

English Language Teaching

ESP

English for Specific Purposes

G/vG

Good, very Good (students)

HOTS High-Order Thinking Skill

LLT

Language Teaching and Learning

MoET (Vietnamese) Ministry of Education and Training
SD

Standard of Deviation

SPSS

Statistical Package for the Social Sciences

TBU

Thái Bình University

ULIS

University of Language and International Studies

VNU

Vietnam National University, Hanoi

iv


TABLE OF CONTENTS
DECLARATION ........................................................................................................ i

ACKNOWLEDGEMENTS ...................................................................................... ii
ABSTRACT .............................................................................................................. iii
LISTOF ABBREVIATIONS ................................................................................... iv
LIST OF FIGURES ................................................................................................ viii
LIST OF TABLES.................................................................................................. viii
PART A: INTRODUCTION .................................................................................... 1
1.

Rationale ........................................................................................................... 1

2.

Aims of study and research questions................................................................ 2

3.

Scope of the study ............................................................................................. 2

4.

Method of the study .......................................................................................... 3

5.

Design of the study ........................................................................................... 3

PART B: DEVELOPMENT ..................................................................................... 4
CHAPER I: LITERATURE REVIEW .................................................................... 4
1.1. Material Development in English Language Teaching ...................................... 4
1.1.1

Materials in English language teaching ..................................................... 4

1.1.2. Process of Material Development in English language teaching ............... 5
1.1.3. Process of Authentic Supplementary Material Development in ELT ........ 8
1.2. Material Evaluation in English Language Teaching ........................................ 12
1.2.1. Approaches to Material Evaluation ......................................................... 13
1.2.2. Principles for Material Evaluation .......................................................... 14
1.2.3. Material Evaluators ................................................................................ 15
1.2.4. Criteria for Material Evaluation .............................................................. 16
1.2.5. Factors relating to Material Evaluations.................................................. 19
1.2.6. Authentic Material Evaluation Framework in English language teaching 19

v


1.3. Review of related studies ................................................................................ 21
1.4. Chapter summary ............................................................................................ 22
CHAPER II: METHODOLOGY ........................................................................... 23
2.1. Context of the study ........................................................................................ 23
2.1.1. Thái Bình University context .................................................................. 23
2.1.2. General English Course .......................................................................... 23
2.2. The study ........................................................................................................ 24
2.2.1. The Development of the Authentic Supplementary Materials ................. 24
2.2.2. The Evaluations of the ASM .................................................................. 29
2.3. Chapter summary ............................................................................................ 34
CHAPTER III: FINDINGS AND DISCUSSIONS ................................................ 36
3.1. Strengths and weaknesses of the ASM (Research question 1) ......................... 36
3.1.1. Theoretical worth of materials ................................................................ 36
3.2.2. Practical Utilizing of the ASM ............................................................... 40

3.3. The impacts of the ASM (Research question 2) .............................................. 42
3.3.1. Impactson linguistic and extra-linguistic skills ....................................... 42
3.3.2. Impactson communicative competence ................................................... 44
3.3. Differences between the evaluationsof thetwo groups(Research question 3) ... 46
3.3. Future use of the ASM .................................................................................... 47
3.4. Chapter summary ............................................................................................ 47
PART C: CONCLUSIONS ..................................................................................... 48
1.

Summary of the findings and discussion ......................................................... 48
1.1.

The development of the ASM ................................................................. 48

1.2.

The evaluation of the ASM..................................................................... 48

2. Conclusions ...................................................................................................... 49
3.

Recommendations ........................................................................................... 50

4.

Limitations ...................................................................................................... 50

vi

5.

Suggestions for further research ...................................................................... 51

REFERENCES and BIBLIOGRAPHY ................................................................. 52
APPENDIX 1: SURVEY QUESTIONNAIRE FOR FIRST-YEAR STUDENTS ..... I
APPENDIX 2: RELIABILITY STATISTICS ....................................................... VII
APPENDIX 3: Mean scores of evaluations on the ASM ..................................... VIII
APPENDIX 4: Results of Mann Whitney test on two groups’ evaluations ............. IX
APPENDIX 5: LEARNING STYLES ................................................................... XI

vii


LIST OF FIGURES
Figure 1-1: Typical Forward design lesson plan................................................................. 5
Figure 1-2: Stages in the Backward design process ............................................................ 6
Figure 1-3: A teacher’s path through the production of new or adapted materials .............. 7
Figure 2.2-1: Steps of developing materials ..................................................................... 24
Figure 3.1-1: General appropriateness of the ASM .......................................................... 36
Figure 3.1-2: The appropriateness of the ASM’s content ................................................. 38
Figure 3.1-3: Difficult level of the ASM’s content ........................................................... 38
Figure 3.1-4: The appropriateness of the ASM’s topics/themes........................................ 39
Figure 3.1-5: Practical Utilizing of the ASM ................................................................... 40
Figure 3.1-6: Classroom interactions ............................................................................... 41
Figure 3.3-1: Impacts of the ASM on macro linguistic skills............................................ 42
Figure 3.3-2: Impacts of the ASM on extra-linguistic skills ............................................. 43
Figure 3.3-3: Impacts of the ASM on linguistic competence ............................................ 44
Figure 3.3-4: Impacts of the ASM on other components of communicative competence .. 45
Figure 3.4-1: Significant differences in the evaluations between two student groups........ 46

LIST OF TABLES
Table 2.2-1: Forms of authentic materials in the ASM ..................................................... 26
Table 3.1-1: Students’ learning styles .............................................................................. 39
Table 3.1-2:Means on Practical Utilizing ......................................................................... 40
Table 3.3-1:Means on linguistic skills ............................................................................. 42
Table 3.4-1: Future use of the ASM ................................................................................. 47

viii


PART A: INTRODUCTION
1. Rationale
In this“post-method” era when an increasingly dominant awareness is there exist no such
things as best teaching method, or perfectcommercial language teaching materials to fit
every unique context (Prabhu, 1990; Kumaravadivelu, 1994; Woodward,2001), language
teachers need to be enabled to write their own materials in order to reduce the dependency
on published materials and as a means of professional development (Jolly & Bolitho 1998,
pp.111-112).Using their own language learning and teaching experience as a personal
knowledge base and gaining theoretical insights on material development and evaluation,
teachers can devise the most suitable materials to their particular teaching context. These
language teachers, in good time, could create and nurture an environment supporting the
development of language knowledge and social skills that their learners need to function in
larger society.
With the desire tooffera motivating, beneficial and serviceable General English course for
first-year students at Thái Bình University (TBU), the integrating of authentic materials
into the curriculum is of utmost importance. The authenticity is felt significant since it
gives learners a taste of the natural language use, cultural issues and real world experiences
(Richard, 2001; Kelly et al, 2002; McGrath, 2002). This taste on one hand enhances
motivation, generates deeper interests, and facilitates comprehension; on the other hand it

providesopportunitiesfor learners to rehearse the language studied in a sheltered
environment for the successful language use in the future.
At TBU - a typical contextofa newly-upgraded institutionin an agricultural province where
hardly could people see someone from an English speaking country on the streets or at
special places, the developing of pertinent sets of ELT materials for students has also been
a matter of concern.Being aware of the above mentioned importance of authenticity, the
school management board has made this an important criterion for material development.
A purposeful and carefulcollection ofauthentic materials will“remind” TBU studentsthat
acquiringan international language like Englishis not anything “luxurious” but
advantageous in their daily communicating, entertaining, studying, and their future
employment. Such materials should be prioritized to develop and evaluated, given the
current course-book with limitations in terms of authenticity. Since there has not yet

1


research targeting at the authentic material development in the context of TBU, this gap
has opened a path for the current study on “The Development and Evaluation of Authentic
Supplementary Materials for First-Year Students in General English Course at Thái Bình
University”. Hopefully, advances in language teaching and learningin given context may
stem from the merits of the developed set of authentic supplementary materials (ASM).
2. Aims of study and research questions
The ultimateaim of this study is to develop a set of authentic supplementary materials for
first-year students at TBU in order to compensate for certain unsuitability of the set coursebook.Secondly, the studyinvolves itself in the process ofinvestigating the evaluations of
TBUstudentsandteachers towardsthe strengths and weaknesses of the ASM and identifying
their perceptions tothe ASM’s effectiveness.Another aim is to explorewhether there is a
relation betweenthe students’ evaluationsand their English proficiency. The research is
conducted to answer the following questions:
Question 1. What are the strengths and weaknesses of the Authentic Supplementary
Materials from the perspectives of first-year students and teachers of English at

Thái Bình University?
Question 2.To what extent arethe Authentic Supplementary Materials effectivefrom the
perspectives of first-year studentsand teachers of English at Thái Bình University?
Question 3. To what extent do the two English proficient student groups differ in their
evaluation on the Authentic Supplementary Materials?
3. Scope of the study
The minor thesis limits its scope to onlyauthentic materials as a source of supplementation
since investigating all kinds of supplementary materials are such broad topics that they
cannot be wholly discussed within the framework of this paper.
The process of applying such materials in real teaching and examining their effectiveness
is within the scope of this small study.Investigating the attitudes of TBU teachers and
targetedstudents towards the materials, the paper also attempts to figure out whether or not
there is difference in the evaluations of student groups based on their English
proficiencylevels.
Subjects of the material evaluation include104 first-year students at two faculties at TBU
who are not English majored, the results of this study hence may be generalized to only
non-English major students at Vietnamese colleges and universities.

2


4. Method of the study
The study is carried out on the basis of relevant theoretical document analysis andauthentic
English material collection for the material development process. Asurvey questionnaire
and semi-structured interviews are used for material evaluation. First of all, intensive and
extensive reading of related-theory documents on material development and evaluation
would provide principledframeworks andcriteriafor the authentic material selection for the
supplementation process. Secondly, after being implemented, the ASMevaluation is carried
out in order to obtain detailed and comprehensive informationfor the decision on the future
use of the ASM.In the second stage, both quantitative and qualitative research methods are

employed in the procedureof analyzing data collected from the survey questionnaire and
interviews.Qualitative method plays itsdominating role in the constructing, paneling and
validating of the surveyquestionnairebefore it is revised for surveying purposes.
5. Design of the study
The study is composed of three parts:
Part A –Introduction–presents the background, aims, research questions, the scope, and
the design of the study.
Part B– Development– is organized in three chapters.
Chapter 1- Literature review– conceptualizes the framework of the study through the
discussion of issues on developing and evaluating authentic materials for English language
teaching.
Chapter 2 – Methodology– presents the study context, then the process of the ASM
development and finally material evaluation subjects, data collection instruments, data
collection procedure, and data analysis.
Chapter 3 – Findings and Discussions–consists of a comprehensive analysis of the data
collected and a discussion on the findings of this study.
Part C – Conclusion– offers a summary of the findings, conclusion, recommendations,
limitations, and future directions for further study.

3


PART B: DEVELOPMENT
CHAPER I: LITERATURE REVIEW
This initial paperis a principled enquiryinto the theoretical backgrounds onELTmaterial
development and evaluation, which engages the research into ideas and practices of
developing and evaluating a set of authentic supplementary materials for a General English
course.

1.1.

Material Development in English Language Teaching

1.1.1 Materials in English language teaching
Tomlinson (1998, p. xi) defines that materials are “anything which presents or informs
about the language being learned” or “anything that can be used to facilitate the learning of
a language”. McGrath’s (2002, p.7) agrees that materials are “all kinds” that “can be
exploited effectively for language learning”.
Regarding the role of language learning and teaching (LLT)materials,a large volume of
research has demonstrated its critical importance in language acquisition. Nunan (1988,
p.98) states that language teachingmaterials are “an essential element within the
curriculum…. they provide concrete models for desirable classroom practice and …..
fulfill a teacher development role.” Richards (2001, p. 251) alsotitles materials as “a key
component in most language programs”.He perceives the functions of materials as the
basis for language input (lesson contents, skills taught) as well as language practice
occurring in language classroom; and as a form of teacher training which provides
teachers with ideas on how to plan and teach lessons.
In terms of classification,(Tomlinson 2003, p.2) notes that materials can be linguistic,
visual, auditory or kinaesthetic. Taking the formsof print or non-print,hand-written or
media-distributed (internet websites, computer software, etc.) (Richards, 2001, p.251;
McGrath, 2002, pp. 125-136), linguisticmaterials may be informative – informing learners
about the target language; instructional – guiding learners in practising the
language,experiential– providing learners with experiences of the language in use,eliciting
– encouraging learners to use the language; and exploratory – helping learners to make
discoveries about the language (Tomlinson, 2012, p.143).
Pedagogical materials – materials designed for language learning and teaching,nonpedagogicalmaterials –authentic materials (e.g. off-air recordings, newspaper articles),

4

teacher-written materials, and learner-generated materials (McGrath 2002, p.7) can
beutilized

ascore

/

main

materials

or

supporting

/supplementary

materials

(Cunningsworth, 1995, p. 1).
In summary, scholars have reached a consensus on the pedagogical benefits of materials in
theLLT process. Accordingly, materials in ELT are anything that serve as a guideline for
not only learners but also teachersabout contentandclassroom practice in English lessons.
It is materials that provide learners with English linguistic knowledge and background
knowledge in English, especially when they do the self-directed learning; and it is
materials that help teachers become more efficient and diverse in ELT classrooms.

1.1.2. Process of Material Development in English language teaching
Material development in the words of Tomlinson (1998, p.2) refers to “anything which is
done by writers, teachers or learners to provide sources of language input and to exploit

those sources in ways which maximize the likelihood of intake”. Tomlinson (2012) makes
the

definition

clearer

when

emphasizing

material

development

as

all

the

processesincluding material evaluation, adaptation, design, production, and exploitation
research, which aremade use of by practitioners who produce and/or use materials for
language learning(pp. 143-144).
Essentially,material development is an educational operation which helps teachers to
identify ways to make up,modify; select, edit,adapt and use the LLT materials in order for
them at their best to be a “key component” within the language program.
1.1.2.1. Approaches to material development in ELT
The development and implementation of ELT materials can be approached in several
different ways, each of which surely has different implications. This paper describes three

approaches, namely Forward design, Central design and Backward design, which differ
from one another when issues related to content, process, and outcomes are addressed
(Richard, 2013).
Resolving issues of contentis the initial stage ofForward design, which has been the major
tradition of material development(Richard, 2013, p. 5). Wiggins and McTighe (2006, p.
15) illustrates this forward-design process with a typical lesson plan.
Teacher chooses a topic for a lesson
selects a resource
chooses instructional methods
chooses questions to assess learners’ understanding of the topic.
Figure 1-1: Typical Forward design lesson plan

5


Differently, Central design begins with classroom procedures and methodology.
‘Innovative methods’ of the 1980s and 90s reflect Central design approach (Richard 2013,
p. 5). Research on teachers’ practices (Clark, 1987; Pennington & Richards, 1997; Richard,
2001; Woodward, 2001) reveals that when planning their lessons,many teachers first
consider theprocedures,teaching techniques, and learning activities (presentation, group
work, pair work, role plays, etc.). Later their attention turns to inputs and learning
outcomes which are addressed as the chosen methods
are implemented. These teachers feel the most important is to get their learners motivated,
keep theminterested and dothings worthwhile in class. If this is achieved, learning will
inevitable happen (Woodward, 2001, p. 187).
The use of classroom activities and processes as the starting point in instructional planning
is strongly criticized by Wiggins and McTighe (2006), who argue for starting with a clear,
careful description of desired results or learning outcomes - Backward design. Backward
Design proceeds in three phases, necessarily in the following order:
I. Identify desired results

II. Determine acceptable evidence.
III. Plan learning experiences & instruction.
Figure 1-2: Stages in the Backward design process
As a well-established tradition in curriculum design in general education, Backward design
has re-emerged as a prominent curriculum development approach in ELT. The Common
European Framework of Reference is a recent example of backward design (Richard, 2013,
pp. 5-20).
It is recognized that, there is no one best approach, and that forward design, central design
and backward design might each work well in different circumstances (Richard, 2013).
Each approach has advocates and practitioners who can cite magnificentexamples of
successful implementation. These approaches actually might also work concurrently in
some circumstances.
This paper would rather follow David Crabbe’s suggestion that “in fact, design goes
backwards and forwards whatever the starting point” (cited in Richard 2013, p.
28).Considering the specific context, starting with the consideration about what students
are able to do after a period of instruction, the ASMin this study corporates task-based
instructions to transform contents into “a blueprint which enables the desired learning

6


outcomes to be achieved”, to use Richard’s (2013, p. 6) words.Before the ASM’s
development isdescribedin detail in the subsequent chapter(see 2.2.1), it now comes to the
need of a principled framework for ELT material development.
1.1.2.2. Principled Frameworks for Material Development in ELT
Investigations into the processes and frameworks of material development carried out by
Nunan (1988), Prowse(1998), Tomlinson (1998, 2003), Richards (2001) reveal that
numerous material developers rely on intuitive feel for activities which are likely to work,
not much on principles of learning and teaching. Richards (1995, p. 105) though referring
his need of inspiration for writing materials, concludes that this process is “90 percent

perspiration”. This raises the need fordeveloping material principled frameworks, one of
which can be found in Jolly & Bolitho (1998).The authors provide sufficient theoretical
justification for aflexible, self-regulatingand coherent
diagram (Figure 1-3).

Figure 1-3: A teacher’s path through the production of new or adapted materials
byJolly, D & Bolitho, R (1998) in Tomlinson (1998:98)
The framework initiates withthe identification of needs for materials. Distinctly, a material
is “at its most effective” when it is turned to particular learners’ needs, their learning
objectives, learning styles and language proficiency, etc. The need exploration of not only

7


learners but alsoother parties involved (teachers, administrators, etc.) will engage material
developers in the linguistic andsemantic exploration as the next vital stageof contextual
realization for materials.It is understandable that LLT practice does not work in isolation
but is influenced by institutional contexts. If linguistic knowledge is integrated into texts of
completely unfamiliar topics or notion, the materialwriters themselves reduce the
materials’ efficacy. Once materials have been selectedfrom appropriate resources, they
need to be organized into teachable and learnable units, accompanied with explicit and
comprehensive instructions for exercises and activities. These are parts of pedagogical
realizationstage beforethematerial physical productionone.It is noteworthy that though the
material appearance and presentation is pivotal for both motivation and classroom
effectiveness, the production is not an end in itself. Material development, as an ongoing
and cyclical process, is pointless without constant reference to classroom practice. Put
another way, developedmaterials need to be used and evaluated. The evaluation will show
whether the materials have to be abandoned,rewritten, or may be reused without delay.
In Jolly & Bolitho’s diagram, they emphasize a variety of “optional pathways”and
“feedbacks loops” which allow material developers to deal in a concrete and flexible way

with reasons for the failure of the materials (if there is) and find out clues to material
improvement.
With reference to the current study on the development of authentic supplementary
materials, the envisaged steps of need identification exploration have already been done by
the institution. The following sectionsdescribethe other stages, though, not in the exact
order.

1.1.3. Process of Authentic Supplementary Material Development in ELT
The currentpaper is an addition whichconcentrates on using authentic materials as a source
of supplementation. Benefits of authentic materials are taken into consideration and made
best use of.
1.1.3.1. Authentic materials
There exists a complex web of definitions as to what exactly authentic materials consist
of.In general, there are two main trends of definitions. One focuses on the non-teaching
purposesand the native originof the materials which expose learners to real language style
and usages, cultures, social situations that learners are likely to encounter when engaging

8


in actual communicative use outside the classroom. The other trend focuses on the real
originand communicative purposesof the materials.
Nunan (1988), Harmer (1991), Richards (2001), Mc Grath (2002),though may define the
term using different words, all agree in that authentic materials are any which are
-

real-life texts,

-

designed by and for the native speakers who would hear, read or use them,

-

not specifically designed for the LLT purposes.

In a broader view, Morrow (1977, p.13) defines authentic text as “a stretch of real
language, produced by a real speaker or writer for a real audience and designed to convey
a real message of some sort”.In terms of purposes, Littleet all(1988, p.27) considers
authentic materials are created to “fulfill some social purpose in the language community
in which it is produced”. In the same line, Filice and Sturino (2002) consider authentic
materials to be materials which fulfilled a social or communicative purpose in the
community.The ideas can be summarized as follows.
Authentic materials are
- a real stretch of language,
- produced by real speakers, real writers for real audience,
- designed to fulfill social or communicative purposes (e.g. convey a real
message)
In a nutshell, there exist differences in the viewpoints on authentic materials;the
meaningful similarity to be extracted is the exposure to real languagewhich may catch
learners’ eyes or earsduring their particular dayand the language use in real
situationsforpurposes other than language teaching and learning.Adopting this similarity
as the base, the following sectionshifts the focus to the term supplementary materials in the
process of developing authentic supplementary materials.
1.1.3.2. Supplementary Materials
According to Tomlinson (1998), supplementary materials are such “materials designed to
be used in addition to the core materials of a course” (p.13). Although textbooks represent
“the visible heart of any ELT program” (Sheldon, 1988, p.237), “no textbook is perfect”;
therefore, in order for those textbooks to be used more effectively and properly, “teachers
should have the option of assigning supplementary materials based on their own specific

needs in their own specific teaching situation” (Ansary & Babaii, 2002, p.6).

9


The sound reasons for teachers to use other materials beside core textbooks have been
identify by Brown (1994), McGrath (2002) and Richards (2005). The scholars argue that
there exits gaps or mismatches between the official textbook and different needs of specific
individuals whose language competence levels may be varied as well. More specifically,
learners may need to expose to a wide range of textual materialsto promote motivation, or
they need more practice of particular skills or grammar components which is not adequate
in quantity in the textbook.
Drawingupontheliteratureonsupplementation,McGrath (2002) classifies between syllabus
driven process and concept driven process. That is, teachers can choose or design worthy
materials after they have identified what they need to supplement for a particular class; or
teachers can choose interesting materials accidentally with the hope that they can use such
materials in their future teaching. Teachers may utilize items from different published
sources or from others’, even learners’ writings or teacher may devise their own
materials. Teacher-prepared materials are likely to be more up-to-date and more relevant
to students’ needs and interests since teachers know their own students well. Differently
viewed, supplementary materials may be either content-oriented which focus on different
topics and information, or form-oriented which emphasize appropriate ways of using the
language components presented in the main course-book.
In sum, a variety of ideas on authentic materials and supplementary materials has been
viewed in order to provide foundation knowledge to supplement the currently-used
textbook “Enterprise 2” to maximize students’ learning potentials. The following section
will devote the literature on criteria for selecting authentic supplementary materials in
ELT.
1.1.3.3. Criteria for SelectingAuthentic Supplementary Materialsin ELT
Using authentic materials in an ELT class does not simply mean putting any pieces

of English produced by certain native speakers. Authentic materials, if appropriately
selected and implemented, can “help bridge the gap” between classroom knowledge
andlearner’s needs as well as their capacities to participate in real-world events (Wilkins,
1976, p. 79).In order to decide on the suitable ELT authentic materials, material developers
should bear in mind at least fourmentioned below criteria(based on Ian Mc Grath 2002, p.
106).

10


Linguistic and cognitive usefulness
Only pedagogically useful materials - effectively centralizing on language, intellectual
and/oremotional development - should be taken into consideration. Although authentic
materials are not originally designed for classroom use, the chosen ones should lend
themselves to conventional levels of linguistic description and/or to differentmodes of
action and behavior which realize this knowledge so that follow-up activities/ tasks may be
well constructed by a creative teacher to give learners opportunities for practice and boost
confidence in their English ability.
Relevance (to learners, to context and to syllabus)
In reference to learners, authentic materials chosen must be engaging topics which are
familiar to the learners’ lives.As Hammer (1991) advances, learners can comprehend the
meaning and usage of grammatical items more deeply and implicitly if they are presented
in a relevant context. With topics necessary for learners but they do not realize that
necessity, teacher's personal anecdotes, notes and other background information should be
shared for learners to follow the materials more easily.
With regard to the situational context, it is needed a balance in the choosing of authentic
materials produced by native speakers and of non-natives. Tomlinson (2005, p. 5)
recommends that most Asian learners of English are unlikely to need to communicate with
native speakers of
English but are likely to need to communicate in English with other non-native speakers”.

With regard to the supplementation, the relevance to syllabus and course-book
strongly affects the materials’ selection; otherwise the supplementary materials may waste
learners’ time.
Intrinsic interest
Being in consideration, if a material is patently not going to interest learners, however
relevant it is, it should be replaced by one will.Granting that learners are so interested in
the topic or activity in the material, they will NEED the language to talk about it (Roberts,
2014), then materials succeeds in the act of learning motivation.The involvement cultural
components using the target language, including culturally based practices, beliefs,
linguistic and non-linguistics behaviors (Richard, 2001, p. 253) may arouse learners’
interest. It should be aware that something completely acceptable in one culture may be
taboo in another so it is advisable to consider theculturalappropriateness.

11


Exploitability (availability, length, legibility, audibility)
Potential problems relating to the legibility of a written text (e.g. small font size texts or
hard-to-read handwriting), or to the audibility of tapes or video clips may lead to the
decision to select alternative ones. Over and above, materials chosen must be available
over a long term and easily obtainable in learners’real life. Learners can encounter them
someday somewhere somehow. Certainly, authentic information and itemsmust be legally
free of copyright restrictions.
In a few words,interesting, culturally engaging and relevant authentic materials should
reduce the likelihood of learners to get bored but at the same time are not allowed to make
learners lose track of the lessons.
Up to now, the paper has presented three approaches of forward, central and backward
design; a principled circular frameworkof six steps for developing ELT materials and four
major selection criteria for authentic materials. The subsequent discussion is about theories
and studies into the material evaluation in ELT, an important stage inthe ongoing process

of material development.
1.2.

Material Evaluation in English Language Teaching

Evaluation is the “systematic gathering of information” to “judge the fitness” of something
sothatpractitioners can harvest a better understanding of “what is effective, what is less
effective, and what appears to be of no use at all” (Weiss,1972, p. 22; Hutchinson and
Waters, 1987, p. 96; Rea-Dickins & Germaine 1992, p. 3).As Weir and Roberts (1994,
p.11) put it, evaluation provides “a tool to indicate the suitability of particular approaches
or techniques under given conditions and whether they meet the claims made for them”in
order forawarenessof the need for changesand even where and when changes can take
place to befostered.
With reference to the LLT practice, Rea-Dickins and Germaine (1992, pp. 5-28) present
two main motivations for evaluation. One is its use as a means of confirming the validity of
existing procedures, the other is explaining and exploring reasons why something is
working well in a classroom and why it is appropriate for a given target audience.
Evaluation, intrinsically, provides a wealth of information for the future direction of
classroom practice, for the course planning, determining learning achievement, the
management of learning tasks and for students’ satisfaction as well asinteractions.

12


Accordingly, material evaluation yield insights into how teachers use materials,
subsequently, suggests directions both for material development and teaching profession
development activities. Brown (1995, p.227) perceives material evaluation as the
systematic collection and analysis of all relevant information necessary to promote the
material improvement, or to assess its effectiveness and efficiency, or participants’
attitudeswithin the particular institution involved”. Tomlinson (1998, p. 3) affirms material

evaluation as the dynamic process with attempts to measure the value or potential value of
materialsormeasure the effects of the materials on their users. That is, ELT material
evaluation is the assessment of whether a collection of spoken and written English is
effective and appropriateor not when compared with the prescribed objectives in the target
ELT context.
To recap, when evaluators judge oneELT material, whether they are learners, teachers,
material developers or experts, they primarily accumulates information to answer
preliminary questions on the effectiveness of the materials in facilitating ELT practice.
Questions may be whether the materials are achieving the setup goals, or they are
responding to learners' needs, or whether learners and teachers are satisfied with the
materials, or whether learners are learning sufficiently from them. The answers afterward
are to make optimum use of the materials’ strong points and compensate for shortcomings.

1.2.1. Approaches to Material Evaluation
Johnson (1989) and Robinson (1991) support that material evaluation can be classified into


preliminary (establish materials’ potential suitability),



formative (examine how materials really work in the classroom), and



summative evaluation (identify strengths and weaknesses of the materials’
performance over a period of continuous use).

McDonough & Shaw (1993) distinguishes between


internal evaluation(focuses on theoretical assumptions behind the materials; the
intended objectives and the extent to which the objectives have been realized; the
justification of
language selection and grading; the design of activities and tasks), and



external evaluation(examine the extent to which a set of materials meets the needs of
a particular group of learners, the syllabus, and the examination).

Cunningsworth (1995) differentiates between

13




impressionistic overview (look through to get an overview of strengths
andweaknesses), and



in-depth evaluation (undertaken subsequently to provide a detailed evaluation).

Ellis (1997) assigns


predictive evaluation (make a decision to choose among the available materials the
mostsuitable one to use for a particular situation); and



retrospectiveevaluation(examine whether the materials used “work well” or not).

Recently, McGrath (2002) introduces the concepts of:


pre-use evaluation (carried out before a material is used to examine the potential
performance of the materials),



in-use evaluation (see whether the materials in use should be replaced or not), and



post-use evaluation (provide retrospective assessment of the materials’ performance
and is helpful to decide whether to use the materials in the future).

In general, there is a good deal of overlap in the frameworks proposed. Accordingly,
whether evaluators have a quick or a detailed look into the materials, whether
examiningbefore, after or whilematerials are in use, they actually judgeif the material
issuitable in its own parameter values or if itworks well for particular learners or specific
purposes.
This minor study desires to identify strengths and weaknesses of a set of authentic
supplementary materials from the perspectives of students and teachers after is has been
put into service so that a decision on the future widespread use of the materials can emerge
for all stakeholders involved.

1.2.2. Principles for Material Evaluation
One of the points emphasized by Rea-Dickins & Germaine (1992, p.4) is“ill-prepared and
ad-hoc evaluations are likely to be unreliable, unfair and uninformative" and they are not
adequate and accurate source documentation on which to base educational decisions.
Otherwise stated, evaluation in educational context must be systematic and undertaken
according to guiding principles. Below are the major principles thata systematic material
evaluation needs to pin on.
Multiple measure and evaluators
An evaluation has to be planned and executed by a number of different people (ReaDickins& Germaine,1992, p. 13). Multiple evaluators and measurewould ensure higher

14


quality, validity, reliability. If different people, using different measure still reach the same
decision, the consistent
evaluation is unhesitatingly ensured.
Specific objectives and systematic criteria
Brown and Rogers (2002, p. 229) claim that “the first step in any evaluation is to figure out
the purpose of the evaluation and the issues involved in achieving that purpose”. When
making evaluative concerns the educational aspects, making explicitthe objectives and
criteria in judgments is of significance. This principle requires evaluators to be extremely
clear about whether determinations are made about the materials’ effect on people using
them; or about materials’ aspect, validity, appeal, flexibility, credibility,or reliability of the
materialsthemselves are evaluated(Tomlinson, 1998, p. 15).
Considering the relationship between the language, LLT process and learners
Concerning learners, each individual is distinguished from all others in terms of
personality, motivation, aptitude, prior experience, interests, needs, and preferred learning
styles, level of proficiency and attitudes (Tomlinson, 1998, p. 18).If this learner has
positive attitude to the language studied and language learning, his evaluation to the LLT
materials may be quite different from ones who do not like the language. Personal factors

indeed much affect one’s evaluation.Systematic LLT material evaluation, therefore, needs
LLT theory and research findings on language learning acquisition as guiding
principles(Tomlinson, 2003).
Considering the contextual circumstances and generalizability
Since materials are developed to suit particular learners and specific context, it is critical
for evaluators to consider the contextual circumstances of the evaluation in order to
determine reliable and effective procedures. Moreover, evaluators must be cautious in
making generalizations of their evaluation. A material can be judged to be suited to this
situation but not necessarily to all others.
1.2.3. Material Evaluators
Material evaluationcan be carried out by insiders (e.g. teachers, learners, course
designers,or materials designers) or / and outsiders (those who are not involved in the
language program, for instance, experts, consultants, inspectors, or administrators).
Material evaluation by outsiders is often said to be objective since outsiders are hoped to
bring “fresh” perspectives into the evaluation (Pascual, 2013). Outsiders may have

15