A study on the construction of a marking scheme for end of semester english oral tests for 10th grade students at cam giang high school in hai duong

VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POSTGRADUATE STUDIES
------------------------

NGUYỄN THỊ OANH

A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR
END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10th GRADE
STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG.
Nghiên cứu xây dựng bảng đánh giá cho bài kiểm tra nói cuối kỳ môn
Tiếng Anh cho học sinh lớp 10 trường THPT Cẩm Giàng, Hải Dương.

M.A MINOR PROGRAMME THESIS
Field: English Teaching Methodology
Code: 60140111

Hanoi – 2017


VIETNAM NATIONAL UNIVERSITY, HANOI
UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES
FACULTY OF POSTGRADUATE STUDIES
------------------------

NGUYỄN THỊ OANH

A STUDY ON THE CONSTRUCTION OF A MARKING SCHEME FOR
END-OF-SEMESTER ENGLISH ORAL TESTS FOR 10TH GRADE
STUDENTS AT CAM GIANG HIGH SCHOOL IN HAI DUONG.
Nghiên cứu xây dựng bảng đánh giá cho bài kiểm tra nói cuối kỳ môn

Tiếng Anh cho học sinh lớp 10 trường THPT Cẩm Giàng, Hải Dương.

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology
Code: 60140111
Supervisor: Nguyễn Thị Ngọc Quỳnh, Ph.D

Hanoi - 2017


DECLARATION
I hereby state that I, Nguyen Thi Oanh, declare the thesis entitled “A study on the
construction of a marking scheme for the end-of-semester English oral tests for 10th
grade students at Cam Giang High School in Hai Duong” is my own research for
the Degree of Master of Arts at the Faculty of Post-Graduate Studies, University of
Languages and International Studies- Vietnam National University, Hanoi. This
thesis is the result of my own research and efforts and it has not been submitted for
any degree at any other university or institution.

Hanoi, 2017

Nguyen Thi Oanh

i


ACKNOWLEDGEMENTS
First and foremost, I owe my deepest gratitude to my supervisor, Dr. Nguyen Thi
Ngoc Quynh for her encouragement, support, and guidance and for giving me

constructive feedback throughout this thesis. She has guided me in searching for
relevant theory to my thesis and has also assisted in collecting data. Consequently, I
have learnt a lot about the assessment of English oral tests and construction of a
marking scheme for oral tests.
Secondly, I would like to thank Ms. Bui Thien Sao, an expert of the Center for
Language Testing and Assessment of the University of Languages and International
Studies for her invaluable assistance during the research time.
Thirdly, this thesis would not have been possible without the enthusiastic
participation of six English teachers and 150 students at Cam Giang High School
where the research was carried out.
Lastly, I would like to thank my family whose love and support help me complete
this thesis.

ii


ABSTRACT
This thesis was conducted at Cam Giang High School. The participants consisted of
150 tenth-grade students and six English teachers at Cam Giang High School. Two
experts in the Center for Language Testing and Assessment of the University of
Languages and International Studies also participated in this study. The research
aims at constructing a marking scheme for the end-of-semester English oral test of
tenth-grade students. Constructing a marking scheme for oral tests is a complex
process. In this paper, a combination of three methods: intuitive, qualitative and
quantitative are employed by the researcher. The research started with writing the
draft of marking scheme. Then, the researcher obtained the judgments from experts
and other teachers. Next, the marking scheme was piloted with 150 tenth-grade
students. Based on analyzing the students‟ scores of the oral tests, the researcher
examined how well the marking scheme works. The findings revealed that the
marking scheme can be used by the teachers effectively although there is still a need

for further investigation to improve and validate the current marking scheme. The
majority of the teachers took a positive attitude toward the marking scheme. They
believed in the efficacy of marking scheme in spite of some difficulties at the
beginning and recommended the continuation of using the marking scheme for the
next school year. Pedagogical implications and suggestions for future studies are
drawn out based on the research findings.

iii


TABLE OF CONTENTS
ơ

DECLARATION ......................................................................................................... i
ACKNOWLEDGEMENTS ....................................................................................... ii
ABSTRACT .............................................................................................................. iii
TABLE OF CONTENTS .......................................................................................... iv
LIST OF ABBREVIATIONS .................................................................................... vi
LIST OF TABLES.................................................................................................... vii
PART A: INTRODUCTION .......................................................................................1
1. Rationale .................................................................................................................1
2. Aims and objectives of the study ............................................................................2
3. Research question....................................................................................................3
4. Scope of the study ...................................................................................................3
5. Significance of the study .........................................................................................3
6. Method of the study ................................................................................................3
7. Design of the study..................................................................................................4
PART B: DEVELOPMENT ........................................................................................5
CHAPTER 1: LITERATURE REVIEW.....................................................................5
1.1 Communicative competence .................................................................................5

1.1.1 Communicative competence in the CEFR ..........................................................6
1.2 What is speaking?..................................................................................................9
1.2.1 Assessing speaking ...........................................................................................10
1.3 Marking scheme ..................................................................................................10
1.3.1 What is a marking scheme? ..............................................................................10
1.3.2 Approach to construct a marking scheme ........................................................12
1.3.3 Steps to construct a marking scheme ...............................................................13
1.3.4 Types of marking schemes ................................................................................15
1.3.5 Structure of a marking scheme .........................................................................17
1.3.6 Available speaking marking schemes ...............................................................18
1.3.7 Previous studies ................................................................................................20

iv


CHAPTER 2: METHODOLOGY ............................................................................22
2.1 Setting of the study..............................................................................................22
2.2 Participants ..........................................................................................................23
2.3 Description of the end-of- semester oral test ......................................................24
2.4 Research design ...................................................................................................24
2.4.1 Rationale for using a multiple-method approach .............................................24
2.4.2 Research procedure ..........................................................................................25
2.5 Data collection instruments .................................................................................28
2.5.1 The interview with the teachers........................................................................28
2.5.2 Sample oral test ................................................................................................29
2.5.3 Data collection procedure ................................................................................30
2.6 Data analysis method ..........................................................................................30
2.6.1 Descriptive technique .......................................................................................30
2.6.2 Statistical technique .........................................................................................30
2.6.3 Data analysis procedure ..................................................................................31

CHAPTER 3: RESULTS AND DISCUSSION ........................................................33
3.1 The finding from interviews with teachers and experts ......................................33
3.2 The finding from scores of students‟ oral tests ...................................................37
PART C: CONCLUSION ..........................................................................................43
1. Summary of the study ...........................................................................................43
2. Pedagogical implications ......................................................................................44
3. Limitation ..............................................................................................................45
4. Suggestions for further studies ..............................................................................45
REFERENCES ..........................................................................................................46
APPENDIX 1 .............................................................................................................. I
APPENDIX 2 .......................................................................................................... XII
APPENDIX 3 ......................................................................................................... XIII
APPENDIX 4A ..................................................................................................... XVI

APPENDIX 4B ...................................................................................................... XIX
APPENDIX 5 ....................................................................................................... XXV
APPENDIX 6: ................................................................................................... XXVII

v


LIST OF ABBREVIATIONS
CEFR: Common European Framework of Reference
MOET: Ministry of Education and Training

vi


LIST OF TABLES
Table 1: Pearson Correlation on Grammar ............................................................... 38

Table 2: Pearson Correlation on vocabulary ............................................................. 39
Table 3: Pearson Correlation on pronunciation ........................................................ 39
Table 4: Pearson Correlation on fluency and coherence ........................................... 40
Table 5: Pearson Correlation on Sum ....................................................................... 40

vii


PART A: INTRODUCTION
This part is offered to introduce the rationale of the study, the problem to be
addressed in the study, the aims and objectives of the study, and the research
questions to be answered. It will also present the scope of the study, significance of
the study, an overview of the employed methods and the design of the study.
1. Rationale
English is the most popular official language in the world and the primary language
of global trade and commerce. It is an international means of interaction and
communication in almost all countries. Proficiency in English is seen as a desirable
goal for a lot of people in the world. In many countries including Vietnam, English
is taught as a compulsory subject at school and it is included in many exams. Of all
four skills: reading, speaking, listening and writing; speaking is generally thought to
be the most important due to the fact that a lot of learners have spent years studying
English but they still cannot speak it fluently. In order to speak a foreign language,
the learners must master the sound system of the language, use appropriate
vocabulary and be able to put words together intelligibly with minimal hesitation.
Moreover, they also need to understand what is being said to them and respond
appropriately to maintain good–natured relation to achieve communication goal
(Luoma, 2004). Although students‟ speaking skills are often practiced and
developed, it is not extensively assessed. Comparing to other skills, speaking is the
most difficult language skill to assess the reliability. The student‟ speaking ability is
usually judged during a face-to-face interaction, in real time between the teacher

and the student. Besides, the factors such as the nature of the interaction, the kind of
tasks, the questions asked, the topic raised and the opportunity given students to
speak in English will all have an impact on the student‟s performance (Luoma,
2004). On assessing speaking skills, the teacher has to take on the role as an
interviewer and assessor at once, which puts him or her under a lot of pressure. This
makes teachers hesitant to assess speaking and focus on assessing other skills
instead (Rychtarik, 2014). However, “if you want to encourage oral ability, then test
1


oral ability” stated by Hughes (1989). Testing and assessment is a very important
part of teaching process, which helps provide necessary information to the students
and teachers about the progress made and the work be done. If assessment is carried
out accurately and fairly, it will have a positive impact on both teachers and learners.
It helps learners define the aim of their learning and contributes to improve the
quality of teaching.
Perceiving the importance of testing oral proficiency, on September 9th, 2014, the
Ministry of Education and Training signed the decision No 5333/BGDT-GDTrH to
promulgate the applying of speaking test in the final term test (account from 20 %
to 30 % of the total score) with the 10th grade students (seven –year program and
ten-year program) and the 6th grade students (ten-year program). However, there is
no standard marking scheme for the teachers to follow. Each teacher evaluates
students‟ speaking based on the marking scheme he/she individually creates and
uses different criteria to assess. The marking schemes produced by teachers consist
mostly or only numbers without descriptor or with very short descriptors such as
very good, good and bad. Besides, other factors such as language level, gender,
status of the teacher, the familiarity between the teacher and the students also affect
the evaluation student‟s performance and the score that the students get. In order to
overcome some of these problems, the researcher is fully aware of the need of
constructing a marking scheme for students‟ speaking ability. Therefore, a lot of

attempts have been made to do a study titled “A study on the construction of a
marking scheme for end-of-semester English oral tests for 10th grade students at
Cam Giang High School in Hai Duong”. Hopefully, the study, to which I will
devote all my efforts, will make a contribution to the English teaching and learning
of the teachers and students at Cam Giang High School.
2. Aims and objectives of the study
The study is aimed at constructing a marking scheme for end-of-semester English
oral tests for 10th grade students at Cam Giang High School in Hai Duong with
hope to find an effective tool for assessing students‟ English speaking competence.

2


In order to achieve this aim, the following objectives will be addressed in the study:
+ To find out the marking criteria and descriptors of the target marking scheme
+ To investigate the feasibility of the proposed marking scheme
3. Research question
The study aims at answering the following questions:
1. What are the marking criteria of the marking scheme for end-of-semester
English oral test for 10th grade students at Cam Giang High School?
2. What are the descriptors for each criterion of the marking scheme for end-of
-semester English oral test for 10th grade students at Cam Giang High School?
4. Scope of the study
The scope of the research has been made quite clear from the title: “A study on the
construction of a marking scheme for end-of-semester English oral tests for 10th
grade students at Cam Giang High School in Hai Duong”.
Firstly, the study only focuses on oral testing, namely the construction of a marking
scheme. To be more specific, it refers to the process to find out the marking criteria
and descriptors for a marking scheme which helps teachers at Cam Giang High
School assess students‟ oral test. This will help teachers at Cam Giang High School

assess students‟ speaking more accurately and objectively. Secondly, the subject of
the study is restricted to tenth-grade students at Cam Giang High School.
5. Significance of the study
The study is of great significance to both the 10th grade students and the teachers at
Cam Giang High School because it may have a great contribution to teaching and
learning speaking. For teachers, the study helps them to find a useful assessment
tool which ensures objectiveness and fairness. A marking scheme is an assessment
tool that clearly indicates marking criteria which let students know what is expected
of them and how to achieve aim. It is hopeful that the study will give students
opportunity to do self-assessment to reflect on the learning process.
6. Method of the study
As the aim of the study to construct a marking scheme for end-of-semester English

3


oral tests for 10th grade students at Cam Giang High School in Hai Duong, the
study is designed to use a combination of intuitive, qualitative and quantitative
methods. Besides, many sources such as books, newspapers and some sources on
the internet have been read by the researcher. The findings are reported based on the
experts‟ and the teachers‟ responses in the interview and students‟ scores on oral test.
Moreover, constant discussions with the supervisor are of great significance.
7. Design of the study
The study is divided into three main parts:
Part I (Introduction) includes the rationale, the aims and objectives, the scope, the
significance, the research questions, the method and the design of the study.
Part II (Development) consists of chapters as follows
Chapter 1 (Literature review) presents the theoretical background of the study and
the review of the available marking schemes and previous studies.
Chapter 2 (Methodology) describes in detail the research methodology which

consists of the context of the study, the information of the subjects, instruments of
data collection, procedures of data collection and methods of data analysis.
Chapter 3 (Results and Discussion) reports the statistical results and the analysis of
the data.
Part III (Conclusion) closes the study by summarizing the whole study with
concluding remarks and offering some limitations and suggestions for further
studies.

4


PART B: DEVELOPMENT
CHAPTER 1: LITERATURE REVIEW
This chapter provides an overview of the literature on the field of communicative
competence, assessing speaking and marking scheme. It discusses communicative
competence, communicative competence in the CEFR, definitions of speaking,
assessing speaking, definitions of marking scheme, approaches to construct a
marking scheme, types of marking scheme and the structure of a marking scheme. A
review of the available marking schemes and previous studies is also presented.
1.1 Communicative competence
During the past few years, the concept of communicative competence has been
discussed

and

redefined by many researchers

and authors. The

term

“communicative competence” was introduced by Hymes (1972) and he emphasized
that language consisted of a wider range of competence while Chomsky (1965) just
concerned about grammatical competence. Hymes stated that language can only be
understood if the rules for grammar, speech acts and discourse are analyzed in
relation to the speech community and the context (Hymes, 1972). According to
Luoma (2004) communicative competence focuses on language user‟s means of
communicating, which may affect the choice of more authentic learning material
and communicative tasks as learning material in language classrooms.
Canale and Swain (1980) proposed one of the first theoretical models of
communicative competence. The model distinguishes between communicative
competence and communicative performance, as communicative competence is
knowledge about grammar, sociolinguistic knowledge, strategic competence, while
communicative performance is the actual communication. However, Canale (1983)
revised this model and used the term “actual communication” instead of
“performance”. He also asserted that: “Communicative competence refers to both
knowledge and skill in using the knowledge when interacting in actual
communication” (Canale, 1983, p. 5).
Bachman and Palmer (1996) developed another model in communicative

5


competence and language testing: the model of communicative language ability.
The model focuses more on the interaction between context and language use
(Fulcher and Davidson, 2007).

The term “language ability” consists of language

knowledge and strategic competence. Language knowledge includes both

organizational knowledge (grammatical and textual knowledge) and pragmatic
knowledge (illocutionary competence and sociolinguistic competence). Strategic
competence involves the ability to assess whether the situation is practicable and to
plan for the next movement (Luoma, 2004). Bachman and Palmer (1996) claimed
that their model can be used as a checklist for developing language tests.
1.1.1 Communicative competence in the CEFR
The CEFR is the Common European Framework for language learning, teaching
and assessment, which focuses on the nature of language use and the language user
and the implications for learning and teaching (Council of Europe, 2001). The
Common European Framework defines levels of proficiency which allow learners‟
progress to be measured at each stage of learning and provides criteria for assessing
four English skills: reading, speaking, listening and writing. Of all skills, speaking
is very important and within communication the learners can learn and apply
various skills. The Framework consist of three proficiency levels which are referred
to respectively as Basic User: A1, A2, Independent User: B1, B2, and Proficiency
User: C1, C2 (Council of Europe, 2001)
Communicative competence in the narrower sense consists of three components,
linguistic competences, sociolinguistic competence and pragmatic competences
(Council of Europe, 2001, p. 108). In the model of communicative competence of
Canale and Swain (1980), there are also three components, grammatical
competence, sociolinguistic competence and strategic competence. Although the
CEFR and Canale and Swain label these categories slightly differently, they use
similar categories to describe competences. In the following paper, the categories
from the CEFR are particular relevance for the assessment of speaking will be
discussed.

6


Linguistic competence is considered the core of the model of communicative

competence. Linguistic competence is divided into lexical competence, grammatical
competence, semantic competence, phonological competence and orthoepic
competence (Council of Europe, 2001, p.109).
Lexical competence is defined as “knowledge of, and ability to use, the vocabulary
of a language, consists of lexical elements and grammatical elements” (Council of
Europe, 2001, p.110). Lexical element includes fixed expression and single word
forms to enhance language on the different levels of meaning. The CEFR also
presents illustrative scale for the range of vocabulary knowledge and the ability to
control that knowledge to specify these competences (Council of Europe, 2001,
p.112)
Grammatical competence is described as “knowledge of, and ability to use, the
grammatical resources of a language” (Council of Europe, 2001, p. 112).
Grammatical competence is the ability to understand and express meaning by
producing and recognizing well-formed phrased and sentences (Council of Europe,
2001). To measure grammatical competence, the CEFR has developed an
illustrative scale demonstrating levels of grammatical accuracy (Council of Europe,
2001, p. 114). Grammatical competence also includes morphology and syntax
which the learners need to be aware of. Morphology deals with the organization of
words and the ways to modifying words forms and syntax deals with the
organization of words into meaningful sentences (Council of Europe, 2001).
Semantic competence is “the learner‟s awareness and control of the organization of
meaning” (Council of Europe, 2001, p. 115 ).
Phonological competence involves the knowledge of the sound-units, words stress,
sentences stress, sentence rhyme and intonation (Council of Europe, 2001, p. 116)
Orthoepic competence involve knowledge of spelling conventions, ability to consult
a dictionary, knowledge of the implication of written form for phrasing and
intonation and ability to solve ambiguity in various context (Council of Europe,
2001, pp: 117-118).

7

Sociolinguistic competence is “concerned with the knowledge and skill required to
deal with the social dimension of language use” (Council of Europe, 2001, p. 118).
Sociolinguistic competences include linguistic markers of social relations,
politeness conventions, expressions of folk-wisdom, register differences, and dialect
and accent (Council of Europe, 2001, p. 119). Canale and Swain defined
sociolinguistic competence as “the ability to communicate appropriately in a variety
of contexts, this includes both verbal and non-verbal communication” (Canale and
Swain, 1980).
Linguistic markers of social relations vary from language to language. They include
use of choice of greeting, use and choice of address forms, conventions for
turn-taking in conversations and the use of choice of expletive (Council of Europe,
2001, p. 119).
Politeness conventions vary from one culture to another and are a frequent source of
inter-ethnic misunderstanding when polite expressions are literally interpreted
(Council of Europe, 2001, p. 119).
Expressions of folk wisdom are fixed formulas about daily life, often used in
newspaper headlines. The expressions include proverbs, idioms, and expressions for
beliefs, attitudes and values and are often used in graffiti and on T-shirt slogans
(Council of Europe, 2001, p. 120).
Register differences refer to “systematic differences between varieties of language
used in different contexts” (Council of Europe, 2001, p. 120). Register differences
express differences in level of formality: frozen, formal, neutral, informal, familiar
and intimate (Council of Europe, 2001, p 120).
Dialects and accents perform people‟s origin, sociolinguistic competences include the
ability to distinguish between various social classes, regional provenances, national
origins, ethnicities and occupational groups (Council of Europe, 2001, p 121).
Pragmatic competences include discourse competence and functional competence.
“Discourse competence is the ability of a user/ learner to arrange sentences in

sequences so as to produce coherent stretches of language” (Council of Europe,

8


2001.p. 123). The CEFR has included illustrative scale with aspects of discourse
competence: Flexibility to circumstances, turn-taking in interactions, thematic
development and coherence and cohesion (Council of Europe, 2001, pp: 123-125).
Functional competence is “concerned with the use of spoken discourse and written
texts in communication for particular purpose” (Council of Europe, 2001, p. 125).
Functional competence also includes knowledge and ability to use the schema
patterns of social interaction) which underline communication, such as verbal
exchange patterns (Council of Europe, 2001, p. 126). The CEFR developed an
illustrative scale for two qualitative aspects, fluency and propositional precision.
Fluency is “the ability to articulate, to keep going and to cope when one lands in a
dead end”. Propositional precision refers to “the ability to formulate thoughts and
propositions so as to make one‟s meaning clear” (Council of Europe, 2001, pp:
128-129).
1.2 What is speaking?
According to Widdowson (1984), “speaking is an active productive skill”. This
means that when someone wants to deliver information, he or she needs to activate
their background knowledge, choose appropriate words and use correct grammar
and pronunciation to gain meaning. This process requires someone‟s brain to use all
knowledge he or she has about the language therefore it is not easy to be mastered.
Bygate (1987) considers learner‟s speech as a process, speaking is a “real-time”
action because the learner has to plan, process and produce the language
simultaneously. The speech process includes planning, selection and production of
speech. Planning is an interactive process which requires learner to have knowledge
about interaction routines to plan the next step of the conversation. Selection is the
stage in which the leaner uses knowledge of language and grammar to decide how

to express oneself. In the production stage, the learner uses the knowledge about
pronunciation and communication strategies to produce language (Bygate, 1987).
The CEFR has distinguished clearly between interaction (spontaneous) and
production (prepared) of language. Interaction activities are mainly spontaneous and

9


are carried out throughout conversation and more or less informal discussion.
Production activities are mainly prepared and rehearsed in advance (Council of
Europe, 2001, p. 178).
1.2.1 Assessing speaking
The categories for oral assessent is enormous and deciding which criteria to use for
assessment is a relatively dificult work. Assessor should determine the most
appropriate criteria that could not only be used to assess students‟ speaking, but also
be relevant to the objective of the course/ lesson, etc (Knight , 1992).
The CEFR classifies 12 qualitative categories relevant to oral assessment. The
CEFR has also developed illustrative scales for assessment and each scale describes
the level of proficiency. There are a number of categories relevant to assess
speaking such as turn-taking strategies, co-operating strategies, asking for
clarification, fluency, flexibility, coherence, thematic development, precision,
sociolinguistic competence, general range, vocabulary range, grammatical accuracy,
vocabulary control and phonological control (Council of Europe, 2001). However, it
is impossible to assess all criteria simultaneously. The assessers need to make
choices for each assessment situation and select only several criteria that are
relevant to the particular context. Choosing no more than 4 or 5 criteria in each
testing situation guarantees feasibility as well as reliability of the assessment
( Council of Europe, 2001, pp: 192-193).
1.3 Marking scheme
1.3.1 What is a marking scheme?

A marking scheme is sometimes referred to as a scoring rubric or a rating scale
which is defined as an explicit set of criteria used for assessing a particular type of
work or performance.
As Davied, Brown, Elder, Hill, Lumly, McNamara (1999) defined it:
A rating scale is a scale for the description of language
proficiency consisting of a series of constructed levels against
which a language learner’s performance is judged. Like a test, a

10


proficiency scale provides an operational definition of a
linguistic construct such as proficiency. Typically such scales
range from zero mastery through to an end-point representing
the well-educated native speaker. The levels or bands are
commonly characterized in terms of what subjects can do with
the language and their mastery of linguistic features (such as
vocabulary,

syntax,

fluency

and

cohesion)…Scales

are

descriptions of groups of typically occurring behaviors, they are

not in themselves test instruments and need to be used in
conjunction with tests appropriate to the population and test
purpose. Raters or judges are normally trained in the use of
proficiency scales so as to ensure the measure’s reliability (David
et all, 1999, pp: 153-154)
From above definition, it can be seen that a rating scale includes both the fields to
be assessed (construct) and the alignment between examiner‟s performance and the
predetermined levels of behavior descriptions. Thus, it is important to consider two
above components when constructing a rating scale. In addition, for different
purposes the construction of scale can be different. Alderson (1991) and Pollitt and
Murray (1996) classify different purposes that rating scales could serve:
user-oriented, constructor-oriented and assessor-oriented. A user-oriented scale is
“designed to communicate information about typical of likely test taker behaviors at
a given level” (Taylor, 2011, p. 190). Constructor-oriented scale “guides test writer
in their choice of tasks to include in a test” (Taylor, 2011, p. 190). Assessor oriented
scale “guides the rating process, focusing on the quality of performances expected”
(Taylor, 2011, p. 190). Because one rating scale is rarely appropriate for all the
purposes above, the purpose compatible with the scale must be prioritized to
measure the sample language elicited from learner‟s performance in particular
testing situation (Nakatsuhara, 2007).
In the present paper, the researcher focuses on constructing an assessor-oriented

11


speaking marking scheme which teachers at Cam Giang high school can utilize to
assess students‟ oral tests.
1.3.2 Approach to construct a marking scheme
Marking scheme construction is recognized to be a complex process (Brindley,
1998, Fulcher, 2003, North, 2000). Traditionally, the design and construction of

rating scales used a priori approach in which assessment criteria and rating scale
descriptors are developed by “experts” using their own experience and intuitive
judgment ( Fulcher, 2003). McNamara (1996) states that marking schemes were
constructed based on the construction of the first scale for the Foreign Service
Institute‟s Oral Proficiency Interview in the 1950s. In 1990s, many authors
supported empirically based approach involving analyzing samples of actual
language performance to construct criteria and marking scheme descriptors (Fulcher,
1996; Milanovic, Saville, Pollitt and Cook, 1996; Shohamy, 1990; Upshur and
Turner, 1995). Fulcher (2003) discusses two basic approaches to rating – scale
development: intuitive approaches and empirical approaches. Intuitive methods
primarily rely on “expert judgment and the principled interpretation of experience”
(Council of Europe, 2001, p. 208). Fulcher also introduced three common
subcategories of an intuitive method as expert judgment, committee and experiential.
The expert judgment and committee are alike in which the experts make decisions
upon issues such as the number of levels and the wording of descriptors based on
existing scales, curriculum, course material and other necessary sources.
Experiential evolves on the basis of the expert judgment and committee by revising
the scale after a further understanding on both the content of existing scale and
sample performance (Fulcher, 2003). Empirical methods include data-based, data
-driven scale development, empirically derived, binary – choice, boundary
definition scales and scaling descriptors (Fulcher, 2003). More recently, together
with scale development for the Common European Framework of Reference
(CEFR), the mixture of qualitative and quantitative methods makes complementary
contribution to rating scale development (Council of Europe, 2001). Quantitative

12


methods rely on statistical analyses and careful interpretation of results while
qualitative methods involve interpretation of information obtained.

There are a number of approaches to marking scheme construction. The best
methods for marking scheme construction are said to combine all three approaches
including intuitive, quantitative and qualitative approaches in “a complementary
and cumulative process” (Council of Europe 2001, 207). Therefore, in constructing
a marking scheme for assessing student s‟ oral test at Cam Giang High School, a
combination of intuitive, qualitative and quantitative approaches was used.
1.3.3 Steps to construct a marking scheme
According to Mertler (2001) building a marking scheme includes following seven steps:
Step 1: Re-examine the learning objectives to be addressed by the task. This
helps to match the teacher‟s scoring guide with objectives and actual
instruction.
Step 2: Identify specific observable attributes that the students demonstrate in
their product, process, or performance.
Step 3: Brainstorm characteristics that describe each attribute. Identify ways
to describe above average, average, and below average performance for
each observable attribute identified in Step 2.
Step 4a: For holistic rubrics, write thorough narrative descriptions for
excellent work and poor work incorporating each attribute into the
description. Describe the highest and lowest levels of performance
combining the descriptors for all attributes.
Step 4b: For analytic rubrics, write thorough narrative descriptions for
excellent work and poor work for each individual attribute. Describe the
highest and lowest levels of performance using the descriptors for each
attribute separately.
Step 5a: For holistic rubrics, complete the rubric by describing other levels on
the continuum that ranges from excellent to poor work for the collective
attributes. Write descriptions for all inter mediate levels of performance.

13

Step 5b: For analytic rubrics, complete the rubric by describing other levels on
the continuum that ranges from excellent to poor work for each attribute. Write
descriptions for all inter mediate levels of performance for each attribute
separately.
Step 6: Collect samples of student work that exemplify each level. These will
help the examiners score in the future by serving as benchmarks.
Step 7: Revise the rubric, as necessary. Be prepared to reflect on the
effectiveness of the rubric and revise it prior to its next implementation.
(Mertler, 2001)
Meanwhile, Nakutsuhara (2007) suggests developing a marking scheme
according to four stages:
Stage 1: Reviewing existing speaking rating scales outside and inside Japan to
collect marking categories and descriptors to be referenced in the later stage.
Stage 2: Examining the course of the study (the guideline of secondary school
education) to decide types and levels of marking categories for the target
population.
Stage 3: Drafting a rating scale based on the existing rating scales, while
obtaining expert judgments from eight experienced upper-secondary school
teachers.
Stage 4: Piloting the scale with 42 Japanese upper- secondary students with
two raters, to examine how well the resulting rating scale functions.
(Nakutsuhara, 2007)
Taylor (2001) proposes the process of constructing a marking scheme is took place
in three phases as outline below:
Phase 1: Intuitive
The marking scheme is constructed by the researcher. The researcher carries out
according to following steps:
+ Review the existing scale in the world and Vietnam, teaching materials,
curriculum, objectives of the course and relevant source materials.

14


+ Propose the criteria
+ Determine the number of scales
+Develop the descriptors
+Discuss with other teachers and revise
+Trail the scale
+ Stabilize the scale -> Version 1
Phase 2: Qualitative
External expert reviewing the existing scale
+ Rank the descriptors in order of difficulty
+ Trail the scale
+ Discuss and revise the scale
+ Stabilize the scale -> Version 2
Phase 3: Quantitative
Standard –setting phase
+ Trail the raters
+ Trail the scale
+ Analyze the scores
+ Stabilize the scale -> version 3

(Taylor, 2011, p. 195)

In this research, the researcher follows a three - phase process suggested by Taylor
(2011) for some reasons. Firstly, according to the CEFR (2001, p. 207) the best
methods for rating scale development are said to take advantage of the strengths of
a range of intuitive, quantitative and qualitative approaches. The best scale, the
CEFR (2001, p. 207) suggests, combine all three approaches in a “complementary

and cumulative process”. Secondly, the phases are logical and detailed. They are
clear for the researcher to follow. In each phrase, steps are described specifically
therefore it helps the researcher have an outline to construct a marking scheme.
1.3.4 Types of marking schemes
There are many divisions of individual types of marking scheme as there are many
authors dealing with this issue. According to “Rubric for assessment” of Northern

15


Illinois University there are several types of rubrics including holistic, analytical,
general, and task –specific. Timothy Farnsworth (2014) of CUNY Hunter College
divided rubric into five types: holistic, analytic, task Fulfillment, performance
decision trees and checklist. However, according to Mertler (2001) and Taylor
(2011) assessment criteria used to evaluate L2 spoken language performance
generally fall into two main categories: holistic and analytic. In this study, the
researcher will focus on describing some advantages and disadvantages of these two
types.
A holistic marking scheme requires the rater to make an “impressionistic
assessment” of the quality of students‟ oral test using a single marking scheme
(Davies, Brown, Elder, Hill, Lumley and Mc-Namar, 1999, p. 75). A holistic
marking scheme is commonly used in assessing L2 speaking and it is described as
“general impression marking” in which the overall properties are more important
than particular features of the performance (Association of Language Testers in
Europe, 1998, p. 147). Using holistic marking schemes can be relatively quicker
than analytic marking schemes (Nitko, 2001). However, the holistic marking
schemes do not provide detail information on student performance for each criterion
and they are not very useful to help plan instruction because they lack a detail
analysis the strengths and weakness of student‟s product (Taylor, 2011, p. 178).
An analytic marking scheme gives separate scores for each criterion of performance.

Each criterion is assessed separately and using different descriptive marking scheme
(Taylor, 2011, p 179). Analytic marking scheme results initially in many scores,
followed by a summary total score representing an assessment on a
multidimensional level (Mertler, 2001). Using an analytic marking scheme helps to
focus rater judgments more narrowly which contribute to rater agreement and rating
reliability (Weir, 1990). Scoring tends to be more consistent across students and
grades. It provides students meaningful and specific feedback on area of strength
and weakness (Nitko, 2001). Besides these advantages, using analytic marking
schemes also has some disadvantages. The use of an analytic marking scheme can

16