Evaluation of Career Exploration project in College Success course through students’ change in attitudes and knowledge A case study = Đánh giá dự án Khám phá ng

VIETNAM NATIONAL UNIVERSITY- HANOI
UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES
FACULTY OF POST- GRADUATE




NGUYỄN THỊ CHI


EVALUATION OF CAREER EXPLORATION PROJECT IN
COLLEGE SUCCESS COURSE THROUGH STUDENTS’
CHANGE IN ATTITUDES AND KNOWLEDGE:
A CASE STUDY

(Đánh giá dự án Khám phá nghề nghiệp trong chương trình Thành
công học đường qua sự thay đổi thái độ và kiến thức của sinh viên:
Nghiên cứu cá thể)

M.A THESIS

Field: English Teaching Methodology
Code: 601410


Hanoi - 2012
1

VIETNAM NATIONAL UNIVERSITY- HANOI
UNIVERSITY OF LANGUAGES & INTERNATIONAL STUDIES

FACULTY OF POST- GRADUATE



NGUYỄN THỊ CHI


EVALUATION OF CAREER EXPLORATION PROJECT IN
COLLEGE SUCCESS COURSE THROUGH STUDENTS’
CHANGE IN ATTITUDES AND KNOWLEDGE:
A CASE STUDY

(Đánh giá dự án Khám phá nghề nghiệp trong chương trình Thành
công học đường qua sự thay đổi thái độ và kiến thức của sinh viên:
Nghiên cứu cá thể)

M.A THESIS

Field: English Teaching Methodology
Code: 601410
Supervisor: Dinh Hai Yen, M.A


Hanoi - 2012
5

TABLE OF CONTENTS
CANDIDATE‟S STATEMENT …………………………………………… ii
ACKNOWLEDGEMENTS …………………………………………………iii
ABSTRACT …………………………………………………………………iv

TABLE OF CONTENTS …………………………………………………….v
LIST OF TABLES & FIGURES ………………………………………… .viii
PART A: INTRODUCTION……………………………………………… 1
I. Statement of the problem and rationale for the evaluation… ……… 1
II. Scope of the study ………………………………… ……………….4
III. Aims of the revaluation and research questions …………………….5
IV. Significance of the study………………………………………………6
V. Methods of the study ………………………………………………….6
PART B: DEVELOPMENT ……………………………………………… .8
CHAPTER 1: LITERATURE REVIEW ………………………………….8
1.1. Career exploration and related terms … ……………………………8
1.2. Importance of career exploration …………………………… … 10
1.3. Research on career exploration …………………… …………… 11
1.3.1. Correlation studies ……………………………………………11
1.3.2. Evaluation …………………………………………………….15
1.3.2.1. Significance and framework of career program
evaluation …………………………………………………… 15
1.3.2.2. Types of career exploration and findings of specific
evaluation …………………………………………………… 19
CHAPTER 2: RESEARCH METHODOLOGY……………….……… 32
6

2.1. The context………… …………………………………………… .32
2.2. Participants …………………………………… …………………33
2.3. Data collection methods and procedures… ………………………. 36
2.3.1. Questionnaire ………………………………………………….36
2.3.2. Test ………………………………………………………… 37
2.3.3. Interview ………………………………………………………38
2.4. Data analysis ………………………………………………………. 39
CHAPTER 3: DATA ANALYSIS AND DISCUSSION ……………… 41

3.1. Single-case analysis and discussion ……………………………… 41
3.1.1. Case 1 (A) ………………… …………………………….41
3.1.1.1. Background of the case …………………… 41
3.1.1.2. Changes in attitudes ………………………… 44
3.1.1.3. Changes in knowledge …………………………… 48
3.1.2. Case 2 (L) ……………… ………………………………… 51
3.2.1.1. Background of the case ……………………………….51
3.2.1.2. Changes in attitudes ………………………………… 54
3.2.1.3. Changes in knowledge ……………………………… 59
3.1.3. Case 3 (H) …………………………………………………… 61
3.3.1.1. Background of the case ……………………………… 61
3.3.1.2. Changes in attitudes ………………………………… 64
3.3.1.3. Changes in knowledge ……………………………… 69
3.2. Cross-case analysis and discussion .…………………………… …71
PART C: CONCLUSION ……… …………………………………… 78
1. Summary of the study …………… …………….……………………78
2. Contributions of the evaluation ……………….…………………… 79
3. Limitations of the study ………………………………………………80
7

4. Suggestions for further study…………………………………………80
REFERENCES ……………………………………………………… 81
APPENDICES………………………………………………………………I
Appendix 1A: Questionnaire for choosing cases (English version)……….I
Appendix 1B: Questionnaire for choosing cases (Vietnamese version)… II
Appendix 2A: Pre-Questionnaire (English version) …… ………………III
Appendix 2B: Pre-Questionnaire (Vietnamese version) ……………… VII
Appendix 3A: Post-Questionnaire (English version) ……………………XII
Appendix 3B: Post-Questionnaire (Vietnamese version) … ………… XVII
Appendix 4A: Pre-Interview scheme (English & Vietnamese version) XXII

Appendix 4B: Post-Interview (English & Vietnamese version) …… XXIII
Appendix 5A: Pre-Test (English version) …… …………………… XXVI
Appendix 5B: Pre-Test (Vietnamese version) ……………………… XXVI
Appendix 6A: Post-Test (English version) …… ………………… XXVIII
Appendix 6B: Post-Test (Vietnamese version) ………………………XXIX


8

LIST OF TABLES & FIGURES
Figure 1: The general scope of educational evaluation …………………….16
Figure 2: Process of career decision-making ……………………………….20
Table 1: Framework of holistic program evaluation ……………………….16
Table 2: Results of reason survey ………………………………………… 35
Table 3: A‟s changes in attitudes ………………………………………… 44
Table 4: Development of A‟s attitudes from her perception ……………… 47
Table 5: A‟s changes in knowledge………………………… ………… 49
Table 6: Development of A‟s knowledge from her perception …………… 51
Table 7: L‟s changes in attitudes ……………………………………………54
Table 8: Development of L‟s attitudes from her perception ……………… 59
Table 9: L‟s changes in knowledge …………………………………………59
Table 10: H‟s changes in attitudes ………………………………………….65
Table 11: Summary of changes of three cases …………………………… 72

9

PART A: INTRODUCTION
1. Statement of the problem and rationale for the study
Evaluation is a tool which helps systematically identify the „merit‟ or
„worth‟ of a program so as to reduce „uncertainty in decision‟ and „facilitate

positive change‟ for advantages (Mertens, 1999, as cited in Mertens, 2005,
p.48). In other words, program evaluation is really critical; however,
according to several scholars (Poel, 2009; Royse, Thyer, and Padgett, 2010) it
has rarely been undertaken. The reasons for avoiding it have not been
officially revealed by researchers, but they may include the concern that
conducting evaluations take time and attention away from the course content.
The situation may be the same with University of International Languages
and Studies (ULIS), where a large number of courses are internally designed
by its staff, but just a few evaluations of their programs and projects have
been undertaken. As such, it is considerably beneficial to take evaluative
research into consideration for the case of ULIS.
„College Success: Taking Charge of Your Personal and Academic
Growth‟ is a new compulsory course specifically designed for first-year
students at ULIS. So far the course has run for three years. During the course,
students are expected to participate in a variety of practical tasks that promote
a number of necessary academic and soft skills (Study skills, 2009; Phung et
al., 2011). The content of the course was revised once in 2010 when the task-
force recognized that the fundamental objectives of version 2009 were to
provide students with opportunities to practise some vital skills such as note-
taking, citing, and presentation skills while most of the tasks were like
reading exercises. Specifically, students were required to read some texts and
then answer some questions as a follow-up activity, which does not reflect the
principle of learning by doing stated in the course description. More
10

importantly, the original course just focused on the academic skills which are
integrated in other courses. For example, the Reading course does involve
practising note-taking skills; the Writing course does involve practice on
referencing and avoiding plagiarism. The „tasks‟ designed for the original
version of College Success are somehow the same as the exercises in the two

courses mentioned above. Version 2010 is similar to version 2009 in that it is
for first-year students to develop a number of essential skills so that they can
better understand themselves and their environment, but it bears remarkable
differences. The first difference lies in the objectives. Besides training in
academic skills; the new version creates opportunities for students‟ personal
growth and provides them with career orientation. The second difference lies
in the task process. Students are not required to do the exercises but take
action in making some specific products such as posters, talk shows, and job-
related reports. The practice of the needed academic skills is integrated into
the process of making these products. For instance, the note-taking skills are
integrated into the task of creating a poster of a role model. In so doing,
students can practise and/ or revise one of the academic skills – note-taking
(select what information to be displayed on their poster). At the same time,
they have an opportunity to share their own stories about what they truly love
and value in their life. Whereby, they can be aware of various values in
human life and the needs of putting efforts into studying and working for
great achievements. Actually, it is impossible for students to acquire and
master all the skills expected in this course; students need time to absorb and
put them in practice for further improvement. Thus, the fundamental aim of
the course is to raise students‟ awareness of some necessary skills that they
need for both academic purposes and personal growth. As such, the new
version is hoped to work better with the given aim. Until 2011, the original
11

course title, Study Skills, has been changed into „College Success: Taking
Charge of Your Personal and Academic Growth,‟ which better reflects the
nature of the new version of the course.
Within the framework of the course, there are two projects which take
place at the same time during the second half of the course – Career
exploration and Dream jobs. Students are divided into groups and asked to

choose one of the two projects for their process. While Dream jobs creates an
opportunity for students to better understand the job they have wished to do
in the future by expressing their dream to the others, Career exploration is a
tool to enrich students‟ knowledge of the various career options available to
them and suitable to their major. Further, the projects both help students be
self-aware of their personality traits, work-related strengths and weaknesses,
have a sense of cooperative work, and practise a number of academic skills,
including information searching, processing, presentation, and teamwork
(Phung, et al., 2011a). It is assumed that the students who choose Dream jobs
are clear about the career path they want to follow in the future, whereas there
is a fact that many students, when applying for ULIS, still are not sure about
their expected jobs. These students just study as scheduled and do not seem to
know where to start with their career. In this regard, Career exploration is a
true opportunity for them to kick off their start; therefore, the evaluator would
like to investigate into its impacts on the students who choose this project.
Moreover, Career exploration was implemented for the first time in 2010 and
has informally received conflicting oral comments from students. Some
claimed that the project was interesting, but others complained that they were
burdened with the task and had to spend such a good deal of time for a mark.
As one of the project designers as well as a teacher involving in its process,
12

the evaluator is encouraged to conduct a formal in-depth evaluation of the
project.
2. Scope of the study
According to Scriven (1991), the evaluation is implemented on the
basis of „relevant standards of merit, worth, or value‟ (p.47, as cited in
Mertens, 2005). In other words, the evaluation process normally involves the
comparison between the outcomes with the intended objectives. Royse et al.
(2010) share the same opinion when they categorize evaluation as one type of

research that helps define whether the needs are met or the participants‟ goals
are attained. In the case of Career exploration project, students‟ achievement
is compared to the planned goals in respect of attitudes, knowledge, and
skills. The first two goals are those expected to be acquired by students
during their involvement without any lectures, while the third involves the
teaching and practice in class. By the nature of the project, its primary aim is
to promote students‟ changes in attitudes and knowledge in relation to their
possible career path. As such, this study is just to evaluate students‟ changes
in attitudes and knowledge while the evaluation of skills attainment and/ or
enhancement is excluded. As regards attitudes, students are expected to
develop: self-awareness of personality traits, work-related strengths and
weaknesses, activeness and criticality in utilizing sources, and cooperative
sense in teamwork. In terms of knowledge, students are expected to have a
grasp of the important features of relevant jobs and their possible career
paths (Phung et al. 2011, p.23). Any other emerging attitude and knowledge
discovered during the course of the study will also be included.
As a matter of fact, it is difficult for students to become different in
their behaviors right after the project, so achievement or „change‟ in this
study refers to the students‟ awareness of the expected attitudes and
13

knowledge that are assumed by the course designers to be needed for
students‟ learning at college and for their future career. Job performance or
personal adjustment was excluded from this scope. Moreover, in this
evaluation, „change‟ is divided into two levels: „acquire‟ and „develop‟.
„Acquire‟ refers to the attitudes and knowledge that students became aware of
in the course of doing the project. In other words, they had never learned the
„acquired‟ attitudes and knowledge before the project. Meanwhile, „develop‟
refers to the attitudes and knowledge that students had learned from
elsewhere other than from the project, but thanks to the project their

awareness of these attitudes and knowledge was heightened.
Another important matter is how the project and the related evaluation
can be located in the field of English Teaching Methodology. In answer to
this question, the approach on which the designers based should be
mentioned: competence-based approach. In other words, the designers would
like to build up learners‟ competences as the outcome of the course or the
project itself. Competences can be understood as the application of skills,
knowledge, strategies, attitudes, etc. in doing some tasks to a standard. Tasks
in language teaching require learners to pay primary attention to the meaning
and make use of their linguistic sources to process and understand language.
Tasks designed within the project require students to process the provided
guidelines which were written in English in order to make a certain product.
In this respect, language (specifically English) serves as a means to help
learners achieve some expected outcomes. With regard to this study, it did not
evaluate the achieved competences; neither did it assess students‟ completion
of specific tasks. What was focused on were the attitudes and knowledge
which are considered part of language learning process. Having adequate
attitudes and knowledge benefits students in bettering their learning; being
14

aware of students‟ attitudes and knowledge benefits teachers in helping them
to the fullest. Putting differently, the project itself as well as the evaluation
was considered practical in English Teaching Methodology.
3. Aims of the study and research questions
The first aim of the study is to find out whether students achieve the
goals in terms of attitudes and knowledge in Career exploration project. If
the achievement exists, the specific attitudes and pieces of knowledge are also
expected to be discovered. Also, attempts were made to see whether students
could obtain any other attitudes and pieces of knowledge out of the expected
ones.

In brief, the study seeks answers to the following questions:
1. In terms of attitudes, after completing the project of career
exploration,
a. was there any change in students‟ attitudes?
b. what attitudes did the students acquire?
c. what attitudes did the students develop? To what extent were those
attitudes developed as perceived by the students?
2. In terms of knowledge, after completing the project of career
exploration,
a. was there any change in students‟ knowledge?
b. what knowledge did the students acquire?
c. what knowledge did the students develop? To what extent was that
knowledge developed as perceived by the students?
15

4. Significance of the study
The study is a significant endeavor in investigating the impacts of the
project on students‟ attitudes and knowledge in respect of their career path.
Such an outcome determines whether or not the project is successful and/ or
indicates any improvement. Additionally, the project is not of rigid theories
but the exploration of the real jobs, so it is important for teachers in-charge to
provide proper guidance and support with real-life experience so as to engage
students in the tasks and foster their awareness of the tasks‟ values (Phung, et
al., 2011, p.i). Therefore, understanding of the impacts may help teachers in-
charge to look back at their decision on guidance and support for the further
improvement of the project.
5. Methods of the study
Case study was employed in this study which was to discover
individual students‟ overall attainment in terms of attitudes and knowledge
after they accomplished the project. It can be argued that in order to evaluate

a project, an overview of as many participants‟ achievement should be drawn
as possible. However, the goal of the project is to help students partly develop
their personal growth (Phung et al., 2011) and individuals are different;
accordingly, the fact that a person can, to a certain extent, grow thanks to the
project also proves its certain success. As a result, case study is considered
adequate for the evaluation.
Three cases for the evaluation were selected among the first-year
students who attended College Success Course in the second semester of the
academic year 2011 – 2012 of ULIS. In order to investigate students‟ change
in attitudes and knowledge, three instruments namely questionnaire, test and
interview were used and delivered to students both before and after the
project.
16

PART B - DEVELOPMENT
CHAPTER 1 – LITERATURE REVIEW
1.1. Career exploration and related terms
The career guidance, according to Hughes and Karp (2004), dates back
to the 20
th
century when vocational education was strongly supported and
promoted. These scholars also acknowledge that Frank Parson, father of the
vocational guidance movement, was the first person to follow scientific and
systematic model of vocational assessment:
In the wise choice of a vocation, there are three broad factors:
(1) a clear understanding of yourself, your aptitudes, abilities,
interests, ambitions, resources, limitations, and their causes; (2)
a knowledge of the requirements and conditions of success,
advantages and disadvantages, compensation, opportunities, and
prospects in different lines of works; (3) true reasoning on the

relations of these two groups of facts (Philips & Pazienza, 1998,
p.2, as cited in Hughes & Karp, 2004).

Basing on this model, career guidance was first designed as the
assessment of individual differences by „experts who would determine the
person‟s best occupational direction (Philips & Pazienza, 1998, as cited in
Hughes & Karp, 2004). Not until 1920s and 1930s did the role of counseling
start, and so far it has been a trend in career orientation. The Comprehensive
Guidance Program was developed in the early 1970s and a lot of reforms
have been implemented since then.
Career orientation is a general term that refers to any types of activities
to help students have better understanding of themselves and the world of
work and/ or prepare them with essential occupational knowledge and skills.
This term and the others, such as career development, career education, career
guidance, and career counseling, can be used interchangeably (Maddy-
Bernstein, 2000, cited in Hughes & Karp, 2004) although they may be used
17

differently by various programmers. For example, career guidance is often
understood as „an umbrella term‟ (Hughes & Karp, 2004) encompassing
many services for students‟ personal and professional growth while career
counseling is only one portion of career guidance, focusing on the interaction
between trained professionals and students so as to help them develop their
career path(s). Accordingly, career orientation can be divided into two
categories: one is career counseling or advising and the other is interventions.
The former includes the activities that provide direction and planning skills
for students, and the latter refers to career courses designed to promote
student knowledge and skills relevant to the world of work.
One emerging term in the late 1970s from the theory of career choice
and counseling is career exploration which is considered as a type of

„information-seeking behavior‟ or as a „career problem-solving behavior‟.
Along with its development, career exploration has been referred to as „a
complex psychological process, which sustains the search of information, as
well as hypothesis testing about self and environment, in order to attain career
goals‟ (Hall, 1998 cited in Zikic & Hall, 2009; Taveira & Moreno, 2003).
Putting differently, career exploration is considered to be a critical step in the
career decision-making process.
Many other terms which can be used in the field of career education
and guidance as suggested by Athanasou (2007) include: “group counseling,
computer-assisted guidance, one-to-one counseling services, career advisory
services, telephone career information and advice, employment services,
career information, recruitment programs, supported employment programs,
human resources development programs, outplacement services,
rehabilitation programs, vocational assessment services, and vocational
guidance” (p.3). However, whatever term is used and although the
18

implementation of specific programs associated with each term is different,
the overall objective of these programs is to involve individuals in the
exploration of the world of work as well as offer them a chance to identify
their own potentials in certain career paths.
1.2. The importance of career exploration
There is no wonder that career exploration is of particular importance
in eliminating initial confusion about what a person should prepare in terms
of academic and vocational concerns and what occupations he/ she can follow
after graduation. Once things are made clear at the very beginning, he/ she
can better plan towards the goal. According to Porfeli & Skorikov (2009), the
selection of a suitable career is presumed to depend heavily on the nature of
the orientation program and the extent to which a person engage in it (p.47).
Schneider & Stevenson (1999) also assert that career development is „vitally

important for today‟s youths‟ (Hughes & Karp, p.2, 2004). They argue that
the youths now have been more motivated and ambitious for their education
and profession, but many of them have been directionless; hence the demand
for purposeful guidance has increased. Occupational Outlook Quarterly
(1999) shares the view that students often wonder why they need to study the
subjects that seem unrelated to their career goals and that they often feel
confused when it comes to making a decision on their career path. Although
career exploration takes time, many believe that it brings positive impacts on
the determination of students‟ future plans. Zikic & Hall (2009) posit that
during the process of career exploration the individual is likely to become
more self-aware of his or her likes and dislikes and possibly figure out the
focus of his or her exploration. Accordingly, the person‟s adaptability to the
world of work can be achieved and/ or increase (p.186). These scholars also
discuss the outcomes of career orientation during a person‟s employment, at
19

which time it helps him or her to discover more occupational opportunities
and become more employable with the increase in adaptability (Fugate,
Kinichi & Ashforth, 2004; Savickas, 2007).
1.3. Research on career exploration
Two most popular trends of research on career exploration, up till now,
have been correlation studies and impact evaluation. Within the scope of this
study, only a small number of examples of correlation studies concerning
career exploration were presented along with significant findings, whereas
most of the space was dedicated to the summary of career program
evaluation.
1.3.1. Correlation studies
One example of correlation studies is „Five-factor model of personality
and career exploration‟ by Reed, Bruch, Haase (2004). The researchers
examine whether there is any relation between the dimensions of the five-

factor model (FFM) of personality namely neuroticism, extraversion,
openness to experience, agreeableness, conscientiousness and specific career
exploration variables which include self-exploration, career information
seeking, stress regarding career exploration, career search self-efficacy
(p.224-225). The correlation study was conducted with the sample of 204
undergraduates at the average age of 20, including 2% freshmen, 46%
sophomores, 44% junior, and 8& seniors. The participants were required to
complete a questionnaire booklet comprising a form to assess the five
personality factors and a series of items to assess the relation between those
factors and career exploration variables. Results from the correlation analysis
revealed the association between conscientiousness/ extraversion/ low
neuroticism and career search self-efficacy/ career information seeking. The
results also proved that openness was associated with a lack of career
20

information seeking and that neuroticism/ openness was related to self-
exploration. In this regard, counseling experts are advised to consider using
the FFM when engaging in career counseling because it can be “a useful tool
for teaching clients how their personality style might facilitate or inhibit their
career development and assessment” (Costa, McCrae & Key, 1995, as cited in
Reed, Bruch, Haase, 2004, p.235). This tool can also help counselor in
optimizing the clients‟ career self-efficacy by facilitating good conditions for
beneficial factors or providing more detailed plans and guidance to minimize
the impacts of detrimental factors. However, the investigators do not intend to
generalize such relationship between personality factors and career
exploration variables to other population and suggest that additional research
is needed (p.235-236).
Nauta‟s (2007) also investigates into the relationship between the five
big personality dimensions and college students‟ career exploration.
However, the investigation of these two sets of variables was not

simultaneously; instead the investigator scored the five big personality
dimensions of the participants and used the results to predict their career
exploration behaviors (self-exploration and environment exploration). After
approximately 18 months, the participants were re-contacted and do a career-
exploration self-report which was put into comparison with the previous
prediction. The so doing revealed that none of the predictors was associated
with subsequent environmental exploration and that the findings related to the
association between personality traits and self-exploration were not totally
consistent with those in Reed et al. (2004). Specifically, openness was still
seen to be positively associated with self-exploration while extraversion was
negatively associated with such exploration. This raises a wonder whether
extraversion just has short-term benefits in promoting exploration. In
21

addition, the relationship between career interests and self-efficacy (which, as
proposed by Holland‟s (1997), include realistic, investigative, artistic, social,
enterprising, and conventional) and career exploration behaviors was
examined with the same procedures. The results showed that whereas
realistic, artistic, and conventional interests and artistic self-efficacy had
positive association with self-exploration, investigating and enterprising
interests were negatively associated with such exploration. „Recognizing such
tendencies early in students‟ careers would allow for interventions to be used
to increase the likelihood that they will engage in self-exploration prior to
committing to a college major or career‟ (Nauta, 2007, p.176). Moreover,
understanding of the mentioned tendencies, career counselors and/ or support
staff could design suitable activities to facilitate self-exploration among
students.
Another example of correlation studies concerning these issues is
„Personality traits, vocational interests, and career exploration: a cross-
cultural comparison between American and Hong Kong students” by Fan,

Cheung, Leong, and Cheung (2012). This study also discusses the
relationship between the personality predictors and career exploration, but it
was put in the context of two different cultures (Western and Eastern
counterparts). For such a feature, Fan at el. (2012) did not employ the five-
factor model to assess personalities; instead they based on the Cross-cultural
(Chinese) Personality Assessment Inventory-2 (CPAI-2), which consists of
four factors namely Social Potency/ Expansiveness, Dependability,
Accommodation, and Interpersonal Relatedness. To a certain extent, these
four factors are similar to five dimensions of the five-factor model; however,
the investigators would like to use a tool that could cover indigenous
personality dimensions of Eastern culture (Hong Kong students). In
22

particular, „the IR (Interpersonal Relatedness) factor assesses the aspect of
formalistic relationship orientation in collective societies. The coverage of IR
is broader than the meaning expressed by Agreeableness in the Big Five and
includes the qualities of humanity and social propriety in Confucian
philosophy.‟ (p.109). As regards vocational interests, the scholars did not
merely examine the relation between different types and career exploration
behaviors as Nauta had done, but hypothesized that vocational interests
mediated the association between the personality and career exploration.
Their thorough analysis revealed that the CPAI-2 indigenous personality
dimensions derived in the Chinese cultural context predicted career
exploration of Hong Kong students only. Significant cultural differences were
also found in the contributions of the other two universal dimensions,
Accommodation and Dependability. The Accommodation personality factor
was more important to career exploration in Hong Kong context whereas
Dependability was more important in American context. Furthermore, there
existed certain mediation. Enterprise interest type mediated the association
between Social Potency and career exploration in Hong Kong sample;

Artistic interest type was the mediator in the American sample. The findings
of this investigation raise significant awareness of cultural differences in
career guidance and the possible mediation of vocational interests in the
contribution of personality traits to career exploration, possibly leading to
better design of counseling plans and/ or career courses.
1.3.2. Evaluation
1.3.2.1. Significance and framework of career program evaluation
Evaluation of career program outcomes is crucial as it can help to
better the effects of wider-scale applications as well as limit the unexpected
impacts on the participants. According to Reese and Miller (2010, p.107), the
23

use of outcome data can serve as an important catalyst for improving career
interventions. So as to prove their argument, the scholars conducted a two-
year study to assess whether modifications made to the course using the
previous year‟s outcome data as a baseline resulted in subsequent
improvements. The study used a pre-post group design that compared
students in a career course to a quasi-control group. In both years of making
modification and examine the related outcomes in comparison with those of
the original course, the scholars found that there was increase in students‟
career decision-making self-efficacy. Although the number of participants
was small (35 students enrolled in the career course and 38 students were of
quasi-control group), the consistency of the results across both years appeared
to minimize such limitation. In a broader sense, outcome evaluation of a
career course can bridge the gap between research and practitioners in terms
of providing career guidance. In addition, Athanasou (2007) suggests that
evaluation can help to recognize unintended effects, for example social
enjoyment, group cohesion or work escape (p.3), whereby certain
modification can be decided. The scholar summarizes the general scope of
program evaluation as follows:

24

Figure 1: The general scope of educational evaluation
(Athanasou, 2000, as cited in Athanasou 2007, p.4)
Anathasou also suggests a model as a holistic evaluation framework for
career education and guidance, including six steps: ethics, coverage, cost,
objective(s), effects, and stakeholders. The detailed description of those steps
can be seen in the following table:



25


Table 1: Framework of holistic program evaluation (p.29)
The above framework, from my perspective, is not only useful for
those who intend to have a holistic look at a program, but it is also useful in
determining evaluation design for each dimension. As regards the fourth step,
Kirkpatrick (1996) provides the hierarchy of evaluation, starting with
reaction, learning, behavior and moving up to results:
- Reaction – program participants‟ estimates of satisfaction
provide an immediate level of evaluation
- Learning – the extent of learning (i.e., skill acquisition,
attitude change) that had been achieved is also of interest
- Behavior – the extent to which learning has generalized to
the work situation or there has been a transfer of skills is
assessed at this level through follow-up after a program
- Results – the wider impact of a program in the community or
the organization is observed at this level of the hierarchy

(p.25-26)
Although there are documented criticism of the validity of the
hierarchy and the relationship between the levels (Anathasou, 2007, p.27),
this is still considered a source of reference when researchers decide the
scope and the design of their evaluation. In particular, the evaluation of
Career Exploration project in ULIS did not aim at whether they transferred
the skills learnt from the project; hence field observation was not really
necessary for the evaluation. It was also not an evaluation of the wider impact
26

in the community, so the evaluation could be limited to certain goals instead
of looking at each and every aspect of the project.
Regarding the effectiveness of career interventions, Brown & Krane
(2000) did not discuss how to determine the effect as Athanasou (2007), but
suggested what should be included as important predictors. According to
them, in order to evaluate the impact of career courses, three out of the
following five components should be assessed: whether the courses (a) allow
clients to clarify career and life goals in writing; (b) provide clients with
individualized interpretations and feedback; (c) provide current information
on the risks and rewards of selected occupations and career fields; (d) include
study of models and mentors who demonstrate effective career behavior; and
(e) assist in developing support network for pursuing career aspirations
(Folsom and Reardon, 2003, p.426).
Another scheme to evaluate career interventions is based on outputs
and outcomes (Burck, 1978; Burck & Peterson, 1975; Peterson & Bruck,
1982; Peterson, Sampson & Reardon, 1991). In this scheme, outputs refer to
the skills, knowledge and attitudes acquired by participants as the result of an
intervention whereas outcomes refer to the effects of course satisfaction,
deciding on a major and timely graduation from college (Folsom & Reardon,
2003, p.427)

1.3.2.2. Types of career exploration and findings of specific
evaluation
As stated above, career development has received great concern from a
lot of stakeholders. Accordingly, various forms of career development have
been in existence namely career-oriented articles and books, online career
decision-making tools, workshops, and career courses. Some of theory books
relating different careers have been easily found on the Internet, for example
27

Career Ideas for Kids Who Like Computer by Diane Lindsey Reeves and
Peter Kent. In Vietnam, a series of books by Kim Dong Publisher introducing
the overview of certain types of jobs have also been available. Career-related
work books such as Career Plan Project Workbook by McGraw-Hill
Companies have been delivered. In the following part, a review of
effectiveness of three most popular forms of career exploration, Internet-
based career exploration and career courses, will be provided.
1.3.2.2.1. Internet-based career exploration
Up till now, a large number of universities and/ or institutions in other
countries have designed useful online tools and offline programs to help
students become aware of their own strengths and weaknesses, get to know
clusters of available jobs, as well as figure out whether or not their interests
and aptitudes are suitable with certain careers. One example of online career
decision-making tools can be found on the website
by America‟s Career Resource Network.
This tool serves to help teachers, counselors, parents assist their children as
well as help students themselves in identifying an appropriate career direction
and selecting or developing an education plan that will provide the
knowledge and skills needed to succeed within their chosen career field. The
process of career decision making goes through six steps as follows: