ASSESSMENT OF SERVICE QUALITY AND CUSTOMER SATISFACTION AT MILITARY BANK

VNU- UNIVERSITY OF ECONOMICS AND BUSINESS
FACULTY OF BUSINESS ADMINISTRATION
---------------------------------------

ASSESSING THE USABILITY OF MARKUS WEBSITE
BY
HOANG THI CHANG

A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE BACHELOR DEGREE OF BUSINESS
ADMINISTRATION
(INTERNATIONAL STANDARD PROGRAM)

Hanoi, May 2016
VNU- UNIVERSITY OF ECONOMICS AND BUSINESS
FACULTY OF BUSINESS ADMINISTRATION
---------------------------------------

ASSESSING THE USABILITY OF MARKUS WEBSITE
BY
HOANG THI CHANG

A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE BACHELOR DEGREE OF BUSINESS


2

ADMINISTRATION
(INTERNATIONAL STANDARD PROGRAM)

Hanoi, May 2016
VNU- UNIVERSITY OF ECONOMICS AND BUSINESS
FACULTY OF BUSINESS ADMINISTRATION
---------------------------------------

ASSESSING THE USABILITY OF MARKUS WEBSITE

Supervisor’s name: Dr. Nguyen Thi Huong Lienn
Student’s name: Hoang Thi Chang
Student’s ID: 11050013
Intake: QH2011-E
Program: International Standard Program

Hanoi – May 2016


3

ACKNOWLEDGEMENTS
First and foremost, I wish to express my sincere thanks to my advisor Dr.Nguyen Thi
Huong Lien, who was thoughtfully guiding me in writing this thesis. Besides, I want to
express my grateful to director and staff at faculty of business and administration who
have support me in this thesis.
Secondly, I would like to thanks Markus School that gave me an internship. I
especially thank Mr. Do Xuan Khoa who was directly introducing me the website
usability and all aspects of Markus School. Besides, I thanks to all staffs of Markus
School for their whole-hearted guides during my internship in Markus School. Last but
not least, I’m grateful to the numerous people who participated in my research.
Student
Hoang Thi Chang

Table of Contents


4

LIST OF TABLE
1.

a.
b. LIST OF FIGURE
2.
3.

4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.

20.
21.
22.
23.

24. ABSTRACT
25. Internet is much more important nowadays which new business environment is.
Every entrepreneur has its own website to present themselves, communicate and
cooperate with stake holder. An efficiency website contributes to the success of
entrepreneur. This study aim to evaluate the usability of Markus website on
user’s perception to learn how satisfaction the website serve user. The usability
would be studied based on Wammi model with two tools: questionnaire
evaluation and performance evaluation. The result of this study will answer how
well it satisfy customer, it meet the expectation of Markus. The result of the
study show that Markus website score a good point at usability. Base on, the
result, the researcher have recommend three point as suggestion for website
development. This suggestion is developed logically to improve limited point
and enrich the good point at Markus website. Although the research solve
research problem, answered the big and sub research questions as well as


5

achieve the purpose of this study, it still have several limitation that can improve
in the future study.
26.
27.
28.
29.
30.

31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.

43.
44. INTRODUCTION
45. Internet in Vietnam was formed and developed in 1997, since then, Vietnam has
always been the world rated as one of the countries with the fastest annual
increase in the number of internet users. The rushing speed development boots
up the e- commerce environment in Vietnam. According TO VECITA
(Vietnam’s e-commerce and information technology agency), in 2014, B2C ecommerce sales amounts is more than $ 2.97 billion. Such a rapid increase have
gain a lot attention of entrepreneurs. More and more company or entrepreneur
have invested in e-commerce environment. One of good example is the
increasing number of website in Vietnam. The internet has modified the
traditional communication process between organization and stakeholders
(Mentes & Turan, 2012). For organization, the web gives access to a lot of
people and enhances functional performance. Web sites are becoming key
factors of an organization's success in the global market. According to VECITA,
2014, 69, 7 % active user use internet for educational website every day in
which 59 % active website in Vietnam belongs to health, education and training
website. Therefore, education website is not an exception. Academic website not

only means technique to connect with various stakeholder but also means for an
institution to shape its image. Gordon and Berhow (2009) claims that most of
academic institutions use their website as a key public relation and marketing
tool to reach potential students. Whether students would apply to institution, the
final decision may base on how useful and comprehensive information on


6

institution’s website is. On the other word, the customer generally satisfied with
institution if the website can help them achieve their goal in a specific context.
Otherwise, they would move to competitive complement. Therefore it is pivotal
to evaluate the satisfaction of customer in general and student in academic
institution in detail on website operation which attract customer at the first sight.
Aldwyn (2013) argues that usability is one factor that satisfies user when surfing
internet or access website. Web design must conform to specific user needs, and
must also make sure that users are satisfied to successfully complete tasks with
web without facing any difficulties (Yan & Guo, 2010). Therefore, in such a
rush development and competitive e-commerce currently, it is really important
to satisfied customer firstly through website which mean, the firm or institution
should pay more attention to evaluate and improve the website usability. It is
much more important to education industry since more and more active user
choose distance or e- learning as a money –time saving education, otherwise
there are no more inactive user who only know to find information directly in
representative office. Nowadays, internet booming save their time and make it
easy to assess information via website, social network. In case of Markus
School which is an academic institution, where they ignite new marketing
thinking and train marketing skills. Their website means brand image, present
their updated course information and useful marketing blog which support their
student. This website is technique to connect lecture, student an operational

support. Enter education market have been 3 three years, however their website
have been invested for one year with the aim of enhancing business. Therefore,
it is necessary to assess the usability of website to be sure whether or not the
website have done well in term of satisfying customer. The result is also the
foundation for next investment: e-learning.
a. Purpose of study
46. In context of this study, the researcher want to evaluate the usability of Markus
website based on user’s perspective then use the result to suggest the
improvement method which increases the usability as well as satisfaction of user
at present when they access Markus website and foundation for future feature
development. To achieve that purpose, the researcher must firstly understand
deeply what usability is and its elements. From investigated literature, the
researcher would assess the usability of Markus website to know how easy the
user feel when the use Markus website based on usability element and practical
accession. From user’s perspective, based on their feeling of Markus website,
the researcher would recommend some method to improve and raise the
usability of Markus website for user based on their strategy in future.


7

a. Research questions
47. In order to achieve the purpose of this research, this study would come to
answer on big question and some sub questions.
48. Big question: “How is the usability of Markus Website?”
49. Sub questions:
50. What is the element to measure the usability in general and website usability?
51. How to measure usability and website usability?
a. Object of study
52. In this thesis, researcher only focus on usability of website on user’s

perspective. A website includes one or more web page that have a common
theme such as people, organization, business (Nur, Adzhar, Norrozila, 2013).
There are many methods, elements to evaluate website. Usability is one of the
major factor contribute to the successfulness of a website. There are a lot of
definition or term about usability. In ISO 9241 – 11 (1992), usability is “the
extent to which a product can be used by specified users to achieve a specified
goals with effectiveness, efficiency and satisfaction in a specified context of
use”. According to HCI (Human computer Interration), usability is designing a
computer system that support people to carry out activities productively and
safely. This definition reflect a change that focus shift to human. Whatever the
aim of website is, but final goal is communicating effectively the information of
content to customer. Therefore, a website design firstly base on customer’s need
and the website only successful if its usability satisfy actual user. On the other
word, assessing usability on user’s perspective reflect the true quality of
website.
53. The study could only focus on Hanoi area in which, the researcher concern
mostly of Student experience and staffs at Markus, who response for building up
website.
54. Markus has start for more than 3 years. However the website was built for about
1 years to meet the demand of development. In this thesis, all data of usability of
Markus website is only applied in year from staring time.


8

a. Task/ Process
55. This thesis has 3 main steps: (1) secondary research (2) primary research (3)
analysis and summary. In secondary research, the researcher investigate
conceptually to understand research problem and figure out the solving method.
In Primary research step, the research applied theory and follow a proven

method to approach objective or respondent to collect primary data. Finally,
research combine the result of 2 previous step to answer the researcher
questions.
a. Contribution
56. By conducting this thesis, the researcher want to gain attention of developer and
business organization about new trend in website design, which is focusing on
the user’s perspective, taking user-experience to be cornerstone. The result will
contribute a comprehensive conceptual of usability for academic researchers and
a easy and cheap method for assessing usability. This method can be easy to
apply by all kind of business, special start up.
57.
58.
59.
60. CHAPTER1: LITERATURE REVIEW
a. 1. 1 Usability
i. 1.1.1 Concept and element of usability
61. Usability is simply the “ease of use” which is a catch phrase that describe a

product that work better for user (Whitney Quesenbery, 2001). Usability can be
applied in numerous objects, such as: software application, book, website, tool,
machine, process, etc. In different context usability has a different specified
definition, however all of them focus toward make product and system easier to
use and match them closely to user’s need and requirement.
62. The term ‘usability’ was introduced in the early 1980s to refer to what was then
a number of vague and subjective attributes of a product, collectively known as
“user friendly elements” (Bevan et al. 1991). However, the cornerstone of
usability conceptual is usability standards provided by (ISO) the International
Standards Organization which has two main categories: (1) product-oriented
standards (ISO 9126, 2001; (2) process-oriented standards (ISO 9241,
1992/2001; ISO 13407, 1999). In ISO 9241-11, 1992 usability is defined as:

“The extent to which a product can be used by specified users to achieve
specified goals with effectiveness, efficiency and satisfaction in a specified
context of use”. ISO 9241-11 explains how to identify the information that it is


9

necessary to take into account when specifying or evaluating usability in terms
of measures of user performance and satisfaction. It also explains how measures
of user performance and satisfaction can be used to measure how any
component of a work system affects the quality of the whole work system in
use. In this definitions, the element of usability are: “effectiveness, efficiency
and satisfactions”. It focus more on human interaction for software product. In
the field of Software Engineering (SE) in ISO 9126-1, usability is defined as
“the capability of the software product to be understood, learned, operated,
attractive to the user, and compliant to conditions”. It has 6 main elements:
“functionality, reliability, usability, efficiency, maintainability and portability
and more sub elements: understandability, learnability, operability, attractiveness
and usability compliance. Usability is not barely functionality and increasing
functionality does not mean improved usability. You should focus on 3 usability
component: user, their goal and usage situation which is affected to improve
usability.
63. Base on ISO 9126 AND 9241, some researcher has created their own model
including different usability element. For example, Benbunan-Fich defined
usability as “how well and how easily a user, without format training can
interact with an information system of a website”. In context of website,
Benbunan-Finch use learnability, efficiency, aesthetics, navigation, content,
functionality, accuracy, consistency, technical adequacy, help and
documentation and error removal for his usability. Similar, in 2005, in term of
digital library, Judy Jeng use effectiveness, efficiency, satisfaction and

learnability as usability elements. Usability consultant Jakcon Nielsen and
computer science,Professor Ben Shneiderman has define usability as usefulness,
which composed of learnability, efficiency, memorability, errors, satisfaction. In
this definition, Nielsen concentrated on memorability, which related to cognitive
aspects, and highlighted error tolerance, a elements that some authors neglected.
64. Beside some common previous elements, some researchers concerned on the
relationships between users and Web technologies (Yu-Hui, Abebe, Carol,
2015). For example, Andreas Lecerof and Fabio Paternò’s definition mentioned
the relevance of a system to the users’ needs, efficiency, learnability, and a
system’s safety features (undo action to prevent error). Whitney Quesenbery
highlighted user experience and affect by including elements: engagement,
pleasure in information seeking and online service context. Furthermore, many
researcher addressed information organization and structure as crucial usability
elements. Jonathan Palmer accentuated website consistency, ease of reading,
information organization, speed and layout. Many experts also regard


10

information organization and structure as crucial usability elements. Jonathan
Palmer emphasized website consistency, ease of reading, information
organization, speed, and layout. In addition, many researcher believed that
content is a core component of website usability. Beside, usability characteristic,
researcher also defined usability testing in a specified environment in which a
particular system is used and tasks undertaken to achieve a user’s goals. For
example Shackel emphasized environment in relation to user support in
fulfilling a specified range of tasks. Tefko Saracevic built an iterative
information retrieval model in which users (with a host of variables of their
own) are related to a situation (task, problem-at-hand) within an environment,
each having a number of elements and dynamics. Nicholas Belkin developed

information-seeking models which consists of three components: the end user,
the information objects, and intermediaries (such as humans, tools, or both) that
support the interaction between the user and the information objects. Another
information-seeking behavior model proposed by Tom Wilson addressed the
interrelation among cognitive, physiological, and affective needs. His model
accentuated the environment factor, focusing on work, sociocultural, politicoeconomic, and physical aspects. Wilson noted that a user’s social role in tandem
with the environment would impact the user’s needs.
65. From precious review, it is clearly to claim that usability is much more pivotal
in human interaction and communication. Usability has various elements in
different filed, which is influenced by many factors: web technology, user,
context, and environment and so on. Day by day when the environment changes
cause usability elements change to adopt a new environment. Nowadays, the
internet booming and technical digital tool development bring the control closer
to user or customer, which cause a change in usability change. There is an
obvious shift toward human interaction as well as user perspective. Usability
only work if it meet user’s need, facilitate user to achieve their goal. Usability is
not only focus on software any more. Specially, web site is use widely for
medium of information and service communication and usability principles was
implementing in web environment and not for software only. User only use the
web site if the web site can achieve their task or goal for searching the
information or using it services more quickly, easily and effectively. This issue
has gain attention of a lot researcher in website usability on user’s perspective.
i. 1.1.2 Website usability concept and its elements
66. Web evaluation has been used in decades to evaluate how well the website

work. Website is a series of page which has a common theme in user, business
or organization. The front page is called home page which composed a lot of


11

hyperlink related to the others webpage. There are many method to evaluate
website performance. However, usability is a key factor whose test result tell
developer and organization how well the website work. From international
standard definition proposed by ISO, many researcher has developed new
concept of website usability. For example, Benbunan – Finch, 2001defined
usability as: “how well and how easily a user without formal training can
interact with information system of website”. This definition developed base on
the key term of usability: “ease of use”. Bazar, 2009 and Jadhav, 2010 has
separately defined that website usability is a quality attribute to assess how easy
user interfaces to use. This definition clarified website usability in two term:
design process and approach to use. In 2013, Samsur Rahman and defined
website usability as an approach to make end-user for using and learning
without requirement training. Usability play a pivotal role in website quality
control. A website is consider usable if it encompass four factors: usefulness,
effectiveness, learnability, user satisfaction (Rubin, 1994). According to
Benbunan- Fich, 2001, the element of website usability are learnability,
efficiency , aesthetics and navigation, content and functionality, accuracy and
consistency, technical adequacy help and documentation and error removel.
Zaphiris and Darin (2001) claimed that web usability as “anyone using any kind
of web browsing technology must be able to visit any site and get a complete
understanding of the information, as well as have the full and complete ability to
interact with the site if that is necessary.” In other words, the web site usability
is a test on the successfulness of web site’s user in doing some task or finding
information in the web site (Yusof et. al, 2010). Mebrate (2010) defined web
usability as the mixed several design goals that make website easy to learn,
remember, and understand find and effective to use. Nielsen, 2012 defined that
‘usability is a quality attribute that assesses how easy user interfaces are to use’.
Nielsen’s definition composed five quality components: learnability, efficiency,
memorability, error, satisfaction. Jabar, Usman, Sidi, 2014 defined website

usability as four main components: Content, organization and readability;
navigation and links; User interface design; performance and effectiveness.
67. There are many website usability definition cover various dimension, however
there are two common elements: (1), ease of use, (2) human interaction. From
1994, Jakob Nielsen has claimed that usability is about basic human capabilities
and user’s need, which do not change nearly as rapidly as technology. He also
claimed that human factors remain the same decade after decade. Therefore, the
use center usability trend is indispensable. More and more research interested in
website usability on user’s dimension.


12

a. 1.2. Usability evaluation method, model and tool
i. 1.2.1 Category of usability evaluation method
68. Usability can be evaluated in different dimensions, such as technology,

developer, and user. There are different types evaluation method used to
examine usability in difference aspects. According to Mack and Nielsen (1994),
the evaluation methods can be classified into four following categories: (1)
Automated – usability was measured by running a user specification through
evaluation software. (2) Empirical – usability is evaluated by testing the
interface with real users. (3) Formal – using models and formulas to calculate
usability level. (4) Informal –usability expert use their knowledge and
experience to assess usability measures. Benbunan-Fich (2001) on the other
hand, clarified usability evaluation methods into four following categories: (1)
Objective performance – measuring how the visitors using the website in terms
of time for task completion. (2) Subjective user preferences – measures the
users’ likability by asking them give opinions or use questionnaire to rate the
system. (3) Experimental – test hypotheses about design and impact on user

performance and preferences based on experiment. (4) Direct observation –
inspect and monitor the users’ behavior while they are interacting with the
system to detect usability problems. Al-Radaideh (2011) simply clarified
usability measure method into 3 categories: (1) The inspection based methods
without end users. (2) The test based methods with end users and the tool based
automated evaluation method. (3) Usability inspection methods often referred to
as professional evaluation where experts’ knowledge is used to assess the
usability of the website, are used to identify usability problems and improve the
usability of interface design by inspecting it against recognize standards.
i. 1.2.2 Model of Usability evaluation
69. A lot of usability testing model was developed for various dimensions.

However, there are only several well-known models which is used and adapted
to measure usability on different field. Esacon Model (1984) is the earliest
usability measurement model, which is proposed by Kennneth Eason in issue of
behavior and information technology. Eason Model defined usability into three
aspect: Task, user and systems. In Task aspect, researcher addressed the
frequency and openness of product or system. Eason concerns knowledge,
motivation and discretion. A usability system in Eason model have to be ease of
learning, use and task match. Eason model could not measure user and their
targeted task. This model can applied to measure usability of product and
system. From this model, many researcher has improved and developed new
different model for various purpose and context. For example, Shackel Model


13

(1986) has 4 elements: effectiveness, learnability, flexibility and attitude. The
researcher did not weight aspect of dimension like Eason model. Because the
researcher believed that the dimension would change on different project.

However, this model accentuated on human factor. Booth adapted this mode to a
new usability definition which composed 4 elements: usefulness, effectiveness,
learnability (ease of use) attitude (likability) He eliminated flexibility since it is
difficult to specify and measure flexibility. Booth’s definition has been accepted
generally by usability community, especially likability was considered to be
important. Until 1998, ISO 9241 built up an international standard for guidance
on usability based on process oriented. This model has 3 main elements:
effectiveness, efficiency, satisfaction. Effectiveness describes the interaction
based on processed viewpoint, efficiency emphasized on results and resources.
Satisfaction means user viewpoint. The advantage of this model is objective
measures of usability. The disadvantage is abstract. However this usability
model is accepted and applied in academic research field. In addition to ISO
model, there are another practical and applicable to real business, which is
Wammi. WAMMI has been scientifically proven whose reliability and data
rating of between 0.90 and 0.93. (Marzanah, Usman, Aisha, 2013). The unique
point of this model is Wammi questionnaire which has 20 statement focus on
user –experience of visitors. 20 questions covers five factors to assess the
usability of websites ( According to
wammi official service, explanation of 5 factors as following:
70. Attractiveness
71. An Attractive site is visually pleasant, and also offers much of direct interest to
the respondents, whether it be functionality or information.
72.

Controllability
73. If a site scores well on controllability the respondents most probably feel they
can navigate around it with ease and do the things they want to do. Poor usually
means a poorly organized site that disrupts the way they normally expect to do
things
74.

Efficiency
75. When respondents give a high Efficiency rating they feel they can quickly locate
and do what is of interest to them in a effective and economical manner. They
feel that the web site responds (possibly, the pages load) at a reasonable speed.
76.

Helpfulness


14
77. A site which is high on Helpfulness corresponds with the respondents'

expectations about its content and structure. A site low on helpfulness can be
misleading about its layout and content.
78. Learnability
79. When Learnability is high, respondents feel they are able to start using the site
with the minimum of introductions. Everything is easy to understand from the
start. When Learnability is low, respondents feel that the site may be using
concepts or terminology which are unfamiliar. More explanations are needed.
80. Besides, there are many other famous model like Nielson Model (1993), ISO
9126 (2001), QUIM (2006).
i. 1.2.3 Usability measure tool.
81. There are a lot of questionnaires developed to support usability measurement.

Usabilitynet.org has listed down several outstanding questionnaire. For
example, Sumi which is a mature questionnaire which is good for desktop
products, but has also been used to evaluate command-and-control
applications. It is a commercial product which comes complete with scoring and
report generation software. Factors Research Group at University College Cork

designed and sold it. Specially, applicable widely Wammi questionnaire,
designed to evaluate the quality of use of web sites. It is backed up by an
extensive standardization database. It is the result of a joint development
project by Jurek Kirakowski and Nigel Claridge. Besides, there are many other
tools like: SUS which is very robust and has been extensively used and adapted.
It is public domain and nobody has published any standardization data about it;
QUIS questionnaire developed by Kent Norman that has been modified many
times to keep it current since its first appearance. It is commercially available
and is championed by Ben Shneiderman in his book “Designing the User
Interface”; USE Arnie Lund developed it and last updated 11/11/1998. It
attempts to create a three-factor model of usability that can be applied to many
situations. However, no reliability or validation data are presented; IsoMetrics
This questionnaire is produced by Guenter Gediga and his team, which attempt
to produce a way of measuring ISO 9241 part 10, with reference to specific
software features. It is therefore good both for summative and formative
assessments. The questionnaire is well researched and detailed statistical
information is given.


15

i. 1.2.4 Educational website usability measurement on user’s
perspective
82. Although there are various usability evaluation method generally or specified

website context, there are few studies in context of educational website (Samsur
Rahman and Zabed Ahmed, 2013) Lautenbach (2006) used 2 element or
element: survey ability (user perceptions of satisfaction lay out) , find ability
( observed ease of use). Mustafa and Al-Zoua’bi (2008), applied 5 usability
factors for university website content, organization and readability; navigation

and links; user interface design; performance and effectiveness; and educational
information. A questionnaire evaluation technique was used in this study,
consisting 24 questions which was divided into four parts. The results showed
the good and the bad of the usability aspects of these websites in terms of the
usability element, the result of the study can be used by website designers and
developers to improve the websites. Abdul Aziz, 2010 investigate the usability
of Malaysian university website including 120 samples of higher education
institution websites from the online portal of the Ministry of Higher Education
(). This study focused on page size, broken links and
web performance are the usability features that the study focused on.
Bairamzadeh & Bolhari, 2010 on the other hand investigates factors affecting
students' satisfaction of university websites. The model composed usability,
trust, perceived usefulness, satisfaction and website innovation. The study by
(Şengel & Öncü, 2010) investigates the usability of the Website of Uludag
University through questionnaire which inquired on how students uses the
university’s website. It was distributed to 445 students in different grade levels
and enrolled in various faculties of Uludag University. There were
dissimilarities in the responses of both males and females. The study provides
vital directions for how future. In other words, Yusof, 2010, web site usability
was test by doing some task or finding information in the website. The results of
study provide a tangible concept on how the aesthetics design and usability
impact each other and methods to balance between aesthetics design and the
usability. Other studies used automated tool, such as Anwar and Tsuji (2011)
used HTML Toolbox and Web Page Analyser, to measure university websites’
internal usability in Bangladesh. This study gathered data from users through a
questionnaire covering 5 usability element in research Mustafa and Al-Zoua’bi
(2008) Caglar & Mentes, 2012, the research analyse the effects of the
information age on universities and measure usability of EUL's (European
University of Lefke) website in terms of student utilization. His studied adapted
Website analysis and measurement inventory (WAMMI) questionnaire, consists

16

of 20 questions and use regression and Non–parametric techniques examine the
data. The study shows discontentment and other usability problems of a EUL
which is situated at Northern Cyprus. Mentes & Turan, 2012 tested usability of
the usability level of Namık Kemal University (NKU) website based on Caglar
and Menttes’s research. Hasan (2012) assessed the usability of nine educational
website with 5 elements: navigation; architecture/organization; (3) ease of use
and communication; design; and content. Several studies measure usability
based on page size, speed and broken links. Recently, in 2013, Sharmistha,
Prasant, Rajib Mall combined Wammi questionnaire method and performance
method to evaluation usability of academic website on user perspective to the 23
usability element which were grouped into five categories and each of the
category deals with one usability aspect.
83. Another study by (Hasan, 2012) developed element for evaluating the usability
of educational websites and consists of 25 design issues divided into five major
categories: content, design, ease of use and communications, organization/
architecture and navigation. Then the relative importance of the developed
element in the evaluation of the usability of educational websites was
investigated from the perspective of 237 students. The study also investigates
based on the major/ specialization and gender of the students that participated in
the research (where students were chosen from two faculties: Administration
and Information Technology). It was observed from the results that the order of
the in the evaluation of the usability of educational websites element from the
most to the least important from the students perspectives was: content,
navigation, ease of use and communications, design, and organization /
architecture. The results also reveal that it is only in one category: contents that
there was a statistically significant difference between among the gender. As

females considered it as the most important category and males considered it as
the second most important category. However, the results showed that there
were no statistically significant differences between the students of the two
selected faculties regarding the relative importance of the element.
84. The literature outlined above shows that there is an interest in investigating
users' perceptions of the relative important of different design factors on the
usability of different types of website.
85. Quite a number of questionnaires have been and still being used in the
evaluation of the websites usability. Some of such questionnaire include: SUMI
(Software Usability Measurement Inventory), QUIS (Questionnaire for User
Interface Satisfaction),
86.


17

87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.

101.
102. CHAPTER 2: METHEDOLOGY
a. 2.1 Analysis framework
103.
104.
105.

Literature review
Research model

106.

107.
108.
109.
110.
111.
112.
113.

Questionnaire- based
method

Website performance based evaluation method
Sample

Data collection
Data analysis

Figure 2.1: Analysis Framework

115.
After doing secondary research, the researcher reviewed all related article
and investigate company data to finger out researcher model: WAMMI. Based
on research model, the usability was evaluated by two method questionnaire
based evaluation and website performance based method. Then the researcher
randomly selected sample to participated two evaluation method. After data
collection is done, it will be analysis the same method in research of Sharmistha,
114.


18

Prasant, Rajib Mall, 2013 in term of performance method and Islam & Tsuji,
2011 in term of questionnaire method.
a. 2.2 Research model
116.

Figure 2. 2: Wammi Model (source: Human Factors Research in 1999)
118.
After reviewing related article, the researcher decided to choose WAMMI
which was developed by Human Factors Research Group (HFRG) in 1999. This
model was chosen for several reason:
119.
WAMMI is a usability evaluation tool for website.
• It is based on a questionnaire filled by visitors of a website, and gives a measure
of how useful and easy it is to use in term of user’s dimension.
• WAMMI has been scientifically proven and has a reliability and data rating of
between 0.90 and 0.93.
• Model include 5 elements: attractiveness, control, efficiency, helpfulness, and
learnability. They are popular element which was used in a lot of articles.

120.
Explain Wammi evaluation tool
121.
WAMMI questionnaire has been thoroughly tested and used on numerous
websites to ensure maximizing user The 20 WAMMI statements are selected
from a large range of questions about users' experiences with websites. Over a
number of iterative test-and-analyse cycles, the current 20 emerged in the form
they now have. The questionnaire allows visitor to speak their own voice of
their real experience. The 20 statement in questionnaire covers 5 following
elements:
122.
Attractiveness
123.
An Attractive site is visually pleasant, and also offers much of direct
interest to the respondents, whether it be functionality or information.
124.
Controllability
117.


19

If a site scores well on Controllability the respondents most probably feel
they can navigate around it with ease and do the things they want to do. Poor
usually means a poorly organised site that disrupts the way they normally expect
to do things.
126.
Efficiency
127.
When respondents give a high Efficiency rating they feel they can

quickly locate and do what is of interest to them in effective and economical
manner. They feel that the web site responds (possibly, the pages load) at a
reasonable speed.
128.
Helpfulness
129.
A site which is high on Helpfulness corresponds with the respondents'
expectations about its content and structure. A site low on Helpfulness can be
misleading about its layout and content.
130.
Learnability
131.
When learnability is high, respondents feel they are able to start using the
site with the minimum of introductions. Everything is easy to understand from
the start. When Learnability is low, respondents feel that the site may be using
concepts or terminology which are unfamiliar. More explanations are needed.
a. 2.3 Designing questionnaire and performance test.
i. 2.2.1 Questionnaire design
125.

Based on Wammi questionnaire, and the purpose of how different user
assess the usability of Markus website. The researcher developed her own
questionnaire including three parts: Personal information, usability evaluation,
extended question. Personal information including 4 questions of gender, age,
occupation and the course user studied. The age question is defined into 4 main
group which has short gap ( 13-22, 23-28, 29-34, >34) which corresponded to
each group targeted customer of Markus, which is student ( junior and seniorz)
graduated and young worker. Usability evaluation composed 20 statement in
Wammi questionnaire. Extended questions includes 4 question which
investigate deeper the user feeling of Markus website, such as what is the best

point, the role of Markus website toward user, their internet skills and finally
their opinion of new field in Markus website. The questionnaire was design in
English then translated to Vietnamese to limit the language barrier since almost
respondents is Vietnamese.
i. 2.2.2 Designing website performance test

132.

In Performance-based evaluation, Remote Usability testing method is
chosen to perform usability tests on websites. This usability evaluation approach

133.


20

also involved participants answering post-task questionnaire – After-Scenario
Questionnaire (ASQ) and post-study questionnaire – Website Analysis and
Measurement Inventory (WAMMI). A website performance test is task
completion experiment. There are several task was assigned to user to complete.
The process user’s performance would be record in term of the number of
completed task, time and the number of clicks. This test was design to support
questionnaire method which still have some limitation. With a real performance
test, we can evaluation exactly what user really do compare what we expected
them to do. It may be different internet level, satisfaction standard and so on
people can feel the website in different usable level. The result of questionnaire
just show how usable the website is generally without a specified metrics.
Therefore, we need a practical task completion test to know detail how user use
our website in how long, many click to complete a task. A small interview with
Markus website developer and user was done to know what the developer want

user to do. Based on that interview, the researcher chosen 4 tasks: (1) find a
course that user want to study (2) Register for one course (3) Find useful
information (4) Contact with Markus staff. The task was built on developer
dimension only in order to evaluate the development process, whether the
design of website with oriented task is suitable to user or not.
a. 2.4 Sample and data collection.
134.
There are two sample were collected, one sample for questionnaire based
method, another for website performance method.
135.
Firstly, the questionnaire’s sample is 50 - 60 people. The researcher want
to have a fresh experience which the user still remember so sample chosen
respondents who recently studied or are studying and going to stud. Since
Markus has only 5 course, each course has around 20 people, totally appropriate
100 people for 5 course there for the scale is only 50 - 60 people which account
50 % and cover equally among course. The respondents were selected through
email then fanpage of Markus. The researcher was allowed to take email of
student in 5 main course then sent them email. In total, there are 200
questionnaire was spread out via email. After that, the research post a public
status on “Welcome to Markus” fanpage to attract more respondents who
studied and are going to study in Markus.
136.
Secondary, website performance based method, there is a small sample of
3 -5 people who respectively a new user who want to study, one user who
studied one course, one or two user who studied more than one course, one user
who work in Markus. Sample of user there is the same internet level and
condition but difference in Markus website experience. Sample was gather to


21

participate in a real experiment. They was assigned 4 tasks: (1) find a course
that user want to study (2) Register for one course (3) Find useful information
(4) Contact with Markus staff. They was asked to record their PC screen during
experiment test.
137.
The result from both process will be collect and analysis based own their
own metrics.
a. 2.5 Data analysis methodology
138.
The result questionnaire and website performance will be collected and
analyzed in a specified method
i. 2.5.1 Questionnaire analysis
139. Table

2. The merit of website usability

140.

141.

142.

143.

144.

145.

Option

Strongl
y
disa
gree

Disagre
e

Fair

Agree

Strongl
y
Agre
e

146.

147.

148.

149.

150.

151.

Merit

0

0.25

0.5

0.75

1

(Source: Islam and Tsuji, 2011)
153.
To evaluate the usability of website from the perspective of students,
responses will be evaluated according to the merit point adopted by (islam &
Tsuji, 2011).
154.
Five options are available for each question. The options and
corresponding merits are shown in table. According to (Abdullah & Wei, 2008),
usability point for a factor x, is defined:
155.
x = [Σ (Merit for each question of the factor)] / [number of questions]
156.
Where factor is:
157.
Attractive: statement 1, 6, 11, 16 in part II of questionnaire. These
statement investigate the customer feeling and opinions on how website visually
pleasant, and also offers much of direct interest to the respondents, whether it be
functionality or information.

158.
Controllability: statement 2, 7, 12, 17 in part II of questionnaire. These
statements ask respondent to express their feeling when they can navigate
around website with ease and do the things they want to do.
159.
Efficiency: statement 3, 8, 13, 18 in part II of questionnaire. These
statements would show how productive the website is based on can quickly
locate and do what is of interest to them in effective and economical manner.
152.


22

They feel that the web site responds (possibly, the pages load) at a reasonable
speed.
160.
Helpfulness: statement 4, 9, 14, 19 in part II of questionnaire. These
statements express the respondents' expectations about its content and structure.
A site low on Helpfulness can be misleading about its layout and content.
161.
Learnability: statement 5, 10, 15, 20 in part II of questionnaire. These
statements considers respondent’s feeling they are able to start using the site
with the minimum of introductions. Everything is easy to understand from the
start.
162.
The usability level of Markus website was determined by using the
corresponding merit values from 0.000 to 1.000. The greater the value shows
greater satisfaction from the respondents whereas the lower the value
determined lower respondents’ satisfaction. Table 2.2 shows the usability points
and the corresponding level for each point. The overall website usability is the

mean value of usability points for the five factor
163. Table 2. : The usability points and corresponding level.
164.

165.

166.

167.

168.

169.

Point
,
x

0<

0.2< x
<=0.4

0.4< x
<=0.6

0.6< x
<=0.8

0.8< x

<=1

170.

171.

172.

173.

174.

175.

Usab
ili
ty
le
v
el

Bad

Poor

Moderat
e

Good

Excelle
nt

x
<=0
.2

(Source: Islam and Tsuji, 2011)
i. 2.5.2 Website performance analysis
176.

The performance metrics used in this study to evaluated collected data
includes:
178.
Task success which is used to measure whether the users were able to
complete the tasks successfully. Since all the tasks that the participants had to
perform on the websites involved in finding something or contact with someone,
it was possible to measure task success.
179.
Task completion time which means task is considered to have
completed as soon as the participant finds the information from the website and
177.


23

provides the same in his response. The time elapsed between the start and end of
task was expressed in seconds. It is a measure of efficiency of the system.
180.
Number of clicks: Efficiency was also assessed based on the amount of

clicks that the users had to perform on the website to complete a task. The
number of clicks was counted during the time between the start and end of each
task.
181.
The metrics was applied to all user then compare in order to figure out
whether the website is easy to all user in difference level.
182.
183.
184.
185.
186.
187.
188.
189.
190.
191.
192.
i.
193.
194. CHAPTER 3: ANALYSIS
a. 3.1 Markus School introduction.
i. 3.1.1 Marketing school introduction
Markus Marketing School (Markus) was found in 11/2012 by CEO “Do
Xuan Khoa” with the aim of igniting marketing thinking and skill. From 2012
until now, Markus have organized more than 30 classes with more than 600
student and enhanced company size in term of courses. Until now, Markus has 5
main courses and several short term course cover all aspects of marketing as
following: (1) Marketing thinking: igniting marketing thinking for everyone, (2)
Copywriting foundation: think and write like a copywriter, (3) Sale and Trade
marketing for everyone and management trainee applicator in fast moving

company, (4) Content marketing: how to generate content like a king, (5)
Inbound marketing: Attract customer in digital environment. Markus also has
several short term courses: (1) How to survive without budget: marketing for
startup and small business, (2) Visual story telling: make content by

195.


24

visualization, (3) Promotion, activation and reward: how to increase sale, brand
and keep loyal customer
196.
Markus School targeted in young customer, special student, fresh
graduate student and worker who are newbie in marketing, want to learn more
practical knowledge in marketing.
197.
Nowadays, Markus has 2 branches, one in Hanoi and another in Ho Chi
Minh City. Although has two location, there are a lean operation system. The
operation system is following:
198.
CEO
199.
200.
201.
CEO - Assistant
202.
203. Figure 3.: Markus operation system
(Source:
Markus

Marketing team 204.
Operation
teamstaff interview)
Project team
205.
206.
Marketing team has 3 people who respond for communication channel
including website, facebook, and blog.
207.
Operation team there are 3 people who respond for contact with lectures,
take care class
208.
Project team has 2 people who respond for market research and develop
new product, such as new course and project.
a. 3.2 Markus website operation
209.
Although, Markus is found in 2012 but until 2015, thinkmarkus.com is
built. From 2012 to 2014, Markus School operation based on fanpage and worth
of moth. In 2015, with the aim of enhancing business, thinkmarkus.com is
developed. The website is designed as a communication and branding tool.
While fanpage and worth of mouth is use as main sale tool which promote
website. It composed 6 main parts: homepage (general introduction and build
brand image to user), about us (introduce deeply about Markus), course (present
information of course with register function), Blog (marketing content sharing),
Opportunity (Special event and job opportunity), Contact (how to contact with
Markus). Website í built based on wordpress foundation.
210.
According to google analytics report in March, 2016, the website has
4774 user, 11681 page view with 73.7 % is return visitor. Those number has
showed that Markus user is general loyal user. The age of user focus from 18 to

34, mainly is male.


25
211.

212. Figure

3.: Demographic user at thinkmarkus.com (Source: Google analytics,
2016)
213.
Website is built for some reason:
214.
Brand building: URL is one important bran element which is presented in
website. Therefore, it is claimed that website is one brand element. Markus
website is added to comprehensive Markus Brand. It build brand image, is a
place for customer come and discover more deeply about Markus
215.
Communication tool: Through website Markus offer a useful information
of Markus and its product, especially they offer a blog where people can find
related marketing knowledge and cooperate with each other.
216.
Marketing channel: Although Markus fanpage and worth of mouth is
powerful marketing channel at Markus School. However, it is not an official
marketing channel which can cover a mass group of student. Markus need more
tool to connect with customer which can cover massive customer. Website is a
medium marketing tool which is powerful when combining with SEO, Adword,
landing page in order to attract potential customer.
a. 3.3 Data description.
i. 3.3.1 Questionnaire based method

1. 3.3.1.1 Part 1: Personnel information
217.
After one week of collection, there are 60 respondents. In which, 43.5 %
is male, 56.5 % is female. Most of them is from 18 -28 age. They are fulltime
worker (41.9 %), student (51.6%).
218.
219.