Public expenditure review for the national program for children protection at the first half of period 2011 2015

UPPSALA UNIVERSITÉT &
VNU UNIVERSITY OF ECONOMĩCS & BUSINESS
______________ _____ __________________7C » t • * ____

VNU-UEB

UPPSALA
UNIVERSITET

MASTER THESIS OF MPPM

PUBLIC EXPENDITURE REVIEW FOR THE NATIONAL
PROGRAM FOR CHILDREN PROTECTION AT THE FIRST HALF
OF PERIOD 2011 -2015

Author:

Lê Thu Hoàng

Supervisor:

PhI). Lars-Torsten Kriksson

Local Supervỉsor: PhD. Hoàng Thị Thúy Nguyệt
Class:

MPPM INTAKE 7

Hanoi, January - 2015

1

ACKNOVVLEDGEMENT
I would like to extend my sincere and heartĩelt obiligation towards all
the personages who helped me in the thesis. Without their active guidance,
help, encouragement, I would not have made headvvay in tltie study.
I offer may sincerest gratitude to my sưpervisors, Dr Nguyet and Dr
Lar-Torsten, who has supported me thoughout my thesis.. One simply could
not wish for better or friendlier supervisors.
I extend my gratitude to CITE Center, VNU Universty o f Economics &
Business and UPPSALA Universitet for giving me this opportunity.
I would like to thank Bureau o f Children Protection and Care,
MOLISA and some local authorities for providing m e vvith reíerence,
interview and data sources.
I

also acknovvledge with a deep sense o f reverence, my gratitude

tovvards my family who has always supported me m orally as well as
economically.
At last but not least gratitude goes to all o f my íriends who directly or
indirectly helped and contributed me ideas and comments to complete this
thesis.
Respectíìilly,

Lê Thu Hoàng


2

ABSTRACT
Title:

PUBLIC

EXPENDITURE

REVIEW

FOR

THE

NATIONAL

PROGRAM FOR CHILDREN PROTECTION AT THE FIRST HALF OF
PERIOD 2011 -2015
Level: Final assignment for Master Program in Public Management.
Authors:Lê Thu Hoàng
Sweden Supervisor:PhD. Lars-Torsten Eriksson
Local Supervisor: PhD. Hoàng Thị Thúy Nguyệt
Date when the thesis is presentedron 6thDecember 2014
Aim: Out o f Vietnam’s 90 miỉlion inhabitants 29 % are children. O f these
about 1.5 million are considered to be children in specially difficult
circumstances. The National Plan for Protection o f Children in 2011-2015
aims at protecting children from violence, abuse and exploitation. The
purpose o f this study is to describe and revievv public expenditure for NPCP
for the first half o f period 2011-2015. However, It need to deíined clearly
that: the result o f review for whom? What is the revievv purpose? Since the
Plan is in the implementation process, the purpose o f mid-term evaluation is

to assess the progress and its results are used for State management o f this
plan. Speciíically, activities are assessed if they serve the objectives o f the
Plan and follow the timelime o f the Plan ữamevvork. There are many methods
to revievv public expenditure, for example, cost — beneíĩt analysis. In this
study, 5 indicators o f evaluation method base on result-based management
will be chose to anaỉysis and review. This research only analyse in scope o f
project 3 about Child protection Service m odels which MOLISA manages and

monitors. Evaluated objects are results o f the program and the degree o f the
program’s implementation.


3

Method:
Firstly, a logic framework is establihed for the Program. NPCP is not
made and shovvn logically from goal, objects to activities, indicators and
criteria. Base on the approved program and knovvledge, experience in
management and monitoring the program, prọịect 3 is revvrote according to
Logical Framework.
A íter that, the frame o f reference rests on the theory o f result-based
evalutation and its five indicators o f efficiency, effectiveness, impact,
relevance and sustainability. Base on these five evaluated indicators,
thisresearch collected secondary data, from mainly the MOLISA database, for
statistics, comparisons o f different periodical and technical evaluations,
íìndings o f surveys on protection and care for children, and in-depth intervievv
(offícers who work on child care and protection). If evaỉuation is made by
another method as cost - beneíìt analysis, results could be đifferent because o f
a different approach. This study does not discuss that what method is better or
more exact and so on. However, in this case, cost - beneíìt analysis is put on

appendix 2 primarily, not deeply and completely. It is just to reference. In
cost - benefit analysis, beneíĩts and costs are expressed in monetary terms,
and are adjusted for the time value o f money, so that all flows o f beneíits and
flows o f project costs over time (which tend to occur at different points in
time) are expressed on a common basis in terms o f their "net present value."
It is very diffĩcult, time - consuming and expensive - costly to implement.
Moreover, the project is not íinish. It should be reviewed base on results
based management. It is the more suitable method which developed
organizations such as WB, UNDP and ư n ic ef are applying.
Finally, SWOT analysis is used and recommendations for the prọịect
in the next time are proposed base on 5 evaluated indicators. Moreover, the
future management method, RBM, will also be suggested.


4

Result & Conclusions:
On the basis o f its cvaluation criteria, the dominant successes are high
reỉevance and relatively good effĩciency o f budget use. E ffectiveness can be
found but not very clearly. Because o f long-term project, its im pact and
sustainabỉe are diffcult to obvious, evaluate and predict. Nevertheless, there
emerges a tendency for systematically sustainable planning and global
m anagement innovation o f RBM can be predicted. High development o f
efficiency and effectiveness will further boost impact and sustainability.
The best solution for fùture managem ent is applying RBM for public
expenditure m anagem ent because o f the relation between good govemance
and RBM. M oreover, logical íramevvork is its best basic tool to make and
implement plan. It also makes clear results o f a prọịecưprogram to monitor
and evaluate better.
Suggestions for future research:

Some shortcom ing are found as follow:
-

Data from local stakeholdres such as children, their parents, local

officers and collaborators are very important and objective. However,
indicators and criteria o f the Program in general, especially the Project 3 have
many problems. They are general problems o f Vietnam public policies. They
does not show goals and objectives o f a prọịecưprogram adapquately. It is
also difficult to apply them to evaluate results at locals. For example,the
overall criteria should be gain at 5,5% CEDC o f total children is not suitable.
In 2013, Ha Giang province gain criteria as 5% CEDC despite number of
CEDC is high and wam ing. It is because that its total children is many in
number. There will be support and guiding administration documents for
local authorities to implement after every public policy is null and void.
Therefore, data which collected from local stakeholdres are ju st used to


5

reference and qualitative analysis. Data are mainly collected and analysised
from State mangement at Central level.

-

Social supportive and protection ĩield also relates to this children

protection and care íìeld. However, relative effects and impacts has not
researched yet.
These above limitations could be suggestions for íìirther researchs.

Contribution of the thesis:
This study helps to practices and inhances work skills, logic and critical
thinking. Applying Logical Framework and 5 indicators o f results base
evaluation makes monitoring and evaluation more effective. This research
contributes to new knovvledge o f results base management for administrative
officers on approving an action plan in general, especially public expenditure
management. It is also íbundation for whom research and make policies,
plans, programs,... These skills and knowlegement are being a trend on the
world vvhich developed organizations and countries has been applying.
Key words: Logical Framewwork, results base management, relevance,
efficiency, effectiveness, impact and sustainable.


6

Table of Content
Abbreviation.................................................................................................................. 8
1. Introduction:.............................................................................................................9
/. /. Background .................................................................................................................. 9

1.2 NPCP Introduction ................................................................................................... 10
1.3 Research scope and ỉimitation ............................................................................... 13
1.4 Research purpose ....................................................................................................... 14
1.5 Research questions ................................................................................................... 15
2 Frame o f reference.................................................................................................. 16
2.1 Theory o f resuỉt-based m anagem ent................................................................. 16
2.2 Theory o f resuỉt-based evaluation......................................................................18

2.3 SWOT Anaỉysis .......................................................................................................... 26
3 Research m ethods................................................................................................... 27

4 Empirical data and an alysis................................................................................ 28

4.1 Reỉevance ỉndicator .................................................................................................. 29
4.2 Effìciency Indicator .................................................................................................. 31
4.3 Efectiveness Indicator ..............................................................................................34
4.4 ĩmpact Indicator ........................................................................................................ 36
4.5 Sustainable Indicator............................................................................................... 3 7
5 C onclusion................................................................................................................ 39
6 Recommendations................................................................................................... 39

6.1 Analysis o f situation o f chiỉd care and protectìon work ................................. 39


7

6.2 Recommendations fo r the proịect pỉan in the next tim e .................................41
6.3 Recommendaíion fo r the fuíure management................................................... 43
7 Contribution and further research.................................................................... 44
Reĩerence ......................................................................................................................46
A ppendix l:Stakeholders analysis............................................................................48
A ppendix 2: Cost —benetĩt anlysis.......................................................................... 51
Appendix 3: Indicators and criterias O ÍN PC P....................................................... 53
Appendỉx 4: In-dept Intervievv an officer.............................................................. 60
Appendix5: Results o f pilot ranking o f provinces/cities in child rights
implementation in 2013 .............................................................................................. 62


8

Abbreviation

CPSM

child protection Service model

CEDC

Children in especially difficulty circumstance

CRC

United Nations Convention on the Rights o f the Child

M O LISA M inistry o f Labor, Invalids and Society Affair
NPCP

National Program for Children Protection in the 2011-2015
periods

RBM

Results Based M anagement


9

1. Introduction

1.1. Background
In 2011, the population o f Vietnam vvas 90 million including 29% child
population(about 25.244.159 people). Thereíore, the State paid attention to

works o f children protection more and more. Forms o f care for CEDC have
been more various and appropriate for each group’s needs.The number o f
CEDC, children o f poor families, ethnic-minority children in especiaỉly
difficult residential areas and children under the age o f six who have received
State assistance have risen quickly. 100% o f CEDC, poor children, ethnicminority children are entitled to legal assistance on fam ily’s demand. All
children o f ethnic minorities receive assistance for their education via tuition
fee w aiver and other forms. Care for CEDC increased to over 80% by 2011.
Hovvever, a number o f Children in especially diffícult circumstance (CEDC)
was still high, nearly 1.5 million, accounting for 5,9% o f child population.
Reported cases o f child abuse, tracfficking, labour and exploitation are
increasing that: 50% o f children experiecing physical punishment, 8% o f
sexually abused children,...
Recognizing the significant o f children and problems they have to face,
Government has expressed their concem s and implemented many special
policies to help care and protection children. In recent times, legislation,
policies and programs on child care and protection has been concemed and
enhanced greatly. In particular, Vietnam was the first country in Asia and the
second nation in the world to ratiíy United Nations Convention on the Rights
of the Child (CRC) on 20th February 1990. Since then, it gains some
remarkable achievement in child protection, care and education. The nation’s
legal system and policies have been íĩne-tuned, State m anagem ent has been
enhanced, greater resources have been allocated to children. However, impact


10

o f disparities economic development and the change in certain social
perceptions and norms still have impacted on children. Children protection
has moved from intervention, assistance into a comprehensive establishment
and implementation o f policies at all three levels: prevention, early detection

and intervention, assist in social reintergration.
To contribute to the nation target of protecting children from violence,
abuse and exploitation, particularly reducing the number o f children in special
circumstance, the National Plan for the Protection o f Children in the 20112015 period was approved by decision number 267/ỌD-TTg on 22nd February
2011. Implementation responsibles are assigned for many Ministries and
fields which M inistry o f Labor - Invalid and Society Affair (Bureau o f child
care and protection) is the main responsible office for implementing, guiding
and monitoring this Plan nationwide. This is the largest-scale child program
that attracts attention and receives the highest annual budget. At the middle of
period o f 2011-2015, NPCP vvas evaluated that it has made positive changes
in both behavior and action of citizen in child protection, care and education.
However, these mid-term evaluations o f National Plan for Child Protection
of, the evaluation bases on traditional method and just íocuses on the
implementation process and outputs o f every specific objectives.

1.2 NPCP Introduction:
This

research

is

conducted

from

MOLISA

State

management

perspective. A ccording to the third and fourth National Report on CRC,
Vietnam has perfonned best in two rights namely protection right and care
right out o f the four íìindamental right groups o f children (protection right,
care right, participation right and development right)
A number o f plans in child protection approved by State prime minister
is large such as National Action Plan for Children 2012-2020, Plan for Child


11

Injury Prevention 2013-2015, Plan for Children affected by HIV/AIDS.
Hovvever, budget for these plans is only for State management at the Central
level and localities have to íĩind themselves for the implementation from local
budget.
It can be said that the NPCP in the 2011-2015 periods is an overall,
comprehensive and consistent plan vvith budget allocated at both Central and
local levels from 2011 to implement its prọịects. As a result, the Plan has
received positive

íeedback

from

local offices, technical offícers and

beneíìciaries.
Objectives o f the program are as follows:
1. Life conditions for children, in particular, children in

especially circumstance arc better to protect, carc and
develop them well in both mind and body.

1. 1. Knowledge, capabilitics and estahlished supportive tools for related administrative
organi/alions, authorities, officcrs, collaborators, voluntecrs and social community on
íìelds of child protcction and carc are enhanccd.

ị
1.1. 1.
Awareness
of families,
schools,
community
social and
children is
enhanced to
change
actions
about
protecting
and caring
children.

1.1.2. Staff,
oíĩicers,
collaborators
and
volunteers
are trained
more

professionall
y about
knowledge,
skills and
experience
on the íĩeld.

1.1.3. Child
protection
services
models are
built and
upgraded
include
counscling
OíTices at
district level,
counseling
spots at
schools,
hospitals,
and
management
Knarrlc

ị'
1.1.4. A
system of
child
assistancc

centers are
built and
developcd to
helps children
in especial or
be hurted
circumstance
recoverable
and
rcintegrated.

1.1.5.
EíTectiveness
of
administrative
management
on child
protection and
care are
strcngthened.


12

Logic framework o f the Program is presented in the Appendix 3. It
includes indicators and criteria vvhich the Progam need to be gain. Hovvever, it
is established not well. M inistries have responsibles for research and propose
public policies to be approved by Community Council. N PCP is made and
propose by Bureau o f child care and protection, a function unit o f MOLISA.
A general problem for almost public program is that indicators and criteria are

very general and difficult to understand, measure and apply for m anagem ent
and implementation. They make related offices, especially local authorities
coníìise to implement. After that, it need to have many adm inistrator
documents to guide for situation o f locals and point o f time. It is difficult to
review result at locals by primary program s which Prime M isnister has been
approved. Thereíòre, the program will only be evaluated base on its available
indicators and criteria at Central m anagem ent level.

Stake holders o f the program include:
- M inistries and the M inisterial-level offíces are the administration
management organization with following duties and authorities: cooperating
cach others to organize and build a action plan, direct, provide guidelines and
monitor local authorities to implement the program.
- Some o f donors on technology and íinance.
- Local People’s Committee and authorities are responsible to organize
action plans to implement the program base on guidelines o f related
Ministries and present regulations.
- Staff, collaborators and volunteers has been participating activities o f
the program directly.
- Children (in particular, children in especially difficulty circumstance,
children have possibilities in diffícult circumstances, abused children and
violent) are the main impacted objects o f the program.


13

The CRC define a child as “any human being under the age o f eighteen,
unless the age o f mạịority is attained earlier under a state’s ovvn domestic
lcgistation” . However, this study base on Viet Nam Law on the Protection,
Care and Education o f Children deíĩnes a child as “a Vietnamese citizens

aged under 16 years” .
As stipulated in Clause 1, Article 3, Law on Protection, Care and
Education o f Children revised in 2004, children classified as in especially
difficult circumstance (shortened in English as CEDC) are those who suffer
from abnormal mental and physical conditions, have limited conditions to
períbrm basic rights and integrate with íamily, and community”
Accordingly, ten groups o f children are included namely helpless
orphans, abandoned children; disabled children; children vvho are victims of
toxic chemicals; children with HIV / AIDS; children do hard, dangerous, jobs
and are exposed to toxic chemicals; children work far from home; Street
children; children who are sexually abused; drug-addicted children; lawviolated children.
-

Social community is all people who be impacted indirectly by the

program in country.

í. 3 Research scope and limitations:
The research is coníined to State management perspectiveof MOLISA.
The author is an officer vvhich works at deparlment o f planning and fmance,
MOLISA. It is limited by acknovvlegement and experiences. It also has
problems with its indicators and criteria. So it is difficult to measure and
compare, review for results o f locals. It is only reviewd for State management.

M oreover, the evaluation will only focus on prọịect 3 with its objective
stated as Child protection services models are built and upgraded include
counseling Offices at district level, counseling spots at schools, hospitals, and


14

management boards. Although NPCP includes 5 prọịects, they are not clear
and scperate about activities and results. Their activities overlap or duplicate
éch other. It is detailed at appendix 3. ỉt can see that activity o f prọịect 1,
organize public iníbrmationand propaganda directly, is overlap with activities
o f prọịect 3, enhance child protection skills for parents o f children and social
community. Thereíore, it is suppose that results o f project 3 can represent the
program. Evaluating concentrately for prọịect 3 is also suitable limitation on
time, knovvledge and experience in research progress.
Forms o f CPSM s can be seen that include: Social work Centers for
Children (at the district and provincial levels); Counseling Offices at district
level; Counseling spots at schools, hospitals, courts and in the communities;
model o f providing regularly services for children who are infected with
HIV/AIDS; prevention and eliminition the vvorst fomis o f child labour;
Community base model for rehabilitation o f children with disabilities; Irỹury
prevention model for children; child helpline m odel;...
As that, Social supportive and protection íĩelds also relate to this field.
However, relative effects and impacts are not research in this study. It could
be a suggestion for íurther research.

1.4 Research purpose
The purpose o f this study is to describe and revievv public expenditure
for NPCP for the first half o f period 2011-2015. However, It need to defined
clearly that: the result o f review for whom? What is the review purpose?
Since the Plan is in the implementation process, the purpose o f mid-term
evaluation is to assess the progress and its results are used for State
management o f this plan. Specifically, activities are assessed if they serve the
objectives o f the Plan and follow the timelime o f the Plan framework. There
are many methods to revievv public expenditure, in this study, 5 indicators o f
evaluation method base on result-based managem ent vvill be chose to analysis

15

and revievv. Evaluated objects are results o f the program and the degree o f the
program’s implementation.
/. 5 Research Questions
A primary question show the research purpose is that:
How is the results o f the prọịecí and the degree o f its implementation?
In order to realize the research purpose which is shovved by this
question, this study vvill try to answer 2 questions below:
(1) How is the project described base on Logic Model? It means thai
rewriting the prcỹect is to make “measure" fo r evaluation.NPCP is not made
and shovvn logicaỉly from goal, objects to activities, indicators and criteria.
Base on the approved program and knovvledge, experience in management
and monitoring the program, project 3 is rewrote according to Logical
Framework.
(2) Seen on 5 evaluated indicators, how are

results o f public

expenditure and its degree reviewed, especially comparing to the prọịecí ’s
objectives? This study just review and focus on a main method as result base
management. If evaluation is made by another method as cost - beneíit
analysis, results could be different because o f a different approach. This study
does not discuss that what method is better or more exact and so on. However,
in this case, cost - beneíĩt analysis is put on appendix 2 primarily, not deeply
and completely. It is just to reference. In cost —beneíit analysis, beneíĩts and
costs are expressed in monetary terms, and are adjusted for the time value of
money, so that all flows o f beneíĩts and flows o f project costs over time

(which tend to occur at different points in time) are expressed on a common
basis in terms o f their "net present value."1 It is very difficult, time -

1 />

16

consuming and expensive - costly to implement. Moreover, the prọịect is not
íìnish. It should be reviewcd base on rcsults based management. ỉt is the more
suitable method vvhich developed organizations such as WB, UNDP and
U niceí are applying.
2 Frame of reíerence
This study will be researched base on 3 basic theories, includes: (1) Theory of
result-based management, (2) Theory o f result-based evaluation and (3)
SWOT analysis.
2.1 Theory o f resuỉí-based m anagem ent
Results Based Management (RBM): A management strategy by vvhich
an organization ensures that its processes, products and services contribute to
the achievement o f desired results (outputs, outcomes & impacts). RBM rests
on stakeholder participation and on clearly defined accountability for results.
It also requires monitoring o f progress towards results and reporting on
performance/feedback which is carefully reviewed and used to further
improve the design or implementation o f the programme.2
RBM is differ from traditional management . Traditional management
íocuses in managing inputs, activities and production o f “outputs” while
RBM íocuses on management tovvards the achievement o f outcomes.
Morcover, they have many different characteristics as follow:

2ƯNFPA, Technical Support Division (March 2004) Progamme Management’s
Planning Moniíoring and Evaluation Toolkit, Tool number2: Deíĩning Evaluation.

17

Traditional management

Results based M anagem ent

Description o f situation beíòre project

Baseline data to deíìne the situation

start

beíore prọịect start

Benchmarks for activities and outputs

Indicators prepared for outcomes

Collect data on inputs, activities and

Collect data on outputs- are they

immediate outputs

contributing to outcomes

Routine reporting on píovision o f

M ore focus on perceptions o f change

inputs and outputs

am ong stakeholders

Directly linked to a discrete prọịect

M ore qualitative & quantitative
analysis o f progress to outcomes

Provides information on

Project and strategic partners look at

administrative, routine

broader development effectiveness

implementation and managem ent
Logical framework is a fìrst basic tool to establish a Results Based
M anagement Process.
-

Logical

Framework

(log

frame):

A

dynamic

planning

and

management tool that summarizes the results o f the logical framework
approach process and communicates the key features o f a programme design
in a single matrix. It can provide the basis for m onitoring progress achieved
and evaluating programme results. The matrix should be revisited and reíĩned
regularly as new iníbrmation becomes available.3

(3) UNFPA, Technical Support Division (March 2004) Progamme Management ’s
Planning Monitoring and Evaluation Toolkit, Tool number 1: Glossary of Planning,
Monitoring and Evaluation Terms, p.6.


18

- Logical Framework Approach:

A speciíìc strategic planning

methodology that is used to prepare a programme

or development

intervention. The methodology entails a participatory process to clariíy
outcomes, outputs,activities and inputs, their causal

relationships, the

indicators with which to gauge/measure progress towards results, and the
assumptions and risks that may iníluence success and ĩailure o f the
intervention. It offers a structured logical approach to setting priorities and
building consensus around intended results and activities o f a programme
together with stakeholders4

2.2 Theory o f resuỉt-based evaluaíion
Programme evaluation is a management tool.

It is a time-bound

exercise that attempts to assess systematically and objectively the relevance,
períormance and successof ongoing and completed programmes and prọịects.
Evaluation is undertaken selectively to ansvver speciíic questions to guide
decision-makers and/or programme managers, and to provide information on
whcther

underlying

theories

and

assumptions

used

in

programme

development were valid, what worked and what did not work and why.
Evaluation

commonly

aims

to

determine

the

relevance,

efficiency,

effectiveness, impact and sustainability o f a programme or project.5
It is one o f important tools for results based management. It differs
tradional evaluation method very much. It is similar to compare betvveen
traditional management and results based management.

(4) UNFPA, Technical Support Division (March 2004) Progamme Management ’s

Planning Monitoring and Evaluation Toolkit, Tool number 1: Glossary of Planning,
Monitoring and Evaluation Terms, p.6-8
(5)ƯNFPA, Technical Support Division (March 2004) Progamme Management ’s
Planning Monitoring and Evaluation Toolkit, Tool number 2: Deíĩning Evaluation,
p.ll


19

2.2. ỉ Relevance Indicator:
Relevance is the degree to which the outputs, outcomes or goals o f a
programme
subsequently

remain

valid

modiíìed

and

pertincnt as originally

ovving to

changing

planned

circumstances

or as

within

the

immediate context and extemal environment o f that programme.6
An assessment o f programme relevance examines the appropriateness
o f results in relation to: the national needs, policies, and priorities; the needs
and priorities o f programme target groups (the local programme context). The
analysis ascertains whether the programme continues to make sense and
identiíĩes any changes that may have occurred in its context during
implementation. The initial problems and needs may no longer exist and
policies and priorities may have changed as a result o f political, economic,
social and other factors, or even because o f programme activities. Ultimately,
the analysis determines vvhether the resuỉts are still valid or should be
reíbrmulated.
To assess programme relevance means that It find answers for some
key question about needs o f stakeholders, policies and priorities of
govemment: Are the results planned by the national needs and beneíìt of
stakeholders? Are they suitable to the priorities and policies o f govemment?
Do the stakeholders consider them useíìil? Are they complementary to other
donor interventions? Should results be modiíied, eliminated or supplement
because nevv needs, priorities and policies may change?7

(6) IJNFPA, Technical Support Division (March 2004) Progamme Management’s
Planning Monitoring and Evaluation Toolkit, Tool number 1: Glossary of Planning,
Monitoring and Evaluation Terms, p.6-8

(7) ƯNFPA, Technical Support Division (March 2004) Progamme Management 's
Planning Moniíoring and Evaluation Toolkit, Tool Number 5: Planning and
Managing an Evaluation, Part II: Deíĩning Evaluation Questions and Measurement
Standards


20

2.2.2 Efficiency Indicator:
Eííĩciency is a measure o f how economically or optimally inputs
(íĩnancial, human, technical and material resources) which are used to
produce outputs.8
An assessment o f programme efficiency measures the “productivity” of
the programme interventions. It assesses the results obtained in relation to the
expenditure incurred and resources used by the programme during a given
period of time. The analysis íìocuses on the relationship betvveen the quantity,
quality, and timeliness o f inputs, including personnel, consultants, travel,
training, equipment and miscellaneous costs, and the quantity, quality, and
timeliness o f the outputs produced and delivered. It ascertains whether there
was adequate justification for the expenditure incurred and examines vvhether
the resources were spent as economically as possible.9
The assessment will answer questions “how” in using resources/inputs:
is it spent as economically as possible to make actual or expected results? its
activities overlap or duplicate other similar interventions o f goverment and
donor organizations/individuals? Is there any ways and means to use available
inputs more efficient?
2.2.3 Effectiveness Indicator:
Effectiveness is a measure o f the extent to which a programme achieves
its planned results (outputs, outcomes and goals).

(8) UNFPA, Technical Support Division (March 2004) Progamme Management 's
planning Moniíoring and Evaluation Toolkit, Tool number 1: Glossary of Planning,
Monitoring and Evaluation Terms, p.3
(9) UNFPA, Technical Support Division (March 2004) Progamme Management’s
Planning Moniíoring and Evaluation Toolkít, Tool Number 1
Planning Monitoring and Evaluation Toolkit, Tool number 5


21

An assessment o f programme effectiveness íocuses on the extent to
vvhich the outputs have been or will be achieved and whether the programme
is likely to contribute to the stated outcomes and impact.

If not, the

evaluation will identiíy whether the results should be modiíìed (in case o f a
mid-term evaluation) or the programme be extended (in case o f a íinal
evaluation) in order to enable achievement o f statcd results.10
Some key questions o f the effectiveness assessment include: Do the
results achieve as expected plan? what is the degree and quality o f

the

results? Who and how is under iníluence o f the results (positive/negative) in
short or middle term?
2.2.4 Impact ìnáicator:
Impact means that positive and negative long term effects on
identifiable population groups produced by a development intervention,

directly or indirectly, intended or unintended. These effects can be economic,
socio-cultural, institutional, environmental, technological or o f other types."
Impact Evaluation: A type o f outcome evaluation that focuses on the
broad, longer-term impact or results o f a programme. For example, an impact
evaluation could shovv that a decrease in a community’s overall matemal
mortality rate was the direct result o f a programme designed to improve
referral services and provide high quality pre- and post-natal care and
deliveries assisted by skilled health care professionals.12

<11) ƯNFPA, Technical Support Division (March 2004) Progamme Management’s
Planning Monitoring and Evaluation Toolkit, Tool Number l
(l2) UNFPA, Technical Support Division (March 2004) Progamme Management’s
Planning Monitoring and Evaluation Toolkit, Tool Number l


22

2.2.5 Sustainability Indicator
Sustainability: Durability o f programme results after the termination of
the

technical

cooperation

channelled

through

the

programme.

Static

sustainability —the continuous f!ow o f the same benefíts, set in motion by the
completed programme, to the same target groups; dynamic sustainability - the
use or adaptation o f programme results to a different context or changing
environment by the original target groups and/or other groups.13
An assessment o f programme sustainability are likely to have lasting
results aíter programme termination ended and the withdrawal o f extemal
resources. The íactors affecting sustainability are examined on the basis o f the
priority assigned to the programme by stakeholders. Their readiness to
continue supporting or carrying out speciíìc activities, or even replicate the
activities in other regions or sectors o f the country, is particularly relevant.
The analysis also assesses the availability o f local management, íinancial and
human resources that would be needed to maintain the programme results in
the long run.14
The key questions o f the sustainabilityassessment is that: is it likely that
extemalities and impact o f programme/project vvill be sustained aíìter it ends?
Are involved counterparts willing and able to continue programme activities
on their own? Have programme activities been integrated into current
practices o f counterpart institutions and/or the target population?15

(l3) UNFPA, Technical Support Division (March 2004) Progamme Management ’s
Planning Monitoring and Evaluation Toolkit, Tool Number 5
<14) ƯNFPA, Technical Support Division (March 2004) Progamme Management ’s
Planning Monitoring and Evaluaíion Toolkit, Tool Number 1
<15) UNFPA, Technical Support Division (March 2004) Progamme Management''s
Planning Monitoring and Evaluation Toolkit, Tool Number 5

23

Depending on intervention level, follow ĩactors decidc that is the program
sustainable:16
1. Support policies o f
govemment
2. suitable technology

If State policies enahles the continuation o f the

project’s results
- If beneíĩciaries and third parties can use the
technology provided when the prọịect finishes

3. enviroment

- If the environment is protected

protection
4. Culture - social

- If socio-cultural íactors can ensure who will

factor

actively participate or take responsibility. Has
there any measure been adopted to ensure the
participation o f all members in the target

group.

5. Institutional and

- What measures have been taken to ensure the

management capacity

continuous operation o f new units or the
strengthening o f old ones thanks to
intervention process?

6. Pinance perspectives

- In the future, if the beneíìts are higher than
the vvhole costs incurred for the target groups?

(l6) MDF Indochina, Training and consultancy Department (2014) www.mdf.nl


24

Table 1: Summary of evaluation process

5 cvaluated
indicators:
1. Rclevance
2. Eíĩĩciency
3. Effective
4. Impaci

5. Sustainablc

Source: FASID (2000) Monitoring and evaluation based on the prọịect cycle
management method, Foundation for Advanced Studies on International
Development, Tokyo, Japan.