Criteria for Good Evaluation

Criteria such as:

1) usefulness,

2) impartiality and independence,

3) credibility, and

4) participation of partner countries,

All of these criteria should be satisfied in order to provide appropriate information worth being utilized; all of these are important for conducting a ‘Good Evaluation.’

(1) Usefulness Evaluation results should be clear and useful in order to be utilized in the decision-making of the organization. The objectives of the evaluation should be made clear, and for feedback to be effective, it is desirable that the evaluation should reflect the needs of envisioned users. Information regarding evaluation results is utilized in various stages during the project cycle and it is also utilized in the formulation of effective strategy, so it is essential that information is readily accessible, and implementation of the evaluation is just in time.

(2) Impartiality and Independence Evaluation should be impartially conducted in a neutral setting. For assuring the credibility of the evaluation information, it is indispensable to assure impartiality in providing standardized analysis of evaluation results. It is also important to gather a broad range of information from various stakeholders as well as from the particular persons and organizations concerned. In addition, the guarantee of impartiality is expected to prevent conflicts among stakeholders. The impartiality and independence of the evaluators is not counter to the sharing of information or feedback among stakeholders. The evaluation study group and departments involved in project implementation each have an independent role, yet share the same objective to improve project management.

(3) Credibility For credible evaluation information, it is essential for evaluators to have specialized knowledge of the target fields and schemes (types of projects) and also to be well informed in the scientific research method. Such an evaluator is capable of producing credible information that is the result of an objective analysis of those factors indicating success or failure. Besides, it is important to share the evaluation process itself among stakeholders of the project, because it assures transparency. In the process, it is important to include different opinions proposed regarding lessons learned and recommendations in the report, and leave it to a public verdict. Transparency is indispensable for increasing credibility with the public, and it also has no small effect on how much the evaluation information is utilized.

(4) Participation of Partner Countries' cooperation projects are conducted in collaboration with partner countries, and jointly with them to conduct evaluation consistently from the ex-ante to terminal stage. Thus, evaluation information should be utilized by both donor and partner sides. Lessons learned and recommendations obtained by the experience of project management are used as feedback for the development strategy of partner countries, and lead to a reexamination of the project. It is necessary for evaluators to communicate with the stakeholders of both donor and recipient sides during each stage of the evaluation.

Comments

Planning, Monitoring and Evaluation

Monitoring Definition

Monitoring - Definition Monitoring is the systematic and continuous collection, analysis, and use of information for quality control and decision-making related to the projects. It involves the ongoing and routine review of projects to verify whether they are developing according to plan and budgetary requirements and whether adjustments may be needed to achieve intended goals. Purpose of Monitoring The primary aim of monitoring is to improve the quality of projects during implementation. Through monitoring, ineffective efforts are identified and rectified, and remarkably effective actions are appreciated. Monitoring enables us to answer these questions and then make important decisions to improve the quality of projects and their implementation: Are project inputs (finance, personnel, materials, etc.) available on time and in the right quantities and quality? Are activities being implemented according to the project design and on schedule Ar...

Definition (Planning)

Planning can be defined as the process of setting goals, developing strategies, outlining implementation arrangements, and allocating resources to achieve those goals. It is important to note that planning involves looking at several different processes: Identifying the vision, goals, or objectives to be achieved. Formulating the strategies needed to complete the vision and goals Determining and allocating the resources (financial and other) required to achieve the vision and goals Outlining implementation arrangements, which include the arrangements for monitoring and evaluating progress toward achieving the vision and goals The expression “failing to plan is planning to fail.” While it is not always true that those who fail to plan will eventually fail in their endeavors, there is strong evidence to suggest that having a plan leads to greater effectiveness and efficiency. Not having a plan—whether for an office, program or project—is in some ways similar to attempting to build a hous...

Objectives of Project Evaluation

It is impor tant to evaluate the outcomes that a project achieves and to feedback the evaluation results, lessons, and recommendations obtained for more effective and efficient implementation of development assistance. The harsh economic and fiscal situations at home have generated a strong call to the organization for more effective and efficient implementation and to ensure accountability. The enhancement of evaluation has drawn attention as one of a major improvement measures. In addition, there are changes in the political landscape such as the adoption of public sector evaluation by ministries and the reorganization of agencies into Independent Administrative Institutions (IAI) that ask for improvements of the evaluation system. Evaluation is a tool for judging as objectively as possible the relevance and effectiveness of organization’s cooperation activities at four different stages during the project cycle: ex-ante, mid-term, terminal, and ex-post. The primary objectivethe...

Database-Based Monitoring/Tracking

In this approach to monitoring, achievements in terms of numbers (of inputs, activities and outputs) are collected and stored in the project/central databases regularly and presented in an identified frequency against the contractual targets in order to establish whether the project . is leading towards achieving its targets. This approach to monitoring authenticates the progress of the project toward the intended targets. It is important to remember the following when conducting database-based monitoring: The reports drawn on the overall target population, but not sampled; Mostly captures quantities only; Not field-based. Usually performed in the capital office; Types-and-scope-of-monitoring, the officer/staff are responsible for the final products; however, database staff must coordinate necessary data gathering, storing, and analysis. M&E practices two types of tracking under this approach to monitoring in general: Indicator Tracking: Tracking contractual indicators is cruc...

Step 4: Creating the Vision of the future

Based on the problem analysis, stakeholders should engage in a process of formulating solutions. This exercise may simply involve rewording the problems and their causes into positive statements or objectives. However, stakeholders should first engage in a visioning process before rewording the problems. The aim of this process is to visualize what the future would look like if the problems were resolved. The benefits of doing a visioning process before rewording the problems include the following: Visioning brings energy to the group. Rather than immediately beginning another detailed process of working on each problem, groups can be energized by thinking positively about what the future would look like if these problems were solved. This exercise encourages creativity and helps ensure that the process is not too analytical and methodical. TIP Visioning process to have prior knowledge or understanding of the results chain or logical framework model. In fact, in the initial stages...

Types and Scope of Monitoring

Types and Scope of Monitoring While there are several types of monitoring, M&E mainly focuses on performance monitoring, that is, the monitoring of project inputs, activities/processes, outputs, and outcomes in the field. Inputs: checking that the resources needed for the intervention are mobilized as planned. Example Indicators: The tools are available in sufficient quality and quantity to enable an efficient CFW process The funds are available on time to the money vendors to enable timely cash distribution to the beneficiaries Activities/Processes: checking that project activities are carried out as planned; assessing the implementation method. Example Indicators: A complaints response mechanism is operational in the areas of interventio % of selected households meets the targeting criteria % of targeted households receiving the correct amount % households reporting good or fair s...

Planning Process

This chapter provides step-by-step guidance on how to undertake planning for results. It focuses on the tasks involved in planning for desired results and includes considerations for operationalizing results. As noted earlier, monitoring and evaluation are closely related to planning. Therefore, in planning, it is essential to consider not only intended results but also how results, and the process of achieving them, will be monitored and evaluated. In particular, planning needs to ensure that planned gives initiatives are evaluation-ready. Planning can be done in many different ways. This chapter is designed to make the persons involved in planning more comfortable with the main steps involved in preparing a plan that can be implemented, monitored, and evaluated. The actions and approaches recommended generally apply to all planning processes, whether for a global, regional, or country program; a project; or ...

Step 5: Creating Dradft Results Map

This step provides guidance on how to create a draft results map using what is commonly referred to as a ‘results mapping technique’. Developing the draft results map can be time- consuming but is extremely worthwhile. The fundamental question that stakeholders in the planning session should answer is “What must be in place for us to achieve the vision and objectives that we have developed in the particular problem area?” Creating a set of positive results A good starting point in creating a results map is to take each major problem identified on the trunk of the problem tree and reword it as the immediate positive result with longer-term positive results or effects. For example, if the problem were stated as “low public confidence and involvement in governance” the immediate positive result could be “greater public confidence and involvement in governance.” This could lead to longer term positive results such as “wider citizen participation in elections, particularly by women, in...

Types of Project Evaluation

The Ministry of Foreign Affairs “Report on Improvement of ODA Evaluation System,” released in March 2000, classified ODA evaluation into three levels: policy-level, program-level, and project-level, as shown in Figure 1-1-2. The report called for enhancement of policy- and program-level evaluation. The Ministry of Foreign Affairs conducts policy-level evaluations such as Country Assistance Program and Sector-specific Initiatives, as well as program-level evaluation by sectors and schemes. JICA conducts program- and project-level evaluations. How to evaluate JICA’s management cycle is illustrated in Figure 1-1-3. There are two cycles: the “small cycle” (project cycle) and the “large cycle” (program cycle)-The information source is from JICA.

Finalizing the result frame work

At this stage of the process, stakeholders should be ready to begin converting the results map into a results framework. In many situations, a smaller group of stakeholders is engaged in this undertaking. However, the wider group can participate in preparing a rough draft of the framework, using simple techniques and without going into the details and mechanics of RBM terminologies CREATING THE DRAFT RESULT FRAMEWORK Table 5 provides a starting point for converting the results map into a draft framework for programme and project documents. It shows how to translate some general terms and questions used in the planning session into the common programming language. The table can be used to produce an initial draft of the results framework with all or most stakeholders. It can be particularly helpful at the project level or in situations involving a diverse group of stakeholders. FORMULATING STRONG RESULTS AND INDICATORS Having a smaller group of persons with greater familiarity wit...

MONITORING AND EVALUATION

Search This Blog