End-Term Evaluation of the Project at Finn Church Aid (FCA)

Background of the evaluation

FCA Kenya is currently implementing a one-year project funded by the Bureau of Population, Refugees and Migration (BPRM) education project namely Right to Inclusive and Safe Education in Kalobeyei (RISE) in Kalobeyei Settlement. The project is implemented across 7 primary schools that have a population of 21,438 learners (56% boys,44% girls) and 229 teachers. The goal of the project is to provide inclusive and equitable quality primary education in a safe environment for refugee and host population boys and girls in Kalobeyei Refugee Settlement.

The project addresses the specific needs of the most vulnerable children, including children with disabilities (CwDs), out-of-school children (OOSC), and overage learners’ children in Kalobeyei Settlement and surrounding host communities in Kenya. The activities aim to improve access to quality education through primary teacher education training, enhance the protection, safety, and well-being of learners in the schools, and increase access to safe and inclusive primary education. The project also focuses on the psychosocial well-being of both the teachers and learners in the schools. The project is ending in July 2022 thus the need for end term evaluation. A baseline survey was conducted in the month of November 2021 and gives an overview of the situation at the beginning of the project.

Overall project impact: The goal of RISE is to provide inclusive and equitable quality primary education in a safe environment for refugee and host population boys and girls in Kalobeyei Refugee Settlement.

Project Objectives:

1. Objective 1: Increased access to safe and inclusive primary education for refugee and host population boys and girls in Kalobeyei refugee settlement.
2. Objective 2: Improved protection, safety, and well-being for refugee and host population primary school boys and girls in Kalobeyei refugee settlement, with a focus on the most vulnerable.
3. Objective 3: Improved quality of primary education for refugee and host population boys and girls in Kalobeyei refugee settlement.

Rational, purpose, and priority objectives of the evaluation

The overall purpose of the end-term evaluation is to assess the continued relevance of the action and the progress made towards achieving its planned outcomes. It will also determine in an objective manner the progress of project outcome indicators to inform the program implementation team on the status of planned milestones based on observed actuals. The results will also be used by FCA for learning to continue to inform the programming of EiE and accountability to the donor.

The scope of the evaluation entails conducting a detailed analysis of the project within its implementation context and establishing the extent to which the project has contributed to the FCA Quality Education mandate. The evaluation will mainly focus on the project activities Kalobeyei settlement of Turkana west sub-county and will target project beneficiaries, implementing partners, staff, and other stakeholders,

The specific objectives of the end-term evaluation are as follows:

• To evaluate the Relevance of the project activities to the context and stakeholders.
• Measure the efficiency of the project in relation to the resources.
• Assess the overall contribution of the action to improved quality education delivered to Boys and girls in Kalobeyei Settlement.
• Document challenges, programmatic lessons learned, and key recommendations for program improvement.
• Analyse the change brought by the project implementation through comparison with the baseline report.

Scope of the evaluation

The end term evaluation will be done across the 7 primary schools in the Kalobeyei settlement managed by FCA. The participation of all the stakeholders of the project is essential. This includes learners, parents, BOMs, PTAs, partners, government/County officials, and project staff. The duration of the assignment will be fifteen (15) days including travel. The evaluation will begin on July 15th, 2022; however, preliminary works may begin earlier such as the development of the tools and review of the final study design. The final report incorporating feedback from Finn Church Aid will be submitted 3-5 days after the presentation of the preliminary findings.

Evaluation question

1. To what degree have project outcomes been achieved? Were there any unexpected outcomes?
2. How did the project perform against SPHERE standards for the construction work?
3. What has worked well, and what has not worked well?
4. To what extent has the intervention been effective and coordinated between different actors, promoted synergies, and avoided gaps, duplications, and resource conflicts?
5. Evaluate the project against the OECD DAC Criteria:
   1. Relevance: to what extent did the intervention objectives and design respond to beneficiaries’ needs and priorities and continue to do so if circumstances change?
   2. Effectiveness: to what extent did the intervention achieve or is expected to achieve its objectives, and its results, including any differential results across groups?
   3. Coherence: compatibility of the intervention with other interventions in the same location?
   4. Efficiency: to what extent the intervention delivers, or is likely to deliver, results in an economic and timely way?
   5. Impact: to what extent the intervention has generated or is expected to generate significant positive or negative, intended, or unintended, higher-level effects?
   6. Sustainability: to what extent to which the net benefits of the intervention continue, or are likely to continue?
6. Evaluate the functionality and areas of improvement of the complaints and response mechanism established by FCA.
7. Assess the awareness of the beneficiaries on the child protection referral pathways.
8. Impact of the project on the safety and well-being of children within the school environment.

Methodology required & available data

The evaluation should employ a mixed-method approach in data collection. It is important to have clearly defined ways of collecting both qualitative and quantitative data. The use of mobile data collection is preferred.

Evaluation process

Preparation Phase:

• Conduct a review of the baseline report, the proposal, and the logical framework of the project.
• Meet with the relevant program team to discuss the current situation of the current project areas.
• Determine the timeline for the survey and the evaluation team needed.
• Write up the protocol of end term evaluation, including background, aims, specific objectives, methodology, sample size, sampling procedures, key indicators, evaluation team, ethical issues, timeline, data collection, field supervision, quality control, data entry, data cleaning, data analysis, and reporting
• Update the baseline evaluation to be used for end term evaluation tools (questionnaires) based on the indicators presented in the log frame.
• Criteria for final evaluation should be scoped on Impact, Sustainability, Relevance / Appropriateness, Effectiveness, and Efficiency.
• Present and discuss draft protocol and tools with relevant members. Make revisions as needed.

Skills and qualifications required

• Post-graduate degree in Education, Social Sciences, Statistics, or related field.
• Minimum of 5 -10 years’ relevant professional experience in education programming and research.
• Strong background and experience in data collection and analysis; experience in and able/willing to use mobile technology (tablets) for data collection
• Proven experience in undertaking evaluations/research or leading evaluation/research teams, with outstanding skills in qualitative and quantitative research and data analysis using relevant software such as STATA or SPSS and ATLAS
• Technical expertise in assessing issues of education demand and supply, and the ability to draw strong and valid conclusions