Endline Survey and Final Evaluation at Handicap International (HI)

Project Background

Since 2013, HI has implemented phase one of the Ubuntu Care project in Rwanda, Burundi and Kenya, which includes activities and research designed to more effectively identify and address factors related to sexual violence against children with and without disabilities. All project stakeholders agreed that the extension of the project to a second phase was necessary; by building on phase one to develop a more integrated system with a view to confronting sexual violence against children. Phase two of the Ubuntu Care project was developed and continues to address the following vulnerability factors: the poverty of families with children with disabilities; isolated children and ‘invisible’ cases; sexual violence in schools; unreported cases and out-of-court settlements between perpetrators and families of survivors; delayed medical consultation; socio-cultural norms and beliefs; obstacles to or lack of justice; and, the absence of appropriate psychosocial support.

This multi-country project, funded by the Agence Française de Développement (AFD), aims to address the root causes of sexual violence against children, including girls and boys with and without disabilities in Rwanda, Burundi and Kenya. The strategy is to effectively translate international commitments on children's rights, in particular the Convention on the Rights of the Child (CRC) and the Convention on the Rights of Persons with Disabilities (CRPD), into concrete actions on the ground through improved mechanisms to respond to sexual violence against children at all levels. This project deploys a model of an inclusive child protection safety net encompassing children, families, communities, services and authorities to protect children from sexual violence and support survivors and their parents/guardians to facilitate access to immediate assistance, and long-term reintegration and sustainable inclusion.

The “inclusive child protection safety net” is a systemic approach that empowers children and child protection actors and promotes interaction between these actors within a defined geographical area. The project’s goal is to create a more integrated protection system (health, education, legal/judicial, psychosocial and community sectors) to provide quality and timely services that are age-appropriate as well as gender and disability sensitive.

2. Final Project Evaluation and Endline Survey

2.1.Presentation of the project to be evaluated**

Project Title

Ubuntu Care : Confronting sexual violence against girls and boys, including girls and boys with disabilities, in Rwanda, Burundi and Kenya

Implementation Date

January 2016- December 2018

Project Locations

Rwanda: Rutsiro District in 4 sectors: Mushubati, Gihango, Ruhango, Boneza

Burundi: Gitega province in 3 communes: Gitega, Makebuko and Itaba

Kenya: Trans Nzoia County in 2 sub-counties: Western and Eastern Trans Nzoia

Operational partners

Rwanda: Association Pour l’Encadrement Sûr des Enfants orphelins et autres enfants vulnérables de Kivumu (APESEK) and Centre Komera

Burundi: Association Burundaise des Amis de l’Enfance- Famille pour Vaincre le Sida /Amade (FVS-/Amade) and Centre Humura

Kenya: Chanuka Youth Development Program (CYDP) and Catholic Diocese of Kitale (CDOK)

Target groups

18,000 children at risk from sexual violence; 1,200 child survivors of sexual violence (750) and/or isolated children with disabilities (450); 4,800 members of families of child survivors of sexual violence or isolated children with disabilities.

Project Budget

1,260,000 Euros

Project Objectives

Overall objective: Reduce sexual violence against children, including girls and boys with and without disabilities in Rwanda, Burundi and Kenya. .

Specific objective: Implement the inclusive child protection safety net model by empowering children, families, communities, services and authorities to protect children against sexual violence, help survivors and their parents/guardians access immediate assistance, and promote their sustainable reintegration and inclusion in Rwanda, Burundi and Kenya.

Estimated results

R.1: Girls and boys, including girls and boys with disabilities, are empowered to become actors in their own protection and provided with support to play an active role in society and relevant services

R.2: Strengthen the role of families and communities in protecting girls and boys, including girls and boys with disabilities

R.3: A multi-sectoral and coordinated child protection system provides inclusive and quality services for children survivors of sexual violence.

R.4: In line with international and regional human rights instruments, GBV and child protection policies related to sexual violence against children are effectively implemented, and the particular vulnerability to sexual violence of children with disabilities is acknowledged and addressed

Main activities implemented

• Build the capacities of child advisory committees to strengthen the participation of children in the development, monitoring and assessment of project activities and ensure their recommendations are taken into account
• Empower children, including children with disabilities, to become agents for change and effective child protection actors
• Support and training of local partners to manage project implementation.
• Support and training of institutions, community chiefs and champions, community members to protect children from sexual violence and provide support to child survivors.
• Strengthen the protective role of vulnerable families through positive parenting education and connection to support services
• Support and training of multi-disciplinary taskforces / one stop centres
• Support and training of schools/specialised centres to develop/implement action plans in order to become safe schools/centres for children.
• Support to hospitals and health centres to define / implement an action plan improving medical and psychological care to SV survivors
• Support local legal services to define/develop an action plan to improve the care, treatment and follow-up of child survivors, disability inclusion and access to legal aid
• Document and publish lessons learned at regional level, models and good practices on the protection of children against sexual violence, including children with disabilities.
• Strengthening of a data management system from local to regional level on sexual violence against children, including children with disabilities.
• Development of a comprehensive and results-based strategy, 3 year action plan and alliances for advocacy from local to international level
• Local and national advocacy on the protection of children against sexual violence, including children with disabilities.
• Regional and international advocacy on sexual violence against children and disability targeting governments, United Nations and other international agencies.

2.2.Theory of change of the project

A theory of change is focusing on modelling, replication and participation of key agents of change (children, families, communities, service providers and policy-makers). To bring about lasting change at local level, the activities build the capacities of these actors and local social systems for children (through committees, clubs and forums for children): at community level through child protection committees to identify and report cases of SV against children and provide survivors with guidance; and at district/province/county level by building the capacities of task forces/one-stop centres to provide care, treatment and follow-up. By building on this momentum, it will be possible to create safe environments and ensure survivors enjoy equal opportunities

2.3.Reasons for the evaluation

The Ubuntu Care project is ending on 31st December 2018 and, in line with the HI Planning, Monitoring and Evaluation Policy and the existing agreement with the donor, a final evaluation and endline survey will be conducted. This will facilitate accountability to beneficiaries and other stakeholders, as well as the donor, while also supporting the gathering and documentation of best practices and lessons learnt. They will also enable the project to evaluate its final progression against individual objectives.

3. Objectives

3.1.General objectives**

The end line survey will include collection and analysis of quantitative and qualitative data corresponding to the level of attainment of indicators.

The final evaluation will assess outcomes and changes arising from project interventions in relation to children, communities, services and policies, as well as, identify key lessons learnt and best practices. It will examine level of achievement of project goal and outcomes, establish project effectiveness, efficiency and sustainability.

3.2.Specific objectives of the Endline Survey**:

• To collect quantitative and qualitative data at the end of the project for the chosen indicators[1] using among other tools, a KAP survey
• To provide a meaningful understanding of progress toward targets and changes in the indicators being measured
• To highlight significant, important and interesting findings that might inform future programming
• To analyse results and produce a comprehensive endline report

3.2.1. Specific objectives of the Final Evaluation:

• To assess the achievements of the expected results, as well as to capture unintended outcomes
• To assess the changes in the lives of children (boys and girls) with and without disabilities, their families, communities and the service providers in the project intervention areas
• To determine and qualify the level and the quality of the interaction/connection between the active actors in child protection safety net
• To highlight the national/regional policies changes due to advocacy work
• To identify and document specific cases on how sexual violence influence disability
• To assess the level of children participation using levels of child participation[2]
• To assess the satisfaction level of beneficiaries: families, children
• To capture lessons learnt and best practices that HI can build on for future interventions
• To produce final report, including recommendations for future improvement

3.3.Evaluation and Endline survey criteria and key questions**

In particular, the consultant will assess the following criteria (please note that this list is indicative and not exhaustive and will be validated during the evaluation scoping stage)

QUALITY CRITERIA

KEY QUESTIONS

Changes

What are the short and medium term changes in the lives of children, their families and communities?

Has the project created anylong term positive effects?

Has the project led to unintended positive or negatives changes?

Has the participatory and empowerment approach of children contributed to any long-term effect on children, their families and communities?

Effectiveness

To what extent wereplanned objectives and results achieved?

Did the project have the necessary resources (human, financial, logistical and technical) to achieve its objective?

Have the project assumptions affected project achievements?

Efficiency

To what extent were project results efficiently delivered in regard to the implementation strategies, the use of resources–learning materials, facilities, funds, equipment etc.?
Were the project implementation approaches efficient?
Could the same results have been achieved by the project with fewer resources?

Capacities

What are the main lessons learnt of this project?

What are the best practices of this project? Why are they best practices?

Are there any innovative practices that this project can capitalize on?

What are key significant changes stories from beneficiaries (children, teachers and parents)?

Sustainability

Did the project help to reduce the vulnerability of children and families and to increase their response capacity?

To what extent are project results sustainable upon project closure/withdrawal?

Could the project be scaled up?

What are the risks facing sustainability of project outputs and outcomes?

Examine the progress towards outputs and Outcomes indicators

To what extent has the project achieved project outcome and output indicators?
What factors contributed to this level of performance?

4. Evaluation and Endline Survey Methodology

4.1.Collection method**

It is expected that the overall methodology adopted for the evaluation and endline survey shall generate both quantitative and qualitative information, and specific methods that will be employed should be participatory and inclusive ensuring participation of all key stakeholders. Where possible, data should be triangulated to ensure a robust assessment.

The endline survey and final evaluation should use innovative and mixed methods. It should thus reflect the following requirements:

A desk review of all relevant documents provided by HI (projects documents, baseline report, PM box, mid-term evaluation, others)

Qualitative methods ensuring a deep analysis of all data collected and highlighting most significant changes. This will include interviews, FGDs and other tools to collect relevant information

Quantitative methods shall include well developed study tools or survey questionnaires (KAP survey) to be administered to beneficiaries to collect figures related to the progress of selected indicators. The lead consultant will be responsible for further development of the methodology and the evaluation tools demonstrating how data for each evaluation objective will be collected. He/she will hire and train the assessors..

Comparative approach: the methodology should contain a comparative analysis between the starting period and the end, the three countries, boys and girls, children with disabilities and children without disabilities, various types of disabilities of sexual violence. It will be necessary to highlight if, how and when sexual violence provokes disability and vice-versa

Disaggregated data: all data should be collected in a disaggregated manner, including age, gender and disability disaggregation

Child friendly approaches: all tools used with children must be adapted to children (age, gender and disability)

Following HI’s evaluation framework: the consultant will be provided with a HI quality framework and is expected to be guided by it

4.2.Actors involved in the evaluation**

For conducting the evaluation, various teams will be involved:

4.2.1. Regional coordination

The regional coordination team will be responsible for the effective planning and implementation of the overall activitiy.

4.2.2. A steering committee (COPIL)

• Composed by the Regional Coordinator, Regional Quality Technical Advisor at Lyon, all operational coordinators in Rwanda, Burundi and Kenya, two regional officers (finance and M/E). Its main task is:
• Select and/or validate the choice of service provider
• Supervising all the activities done by the consultant,
• Participate in and/or validate the scoping exercise and inception report
• Induct the evaluation team
• Monitor the data collection and/or validate the results
• Contribute to formulate and/or validate the conclusions of the evaluation
• Contribute to formulate and/or validate the recommendations
• Contribute to formulate and/or validate the action plan for implementing the recommendations

4.2.3. Partners and beneficiaries

The partners and beneficiaries will be contacted to be participate in evaluation/endline data collection activities.

5. Principles and values

The consultant must be compliant with HI’s ethical rules and protection policies (child protection, protection from sexual exploitation and abuse, fight against bribery), including respecting HI’s main values: humanity, inclusion, integrity and engagement. The consultant has to involve beneficiaries and consider their views as HI is accountable for them. All the outputs e.g. reports, documents, information etc produced by this evaluation will be treated as HI’s and its partners property and consequently confidential. Therefore the above mentioned outputs or any part of it cannot be sold, used or reproduced in any manner.

6. Expected deliverables and proposed schedule

6.1.Deliverables**

Inception report: it should outline project context, evaluation and endline survey objectives and issues, the evaluation line of questioning, work plan (data collection tools, data analysis methods, procedures for formulating conclusions and recommendations) and the procedures for Exchange with COPIL (steering committee).to be approved by COPIL prior to engaging in field work

A debriefing report: summarizing the main findings from the evaluation and endline survey

A draft final evaluation and endline reports : prepared by the consultant following HI’s requirement and discussed with COPIL

An executive summary : outlining main lessons and recommendations

A final evaluation report incorporating COPIL’s comments

A final endline report prepared by consultant and approved by COPIL, incorporating COPIL’s comments

The consultant shall ensure the completion of evaluation activities in accordance with the terms and conditions of the consultancy contract. The consultant will constitute and coordinate evaluation team members to conduct the endline evaluation and conclude it to the required quality standard. The consultant is expected to coordinate field level data collection, the day-to-day management of evaluation activities, data entry and analysis and writing the evaluation report.

N.B: All the reports have to be produced in both languages French and English

6.2. Timeframe (without travels)

Specific tasks

Deliverables

• Desk review and prepare inception reports
• Inception report: evaluation tools and a detailed survey plan
• Induction workshop
• With steering committee (COPIL)
• Training data collectors
• With all data collectors
• Field visits (FDGs, interviews, survey plan for endline)
• Data analysis
• Workshop preparation and facilitation
• Present key findings
• Conduct participative analysis of key findings with project staff
• Final report draft
• Draft Evaluation and Endline reports: complete set of data collected during end line survey
• Incorporation of comments and submission of final reports
• Submit final report evaluations and endline survey (brief and detailed) should include an executive summary, recommendations and annexes

7. Resources

7.1.Expertise required from the consultant

Required

Desirable

• Master’s degree in social science or relevant field, or recognized equivalent work experience.
• Experience in designing and conducting baseline, end line, and in project assessment preferably in child protection
• Experience in child protection
• Understanding of disability inclusive development, including the social model of disability
• Excellent language proficiency in English and French language (speaking and writing)
• Proven excellent communication skills
• Proven experience in participatory, child-friendly methods and facilitation (interviews, FDGs and other relevant methods)
• Proven strong analytical and report writing skills
• Experience working with children with and without disabilities
• Experience in working on sexual violence against children
• Good understanding of child protection systems in Sub Saharan Africa (preferably in Rwanda, Burundi and Kenya)
• Fluency in local language (Kinyarwanda, Kirundi and Swahili) or hiring local translators
• A demonstrated commitment to the rights of children and beneficiaries with and without disability
• safe and fair working conditions, respect, dignity and equality
• Understanding of gender equality and gender based violence
• Understanding reciprocal link between sexual violence and disability

7.2.Consultancy fees**

The total budget allocated for this consultancy is a maximum of 18,000 Euros. Expenses for international travels, data collectors, participants’ perdiem and transport, accommodation and translation fees are included in this budget. Other expenses such as project’s team perdiem and accommodation are paid by HI. Payment will be released in two instalments:

30% upon signature of contract

70% upon submission of final validated report

7.3.Resources available to the evaluation

HI’s vehicle for transportation (the consultant, data collectors and HI team)

HI’s staff for accompaniment

Project modules: proposal, baseline, logical framework, mid-term evaluation, project review, report template, HI quality framework, HI policies…

[1] See Annex 1

[2] UNICEF : Florence (1992) ; Children Participation : from Tokenism to Citizenship, levels of child participation adapted from Hart, Roger A.