Endline Evaluation Consultant at HIAS Refugee Trust of Kenya

Terms of Reference to Conduct a Endline Evaluation

Endline Evaluation for the “Increasing Protection for vulnerable refugees, including LGBTQI+, and host community in Nairobi.” Project.

With funding from the Bureau of Population, Refugees, and Migration (BPRM), HIAS is implementing the “Increasing Protection for vulnerable refugees, including LGBTQI+, and host community in Nairobi”, program over a one-year period since September 2021. The program’s goals are 1). To improve the safety, well-being and inclusion of vulnerable refugees and host communities in Nairobi through community-based protection and 2). To enhance the safety, well-being and inclusion of vulnerable refugees and host communities in Nairobi to prevent, prepare and respond to COVID-19. The program aims to achieve the following objectives:

 Goal one:

1. To enhance the protection of survivors and women and girls vulnerable to GBV, including LGBTQI+ persons, through GBV prevention interventions and response services.
2. To enhance the protection of vulnerable refugees including children and LGBTQI+ persons through the provision of integrated MHPSS community-based interventions.
3. To enhance the social and economic protection of vulnerable refugees including youth and LGBTQI+ persons through integrated services.

Goal two:

1. To reduce the risk and mitigate the consequences of GBV resulting from COVID-19 by strengthening community safety, response, and awareness.
2. To reduce the MHPSS impacts of COVID-19 and support individuals and communities to use appropriate MHPSS care and strengthen protective environments.
3. To support economic recovery activities for refugees and host communities in Nairobi whose livelihoods were affected due to COVID-19.

KEY PARTNERS INVOLVED IN THE PROJECT

The project supported and complemented the work of the Kenyan Government, UNHCR and other actors targeting critical service gaps. HIAS worked with the Government of Kenya through the Refugee Affairs Secretariat Now Department of Refugee Services (DRS) under the new Refugee act 2021. HIAS coordinated with UNHCR in different forums including monthly coordination meetings with the Urban Refugee Protection Network (URPN) and co-chaired the GBV Working Group as a lead implementing agency of GBV interventions, the MHPSS working group, the child protection working group and the LGBTQI working group in Nairobi. HIAS engages refugees as community workers and staff work closely with refugee community leaders to ensure their participation in community outreach and education activities. 

PURPOSE AND SCOPE OF THE EVALUATION

The purpose of the evaluation will be to understand how and to what extent project activities achieved the stipulated objectives including contribution of the project towards the overall goals stated above. The findings of the evaluation will also contribute to learning on the best practices on increasing protection for vulnerable urban refugees. The evaluation findings will inform HIAS’ and partners in future programming and strategic policy in related issues. The targeted users of the evaluation are HIAS, implementing partners, targeted project beneficiaries and other key stakeholders.

The detailed purpose of the evaluation is as detailed below:

• To evaluate the entire project (from start to end date), against the effectiveness, relevance, efficiency, coherence, sustainability and impact criteria, including documentation of the end-of-project value for the following cross-cutting key performance indicator, which measures the protection outlook for people served under each of the project objectives: “Percentage of beneficiaries who report an improved sense of safety and well-being at the end of the program, disaggregated by age and gender.”
• Accountability to stakeholders, including project acceptance at community level, whether and how local communities and the intended target population were consulted, whether and how the project adapted to ensure inclusion of the target population needs in the changing COVID-19 context, and how the project is/was perceived by local organizations and actors.
• Overall effects of COVID-19 on achievement of the program goals and objectives, including whether and how HIAS’ programmatic response cushioned COVID-19 shocks, supported coping strategies in intervention areas, and affected absorptive, adaptive and transformative resilience, and provide recommendations on how to improve the response to such emergencies given the operational constraints imposed by the pandemic.
• To identify key lessons and emerging good practices that can be replicated or expanded.
• Provide actionable recommendations to inform future program design for similar projects by HIAS, other project stakeholders more broadly.
• Examine critical factors that enabled or hindered effective achievement of intended results

EVALUATION QUESTIONS

The key evaluation questions to be addressed include:

Evaluation Criteria

Sample Evaluation Questions:

Effectiveness

1. To what extent were the intended project goal, outcomes and outputs (project results) achieved and how?
2. Did the project achieve its objectives and expected results in ways that contribute to improved safety, well-being and inclusion of vulnerable refugees and host communities in Nairobi? If so, how?
3. What were the major factors influencing the achievement or non-achievement of different the objectives?
4. Were there differential results for different people? If so, how and why?
5. To what extent did the mechanisms put in place achieve HIAS’ COVID -19 response strategic objectives?

Relevance

1. To what extent do the achieved results (project goal, outcomes and outputs) continue to be relevant to the needs of project beneficiaries?
2. Was the intervention designed in ways that respond to the needs and priorities of targeted beneficiaries? If so, how?
3. Are the assumptions that went into the design of the project still relevant?
4. To what extent was HIAS Kenya, as an institution been strengthened to provide a timely and relevant response to target population needs and priorities in the COVID-19 context?

Efficiency

1. The extent to which the intervention delivered results in an economic and timely way

Sustainability

1. To what extent will the net benefits of the intervention be sustained after this project ends?
2. What were the major factors which influenced the achievement or non-achievement of sustainability of the program or project?

Coherence

1. The compatibility of the intervention with other interventions in the country office and sector. What lessons can be learned?

Impact

1. To what extent has the project contributed to improve the safety, well-being and inclusion of vulnerable refugees and host communities in Nairobi (both intended and unintended impact)?
2. What real difference/changes has the project made to the lives of the target groups?
3. Were there equal or varied impact of the project on for all gender groups, vulnerable groups etc.? What were the factors for either case?

Learning

1. What best practices (approaches, modalities) worked best and can be applied to similar future programss?
2. What should have been done differently and should be avoided in such programs in future

 

SCOPE OF THE EVALUATION:

• Timeframe: this evaluation will cover the project duration: September 2021 to August 2022
• Geographical Coverage: The project locations are in Nairobi County, which include Kawangware, Kayole and Eastleigh field offices and its neighboring areas
• Target groups to be covered: this evaluation will cover all the direct beneficiaries, project implementation partners and other stakeholders.  

 METHODOLOGY

The consultant will develop a detailed research methodology in consultation with HIAS program and M&E team. The following methodologies will be utilized but should not be limited to:

1. Desk review: The consultant will review project documents, baseline evaluation report and other relevant secondary data sources.
2. Quantitative data to be collected through a representative survey.
3. Qualitative approaches, such as focus group discussions and key informant interviews, participatory exercises and approaches will be used.

Triangulation of information gathered during the quantitative and qualitative research is crucial in this study, with reflection on how the findings relate to the secondary resources. The assessment is to be consistent with SPHERE standards[1]. The endline evaluation will be organized in a participatory way, including HIAS staff, local actors and project beneficiaries. All data must be collected in a COVID-19- safe manner as per HIAS’ COVID-19 safety policies and as aligned with all national and local directives and guidelines in Kenya.

KEY DELIVERABLES

The expected key outputs and deliverables are:

1. Inception Report: This report will present the detailed methodology, literature review, data collection tools/instruments and a data collection and analysis plan. The inception report will be discussed with the HIAS evaluation team for inputs.
2. Training of enumerators
3. Pilot testing and actual data collection
4. Draft evaluation report for stakeholder review—clearly identifying information relevant to the evaluation objectives; as well as data gaps and areas that may require further assessment.
5. Raw and cleaned final dataset(s) containing all data collected for the evaluation, including survey responses and KII/FGD transcriptions.
6. Final report (incorporating inputs from the HIAS technical and program team). Report needs to meet the minimum requirements and structure (to be shared by HIAS), for HIAS’ review and approval. The main body of the report should be a maximum of 40 pages in length, excluding table of content, tables and annexes).
7. The findings will be presented in a validation meeting with key HIAS staff and stakeholders:

MANAGEMENT OF THE EVALUATION

  HIAS National M&E team and the project team (program managers and project officers) will manage the evaluation team to ensure timely delivery of quality outputs. In addition, Program Director, Country Director, and Regional MEAL. Economic Inclusion, MHPSS and GBV Advisors will provide all the necessary technical support required throughout the evaluation process. HIAS will provide:

• Relevant documentation and background information.
• Contacts of relevant stakeholders and set up meeting(s).
• Consolidated feedback/guidance on draft reports and strategy.

REQUIRED EXPERIENCE AND QUALIFICATIONS

We are looking for a consultant/team with the following skills and qualifications:

• Advanced university degree (Master’s degree or equivalent) in mental health, psychology, social work, law, political science, international relations, research, or other relevant disciplines.
•  At least five years' experience in designing, implementing and conducting evaluations of humanitarian and/or emergency programming, preferably working with refugees and/or forcibly displaced populations.
• Demonstrated in-depth understanding and practical evaluation experience in one or more of the following areas, mental health, psychology, gender-based violence, economic inclusion, or protection of refugees.
• Good understanding of the relevant international/regional frameworks pertaining to refugee protection particularly around refugee operations in Kenya.
• Experience conducting and facilitating interviews and group discussions in cross-cultural contexts.
• Experience transcribing, coding, and analyzing qualitative data, including use of relevant analysis platforms such as Nvivo, Atlas etc. Capacity to use mobile data collection systems such as KoBo Collect, and analysis of survey results.
• Excellent and demonstrated understanding of ethical and legal issues in research, including child protection and data protection regulations.
•  Excellent research and communication skills (oral and written) Oral and written fluency in English required; good command of Kiswahili language is desirable.

Qualified and interested parties are asked to submit the following:

• Letter of interest in submission of a proposal
• A detailed technical proposal clearly demonstrating a thorough understanding of this ToR and including but not limited to the following:
  • Consultant/Company Profile
  • Proposed methodology including areas of piloting, sample size determination, and a proposed consultancy work plan.
  • Demonstrated previous experience in similar assignments and qualifications outlined in this ToR (with submission of the most recent relevant evaluation report(s))
  • Proposed data management plan (collection, processing, and analysis).
  • Team composition, competencies (include CVs of each team member) and level of effort of each proposed team member
  • A financial proposal with a detailed breakdown of costs for the study quoted in Kenya Shillings.

Applications that fail to include these elements will not be considered.

TIME FRAME

Probable date for the endline evaluation to commence will be August 15, 2022 and is expected to take a maximum of 21 working days starting from the contracting date, which includes desk-review, preparation, and implementation, report-writing