Consultancy to Conduct End of Project Evaluation using Outcome Harvesting Methodology at Concern Worldwide

Background and context

DFID’s Deepening Democracy Programme (DDP) has been funding two (2) projects under the Climate Change Governance (CCG) thematic area in Marsabit County, Kenya.

The 1st Project, the Accountable and Inclusive Management of Natural Resources and Livestock Production Systems in Marsabit County is a 21-month project implemented by Concern Worldwide. The project duration is January 2018 to September 2019.

The project seeks to address how to protect and promote pastoralist livestock production by facilitating sustainable management of natural resources and access to affordable and reliable veterinary services. The project also seeks to mobilise community-based institutions (customary and emerging) to engage with the county government in the management of natural resources and veterinary services. In doing so, it is building the capacity of communities to participate in natural resource management, facilitating communities to map natural resources that are critical to their livelihoods, and promoting the recognition of customary natural resource management practices by the county government. Concern Worldwide and the County Government of Marsabit are jointly implementing the Project, specifically, the Departments of Environment, Water and Natural Resources and the Department of Agriculture Fisheries and Livestock Development. The project is implemented in 10 wards spread across 4 sub counties namely Moyale sub-county (Sololo,Uran wards); Laisamis sub-county (Korr/Ngurnit, Logologo & Laisamis wards); and North Horr sub-county (Dukana, Maikona, Turbi, North Horr, Illeret wards).

The 2nd Project, Strengthening Citizens-County Government Collaboration in Water Governance for Enhanced Community Resilience in Marsabit County is a 19-month project focusing on water governance and management sector in Marsabit County implemented by Christian Aid (CA) in partnership with a local partner based in Marsabit – Marsabit Indigenous Organization Network (MIO-Net). The main aim of this project is to contribute towards equitable and inclusive climate resilient development in water security for Marsabit and seeks to achieve three (3) key results, namely; (i) Increased citizen voice and participation in water governance, ii) improved county led collaboration of water sector players within the county, iii) functional community level structures in water management and development of a water resource database.

The 2nd project is implemented in 20 wards spread across 4 sub counties namely Moyale sub-county (Butiye, Sololo, Helilu/Manyatta, Golbo, Moyale Township, Uran & Obbu wards); Laisamis sub-county (Loiyangalani, Kargi/South Horr, Korr/Ngurnit, Logologo & Laisamis wards); and North Horr sub-county (Dukana, Maikona, Turbi, North Horr, Illeret wards) Saku sub county (Karare, Mountain, Sagante/Jaldesa).

Project outcomes and outputs

The Projects seeks to contribute to the following:

• Project 1: Accountable and Inclusive Management of Natural Resources and Livestock Production Systems in Marsabit County
• Project 2: Strengthening Citizens-County Government Collaboration in Water Governance for Enhanced Community Resilience in Marsabit County
• Impact: More accountable and inclusive management of natural resources and veterinary services in Marsabit County.
• Impact: Accountable governance in water security in Marsabit County
• Outcome: Improved citizen engagement in the sustainable management of natural resources and improved veterinary services in Marsabit County.
• Outcome: Effective, accountable and inclusive water governance in Marsabit County

Outputs

• Output 1: Increase capacity of customary and emerging institutions on sustainable natural use and engagement with county government for recognition and enforcement of indigenous natural use practices.
• Output 1: Increased citizen voice and participation in water governance
• Output 2: Support coordination and policy development for sustainable natural resource management and veterinary services at the County Level.
• Output 2: Functional community level structures in water management
• Output 3: Learning and adaptation integrated into Programme management.
• Output 3: Increased engagement and responsiveness of County Government in Water Management

Evaluation purpose and objectives

The purpose of this evaluation will be three-fold as follows 1) To provide an independent assessment of the projects in order to identify and understand extent to which the projects implemented by Concern Worldwide and Christian Aid have achieved the expected results 2) Assess the intended and unintended outcomes of the project and to determine the level of the project’s contribution to these outcomes and 3) Identify relevant lessons learned related to the project operational contexts to inform design and implementation of future Climate Change Governance projects and Concern’s and Christian Aid’s programming.

The specific objectives of the evaluation are to:

• Examine the logical frameworks to develop detailed evaluation questions.
• Identify and describe the outcomes (expected and unexpected) from the project implementation.
• Establish the contribution (causal link which is either direct or indirect) of the project to the identified outcomes.
• Identify good practices and lessons learnt related to the project operational contexts, actors engaged, strategies applied in the implementation and changes observed.
• Establishing what changes occurred and contribution by the project and assess the significance and sustainability of the outcomes

Evaluation questions

• Using an Outcomes Harvesting approach, the evaluation seeks to answer the following evaluation questions:
• To what extent have the Projects contributed to a). improved citizen engagement in the sustainable management of natural resources and improved veterinary services and to strengthening Citizens-County Government Collaboration in Water Governance for Enhanced Community Resilience in Marsabit County?
• To what extent have the Project contributed to more participatory and responsive government processes?
• What do the outcomes of the Projects imply for how governance in strengthening programs in societies in transition should be designed?

To what extent has the project contributed to DDP – ICF Climate Change (CC) Key Performance (KP) indicators of:

• strengthening county CC policies and institutional frameworks, community CC systems; strengthening capacity of citizens and county officials – and ultimately, extent to which the Project has contributed to systemic improvements in planning and decision-making on CC in Marsabit
• Level of integration of climate change (water, environment, natural resources into County Government planning.
• Level of knowledge of climate change, water governance and natural resources among formal institutions within the County.

To what extent are each of the identified intended or unintended outcomes significant to the project context?

What is the project contribution to each of the outcomes? What was the project’s investment to achievement of the outcomes? Who else contributed to these changes? What is the evidence of the project’s and other sources of contribution?

How sustainable are changes observed?

The evaluator will be free to adjust or add to these questions to ensure sufficient data is collected.

Scope of the evaluation

The evaluation will cover all project activities from March/April 2018 –June/July 2019 (Implementation phase)

Evaluation approach and methodology

This evaluation will use the Outcome Harvesting (OH) evaluation approach. The focus on the Outcome Harvesting evaluation is to interact with various categories of target actors with the aim of establishing how they have acted differently because of the project activities. The evaluator will engage the project team, local partners and target actors in identifying and documenting significant outcomes realized during the project implementation. The evaluation team will clearly describe the outcome-monitoring methodologies and clearly outline how these will used practically during data collection. Throughout the evaluation, the evaluation team will be guided by the four standards of evaluation of the American Evaluation Association:

• Propriety: Ensure that the evaluation is conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.
• Utility: Ensure that the evaluation serves the information needs of intended users and be owned by them.
• Feasibility: Ensure that the evaluation is realistic, prudent, diplomatic, and frugal.
• Accuracy: Ensure that the evaluation reveals and conveys technically adequate information about the features that determine worth or merit of the program being evaluated

The outcomes information will be collected through:

• Reviewing the project documents including monitoring data on the outputs and outcomes and identify preliminary outcomes
• Engaging with “focal points” (project staff and county government officers)
• Conducting interviews with project team to expound on the identified outcomes and generate more significant outcomes
• Conducting interviews with project team (staff and county government officers) across 10 target wards to further discuss already harvested outcomes and further identify more outcomes. The will visit the 10 wards and hold beneficiary interviews and focus group discussions.
• Holding focus group discussions with target beneficiaries at ward level to further generate evidence on the documented outcomes
• Holding analysis workshops with project team to make sense of the identified outcomes and sample outcomes to be developed further
• Collecting stories of change using the Most Significant Change methodology. Collect at least 1 change story per ward and select 3 Most Significant Change stories that will be part of the final evaluation report.

Evaluation outputs and deliverables

The consultants are expected to undertake the following tasks:

• Participate in briefing and consultation meetings to discuss the assignment, become familiar with the Climate Change Projects in Marsabit and to collaborate with Concern Worldwide and Christian Aid project and partner staff and other project implementing partners on the development of the OH Evaluation design and implementation plan.
• Develop an inception report detailing the evaluation methodology, work plan/schedule and draft data collection tools.
• Develop and submit an acceptable Draft OH Evaluation Report for feedback by Concern, Christian Aid and DDP staff following an agreed upon format. The report should be comprehensive and provide detailed, specific findings for each of the evaluation questions and considerations/recommendations.
• Plan for and facilitate a Validation Workshop of evaluation findings with project partners and stakeholders after generating the draft OH evaluation report. The evaluator shall share a short presentation of outcome findings, insights into contribution pathways and feasible recommendations with the project team at least 3 days before the validation workshop.
• Develop and submit a Final OH Evaluation report of no more than 40 pages (excluding executive summary, references & annexes) following an agreed upon format.
• Most Significant Change Stories: At minimum of 3 - one-page Most Significant Change stories for each of the projects must be part of the annexes.
• Develop and provide a summary of key findings in the form of Data Infographics format to be used as a quick and engaging way of sharing the key findings.
• Develop and provide a summary of findings in (MS PowerPoint) to be used for subsequent dissemination.
• Submit other documents/materials such as raw data files, quantitative data files, transcripts of FGDs, photographs taken, outcomes matrix, at least one change story per ward, per project should also be submitted to Concern and Christian Aid offices in Nairobi for reference and future use.

Team requirements and qualifications

The consultancy/harvest team should comprise a minimum of three consultants with vast knowledge of outcome harvesting approach, climate change governance, natural resource management and water governance in arid and semi-arid contexts. The team should possess the following skills and experience

• Evaluation expert
• Degree in Monitoring and Evaluation, Social Sciences or any other relevant field
• Demonstrable previous experience in conducting end of project evaluations using outcome harvesting approach
• Have vast knowledge on devolved governance, natural resource management and water governance in a pastoralist context in the arid and semi-arid areas of northern Kenya
• Strong understanding of communities and their interaction with government
• Excellent report- writing skills (Concern and Christian Aid will not provide editing services)
• Team player and willingness to lead the evaluation with Concern's Governance and Christian Aid’s project Team
• Fluency in English, Kiswahili
• Climate Change Governance Expert
• Masters in Environmental Sciences, Natural Resource Management in or any other relevant field
• Demonstrable previous experience in conducting end of project evaluations using outcome harvesting approach
• Have vast knowledge on devolved governance, natural resource management in a pastoralist context in the arid and semi-arid areas of northern Kenya
• Strong understanding of communities and their interaction with government
• Excellent report- writing skills (Concern and Christian Aid will not provide editing services)
• Team player and willingness to lead the evaluation with Concern's Governance and Christian Aid’s project Team
• Fluency in English, Kiswahili

Water Governance Expert

• Technical background in Water Governance, Resilience Building, Climate Change Adaptation (at a minimum of Master’s Degree level)
• Demonstrated experience and expertise in designing and managing evaluations using Outcome Harvesting Approach for similar projects and in delivering agreed outputs on time.
• Demonstration of experience in implementing rigorous research using Qualitative and participatory evaluation methodologies
• Fluency in English and Kiswahili
  good knowledge of the context in Kenya and Marsabit County is an added advantage

Management and coordination

The Senior Manager Monitoring and Evaluation at Concern Worldwide together with the Monitoring and Evaluation Officer at Christian Aid will manage the consultant in close consultation with the Senior Managers for Governance and Livelihoods at Concern Worldwide and other key program staff at Christian Aid. The consultant will also work closely with the County Government of Marsabit Departments for Environment, Water and Natural Resources and the Department for Agriculture Fisheries and Livestock Development.

Timeframe

The evaluation will take place 10 July 2019 to 22 August 2019. The evaluator will submit the first draft of report by 07th August 2019 for review and feedback by the project team, second draft by 15 August 2019. Final report must be submitted by 22nd August 2019.

Key deliverables

Timelines

• Selection of consultants By 08th July 2019
• Contacting of selected consultants By 10th July 2019
• Inception meeting By 12th July 2019
• Field work From 15th July - 29th July 2019
• Submission of draft report By 07th August 2019
• Presentation of findings in validation workshop By 15th August 2019

Submission of final report to Concern & CA By 22nd August 2019