End of Project Evaluation of the Transforming Rural Economies and Youth Livelihoods (TREYL) Project at Practical Action Kenya

Objective of assignment
The purpose of this assignment is to assess the current and potential sustainability and impact of the project and the extent that the project has contributed towards systems change. Across the different elements described below, the evaluation will explore the extent to which the program was gender transformative.

• Impact- The extent to which the interventions have generated significant positive or negative, intended, or unintended, higher-level effects
• Sustainability – The extent to which benefits of project interventions are likely to continue beyond the life of the project and the impacts to be sustained
• Systems – The extent to which the project has leveraged changes in the social and economic environment so that youth have can utilise and engage services, information, finance, and market actors.
• Although any learnings on all of the Organisation for Economic Co-operation and Development (OECD’s) Development Assistance Committee (DAC) criteria should be noted, the focus of the evaluation is on sustainability and impact. And the purpose is for learning so that Practical Action can understand how we can further increase our impact in future projects.

Evaluation Process:

• The consultant will provide an innovative methodology to be used in addressing the objective of this assignment. However, the consultant will be responsible for the following:

Desk review and preparation

• Complete a desk review of project documents to gain a sound understanding of the project’s original plan, objectives, and assumptions
• Complete desk-based analysis of the project’s monitoring data in preparation for the primary data collection (previously gathered by Practical Action)
• Compare the intended impact with the experienced results, identifying areas where there is misalignment and potential reasons for this.

Develop the Evaluation Design

• The consultant will develop the study design based on the findings from the design review and the project’s monitoring data, and their experience of engaging evaluation methodologies. The team will discuss and agree the following with Practical Action Kenya

The evaluation methodologies chosen, develop tools and questions to use with each group and specific methodology.

• Specific groups and numbers within the community and stakeholders to engage
• Refining the schedule and refinement of the evaluation team
• Practical Action believes using engaging and empowering qualitative evaluation methods. Therefore, focus group discussions and key informant interviews can be used. We would be looking for the inclusion of other methodologies that could include:
• Participatory Learning and Action tools
• Most Significant Change Methodology
• Outcome harvesting – especially for assessing the systems change component
• Photovoice
• Transect Walks
• Qualitative counterfactuals
• Change stories
• Field Work – in Kisumu and Homabay Counties
• Using the schedule and the chosen evaluation methodologies, both quantitative and qualitative information will be gathered. The evaluation team will be responsible for facilitating most of the activities. However, the consultant is responsible for:
• Training the evaluation team on the evaluation methodologies. Then? observing how they are being used and guiding improvement. This will include a day where the team will use tools in a test community, and then on-going observing and strengthening in consecutive days
• Leading on at least eight key informant interviews in each of the counties. Interview with County Government and private sector stakeholders is likely to provide some additional secondary data sources which need to be considered.
• Leading at least two reflection session with the communities (one for each county) to reflect on the findings so far, verifying if they are correct or not, and then gaining deeper understanding to the reasons.

Data Analysis

• The consultant will be responsible for data analysis. If relevant this may include some quantitative analysis of qualitative data too. The key findings and recommendations will then need to be drawn from this analysis.

Draft Report

• The consultant will be required to develop a draft report.

This should follow the template in Annex B of this TOR

• Both the systems change component and the Gender lens need to be pulled out clearly in the report
• The details of the evaluation tools (explaining the method and the questions) should be included in an annex
• It should focus on analysis more than stating facts (data can also be included in an annex) with clear findings
• Clear and actionable recommendations
• Sharing of Findings
• The consultant will be required to hold a session to share the findings with key staff in Practical Action. The method is up to the consultant, but it is important to make it visual and engaging giving time for participants to ask questions and interrogate the findings.

Final Report

• The consultant will update the draft report based on any feedback given by Practical Action either during the sharing session or via e-mail. This will then be submitted to Practical Action

Deliverables:

• Brief report on findings from the desk review and review of project monitoring data
• Detailed evaluation design
• Training of evaluation team carried out including a day of piloting the tools
• Field work
• Draft report
• Report review session
• Final Report

Skills Required:

• Engaging Evaluation Methodologies - Practical Action is looking to use good practice modern evaluation methodologies which are engaging and empowering of communities. We are looking for a consultant with experience in using methods such as: participatory learning and action tools, outcome harvesting, most significant change methodology, photovoice etc.

Other skills required:

• Ability to lead and guide an evaluation team
• Strong analysis skills
• Confidence to engage with county and national stakeholders
• Technical knowledge and experience of regenerative agriculture.
• Technical knowledge and experience in market systems development/systems thinking.

Terms of Payment:

• 30% - Inception report -: findings from the desk review and review of project monitoring data as well as the evaluation design
• 40% - Completion of the field trips and submission of first draft
• 30% - Validation of report and submission of acceptable final report