External Evaluation of the Project Empowered Communities of Mt Elgon Working Together for Nutritious Food, Savings and Resilience Implemented by SOET at Act for Change Consulting Ltd

The project works through solidarity groups as units of community entry and implementation. By 2022, 961 (M209, F752) members had been mobilized into 48 functional groups. In the course of implementation, the project had empowered targeted community members and had recorded several results:

• Common labor was being practiced by all the 48 SGs on their fields in addition 42 SGs undertaking common labour at three community seedling nursery sites.
• The 48 SGs had been trained on Bio-intensive kitchen gardening and 961 kitchen gardens had been established with diverse crop varieties. A total of 191,400 vegetable seedlings and 50kg of assorted seeds were planted.
• Member of 41 SGs had been trained on food forest establishment and management. A total of 12,180 trees had been planted with 820 food forests with the seven layers having been established
• Three community nurseries had been established and were functional.
• 11 SGs had been trained on Fastenaktion approach on saving procedures and credit management.

 

The solidarity groups have accompaniers referred to as 'community animators' while the nurseries are managed by appointed community members called 'nursery managers'. The animators facilitate training of groups, offer close monitoring and ensure the implementation of the trained approaches. The animators also ensure that the groups are well coordinated; conduct trainings at group level and regularly conduct home visits to group members to ascertain progress of each group and each individual/member and address any emerging challenges. The community animators continuously visit the farms of their respective farmers to offer technical support and report on progress of each group on monthly basis outlining progress made and any challenges observed.  

The project team constitutes of: Programme Coordinator who is also the CEO of the organization, a Project Officer, Technical Advisor - Agroecology, Project Accountant, Nursery Managers and Community Animators. They meet once monthly for reporting and planning and also exchange on challenges and lessons learnt.

PROJECT CONTEXT

The target project areas are characterised by good climatic conditions (soils & weather pattern) suitable for vegetable and fruit production, cash crop growing and livestock production. The area experiences two rain seasons with the main one running from March to July and the short rains September to October recording approximately 1400mm to 1800mm annually. Soils are well drained and characterized by shallow rocks. Despite these, the area has been impacted by effects of climate change due to deforestation and harmful farming practices that have resulted to increased soil erosion. Of late, the area has experienced recurrent hail stones which had been destroying crops.

Due to high inflation rate and difficult global economic situation, the country has experienced an increase in price of basic commodities and this has affected the well-being of the people targeted by the project as well as negatively affected the operations of the project. An increase in the cost of production means that the people can only produce less food further deepening the food crisis. The major crops grown are maize and beans mostly for home consumption but the surplus is sold. The Mt. Elgon people have been a farming community who continue trading away their produce at reduced prices to brokers and middlemen. According to Bungoma County Integrated Development Plan, poverty level in Mt. Elgon is estimated at 56 percent.

The socio-cultural aspect of the community is characterized by most men meeting in social clubs and outdoor activities while women mostly engage in domestic activities and economically productive work. They thus are like the heads of their households even though women are also subordinated in most homes and in community decision-making processes. The issue of land ownership and control of resource’s is still capitalised by men which to some extent affects production.

EVALUATION PURPOSE

The evaluation should in a very clear manner:

• Assess and document the extent to which the project objectives have been achieved (outcome and impact level), clearly outlining and describing the impacts, both positive and negative (intended and unintended), realized so far per each specific objective.
• Give insight into the key elements of the implementation process which explain why the reported changes occurred and shed light on how such changes were achieved or caused. We expect that an assessment of impact and understanding of key processes will support the generation of Project recommendations that could be considered for the design of future project phases.
• Elaborate strengths and weaknesses of the project, and document best practices and lessons learnt. The evaluation should clearly document and highlight any programming adjustments that might be required to achieve the project desired objectives.

 

3.USERS OF EVALUATION

The primary users of the evaluation are:

• SOET: Project team and management
• Fastenaktion: Country Programme Unit, Programme Manager and Head Office Programme Development Team
• Fastenaktion's Programme Development section
• Right holders especially members of solidarity groups involved in the project to best understand effective strategies for bringing about desired change.

 

4.SCOPE OF EVALUATION

This being an external evaluation, it is expected to comply with the international evaluation standards (OECD), provide a space for learning around best practices for supporting target communities. The evaluation will be undertaken in Elgon Ward with a sampling frame of 19 villages.  Fastenaktion is seeking to identify a consultancy team, which will undertake this evaluation within the agreed terms of reference. Selection of the study sites will be agreed upon with the consultancy team. The consultancy team has to prioritize the field research questions in order to maximize on time. The evaluation process will include engagement with direct and indirect beneficiaries as well as other stakeholders.

5.EVALUATION QUESTIONS

In line with objectives of the evaluation, the evaluation will be expected to answer the following questions:

 

6.1 RELEVANCE

The evaluation should conduct an analysis of the appropriateness of the project design, activities, strategies and approaches in the light of the operational context, the timeliness of the response and its adaptation to the livelihoods security situations. It will involve assessing the extent to which the activities are suited to the priorities of the target groups, target households, and donors as well as the prevailing policy environment. Key questions to be addressed:

1. To what extent are the intended project outcomes and actual activities in line with the needs of the target group (women and men)?
2. Has the project adapted appropriately to relevant changing needs and context after its onset?
3. How are the different aspects of the project implementation relevant in attaining cross cutting themes of gender equality, Leave No One Behind (LNOB), and climate change adaptability?

6.2 EFFECTIVENESS AND IMPACT

The focus will be on the extent to which project outcomes and objectives are being achieved. Key questions to be addressed:

1. How has the food security and wellbeing of the target group (women and men) changed due to the interventions? What other context factors had an effect on possible changes and to what degree has the project tampered with or reinforced them? (cases of success stories can be highlighted and annexed)
2. What is the degree of achievement of the planned results at outcome and impact level of the project?
3. Are there any unintended project results (positive or negative)? eg. multiplier effects
4. Did the project results reflect contribution to the principle of LNOB? How and to what extent? 

6.3 EFFICIENCY

The evaluation will outline the extent to which the interventions have delivered, or are likely to deliver results in an economic and timely way. Key questions to be addressed:

1. Are costs (including time and human resources) and benefits in an appropriate relationship?
2. Are there organizational and/or management issues that hinder implementation and/or effectiveness of the project?  
3. Are there unique or practical ways in which the quality of the project can be improved in a next phase? Which are the proposed approaches and how can they be executed?

6.4 SUSTAINABILITY AND LESSONS LEARNT

Within the scope of this assignment, the evaluator(s) should assess the extent to which the project interventions took into consideration longer term needs of the target population and to what extent project results or benefits will be sustainable after project closure. Key elements to be reviewed include environmental, operational and financial sustainability of the project. Key questions to be addressed:

1. Which benefits are likely to sustain beyond the life time of the project?
2. Did the project incorporate adequate measures and strategies to ensure sustainability of results over time (regarding the partner organisation and the target group)? Which measures and strategies are these and how effective are they in influencing the sustainability prospects of the gained benefits?
3. Can this project be expanded in other geographic and programmatic areas in Kenya based on need?

 

6.6 ACCOUNTABILITY

1. How effective was the community feedback mechanisms that was in place and the capacity of the organization to respond to comments received? Is there any evidence that feedback has or is influencing decision making?

 

6.7 GENDER

1. Did the interventions contribute to furthering (or hindering) gender equality within the project area? If so, how and to what extent?  If it did contribute to furthering gender equality, why not?

6.EVALUATION METHODOLOGY

The evaluator(s) is welcome to suggest an appropriate methodological approach to successfully deliver a quality evaluation.  The evaluation methodology should be consultative and participatory, entailing a combination of desk review, key informant interviews, face to face household surveys, interviews and/or FGDs. While interviews are a key instrument, the evaluators will triangulate a range of data sources to ensure that the evaluation is sound and objective. The evaluator(s) will further elaborate on the method and approach in a manner commensurate with the assignment at hand and reflect this in the inception report, which will subsequently be approved by the programme team in consultation with key stakeholders.

A final agreement on the evaluation design and methodology will be discussed on the basis of the submitted offer and/or the inception report.

Some of the key primary data sources include:

• Interviews with key staff such as the Project Coordinator, Project Officer, Project Technical Advisor, Community Animators, Nursery Managers and others as deemed necessary by the evaluation team.
• Interviews with stakeholders including respective county government departments, and community leaders
• Household surveys, interviews and/or FGDs with women and men of the target group, and field observation in project sites within the project locations

Secondary data will be made available and it is expected that the evaluator(s) will have a comprehensive desk review of project documents. Some of the documents include project proposal and budgets, Log frame, baseline report, annual and quarterly reports, study reports among other as shall be deemed necessary by the evaluation team.

DELIVERABLES AND REPORING DEADLINES

The following deliverables are expected to be produced by the evaluation team:

• Inception report (after initial meeting with the partner organization and Fastenaktion)

The report will include a detailed plan of the evaluation process including a revised evaluation approach, an evaluation matrix stipulating for each evaluation question the data/methods that will be used, as well as the sampling method, data collection plan including methods and timelines, and feedback loops.

 

• Survey tools (with the inception report)

 

The tools should be developed in cooperation with Fastenaktion and the Partner organization. The field data collection should be informed by the evaluation objectives, evaluation questions and criteria. The survey tools and all raw data, statistical tables and coding lists (as appropriate), as well as transcripts have to be provided to Fastenaktion and the partner organization at the end of the evaluation. Data has to be disaggregated according to location and gender (minimum requirement). The evaluation team can decide to further disaggregate data in a way that improves answering of the evaluation questions.

 

• Draft evaluation report (not more than 30 pages) pages excluding annexes (after the field work) plus a power point presentation

 

The draft evaluation report is to be presented to the partner organization during a staff meeting to collect feedback. A PowerPoint presentation should be prepared for this purpose.

 

• Final evaluation report including PowerPoint presentation

The final evaluation report shall incorporate the feedback from the partner organization and any other stakeholder. The PowerPoint presentation shall be adapted to these revisions.

SUGGESTED OUTLINE OF THE EVALUATION REPORT:

1. Cover page with name of the project, logos for partner organisation and donor, name and contact details of evaluator(s), including one representative photos of the project (best option: photo from the evaluation exercise).
2. Executive summary (maximum 5 pages). This should be a section that can be used independently from the full report and should therefore cover: background to the programme, brief overview of aims of the evaluation, brief summary of the methodology, key findings per evaluation criteria and on the achievement of indicators, best practices and lessons learnt, conclusions, recommendations, and a summary of management response
3. Introduction (1 page)
4. Description of evaluation methodology (maximum 2  pages)
5.  Situation analysis with regards to the outcomes, outputs and partnership strategies (maximum 2 pages)
6. Presentation and discussion of the findings (this will be the main part of the document). This section of the report should be clearly structured to show evidence-based answers to the evaluation questions (maximum 15 pages)
7. Key findings, best practices and lessons learnt, and conclusions derived from the findings (maximum 5 pages)
8. Clear and concise recommendations derived from the findings and conclusions to provide guidance for future programming (maximum 5 pages)
9. Annexes: Terms of reference; Data-collection instruments, incl. information on informed consent handling; Project planning matrix / logframe; Travel and work schedule; debriefing power point presentation (final version); Sources (e.g. bibliography, people interviewed); Management response matrix; maps, photos

DISSEMINATION OF FINDINGS 

After a validation and stakeholder workshop, the final report will be submitted to Fastenaktion headquarters for final review and endorsement by the Programme Development Unit. This process will declare the report to the donors and the wider public.

CONFIDENTIALITY OF DATA

All documents and data collected will be treated as confidential and used solely to facilitate this evaluation. Interviewees and their pictures will not be quoted in the reports without their permission. It is expected that the evaluators will used consent forms as appropriate, and hand over to the project team. The deliverables as well as all material linked to the evaluation (produced by the evaluator(s) or the organization itself)is confidential and remains at all times the property of the contracting party.

TIMEFRAME AND REQUIREMENTS

 The Evaluation should take 15 days preferably beginning from September 4th 2023.

DAYS	TASKS	RESPONSIBLE
4 days	Initial briefings of the Programme Desk review of key documents  Draft inception report detailing evaluation tools Final inception report before the start of the consultancy	Consultancy team for interview schedule and inception Fastenaktion is responsible for sharing all relevant documents and arranging meetings with key staff and stakeholders
5 days	Field work Data collection	Consultancy team SOET Coordination office
6 days	Data analysis Draft evaluation report which is circulated internally for comments Presentation of key findings , feedback on draft evaluation, report, preparation of final report/presentation	Consultancy team SOET, Fastenaktion, coordination office

EVALUATION TEAM COMPOSITION AND COMPETENCIES

The suggested team for this assignment includes:

1. An agroecology or sustainable agriculture expert (essential)
2. MEAL (Monitoring, Evaluation, Accountability and Learning) Expert with experience in qualitative and quantitative research methods and data analysis (essential)
3. A gender and inclusion expert with experience in rural livelihoods / development (essential)

The consultancy team should be mixed in gender.

INTELLLECTUAL PROPERTY

All the materials, information and reports, the output of the evaluation exercise shall be the property of Fastenaktion and the consultancy team is bound by Fastenaktion’s confidentiality requirements. All data sets and transcripts must be provided to Fastenaktion Coordination Office and Head Office in electronic copies and on a CD (2). The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his or her own or to make use of the evaluation results for private publication purposes.

LABOUR LAW AND HUMAN RIGHTS

The evaluation team commits itself to conclude (and any one contracted by the team) must commit to execute the contract in accordance with the local labour law, respect human rights, and not to allow discrimination based on gender, lifestyles, ethnic origin, religious affiliation or political opinions. The partner organization will not tolerate sexual or other abuse. The Fastenaktion Guideline on preventing and combating the abuse of power (2020) will apply to the contract.

 

14.HOW TO APPLY:

Submission of Proposals

Based on the above, Fastenaktion is inviting interested parties to submit expressions of interest entailing technical and financial proposals. Individuals or firms applying shall detail the following:

• Consultants’ profile and capability statement describing the technical capacity and experience of the firm or group of individuals;
• Names and resume of individuals or team members proposed and their roles in the achievement of the assignment. This should also entail the proposed team structure for the evaluation;
• 3 professional referees (preferably previous clients) and sample reports of similar assignments taken in the recent past;
• Understanding of the Terms of Reference (TORs);
• Detailed evaluation design with implementation plan and timeframe; and
• A detailed budget, which should be presented in Kenya Shillings and must include all taxes and other anticipated expenses,
• Quality Control and Quality Assessment for the Evaluation

All field visits costs including transport in the field, meals and accommodation during field work will be catered by the contracting agent.