Request for Proposal for an Endline Evaluation Consultant at Aga Khan Foundation

Objectives of the Endline Evaluation

The endline evaluation aims to:

• Assess the overall impact and effectiveness of the Accelerating local ECD solutions to scale.
• Evaluate the achievement of project outcomes and objectives outlined in the project's design.
• Understand the factors contributing to project success and identify challenges and lessons learned in delivering innovations in the community.
• Ascertain the extent to which the programme achieved its intended objectives during the implementation of activities.
• Ascertain the most efficient approaches and strategies used in achieving the project goals.
• Assess the extent to which the program achievements and systems can be sustained and how.
• To document stories of change

Scope of the Evaluation

• The endline evaluation data collection will be conducted in 60 ECD centers in two counties in Kenya (Mombasa and Kwale) from September -mid October 2023.
• The study will collect and analyze data for the following indicators from the project start up form.

Indicators

Data Sources

Data Collection Methods

Ultimate Outcome

• Proportion of boys and girls with improved gross and fine motor skills, cognitive functions, receptive and expressive language, and socio-emotional capacities after 2 years of intervention
• Pre-Primary 2 Learners
• Child Assessment Tool

Intermediate Outcomes

• Proportion of male and female pre-primary teachers with improved classroom practices disaggregated by sex and county
• ECD teachers
• Classroom Observation Tool
• Proportion of ECD centres with improved quality learning environments disaggregated by county
• ECD centres
• Learning Environment Assessment Tool
• Level of implementation of innovations in pre-primary schools
• Design teams and ECD centres
• Innovation Tracking Tool
• Focus Group Discussion
• Level of satisfaction of parents /community members with quality of ECD services
• Parents
• Parents Survey Tool
• Focus Group Discussion
• Proportion of male and female County ECD officials providing quality mentoring and support to ECD teachers as part of their core supervisory duties disaggregated by sex and County
• ECD Teachers
• Teacher Survey
• Frequency of school level visits by county ECD officials
• ECD Teachers
• Teacher Survey

Methodology

• The methodology for this endline evaluation will be a mixed-methods approach, combining quantitative and qualitative research methods:

Quantitative Methodology

• The consultant will review and refine existing quantitative data collection tools to ensure relevance and alignment with evaluation objectives.
• Quantitative data will be collected through surveys, questionnaires, or other appropriate instruments.
• Statistical analysis will include descriptive statistics, inferential statistics, and correlation analysis to measure quantitative impact.

Qualitative Methodology

• The consultant will enhance existing qualitative data collection tools and develop new tools, such as interview guides and focus group discussion outlines.
• Qualitative data will be collected through key informant interviews and focus group discussions.
• Data analysis will involve thematic analysis to extract key themes, patterns, and narratives from qualitative data.

Integrative Analysis

• Integrative analysis will involve merging quantitative and qualitative findings to gain a holistic understanding of the project's impact.
• Connections between quantitative trends and qualitative narratives will be explored to derive deeper insights.
• The consultant will interpret findings to draw comprehensive conclusions that reflect both data streams.

Deliverables:

The Endline Evaluation Consultant is expected to deliver the following:

• Final inception Report: Presentation of a finalised inception report for validation
• Data Analysis Plan: Develop/review a comprehensive data analysis plan outlining the statistical and qualitative techniques that will be employed to analyze the collected data. This plan should include details on statistical software and methodologies to be used.
• Strengthened qualitative data collection tools and approaches, with a focus on capturing diverse beneficiary perspectives.
• Dummy Tables: Provide a set of dummy tables as part of the final Endline Study Report
• Draft endline report: Submission of a draft endline report and successful completion of a validation workshop
• Final endline report: Completed study report (the report should be disaggregated at county and gender) including quantitative and qualitative study findings, completed dummy tables, data visualizations, and analysis of findings.
• Study datasets: Raw data and complete datasets in English. The Consultant will also be expected to provide a fully ‘cleaned-up’ dataset in an agreed file format (e.g., XLS, SPSS) accompanied by the code used to carry out analysis and a variable codebook.
• Transcribed transcripts: Complete, edited transcriptions and translations in English of all focus group discussions and key informant interviews conducted.
• Detailed assessment of project impact, outcomes, challenges, and recommendations.

Timeline:

• The consultancy is expected to span approximately 10 weeks, starting from 20th September 2023. A detailed timeline, including milestones and deadlines, will be collaboratively established prior to the project's commencement.

Ethical Standards and Safeguarding

• The consultant must conscientiously abide by AKF safeguarding Manual, and all members of the Consultant team must sign AKF/MECP-K’s Safeguarding Statement of Commitment upon contracting. The design and implementation of the studies must be in line with the research ethical standards including protection of participants from potential research-related harms, confidentiality, informed consent, beneficence, and safeguarding principles to protect respondents from potential harms such as sexual exploitation and abuse, sexual harassment, and bullying.

Rights to Deliverables

• Upon successful completion of the consultancy, all deliverables, including evaluation tools, reports, visualizations, and findings, will become the exclusive property of MECP-K.

Qualifications and Expertise:

The ideal candidate for this consultancy should possess the following qualifications:

• Minimum of 5 years of experience in conducting evaluations, collecting data, and producing quality mixed-methods baseline/end line education focused study reports, preferably for non-profit organizations.
• Demonstrated experience in designing baseline and end line studies including proven experience in sound sampling, mixed methods approaches (quantitative and qualitative), tool development, enumerator training, etc.
• Proficiency in statistical analysis tools (e.g., SPSS, STATA) and qualitative analysis software (e.g., NVivo, MAXQDA, ATLAS.ti).
• Demonstrated experience in employing KoBo Collect or other computer assisted personal interviewing software.
• Demonstrated experience in data management, including designing and managing gender- and age-disaggregated data and information systems capable of handling large datasets for study purposes.
• Strong analytical skills to interpret complex data sets and derive meaningful insights.
• Demonstrated experience safeguarding and respecting vulnerable populations.
• Prior experience in pre-primary education evaluations is a strong asset.
• Fluency in English is mandatory. Additional relevant languages (e.g., Swahili), an asset.

Reporting and Coordination

• The Endline Evaluation Consultant will be supervised by MECP-K MERL Coordinator for guidance, feedback, and project coordination.