Job Vacancies at Christian Blind Mission

Program/Project : Eye-Health Promotion Program for Children in Kenya (EPPCK)

Partner Organisation(s) : Sabatia Eye Hospital, PCEA Kikuyu Hospital Eye Unit, Kwale Eye Centre, Tenwek Hospital Eye Unit and COECSA.

Project start and end date : 1st January 2016-31st December 2018

Evaluation Purpose : To inform CBM about the project towards improving effectiveness, accountability and learning for future programming by analyzing output, outcome and impact of the project on the project partners and community.

Evaluation Type : End-term evaluation

Primary Methodology : Participatory, qualitative and quantitative

Proposed Evaluation Duration : 21 Days

BACKGROUND OF PROJECT

overview

For the past three years (2016 to 2018) CBM Kenya Country Office has implemented the Eye-Health Promotion Program for Children in Kenya (EPPCK). This was a public health initiative with the overall goal of promoting child eye health Promotion, prevention, treatment and rehabilitation in order to reduce the burden of visual impairment and avoidable blindness in children in Kenya. The direct beneficiaries of the project were 2,100 people who accessed cataract surgeries, 1,000,000 people through awareness campaigns, screening of 300,000 children and 3 HeMis personnel that received training.

The EPPCK project was implemented by five project partners i.e. Sabatia Eye Hospital, PCEA Kikuyu Hospital Eye Unit, Kwale Eye Centre, Tenwek Hospital Eye Unit and College of Ophthalmology for Eastern, Central and Southern Africa (COECSA) through a cluster approach with CBM Kenya Country Office taking lead of overall project coordination. The project was on a Multi-Year plan for three years with a total budget of Euro 1.46 million, and was designed to address gaps identified from Implementation a previous East Africa Child Eye Health Project through a consortia made up of CBM, Sightsavers, and College of Ophthalmology for Eastern, Central and Southern Africa (COECSA), and Fred Hollows Foundation commonly known as Seeing is Believing (SIB).

The SIB project intended to indirectly benefit over 45 million children (total estimated population of children in East Africa) through appropriate changes in national policies and strengthening of national coordination, promoting child eye health.

The provision of quality, child friendly and child centered eye health services in the catchment areas of the programme, anticipated serving over 1 million children directly. However during the life of the project various gaps were realized that warranted further intervention. An end of project evaluation was conducted to establish relevance, efficiency and effectiveness of the project. The findings and recommendations highlighted the relevance of child eye health programming in East Africa. This provided the basis for the project design for EPPCK to address the gaps from the evaluation. The design of the project focused on quality programming to enhance child eye health service delivery through improved infrastructure and equipment, development of human resource, strengthening health systems in collaboration with national government in Kenya.

EVALUATION OBJECTIVE, SCOPE AND INTENDED USE OF THE END TERM EVALUATION

The objective of this call is to conduct an end of project evaluation of the EPPCK project that ran from January 2016 to December 2018 and to make an independent assessment of the project performance against set objectives. The result of this evaluation will help determine to what extent the project’s relevance (meeting the needs) and fulfillment of objectives, developmental efficiency, effectiveness, impact and sustainability (both retrospective and prospective) of efforts supported and also assist in improvement of future projects.

The evaluation will also focus on the quality of services provided, lessons learned, and the exportability of any part of the project (innovative design, process, products, and cost effectiveness).

Scope of the Evaluation

This evaluation shall cover the period starting January 1st 2016 to December 31st 2018. The exercise will be performed at Kwale Eye Hospital, Kwale, Sabatia Eye Hospital, Sabatia, Tenwek Hospital, Bomet, PCEA Kikuyu Hospital, Kikuyu and COECSA, Nairobi. The Programme evaluator should confirm and coordinate the locations for the exercise with partners prior to the start of the fieldwork and ensure that relevant supporting documents as well as key staff will be available during the assignment.

The Evaluator should obtain a preliminary understanding of the engagement context by reading the following documents:

• CBM Standard Contract, and Supplementary MoU signed by CBM and the partners
• The original Proposal, MYP and other related supporting documents
• The EPPCK Project Progress Reports
• The EPPCK Narrative Reports
• The evaluator will inform CBM as soon as possible about any limitations in the scope of work he/she may find prior to or during the programme evaluation.
• Start of the evaluation
• The date of signature of CBM Kenya Country Office on the contract is the official starting date of the evaluation. The Evaluator should contact partner as soon as possible to prepare the evaluation and to agree a date to start the fieldwork but not before the evaluation has been formally announced by CBM Kenya Country Office and the evaluator notified of the same.
• Procedures for the planning and conduct of the evaluation
• The evaluator must include a description in their proposal on how they plan to conduct and control the evaluation, but an appreciation of mixed methodologies with both qualitative and quantitative analyses, and data collection techniques including Key Informant Interviews, Questionnaires, Focus Groups, and document reviews.
• Complementary letter
• The evaluator may at any time during the evaluation process draw up a complementary letter if he/she considers that CBM Kenya Country Office should be informed about facts and issues that are or may be urgent or of particular interest and importance to CBM Kenya Country Office.
• This letter shall solely be addressed to CBM Kenya Country Office.

Reporting schedule

The execution of assignment, debriefing, submission of draft and final evaluation reports shall be completed in the following stages;

• Contract signing and inception meeting
• Inception Report submitted to CBM Kenya Country Office
• Detailed work plan and evaluation tools.
• Data collection tools for Field Work shared with CBM Kenya Country Office
• 1st Draft evaluation report submitted to CBM Kenya Country Office
• Feedback Received and Incorporated by Evaluation Team
• Revised Draft evaluation report submitted to CBM Kenya Country Office
• Final Evaluation report submitted to CBM Kenya Country Office.
• Target Audience and Intended Use
• The target audience will comprise of the five partners Kwale Eye Hospital, Kwale, Sabatia Eye Hospital, Sabatia, Tenwek Hospital, Bomet, PCEA Kikuyu Hospital, Kikuyu and COECSA, Nairobi hospital staff, government line ministries, beneficiaries as well as the local community and CBM staff.

Evaluation Criteria

Relevance and quality of design:

• To what extent were the objectives of the project valid in relation to fulfilling the gaps identified in the Seeing is Believing (SiB) Project?
• Are the activities and outputs of the project consistent with the overall goal, attainment of its objectives, targets and intended change?

Effectiveness:

• To what extent were the objectives achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?
• Have assumptions affected the project; has there been a risk-management in place from the beginning and how effective has this been?
• Assess the quality of the project planning documentation (situation and stakeholder assessment, time frame and milestones, personnel planning).
• Working relationships and communication between stakeholders.
• Mechanisms set up to ensure smooth implementation of the project including participation of persons with disability.
• How was information generated and used?

Efficiency or cost-effectiveness (of planning and of implementation):

• How much did the project cost to implement? Was this justified? Were activities cost effective?
• Were funds spent appropriate, and objectives achieved on time?
• Was the project or programme implemented in the most efficient way compared to alternatives?
• How far was the strategy of the project appropriate in filling the gaps identified by SiB?
• Were existing learning processes such as reflection, internal review used sufficiently to keep project plan updated?

Impact - Contribution to change:

• What changes have been brought about as a result of the project in achieving Government priorities?
• What difference has the project made to the children as target beneficiaries, and individual households?
• How many people have benefitted? It is important to use disaggregated data for gender, age, type of disability and type of assistive device provided.
• What are the positive changes that had not been anticipated or expected of the project?
• Were there also negative effects and consequences of the project?
• Is there any plan to replicate the model of this project or share best practices to influence programming in eye health?
• How is change evidenced within the institution, individual and community?
• How was the Inclusive Eye Health strategy implemented during the programme period?
• Has the project played any role in building the capacity of the partners in?

Sustainability:

• How did the partner organisations ensure that the beneficiaries are still being addressed after the project ends?
• Did the project influence sustainability of the Partners?
• How have the partner organisations ensured that the beneficiaries’ needs are still being addressed after the project ends?
• What are the challenges identified after the funding ended and how are the partner organisations adapting to them?
• Which specific results of the project will remain and continue to exist even after the project ends?

Gender

• Did the project put in place mechanism for boys and girls as beneficiaries proportionately?
• How did the various partners reflect gender sensitivity in their programming? (Balance/equality issue appear in the partner organization’s documents such as its constitution and bylaws)
• Did Gender issues affect uptake of services?
• In case Gender issues affected uptake of services what interventions were put in place to bring about equity?

Child Safeguarding

• Do the various partner organizations have and implement their own policy that safeguards children? Do they train/ induct staff on it?
• How did the project help in making the partner/work environment child friendly and safe?

Inclusion

• How was inclusion accounted for and incorporated in the project design and implementation (please relate to CBMs DID principle)?
• Do the partners implement any policies supporting inclusion?

Methodology

• The evaluation team will conduct key informant interviews with all 6 partners as implementers of the project as well as relevant stakeholders identified in collaboration with CBM and evaluators’ own mapping. This will include respective CBM KCO financial and program staff, MA project officer, Regional adviser, and government officials, among others.
• The evaluation team will source additional information through desk review of existing project planning, implementation and reporting documents and other relevant sector literature.
• The evaluation team is expected to review the partner’s data management system and the use of the data related to overall project management/implementation. The evaluation team should also correlate its field findings with the data available as documented by the various partner organizations.
• The evaluation field work should be conducted in a participatory manner to foster learning among those involved in the evaluation process. See also discussions under “Determine the target audience of the evaluation”

Limitations

• The possibility of time constraint is great in this evaluation since all the cluster partners are in different regions within the country.
  Another limitation may be the availability of government officials to be interviewed.

Evaluation Team and Management Responsibilities

• Commissioning responsibility: The CBM Kenya Country Office is responsible for commissioning the evaluation.

Evaluation Team

• The evaluation team should comprise of people with Technical skills in paediatric eye health or at least a strong background in public health.
• Strong prior experience in conducting programme evaluation, Experience in designing and managing, implementing successful programmes is required to provide well thought judgement to the evaluated project.
• Strong analytical skills and ability to clearly synthesize and present findings
• Proven ability to ensure the effective implementation of and compliance with CBMs code of conduct, including principles of transparency and accountability
  Management of the evaluation and logistics

The CBM Kenya Country Office shall:

• Coordination with partners ensuring they provide adequate support to the process
• Organising in-country logistics in consultation with the partners
• Reviewing the draft final report to ensure correctness of terms, standard procedures and the likes cited in the report.
• Providing documents available at CBM’s CO relevant to the project
• The Project Partners shall be responsibility for:
• Ensuring that data are available for checking by evaluation team.
• Organizing interviews with stakeholders, ensuring their presence
• Providing access to all documents and project staff members required by the evaluation team.

Expected Results (deliverables)

The Evaluation Team is expected to submit the following to CBM in line with agreed timelines:

• An inception report
• A detailed work plan including Task/activity, Location, Number of Days per activity and deliverables per set of activities.
• A draft evaluation report to CBM Country office Kenya using CBM’s evaluation report template;
• A presentation of the draft findings to be presented to CBM KCO prior to final report writing
• Final Report with all relevant attachments (data collection protocols/consent forms as needed, data sets analysed).

The report shall be presented in the standard evaluation report format of CBM.

Language: The report shall be presented in English with an executive summary in the same language.

Duration

The evaluation is envisioned to take a maximum duration of 23 days. An inception meeting will be planned to review the proposed work plan, methodology and data collection tools prior to commencement of the fieldwork.