TOR for Evaluation of a Pilot Project: Uptake of an Innovative Water Testing Kit in Kenya at Action Against Hunger International

• The project aims that Action Against Hunger and local partners will evaluate the adoption of WaterScope’s bacterial testing system for drinking water integrating into ongoing humanitarian responses or programmes in Ethiopia and Kenya.

The project aims to test hypotheses and to draw key learning objectives while conducting water quality testing and monitoring using WaterScope WQ testing kits. The project has three hypothesis:

• Hypothesis-1WaterScope can be easily adopted by field staff and local partners, allowing them to conduct water testing at regular intervals.
• Hypothesis-2 is the more frequent water quality testing leads to improved decision-making to ensure water is safe for consumption. And
• Hypothesis-3 is that WaterScope is more cost-effective than locally available testing options.

The project also set three learning objectives (LOs):

• To assess the potential for collaboration and partnerships between water Scope developers and local stakeholders, to enhance the technology's effectiveness and reach.
• To measure the performance and comparative advantage of the water Scope water testing system against conventional water quality testing kits in emergency contexts.
• To learn about the factors regularly affecting the application of water testing practices throughout the water supply chain in emergency contexts among water offices and humanitarian organization

Objectives of the evaluation

• The objective of this evaluation is to test & verify the hyphothessis’ and evaluate key learning objectives to generate evidences and draw lessons that enable us to use the WaterScope kit sustainably in the humanitarian setting,

Specifically,

• To asses ease of use by field staff of waterscope water quality testing kit in terms of its bacteriological analysis accurately (effectiveness), how quickly users can perform the analysis process (efficiency), how appropriate users feel the system is (engagement), how often and when users make mistakes (error tolerance) by using blanks and Duplicates, and whether testing improves over time (ease of learning), transportation (portability), sample handling, on-site accomplishment, and sterilization with other existing kits.
• How familiar are field staff and local partners with the WaterScope technology?
• Have they received adequate training on how to use WaterScope for water testing?
• What challenges, if any, have they encountered in adopting WaterScope in their regular routines?
• Are there any specific features or aspects of WaterScope that they find difficult to use or understand?To Asses how frequent water quality testing leads to improved decision-making to ensure water is safe for consumption
• How often are water quality tests currently conducted?
• In what ways do frequent water quality tests contribute to better decision-making?
• Have there been instances where more frequent testing has identified potential water safety issues?
• What specific decisions or actions have been taken based on the results of water quality tests?
• Is there a change in frequency of water quality testing after WaterScope?
• To assess cost effectiveness of waterscope testing kit in terms of its initial cost, consumable, cost per sample in comparison with other testing kit and the potential for scalablity of the WaterScope technology. Some of the questions to be addressed include:
• What are the current costs associated with locally available water testing options?
• How much does it cost to acquire and maintain WaterScope equipment?
• Are there any hidden costs or ongoing expenses associated with using WaterScope?
• In what ways can the cost-effectiveness of WaterScope be measured against traditional testing methods?
• Compare the WaterScope kit based on the above tools with another water quality test kit based on the data generated by the project
• Draw lessons for the three learning objectives (the potential for collaboration and partnerships between Water-Scope developers and local stakeholders, the performance and comparative advantage of the Water-Scope water testing system against conventional water quality testing kits in emergency contexts; the factors regularly affecting the application of water testing practices throughout the water supply chain in emergency contexts)
• Based on findings, generate evidences, identify result dissemination platforms and methods to the wider WASH sector, produce suitable pieces of the findings for dissemination purpose, and estimate costs needed for the identified result dissemination and sharing activities

Scope of the evaluation

Geographically, the evaluation will be conducted in North Pokot subcounty of West Pokot County where ACF and Yangat are implementing ElRHA Project. .

• Assessment of cost for existing water quality kits and consumables , Aquasafe, ) , in terms of initial investement cost (to buy kit itself) and waterscope kit, and if there are any hidden costs as well cost of testing at existing water quality laboratories
• Assessment of availability of existing water quality kits (DelAgua, Aquasafe),waterscope kit and consumables, and reputatable water quality laboratory including government laboratories
• Comparative assessment of cost per sample for the existing water quality kits,testing at the government laboratory(Aquasafe and waterscope kit)
• Comparative assessments and analysis of the effectiveness, efficiency, engagement, error tolerance, ease of learning, portability sample handling, on-site accomplishment, and Sterilization of testing water quality testing kit
• Compare the waterscope testing kit with other testing kit, at the existing water quality laboratory based on the above assessment.
• Based on findings, generate key evidences that waterscope can be used sustainablyor not.
• Finding dissemination plan, budget plan and material production
• Inception report and final report

Methodology

Methods of data collection

• The assessment team will entirely rely on the primary and secondary information source and methods to compile both quantitative and qualitative data for the assessment.

Primary data

• Primary data sources will include mixed methodologies, using multiple means of collecting information. This will include;

Key Informant Interviews (KII):

• KII with Importers/Suppliers and local private sectors, Ministry of wáter, WaterScope, national standard Agencies - to assess the potential for collaboration and partnerships between Water-Scope developers and local stakeholders, to enhance the technology's effectiveness and reach
• KII with relevant line ministries in West Pokot County and ACF and Yangat staff who participated in the testing process - to measure the performance and comparative advantage of the WaterScope water testing system against conventional water quality testing kits in emergency contexts and to test hypothesis that WaterScope can be easily adopted by field staff and local partners, allowing them to conduct water testing at regular intervals
• KII with Water Resource Authority officers(WRA) Public Health Officers, County/Sub county WASH Cordinators, WASH actors in West Pokot and regional level WRA/Water quality officers(Nakuru/Kakamega), - to learn about the factors regularly affecting the application of water testing practices throughout the water supply chain in emergency contexts among water offices and humanitarian organizations.
• KII with HHs targeted during the testing, with water committees, Community Health Volunteers, Water companies- to test hypothesis that more frequent water quality testing leads to improved decision-making to ensure water is safe for consumption.

Focus Group Discussion (FGD): Structured group discussions will be conducted, with small homogenous groups of people who will be identified by the preselection criteria.

• FGD with ACF staff, Yangat staff and water office staff, Public Health officers/WASH cordinator – to test hypothesis that WaterScope can be easily adopted by field staff and local partners, allowing them to conduct water testing at regular intervals
• FGD with ACF staff, partner staff and water office staff, with Water committees,CHVS– to test hypothesis that more frequent water quality testing leads to improved decision-making to ensure water is safe for consumption

Water quality test data analysis:

• Comparative analysis and visualization of the water sample test data collected by the project by using different available water testing kits - To measure the performance and comparative advantage of the Water-Scope water testing system against conventional water quality testing kits in emergency contexts

Secondary data

• Review of literatures and similar pilot Project reports on WaterScope and similar water testing systems mainly those under consideration under this Project
• The pre-pilot experiences baseline report on water quality testing for the Project
• Review meeting reports and Project progress reports
• Get Government approvals

Sampling

• Purposive sampling for the Key Informants Interview (KIIs) and Focus Group Discussion (FGD) in the qualitative and quantitative phase

Consultant

• Develops and submits the inception report and data collection tools and should get validation.
• submits the first draft report (s) in accordance with the report format given below to Action Against Hunger not later than 14 calender days after the completion of data collection.
• An innception report: this report should incorporate understanding the work, objectives, methods, tools, work plan, conclusions, dissemination
• A draft report: this report should incorporate findings in line with the objective and scope of the work, suggestions and recommendations, result dissmenination plans and budget
• Final report: the second round of revision may be required before the final version of the final report can be prepared. The final report should be submitted in accordance with the Action Agaist Hunger format.
• Debriefing: the lead consultant will present the results of the assessment and will document the discussion, which will form part of the final report.
• Validation meeting: The lead consultant will be requested to present findings and recommendations to Action Against Hunegr and stakeholders at capital and base level.

Qualifications of the Consultant(s)

The consultant(s) should have the following qualifications and experience:

• The lead consultant should have a minimum Master’s degree WASH/humanitarian/economics/statistics/public health with very good knowledge of WASH, water quality, water quality testing kits. WASH background, particularly water quality monitoring, is mandatory and to be demonistrated within the consulting firm.
• Professional experience in monitoring and evaluation of water quality and safety
• Excellent research experience, including conceptual and analytical skills; evaluating water quality programs and projects, reports and presentation
• Knowledge and/or proven expertise in participatory approaches in conducting assessments and facilitating project result-harvesting meetings
• Legally registered firm with renewed license in related field of business, VAT registration and Pin Number
• Experience in working with INGOs and in similar projects is an asset

Ethical guidelines

It is expected that the evaluation will adhere to ethical guidelines as such as ensuring confidentiality data and as outlined in the American Evaluation Association’s Guiding Principles for Evaluators. A summary of these guidelines is provided below.

• Informed Consent: All participants are expected to provide informed consent following standard and pre-agreed upon consent protocols. Coincide
• Systematic Inquiry: Evaluators conduct systematic, data-based inquiries.
• Competence: Evaluators provide competent performance to stakeholders.
• Integrity/Honesty: Evaluators display honesty and integrity in their own behavior, and attempt to ensure the honesty and integrity of the entire evaluation process.
• Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders. It is expected that the evaluator will obtain the informed consent of participants to ensure that they can decide in a conscious, deliberate way whether they want to participate.
• Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.