Digital Learning Assessment at Mercy Corps

Project Description:

Under YES!’s Innovation and Digital Work stream, we are pursuing a pilot with various technical partners, to place locally accessible educational content about youth employability on a local network for viewing by youth on their smartphones at youth centers, restaurants, bus stops, and more across and outside Nairobi. YES! has contracted service providers to create video content in English and Kiswahili drawing from the YES! curriculum for deployment during this pilot phase. The same content may also be deployed online through various employment-related sites.

Objectives of the Consultancy:

YES! is committed to effectively tracking its impact and developing bold insights to contribute meaningfully to the global conversation on youth economic empowerment by learning from what we do, and putting that learning back into practice.

This particular project is an exciting opportunity for YES! to continue building its evidence base on what works and what does not as we test out different paths to scale in youth employment programming. Broadly, our learning agenda for this collaboration is as follows:

A. Usage

1. Who is accessing the content? What are their demographics?

2. What are the barriers/constraints to youth accessing the content?

3. How do they reach the content? How does it compare to other activities on the platform?

4. How do young people interact with the YES video content?

5. Which particular YES! content was preferred and why?

6. How do sex, age, education, poverty level, employment status influence all of the above?

B. Learning and livelihood outcomes

1. To what extent do participating youth demonstrate learning as measured through a change in knowledge, attitudes and practices as linked to content delivered?

2. Is content likely to impact on livelihoods according to YES!’s four domains of change: skills, diversity, income and savings, and meaningfulness?

3. Which video content proved more effective in facilitating the desired outcomes?

C. Recommendations

The findings from this assessment will inform iteration of the service and YES! video content.

Research Methodology:

The consultant will develop their own approach, in collaboration with the YES! Results, Learning & Research Coordinator. S/he will employ research methodologies with sufficient rigor to generate actionable answers to the research questions within the resources and timeframe available for this effort. S/he will have access to backend analytics of technology usage from our tech partner, but much of the research will involve on-the-ground observation, interviews, surveys, focus groups, participatory activities and more. The assessment should actively and respectfully engage primary stakeholders throughout the research cycle. This could involve engaging further enumerators.

Sample research methods include:

Determining usability and utility of service:

a) Quantitative: Service tech analytics; targeted surveys; heuristics evaluations

b) Qualitative: Field study; structured interviews; Focus group discussions

Determining learning and livelihood outcomes:

a) Quantitative: Performance on site quizzes and tests; Targeted surveys

b) Qualitative: Desk review; structured interviews; Focus group discussions; Field study

Deliverables:

The consultant will finalize the scope and purpose of the research; design and test data collection tools; identify and acquire necessary secondary data sources; carry out the collection of data; analyze data; and draft actionable findings and interpretations. The consultant will have responsibility for ensuring quality and rigor of the research findings. The consultant will engage in the following activities and others as required.

Week 1

Project phase & timing: Development of detailed research methodology

Activities:

• Desk review
• Consultations with YES! team and content creators
• Stakeholder requirement analyses
• Data analysis of existing backend analytics

Outputs/Deliverables: Completed methodology, data collection instruments & plan

Week 2-3

Project phase & timing: Field research

Activities: Field study

Outputs/Deliverables: Summary brief of findings for discussion

Week 4

Project phase & timing: Write-up

Activities:

• Stakeholder meetings for triangulation
• Written report

Outputs/Deliverables: Evaluation report

Suggested Time Frame:

The consultancy will take place over one month in August 2-17.

The Consultant will report to:

The consultant will report directly to YES! Results, Learning and Research Coordinator.

The Consultant will work closely with:

The YES! Program Director, the Innovation and Digital Coordinator, as well as tech and content creator partners. 'Ù”

Applicants should possess the following qualifications:

• Advanced university degree in education, programme management or other development related fields
• Experience in applying qualitative and quantitative evaluation methods, with a minimum of 5 years, demonstrating a strong record in designing and conducting/leading evaluations, particularly in the area of education
• Experience carrying out participatory assessments in Eastern Africa, including assignments focused on digital or mobile learning
• Excellent language skills in English (oral communication and report writing); knowledge of Kiswahili will be considered an advantage