Consultancy: Independent Mid-term Evaluation for Better Regional Migration Management Programme at International Labour Organization

Project objectives expected results and beneficiaries
The project is covering three inter-related components of intervention namely:

• Labour migration policies and programmes that support fair and productive migration in East Africa and Horn of Africa are evidence-based and gender sensitive.
• Labour migration governance is fair, equitable and effective recognizing skills and facilitating social inclusion and decent work.
• Labour migration governance is strengthened through being more inclusive of social partners.
• The first component is scaling up and expanding the work on labour migration statistics, focusing on building the capacities of National Statistical Offices to collect and disseminate labour migration statistics.. It also looks at selected administrative sources for labour migration data. The work under the statistics is being implemented in line with ICLS 19 & 20. It is further using the ILO’s Social Security Inquiry (SSI) for the collection of social protection statistics. Hence, it seeks to ensure a sustainable and comprehensive approach and resulting in having more and up-to-date statistical information for the design and implementation of labour migration policies.
• The second component has continued to work on integration of potential and returned migrants into the labour market. In this context, a particular attention is devoted on developing MSMEs and social finance. Further, the feasibility assessments, conducted under the Project Phase I, for enhancing skills and qualification portability, at sector level, are operationalized and scaled up. This component further seeks to strengthen access to social protection benefits for migrant workers, thus delivering a holistic approach to labour migration. This will contribute to better skills matching and development, improved rights protection, and fair recruitment processes.
• The third component focuses on enhancing labour migration governance, with the strong engagement of employers’ and workers’ organizations. It will continue to build their capacity for the effective implementation of ILO migrant-related labour standards, recommendations, and frameworks. It will further strengthen the capacities of Government, employers’ and workers’ organizations to actively contribute to labour migration policy design, implementation, monitoring and evaluation at national, regional, and continental levels.

The three inter-related components of intervention of the project and the country of focus:

Outcomes

• Outcome 1 - Labour migration policies and programmes that support productive migration in East Africa and Horn of Africa are evidence-based and gender sensitive. - Ethiopia, Djibouti, Uganda, Kenya, and Tanzania
• Outcome 2 - Labour migration governance is more equitable and effective recognizing skills and facilitating social inclusion and decent work; - Ethiopia, Kenya, Tanzania, EAC, AUC
• Outcome 3 - Labour migration governance is strengthened through being more inclusive of social partners. - Ethiopia, Kenya, Uganda, Tanzania, Somali, South Sudan, Djibouti, AUC
• The imperativeness of strengthening each of these components has been recognized widely by both countries of origin and destination, as well as by employers and workers’ organizations, providing a basis for dialogue and action within and across countries.

The BRMM project stakeholders are:

• Governments, and primarily ministries of labour, education, foreign affairs, interior, national statistics offices, Technical and Vocational Education and Training (TVET) Institutions.
• African Union Commission and Regional Economic Communities, namely IGAD and EAC.
• Regional and national Employers’ and Workers organizations and their members in countries covered by the project.
• National and regional social security institutions and associations such as the East and Central Africa Social Security Association (ECASSA).
• Private employment agencies, civil society organizations and research / academic institutions.

International and developmental partners.

• UN Organizations.
• Project implementation strategy.

Project management structure
The project implementation approach is decentralized and led by the ILO Country Office for Ethiopia, Djibouti, Somalia, Sudan, and South Sudan, and for the Special Representative to the AU and the UN Economic Commission for Africa (CO-Addis Ababa), in collaboration with:

• ILO Country Office for the United Republic of Tanzania, Burundi, Kenya, Rwanda, and Uganda (CO-Dar es Salaam).
• ILO Decent Work Team in Cairo and Pretoria.
• ILO Regional Office for Africa.
• ILO HQ.
• The above modalities guaranteed effective and prompt implementation by collaborating directly with the targeted beneficiaries at country and regional levels. The Project staffing structure consists of a dedicated Chief Technical Advisor (CTA) in Addis Ababa, who is responsible for the overall management and coordination, and is supported by a Technical Officer and a Technical Specialist on Skills based in Addis Ababa and a Senior Statistician, based in Kampala, to follow the overall implementation of the project and the skills and statistics component. Then, National Project Officers are deployed in Kenya, Tanzania, Somalia, and South Sudan to ensure smooth project implementation and follow-up. A Procurement Officer and Finance and admin assistant is also part of the team to guarantee smooth and efficient implementation of the project. The project also has an M & E officer to ensure that systematic monitoring frameworks are in place to improve the qualitative and quantitative evidence gathered by the Project and a Communication Officer to follow up on communication and knowledge sharing issues and ensure visibility of the program.
• Technical backstopping is provided by the regional specialist and ILO HQ in Geneva, in particular MIGRANT Branch. Moreover, the project also receives technical backstopping from the ILO Statistics Department, the ILO Skills and Employability Branch and ILO Social Protection Department on the specific thematic activities, as well as ACTRAV and ACTEMP colleagues in the Region & HQ. Some of the capacity building activities, are implemented in partnership with ILO International Training Centre (ITCILO) in Turin, Italy.

The Project has also set up the management frameworks to govern and guide the implementation and M&E (monitoring and evaluation) of the Programme at two distinct levels:

• Tier 1 – Labour migration advisory group (LMAG) at Political level (Regional); and
• Tier 2 - Technical Working group (WG) at technical level (National).
• The project’s multi-stakeholder comprehensive approach requires strong partnerships with a range of actors – both directly delivering the programme as well as others that complement this work. CO-Addis Ababa is playing a coordination role for the various regional and national partnerships and other technical departments. The office also coordinates closely with the relevant ILO Offices in Africa, and East Africa in particular, to oversee procurement operations and management of local partners.

Evaluation Background

• ILO considers evaluation as an integral part of the implementation of technical cooperation activities. As per ILO evaluation policy and procedures all programmes and projects with budgets above USD 5 million must have to go through two independent evaluations. will go through two independent evaluations. Both evaluations are managed by an ILO certified evaluation manager(s) and implemented by independent evaluators. Evaluation in the ILO is used as an accountability and organizational tool, where lessons are learnt and knowledge is built and it helps ILO constituents, ILO staff and other project stakeholders to improve on their work on decent work and social justice. It is also used as part of its accountability to donors and the ILO’s Governing Body, and to contribute to enhanced learning.
• The evaluation is conducted in the context of criteria and approaches for international development assistance as established by the OECD/DAC Evaluation Quality Standard; and the UNEG Code of Conduct for Evaluation in the UN System.
• Considering that the project has attained mid-point, an independent mid-term evaluation will be conducted. This evaluation will follow the ILO policy guidelines for results-based evaluation; and the ILO EVAL Policy Guidelines Checklist 4.6 “Preparing the inception report”; and Checklist 4.2 “Preparing the evaluation report”.
• For all practical purposes, this ToR and ILO Evaluation policies and guidelines define the overall scope of this evaluation. Recommendations, emerging from the evaluation, should be strongly linked to the findings of the evaluation and should provide clear guidance to stakeholders on how they can address them.

Purpose and objectives of the evaluation

• The purpose of the independent midterm evaluation is to assess the implementation of the project so far, review the relevance of interventions as well as the progress made towards achieving planned objectives. It will contribute to enhanced learning and provide opportunity to make modifications to ensure successful achievement of the project’s objectives within the planned lifetime. It will also provide an opportunity to ascertain if the different interventions are coherent with ILO’s strategic objectives, are relevant and useful to the key stakeholders and are being conducted in an efficient and effective manner according to the agreed PRODOC and ILO standards.

Specific Objectives include:

• To review the project’s results ( outputs and outcomes) by assessing progress made so far, identifying factors affecting project implementation (positively and negatively) including factors beyond the projects control. If necessary, propose revisions to the expected level of achievement of objectives.
• To examine the appropriateness of strategies and approaches used for the project’s implementation, especially the effectiveness in achieving outcomes and results.
• To assess the internal and external coherence, and strategic fit as well as contribution of the project with broader ILO work (e.g., ILO’s Decent Work Country Programme), as well as the United Nations Cooperation Framework (UNCF)/Sustainable Development Goals (SDG) and BRMM target countries policies and strategies on labour migration.
• To analyse the intervention logic – Theory of Change (TOC) and Log frame, with particular attention to the linkages (i.e., complementarities, synergies, etc.) of the various projects implemented in the region.
• Review the institutional set-up, capacity for project implementation, coordination mechanisms and the use and usefulness of management tools including the project monitoring tools and work plans.
• To assess the efficient use and adequacy of resources (finance, human and assets) used to implement planned activities and achieved results - outputs and outcome.
• To analyse the effectiveness of the collaboration between the different components, and stakeholders of the project.
• Provide strategic recommendations for different key stakeholders to overcome challenges, improve implementation of the project activities, and increase the project’s achievement of outcomes, objective, and impact.
• To assess sustainability strategies adopted.
• To document lessons learned and potential good practices.

Evaluation scope

• The mid-term evaluation will cover one and half-year of program timeframe from July 2022 to Dec 2023. The geographical coverage includes East and Horn of Africa Region, with focus on Djibouti, Ethiopia, Kenya, Somalia, South Sudan, Tanzania, Uganda. The three intervention components of the project will be covered in the evaluation, and particular attention will be paid to the interconnection of these components and contribution to national policies and programmes.
• The evaluation will discuss how the project is addressing the ILO cross-cutting themes including gender equality and non-discrimination, disability inclusion and environmental impact throughout data collection and analysis of the evaluation process.

Clients of the evaluation

• The primary clients of the evaluation are, the ILO, specifically the Better Regional Migration Management (BRMM) Programme team, the ILO tripartite plus partners at national, sub regional and regional level. At regional level, the clients are Business Africa, ITUC- Africa (ATUMNET), African Union, and EAC. At national level, it is mainly the Ministry of labour, national Statistics offices, Workers and Employers organizations, and different project implementing partners in the respective countries. Other relevant clients include the donor (Foreign Commonwealth and Development Office (FCDO) of the United Kingdom.), the ILO CO-Addis Ababa, the ILO CO-Dar es Salam, DWT-Cairo, DWT-Pretoria, ROAF, and HQ branches namely MIGRANT, SKILLS, STAT, SOCPRO, SOCIAL FINANCE, SME, ACTEMP and ACTRAV.

Evaluation criteria and questions

• The evaluation should address the overall ILO evaluation concerns such as relevance and coherence, effectiveness, efficiency, sustainability, and impact as defined in the ILO Policy Guidelines for results-based evaluation, 2017 following OECD-DAC evaluation criteria[1]:
• The ILO cross cutting themes (tripartism, social dialogue, international labour standards, just transition to environment and gender and non-discrimination) should be integrated in the evaluation question. Other aspects can be added as identified by the evaluator in accordance with the given purpose and in consultation with the evaluation manager. Any fundamental changes to the evaluation criteria and questions should be agreed between the evaluation manager and the evaluator and reflected in the inception report.

Relevance, coherence, and strategic fit.

• Is the projects objectives and strategies consistent with the ILO constituent policy and programme frameworks (i.e., National development plans and the DWCPs) and other national key partners’ needs, and the country needs, the final beneficiaries (i.e., men and women)?
• How is the project contributing to the relevant ILO Programme & Budget Outcomes and development priorities in Ethiopia, Kenya, Uganda, Tanzania, Somali, South Sudan, and Djibouti, the UNDAFs /UNSDCFs and the SDGs?
• To what extent did the project build on previous experience of the ILO, UN agencies and the donor in Ethiopia, Kenya, Uganda, Tanzania, Somali, South Sudan and Djibouti, and relevant experience of other local and international organizations? Are the projects relevant for the national policies and priorities in Ethiopia, Kenya, Uganda, Tanzania, Somali, South Sudan, and Djibouti?

Validity of project design.

• Has the project developed and follows a comprehensive, Theory of change integrating external factors, and based on a systemic analysis? Do outputs causally link to the intended outcomes and impact? Have external factors (assumptions and risks) taken realistically into account?
• To what extent was the project design adequate and effective in the coherence and complementarity between the different project components?
• To what extent did the project build on the comparative advantage of the ILO in the field of employment / migration?

Effectiveness of the project in relation to the expected results.

• To what extent are the project objectives and outputs being achieved?
• Have unexpected positive and negative results taken place?
• What were the main internal and external factors that influenced the achievement or non-achievement of results?
• To what extent has the project management and governance structure put in place collaborated strategically with tripartite constituents, stakeholders and partners in the project, ILO, and the donor - to achieve project goals and objectives?

Efficiency of the resources used.

• Have resources (financial, human, technical support, etc.) been allocated strategically to achieve the project outputs and outcomes? If not, why, and which measures was taken to work towards achievement of project outcomes and impact?
• Are the project’s activities/operations in line with the schedule of activities as defined by the project team, work plans and budgets?
• Have the project developed an M&E strategy that enhance accountability, learning and feed into management.

Impact orientation and sustainability

• What can be identified as project sustainable impacts in the target groups and other actors as relevant? Are the results integrated or likely to be integrated into national institutions, target populations, and will partners be able to sustain them beyond the project (institutionalization of project components)?
• To what extent are the results of the intervention likely to have a long term, sustainable positive contribution to the relevant SDGs and targets (explicitly or implicitly)?
• What measures and actions have been put in place to ensure ownership of the project's results? Is the project implementing exit strategies as necessary?
• Can the project's approach or parts of it, and results be replicated or amplified by national partners or other actors considering institutional and financial dimensions?

Cross-cutting themes

• To what extent has the project identified and integrated into its actions the operational and strategic needs and priorities for men, women, and other vulnerable groups?
• How has the project been able to meet the specific needs of men, women, and vulnerable groups?
• Do the tools developed by the project integrate gender and non-discrimination issues?
• Has the project considered tripartism, social dialogue, international labour standards and a fair transition to environmental sustainability in its design and implementation?

ILO evaluation policy 2017, ILO policy guidelines for results-based evaluation: Principles, rationale, planning and managing for evaluations, 4th ed.

Evaluation Methodology

• The methodology will be participatory and use mix-methods and Theory of change appr. Quantitative data is captured to analyze mainly, but not exclusively, the efficiency and effectiveness of projects, both at the level of desk review and field data collection. While qualitative data is captured to analyze mainly, but not only, the relevance and orientation towards the impact and sustainability of the projects. Data triangulation is required through the combination of various sources of information and various data collection techniques (documentary review, focus group and interviews, field observation, electronic surveys, etc.).
• The evaluation should be conducted in adherence with the relevant parts of the ILO Evaluation Framework and Strategy[1]; ILO Policy Guidelines for Evaluation: Principles, Rationale, Planning and Managing for Evaluations [2]and UNEG Principles.
• The evaluation team will ensure that crosscutting themes (gender and non-discrimination such as women, youth, people with disabilities, etc.) International labour standards, social dialogue and fair transition in environmental issues are integrated in the evaluation questions and data collection tools, as well in the evaluation report. Special consideration should be made to capture women’s views and perceptions.

The evaluation will comprise the following key steps:

Step 1: Inception phase:

• Desk review of all project documents and progress reports, preliminary interviews with project CTA and the donor focus on logistics and evaluation expectations, and preparation of inception report (following EVAL Checklist 4.6) for clearance by the evaluation manager before moving to step 2.

Step 2: Data collection:

• Fieldwork, both physical and virtual will be conducted considering the following techniques of data collection.
• Desk review of project technical and financial documents, databases, and all products created by the project, followed by discussions with project team, ILO at various levels (Country Office, DWT, ROAF, HQ), and national stakeholders and the donor.
• On-site interviews with stakeholders and focus group discussions with project beneficiaries in Ethiopia and Kenya.

Step 3: Development of the draft evaluation report:

• Based on step 1-2 the evaluator will develop a draft report in English following Checklists 5 and 6 of EVAL. The report is submitted to the evaluation manager, who will do a methodological review. After his approval he will share this with key stakeholders. The comments received by the evaluation manager will be provided to the evaluator for consideration around 2 weeks after reception of the first draft.

Step 4: Presentation of preliminary results of the evaluation:

• A presentation of preliminary findings, conclusions and recommendations will be presented to all project stakeholders including the national key stakeholders, project partners, ILO DWT, ROAF and HQ, and the donor. This will allow addressing factual errors, clarifying ambiguities or issues of misunderstanding or misinterpretation. The workshop will be virtual or hybrid.

Step 5: Report writing.

• The evaluator will develop a draft report in English following Checklists 5 and 6 of EVAL. The report is submitted to the evaluation manager, who will do a methodological review. After his approval he will share this with key stakeholders. The comments received by the evaluation manager will be provided to the evaluator for consideration around 2 weeks after reception of the first draft.
• Then the evaluator will develop the last version of the evaluation report and will present clearly (a separate comments log or using track-changes mode on MS Word) how the comments have been addressed in the revised draft. The final draft will be reviewed by the Regional Evaluation Officer for Africa. After approval by the evaluation manager and the regional evaluation officer, it will be submitted to EVAL for final review and approval.

Deliverables
The following products will be delivered by the evaluation team:

• Inception report in English (incl. methodological note) (refer to Checklist 3 “Writing the Inception Report”[1])
• A first draft of the evaluation report in English and French (30-40 pages plus executive summary and annexes), answers the questions related to the evaluation criteria, including the recommendations, lessons learned, good practices. The report will address for guidance on the structure and content Checklists 5 and 6: “Preparing the evaluation report”[2] and “Rating the quality of evaluation report”.

The report will be developed under the following outline:

• Cover page with key project and evaluation data
• Executive Summary
• Acronyms
• Context and description of the project including reported results.
• Purpose, scope, and clients of the evaluation
• Methodology and limitations
• Findings (this section’s content should be organized around evaluation criterion), including a table showing output and outcome level results through indicators and targets planned and achieved and comments on each one.
• Conclusions
• Recommendations (i.e., for the different key stakeholders), indicating per each one priority, timeframe, and level of resources required.
• Lessons learned and good practices.

Annexes:

• TORs
• Evaluation matrix
• List of people interviewed
• Schedule of work
• Documents examined
• Data collection tools
• Lessons learned and good practices (under EVAL formats)
• Others reports
• The final evaluation report, will follow the structure of the draft report, addressing comments from the stakeholders.
• A summary of the final evaluation report (ILO/EVAL template) in English and French.
• Management arrangements and work plan (including timeframe)

Evaluation Management
Evaluation Manager:

• The evaluation will be managed by Benson Mang’eni (mangeni@ilo.org), ILO officer who has no prior involvement in the project, and oversight by Dessero Pacome (dessero@ilo.org) ILO Regional Office Monitoring and Evaluation, Knowledge Management Officer.

The evaluation manager is responsible for completing the following specific tasks:

• Draft and finalize the evaluation TOR with inputs from key stakeholders.
• Develop the Call for expression of interest and select the independent evaluator in coordination with EVAL.
• Brief the evaluator on ILO evaluation policies and procedures.
• Initial coordination with the project team on the development of the data collection process and the preliminary results workshop.
• Circulate the first draft of the evaluation report to the key stakeholders requesting written comments within 10 working days.
• Consolidate the received written comments received into a master evaluation report to send the evaluation team; and
• Ensure the final version of the evaluation report addresses the stakeholders’ comments (or an explanation why any has not been addressed) and meets ILO requirements.

The BRMM project team will be responsible for administrative contractual arrangements with the evaluator and provide any logistical and other assistance as may be required. The BRMM project team will be responsible for the following tasks:

• Provide programme background materials to the evaluator through the Evaluation Manager.
• Prepare a comprehensive list of recommended interviewees.
• Coordinate in-country logistical arrangements.
• Provide inputs as requested by the evaluator during the evaluation process.
• Review and provide comments on draft evaluation reports.
• Organize and participate in stakeholder consultations, as appropriate; and
• Provide any other logistical and administrative support to the evaluators as may be required.
• Evaluation Timetable and Schedule
• All intellectual property rights arising from the execution of this mandate are attributed to the ILO. The contents of the written documents obtained and used in connection with this assignment may not be disclosed to third parties without the prior written consent of the ILO.