Somalia: HARAAD REEB (QUENCHING THE THIRST) PROJECT - SOMALILAND FINAL EVALUATION

CARE SOMALIA

HARAAD REEB (QUENCHING THE THIRST) PROJECT - SOMALILAND

FINAL EVALUATION TERMS OF REFERENCE (TOR)

1. Introduction

a. The organization
CARE International had been Operating in Somalia since 1981 in delivering humanitarian service to the most vulnerable members in the community. CARE currently works through three main programs: First, the Rural Women program which supports poor, rural women and girls in addressing long term underlying causes of poverty and vulnerability as well as social, cultural, political and economic obstacles towards positive change. We help women and girls improve their economic status, access education and support them to play a greater role in local leadership and conflict resolution. Second, the Urban Youth program focuses on job creation and livelihood opportunities for poor youth through interventions such as secondary education, vocational training, small business development and microfinance. Third, the Emergency program provides direct humanitarian relief to victims of drought and conflict in Puntland, Mogadishu, and Lower Juba.

b. The project
CARE Somalia is implementing a WASH project in Sool, Sanaag and Togdher regions of Somalia/Somaliland under the Rural Women Program. The objective of the project is to contribute to the resilience and drought recovery of Sool, Sanaag and Togdheer and reduce vulnerability to future droughts with specific focus on the role of vulnerable/women headed households. The project is an expansion of sustainable environmental services (WASH) for 15 selected rural villages in the 3 regions.

The project was designed to address the needs of women, children and the vulnerable, consistent with CAREs programme’s target group, for having enough safe water close to the home, safe faecal management and dignity through a household latrine, inclusion through community hygiene promotion, children’s hygiene and community led total sanitation (CLTS) training. In addition women will be given a chance to take on a lead role in water related SMEs as the government puts in place systems and structures to manage water in the communities. Overall, the project will reduce risk against the multiple factors causing diarrhea, reportedly the main cause for child deaths in Somaliland and improve the public private partnership for future sustainability.

c. Project Objectives
The specific objective for CARE’s Haraad Reeb (Quenching the Thirst) project is to have an expansion of sustainable environmental services (WASH) for 17 selected rural communities in Sool, Sanaag and Togdheer.

The project will directly contribute to 3 Expected Results as outlined below:

Expected Result 1: Enhanced access to water by improving water infrastructure

Indicators
a. 30% of beneficiary households with average water use for drinking, cooking and personal hygiene is at least 15 litres per person
b. 36,000 people gaining access to an improved drinking water source, 30% will be women and girls
c. 100HH targeted for rain water harvesting mechanism

Expected Result 2: Improved operations and maintenance of rural water supply

Indicators:
a. The water act disseminated and adapted by key stakeholders in rural water supply
b. 16 water systems managed through public-private partnerships in a cost recovery mechanism
c. Community technicians are able to service and maintain water systems.

Expected Result 3: Sanitation and hygiene practices improved in the rural community
Indicators
a. 5 villages certified as “open defecation free” communities
b. 15% of people who observe the four key hygiene practices (washing hands after defecation, after cleaning a child’s bottom, before eating and preparing food)

To address the longer-term water, sanitation and health issues faced in the targeted villages, CARE will work with communities, Somaliland government authorities and the private public partnership to implement community-based sustainable WASH systems. This will be done through the formation and capacity building of development committees and associations. To achieve this strategy, CARE has facilitated the formation of Development Committees, women groups and User Committees in each village. CARE is undertaking capacity building to ensure institutions are effective in addressing the concerns of the communities.

1. Objectives of the evaluation

The overall objective of the final evaluation is to assess the progress made towards achievement of the overall and specific objectives of the project, and achievement towards the indicators of the project. The evaluation will assess the overall impact (positive as well as negative) observed as a result of the project intervention. The terminal evaluation is expected to generate well documented findings, lessons learned and recommendations.

CARE shall hire an external consultant for 17 days to carry out the final evaluation.

The project’s final evaluation is a part of the agreement with the Federal Ministry for Economic Cooperation and Development (BMZ). The evaluation will be done to assess the WASH situation at the end of the project provide recommendations based on lessons learnt from the project interventions.

The assessment will help determine some impact signs the project has made on the main target communities, in terms of improved hygiene practices and increased access to safe drinking water and increased access to sanitary facilities. The final evaluation will also gauge the level of community and other stakeholder participation and ownership of the implementation process including the identification of the intended and unintended outcomes, best practices and lessons learned as well as challenges arising from programme implementation and provide recommendations on the way forward for future programming.

A. The purpose of the final evaluation is primarily:

• To assess extent to which the project has achieved its objectives and created the expected changes in target communities lives.
• To review the achievements against targets of indicators established in the project’s logframe
• Assess whether the project objectives and strategies were relevant in addressing the identified problem(s).
• To evaluate the effectiveness and efficiency of the different approaches used to implement the project
• Establish the extent to which the results of the project are sustainable
• To generate learning (positive and negative) and good practices in implementing the project that could be applied in future programming.
• Based on the findings of the evaluation make specific recommendations for different stakeholders for improvements for community-based sustainable WASH systems.

B. An assessment of a number of critical elements in the project approach and methodology
• To assess the increase in knowledge, attitudes, behaviours and practices of community members on key hygiene practices
• To assess capacity development/building and strengthening of government and community systems and structures in rural water supply management such d private sector linkages component of the WASH project, and provide suggestions to improve these linkages
• To highlight the challenges and lessons learnt in applying the community-led total sanitation (CLTS) approach in theSomaliland context
• To review advocacy positions on the implementation and enforcement of the Water Act with the Ministry of Water and its local representations
• To assess the effectiveness of measures put in place to ensure project achievements are sustained and review existing PPP arrangements and give recommendations and suggestions for improving the sustainability of the project

C. An assessment of the achievements of the project so far against log frame indicators and Organisation for Economic Co-operation and Development (OECD) criteria
The consultant shall assess the project achievements against the following criteria (relevance, efficiency, effectiveness, impact and sustainability):
Relevance
The consultant should look at the design of the project and assess the extent to which the stated project objectives address the identified problems or real needs.

Efficiency
• Analyse the quality of day-to-day management (adequacy of project budget, management of personnel, project properties, communication, relation management with elders, community leaders, other development partners, etc)
• Local capacity building: To what extent the project is able to strengthen the institutions such as communities, government, women groups and the private sector in coordination with other agencies implementing similar projects

Effectiveness
• Assess whether the beneficiaries (communities and LNGOs) perceive that the planned benefits are appropriate
• Assess the appropriateness of the indicators (OVI’s) and make recommendations on any possible changes to be made.
• Assess the robustness of the monitoring protocol and data collection & compilation by project staff based on the log frame indicators.

Impact
• To assess the planned overall objectives against the logframe indicators;
• To assess how the project would impact on the access of beneficiaries to WASH services and, as a result, their health;
• To assess the extent to which strengthening the government capacity in management of water systems and structures would impact on sustainability of the project;
• To analyse the project approach to gender and its possible impact on gender equity and related issues;
• To assess the possible intended or unintended impact on environment.

Sustainability
• Ownership of objectives and achievements: to what extent have the stakeholders been consulted and involved in defining the objectives, the selection process for activities and beneficiaries, implementation?
• Institutional capacity: Assess the degree of commitment of stakeholders, community and institutions in owning the project, the measures taken to strengthen their capacity and suggest improvements for the future.
• To analyse the capacity building component of the project including appropriateness of training methods and suitability of the approach in implementing the project.
• To assess the economical and financial sustainability of the interventions.

At the end of the final evaluation, the external consultant shall produce a comprehensive report and present it to CARE and BMZ.

1. Scope of Evaluation

   The terminal evaluation will be implemented in 25th January – 11th February 2016 over a period of 15 working days. While the main emphasis of the evaluation should be on measuring the final achievements, results and impact of the project against baseline values on each of the indicators, the final evaluation should also cover the relevance, effectiveness, efficiency and sustainability.

   The geographical scope of the evaluation is Sool, Toghdeer regions and Sahil in Somalia/Somaliland.

2. Methodology
   The consultant shall use mixed methods including desk review, key informant interview sample survey, and observation using simple but numerically sensitive tools to collect data. The sample size must be statistically representative of the population. The analysis will involve statistical and content analysis using appropriate packages as deemed fit by the consultant. The analysis among others should show trends and to the extent possible should be disaggregated by gender and location. The BMZ WASH project is part of the Vulnerable Rural Women Program. In consultation with the Program Coordinator the consultant will include relevant program-level indicators from the Vulnerable Rural Women Program Strategy to measure the extent to which the project is contributing to the following strategic objectives:
   i. Over 70% of rural HHs have access to a reliable, safe and affordable water source
   ii. 70% of rural HHs have access to improved sanitation facilities

3. Tasks

The consultant will undertake the following tasks:

1. Conduct meetings with key project staff of CARE, partners and Ministry of Water resource
2. Design the evaluation, including data collection tools for the project and relevant program indicators and sampling technique and size.
3. Obtain feedback on data collection tools and evaluation design from key CARE, Ministry of water Resource and finalize data collection tools and evaluation design
4. Conduct desk reviews of secondary information and project documents including the project proposal (including logframe), implementation and M&E plans of the project, project financing agreements and any other relevant documents.
5. Train enumerators who will pre-test the data collection tools. During training, the consultant will translate the tools into Somali.
6. Collect data from a representative sample of individuals from the target groups and key stakeholders, including, Ministry of Water, women’s groups, village committees, local authorities and relevant private sector stakeholders, through e.g. household questionnaires, key informant interviews (KII) and Focus Group Discussions (FGDs)
7. Assess quality of on-site water and sanitation facilities and associated infrastructure previously constructed by CARE;
8. Data processing (data entry, verification and analysis);
9. Report writing and submission of first draft report
10. Presentation of findings and recommendations to and validation by key stakeholders
11. Finalize the report incorporating feedback and submission of final report.
12. Hold community and stakeholder feedback sessions to present findings and recommendations of the study

There should be adequate women representation and participation throughout the data collection process. Where necessary, especially in rural areas, focus group discussions should be conducted separately for men and women.

1. Deliverables

The consultants will produce the following specific deliverables after signing the contract:

Deliverable Days

1. Methodology, including work plan and draft data collection tools 2 days
2. Data collection/draft report, including all annexes (see below) 8 days
3. Presentation of findings and recommendations to CARE and partners 1 day
4. The consultant, upon completion of the assignment will, verify findings with key stakeholders 1 day
5. Final report, including all annexes (see below) 3 days
6. Travelling 2 days
   Total 17Days

The first draft report is to be received by February 20th 2016

The draft and final report will have the following structure:

1. Executive Summary (max. 2 pages)
2. Introduction.
3. Methodology, including sampling.
4. Analysis and findings of the study.
5. Conclusions, recommendations and best practices.
6. Annexes
   a. Performance indicator tracking table reflecting the status on each indicator against target
   b. Recommendations for revisions to logframe, including biannual benchmarks for the project duration, as well as final targets
   c. Relevant maps and photographs of the study areas
   d. Bibliography of consulted secondary sources
   e. Finalized data collection tools (in English and Somali)
   f. List of key informants
   g. Raw data in an agreed format

The report will be written in English.

The consultant needs to produce 3 hard copies of all deliverables. The final report will also be provided in electronic copy (both PDF and MS Word format) on a CD.

1. Expertise required

The evaluator should be an experienced and independent consultant with the following expertise:

• Minimum of 10 years professional experience in developing countries, including previous experience working in Somalia/Somaliland;
• University degree in project management, water engineering or other relevant related subject;
• Excellent understanding of water structures, problems related to personal and environmental hygiene, institutional governance and public-private sector partnerships.
• Be conversant with community basic WASH systems and designs in water related structures.
• Demonstrated experience in community development / community targeting / rapid rural appraisal.
• Have a good understanding of Somalia/Somaliland and its history, and the resulting impact this may have on the working environment.
• Have an understanding of operating conditions in an insecure environment
• Have practical experience in assessments, planning and implementation of interventions using participatory methods such as Participatory Rural Appraisal (PRA).
• Familiar with Project Cycle Management and evaluations
• Have excellent analytical and writing skills
• Be willing to travel extensively in the working areas of the project.
• Be a team worker who can produce a report and presentation together with other consultants involved in the evaluation.
• Fluent in English (both reading and writing).
• Understanding of the Somali language is a strongly preferred. If not, the consultant will need to arrange for translation/interpretation.

1. Work plan and Time schedule

The consultancy is expected to take place in the months of 1st February 2016 – 17th February 2016 in a total of 17 days, including preparation, evaluation design, field work, report writing and presentation of findings and recommendations to CARE, partners and external stakeholders.

The first draft report is to be received by February 20 2016.

1. Guiding Principles and Values

The consultant shall adhere to the “Do No Harm” principle and any other humanitarian principles. The consultant will be required to follow CARE Somalia’s security advice.

Applications will be evaluated based on the following criteria:
• Technical experience and expertise
• Quality of proposal
• Cost-effectiveness of proposal

The consultant shall ensure at all times the confidentiality of data, respect the privacy of all individuals concerned and make all data collected available to CARE in a usable format.

1. Terms and Conditions:
   Logistics: The consultant’s travel from base to the field and back after the end of the contract (including airport tax), food, and accommodation will be covered by CARE.

All applications should include the following;
• Cover letter (maximum 1 page) and updated CV’s of all study team members
• Technical proposal: Which should include (i) brief explanation about the Consultant with particular emphasis on previous experience in this kind of work; (ii) profile of the Consultant to be involved in undertaking the evaluation, (iii) Understanding of the TOR and the task to be accomplished, (iv) draft work and plan
• Financial Proposal: Which should include consultancy fees but excluding: accommodation and living costs; transport cost; stationery, and supplies needed for data collection; and costs related to other persons that will take part from consortium partners and government authorities during survey process, workshops.
• Application Deadline

The applications should be submitted by 28th January 2016 to somconsultants@care.org and quoting “haraad reeb (quenching the thirst) project – Somaliland-final evaluation”