Position Title: Programme Evaluator - Project based/Consultant
Closing date: 1 Nov 2022
IsraAID is an international non-governmental humanitarian aid organization based in Israel. Since we were founded in 2001, our teams have worked in emergency and long-term development settings in more than 50 countries around the world. From earthquakes and hurricanes to epidemics and forced displacement, IsraAID has been at the forefront of responding to major humanitarian crises worldwide. We have worked in more than 50 countries and at any one time have around 300 staff members worldwide. The intervention in Kenya’s Kakuma Refugee Camp and Kalobeyei Integrated Settlement is focused on prevention and response to child protection needs while strengthening community-based mechanisms in the refugee operation and its host communities.
- PROGRAMME BACKGROUND There are approximately 235,000 refugees living in Kakuma camp and Kalobeyei settlement. Based on recent UNHCR data, there are 95,471 children in the age group of 5 - 17 years, 45.7% girls and 54.3% boys. 2,611 children in Kakuma and Kalobeyei are registered as unaccompanied minors, while 11,873 are registered as separated children. Roughly 80% of the children have lost one parent, and 50% are double orphaned. Currently, only 5% of children in the area have access to Child Friendly Spaces (CFS) run by different agencies in Kalobeyei and Kakuma. IsraAID in partnership with Comic Relief has been implementing the project “Bridging the Gaps: Strengthening Mental Health Support for Children and Young People”. Working with trained community outreach workers, this project sought to improve awareness to mental health support for children in remote areas through early assessment, identification and referral of mental health and psychosocial (MHPSS) needs of children, in addition to increased community awareness and response to MHPSS needs, including MHPSS warning signs, community roles in MHPSS, and referral pathways available to access specialized services.
The project`s objective was to strengthen mental health support for children and young people through the following inter-linked outcomes: a) Increase resilience and well‐being of children through participation in the child-friendly activities at the MCFS. Children’s resilience built through provision of age appropriate emotional, physical, and mental activities. Equipping them with coping skills and linking them to service providers for relevant essential services as needed including psychosocial, protection or medical support while assisting parents and caregivers, in creating supportive environments for children. b) Improvement in community awareness, knowledge, and attitudes on MHPSS and Child Protection (targeting members of caregiver’s councils, community workers, community focal points, child protection committee members etc.) c) Strengthened capacity of existing formal/informal child protection committees, to develop and implement community specific CP and MPHSS action plans. To ensure that the referral pathway is streamlined all the way to the community level (identified committee structures that currently exist include the Area Advisory Councils within the host community and informal CP structures within the refugee community). d) Contribute to strengthening the capacities of IsraAID Kenya staff on MHPSS and Child protection. The project will end on December 05, 2022, and an external final evaluation is planned to take place in November 2022.
- EVALUATION PURPOSE The evaluation is the final evaluation of IsraAID’s “Bridging the Gap” programme to be conducted through an external firm or consultant. The intent is to inform IsraAID and Comic Relief on the effectiveness, efficiency, relevance, and overall performance of the project. The evaluations specific objectives aim to: a) Assess the achievements of the project's indicators against the targets, as articulated in the logical framework. b) To assess the children psychosocial resilience in the project area. c) To assess the level of community awareness, knowledge and attitudes on MHPSS and Child Protection d) To assess the impact of capacity-built project staff in increasing the effectiveness of the child protection project especially the impact of the mobile approach in our programmes. e) Provide input to similar programmes or scale ups and learning on assumptions and pathways, to support and guide IsraAID in developing theory of change on promoting and sustaining positive child protection outcomes. f) To document and make recommendations backed by solid evidence on what worked (or didn’t work) as well as sustainability of the results.
- EVALUATION CRITERIA The project evaluation should be organized taking into consideration programme quality standards. The evaluation will also consider key sector and sub-sector indicators for assessing the performance. The following provides as a guide to the questions to be addressed by this evaluation, under each of the criteria below: Relevance and Appropriateness: To what extent did the project activities respond to recipients/target persons of concern and the identified needs and priorities as defined by local actors? Analyze whether and how the project activities responded to any changes that occurred during the project’s implementation
How did the project activities complement and align to similar activities and, influence efforts implemented by local and national actors? Analyze the relevance of the designed project activities to the project overall objective and outcomes. Furthermore, were they relevant to the project theory of change and the identified outcomes and outputs? Was the project design appropriate to the risks and challenges identified through the project design or those that may have emerged during the project duration. (An analysis of challenges and risks within the internal and external environment needs to be provided) Impact: To what extent has the project generated (or is expected to generate)significant positive or negative, intended or unintended changes in the lives of recipients and in their environment? What are the broader effects of the project on individuals, gender and age groups, host-communities and institutions? What were the significant factors influencing the achievement of the project's outputs and outcomes? Effectiveness: To what extent has the project achieved its objectives, and its results, including any differential results across different groups? What factors have contributed to achieving or not achieving intended project outcomes and set objectives? Have the M&E system delivered robust and useful information that could be used to assess progress towards outcomes and contribute to learning? Assess how the partner portfolio and the mix of different types of partners have contributed to the objectives of the strategy. Efficiency: Was the project designed and/or amended throughout the implementation period to provide the best value for money? Were the resources for running all the activities available, adequate and was this the best use of resources to achieve results? To what extent has effective coordination and collaboration with existing interventions and partners been addressed and achieved?
Sustainability: To what extent can the activities of the project continue after donor funding ceased? Is there an exit/sustainability strategy in place or planned? Is it likely that the benefits of the project (capacities developed; linkages, mutual learning and knowledge and experiences shared) would be sustainable or is there any action required at the endpoint? To what extent do project recipients and stakeholders have ownership, capacity, and resources to maintain the activity results after external funding ceases?
- EVALUATION APPROACH AND METHODOLOGY IsraAID recommends an inclusive mixed methods approach, a combination of both quantitative and qualitative research methods that are relevant to the nature of the project and the evaluation objective. It should be utilization-focused, gender-responsive, and explore the possibility of utilizing participatory methods for developing case studies and/or examples to support the finding of the evaluation process. Data should be disaggregated by sex and according to other relevant project-specific parameters. Therefore, the data collection for this evaluation will contain various tools and methodologies (both quantitative and qualitative approaches). The consultant is expected to develop a detailed methodology (to be endorsed by IsraAID) for this evaluation. We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders including children. Some of the data collection tools expected to be used during the evaluation are: a) Desk review: The consultant(s) will also conduct a desk study/ literature review of all available documentation in preparation for the evaluation, including review of different programme documents, such as monitoring, quarterly, annual, and study reports, progress reports, mid-term evaluation report, technical training reports, project proposals, MEAL framework, etc. and analyzing secondary data. The evaluation methodology should make use of different data already generated by IsraAID programme and MEAL team. The information and findings of the desk review should be integrated with the data and findings from primary data collection and analysis. b) Interviews with Key Informants: The team are expected to conduct a range of interviews with key informants and stakeholders and will visit and interview relevant project staff, partner staff, governmental actors who were included in the project activities, IsraAID project staff in Kakuma including community outreach workers. A number of meetings will need to be conducted IsraAID focal points to guide the consultant to prepare the inception report and identity actors for the data collection. c) Focus group discussions: The team will conduct focus group discussions with direct and indirect persons of concern of the project including project persons of concern, and project staff.
d) Persons of Concern Surveys: The team is expected to conduct surveys of the persons of concern of the project especially using various children and youth resilience measures and other relevant children friendly survey tools. The consultant will devise an appropriate sampling method to select sample persons of concern from the different target groups and they will conduct the survey with the samples. We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include: Targeted persons of concern Partners and actors directly involved in the project at different levels The local implementing partners Community leaders and gatekeepers (if applicable) and representative bodies of the target population (with due consideration of the possible lack of women’s representation among leaders and bodies). Relevant local and county authorities and other relevant duty bearers. Data collection tools and appropriate sampling methodology should be prepared by the consultant and shared with IsraAID’s focal point for comment and approval. The data collection tools proposed by the consultant should be able to respond to the key evaluation questions mentioned above. Samples for surveys should be selected based on a statistically relevant sample per the survey system, that is, a 95% confidence level and a confidence interval of 5 with the total persons of concern count per intervention as the sampling frame. The evaluators/external consultants should take into account the following methodological considerations; The evaluator’s understanding of the evaluation questions. Description of the phases related to the evaluation approach proposed. Sources of information for primary and secondary data collections. Sampling strategies, including area and population group represented in the sample, sampling procedures, and sample size. Instruments to be used for data collection. Different types of data analysis will be carried out. Reference indicators and benchmarks for each evaluation question (if relevant).
- EVALUATION TIMEFRAME AND DELIVERABLES. The evaluator should produce: a) Inception report as per the firms/consultant’s template including the tools/materials/ templates. The inception report MUST include the detailed evaluation work-plan and evaluation matrix. b) Sharing detailed data collection plans and tools in time for IsraAID to be able to conduct quality assurance.
c) Sharing ALL the cleaned qualitative and quantitative data collected from the different sources. This should include code books. d) Briefing and debriefing meetings in addition to the routine meetings and discussions with the IsraAID team. e) The final evaluation report (based on the firms/consultant`s template, incorporating comments and technical inputs from the reference group) with a summary of the evaluation brief and a power point presentation detailing the purpose, process, findings, and recommendations of the evaluation. f) Updated results framework that clearly shows the status/value of project indicators vis-a-vis the targets. To facilitate the process, IsraAID will ensure; a) Timely review and validation of the inception report. b) Timely review and validation of data collection tools and data collection approaches. c) Provision of project scope as well as households and CFS centers list to facilitate sampling for the person of concern surveys are provided. d) Provision of requisite documents for the desk reviews i.e., project reports, evaluation reports, project proposal and theoretical frameworks for the project. The timeline provided for this is 3 weeks from awarding of the tender.
- REQUIREMENTS A consultancy firm with valid registrations, and it should have legal registration paperwork in Kenya and formal access to the locations of the programme implementation. The selected firm/consultant should possess the following minimum qualifications as follows:
Qualifications and experience Master's degree in advanced applied research/evaluation methods/ economics, social sciences, or any related academic discipline or an affiliation with a research institution or a university, holding a PhD, or being in pursuit of a PhD in a child protection/ MHPSS field is an advantage. Previous Experience Ten years of evaluating development Programmes At least five evaluation contracts of similar value, nature and complexity implemented over the last three years or more.
Strong background in monitoring and evaluation techniques and ideally experience in child protection programming in humanitarian contexts. Conversant with the context in Kakuma Refugee Camp, Kalobeyei Integrated Settlement and in Turkana West is an added advantage. Experience in developing and implementing evaluations with the UN, International NGOs or donors. Excellent knowledge and experience in survey design, implementation of surveys and statistical data analysis. Excellent analytical, communication, writing and presentation skills in English and Kiswahili. Ability to analyze complex interventions. Accountability Creates a respectful office environment free of harassment and retaliation and promotes the prevention of sexual exploitation and abuse (PSEA). Accepts and gives constructive criticism. Follows all relevant procedures, processes, and policies related to the organizational principles. Meets deadline, cost, and quality requirements for outputs. Monitors own work to correct errors or incorporate inputs. Takes responsibility for meeting commitments and for any shortcomings. Orientation Identifies the immediate and peripheral programme staff of own work. Establishes and maintains productive working relationships with staff. Identifies and monitors changes in the needs of evaluation, including donors, governments and project beneficiaries. Keeps staff/managers informed of developments and setbacks related to the evaluation.
How to apply
The interested firm/consultant should submit a technical proposal with a detailed evaluation methodology, indicative work plan, and the overall approach to the evaluation and an all-inclusive budget proposal no later than 1st November 2022.
The submission of proposals (technical and financial) and/or related questions should be directed via email to: ekarimi@israaid.org.
The submission should include the following: Company/Consultant profile including a history of similar projects (if applicable) A cover letter CV and biographies of independent consultant/consulting firm and key assessment team members (if any) At least two proof of previous experiences for contracts of similar value, nature and complexity implemented over the last three years or more. References for each evaluation team member or the firm; An example of a recent evaluation report.
Proof of registration of consultancy firm