Caritas Austria is part of the Caritas Internationalis family, with a core mission of identifying and acting to resolve hardship. With international programs in Africa, South Asia, the Balkans, Europe, and the Middle East in addition to domestic programs in Austria, Caritas is an active partner in helping vulnerable and marginalized communities access economic independence, education, food security, and inclusive and safe places to call home. One such program is the Regional Holistic Education Program, which supports young children and their families in Jordan, Lebanon, and Syria in enabling access to primary school, early childhood education (in Lebanon and Jordan), and extra learning support. The RHEP II program operated from 2018-2021 on the four main objectives of enabling children’s access to education, teaching through learner-centered principles, creating protective and inclusive environments, and engaging members of the education community in supporting and caring relationships. As part of an overall learning journey that aimed to develop the next long term organizational strategy for Caritas Austria, Catalystas conducted an impact evaluation of the RHEP II program in all three countries of implementation.
Conducting multi-country evaluations always requires carefully planned methodological approaches and strategies for cross-comparing data and analyzing evidence. However, the summer of 2021 presented a challenging and complex reality in each of the countries in question. Syria, still in the depths of ongoing conflict, presented key challenges when it came to access and clarity. It remains extremely difficult to travel from city to city, with multiple permissions required and checkpoints prevalent, and the country remains under international sanctions and pressure, with security risks still a clear and present danger. In neighboring Jordan, the situation was much calmer, but there were signs of struggle simmering below the surface; the Covid-19 pandemic has taken its toll, and the influx of Syrian refugees – over 600,000 – has put a strain on Jordanian resources. Accommodating these families alongside host communities has proved a difficult task for an overburdened government and civil society. While the economy remains largely stable, the small nation’s proximity to collapsed Syria and imploding Lebanon present a risk to the continued stability and prosperity of the kingdom. On the contrary, Lebanon presented a quickly evolving and challenging environment, with the country’s spiraling economic collapse exacerbated by the resonating impacts of the Beirut Blast, ongoing political catastrophe, the Covid-19 pandemic, and continued struggles surrounding supporting their own influxes of refugees – both old and new. In such a complex and quickly shifting context, Catalystas had to ensure that conflict sensitive and trauma informed approaches were at the core of our evaluation methods, remaining adaptable and aware of the swiftly changing needs and concerns of the communities served by the RHEP II program.
Catalysta Judith poses with school staff while conducting site visits in Jordan.
Against this backdrop of complex environments and multifaceted challenges, Catalystas dove in headfirst to capture an accurate picture of Caritas’ impact among communities on the ground in each location. Taking each specific context into account, we deployed our field missions, working closely with local counterparts to ensure a comprehensive understanding of cultural nuance paired with unbiased outside perspective. In each location, our teams met with the local Caritas sister branches first to gain a better understanding of local operations, program implementation on the ground, and dynamics between the Austria team, local partners, and beneficiary communities. Following our initial introductions, our experts carried out more in depth interviews with key partner staff, alongside interviews and focus groups with teachers and counselors working at the schools served by the program, as well as students and parents. Keeping both child protection guidelines and participatory approaches at the core of our evaluation methodology, we ensured that all participants understood the purpose of the discussions, that school staff or parents were always present where children were involved (albeit far enough away so that the children could speak freely), and that we left space for feedback and open sharing from each group and individual. Our approach looked to measure the impact of the program against the OECD-DAC criteria of effectiveness, impact, efficiency, relevance, coherence, and sustainability, along with the Core Humanitarian Standards of strengthening local capacities and avoiding negative effects, and basing response on participation and feedback & welcoming and addressing complaints. We also looked to examine challenges, lessons learned, and best practices, alongside developing recommendations with an emphasis on preparing future strategic programming for improved interconnectivity of education and community development aspects and inclusion of children with disabilities.
Across the locations in each country, we heard similar stories: while families were often facing many challenges, the schools supported in the Caritas program provided a safe and nurturing environment for children, and students were openly enthusiastic about attending classes. At the end of each day in the field, our experts relayed initial findings to our HQ team for real time triangulation and cross comparison, allowing us to revise and adapt our tools to ensure tailored materials for each location and context. Conducting real time validation sessions also enabled our team to keep that Caritas Austria team apprised of our progress, and we held regular check-ins throughout the process to share and receive feedback and make adjustments to our approaches and tools as needed. Regular communication between HQ and the field also allowed our HQ team to analyze and triangulate findings across locations and Caritas teams, incorporating our learnings into building a foundation for a new strategic framework for future education programming in the MENA region.
However, we also faced a number of challenges that required flexibility and adaptive practices in each location. In Lebanon and Jordan, for example, the Covid-19 pandemic meant that schools were being taught virtually, and so special arrangements were made to hold focus groups with students and parents, with health safety precautions taken highly seriously. In Lebanon, traveling between schools was difficult due to the numerous checkpoints, increasing fuel prices, and long distances from Beirut, where our field team and the Caritas Lebanon office were based.
Students joined our team in Lebanon for a focus group.
However, we successfully conducted site visits to five out of six of the Lebanese RHEP schools, with our local consultant conducting a final visit after the official end of our field mission in order to accommodate school and Caritas staff. In Lebanon our experts also witnessed firsthand how swiftly the needs and priorities of local staff and communities were shifting, with emergency humanitarian assistance quickly becoming a higher priority than long term development work, as crisis after crisis continues to sweep the struggling nation. This perspective enabled our team to adjust the strategic roadmap in order to incorporate and accommodate the changing situation and address the ways in which international organizations, with Caritas as a leading player, can support local communities facing new and changing needs and challenges.
Alongside the evaluation process, Catalystas also helped Caritas Austria to zoom out, guiding the team on a learning journey to support the development of a clear strategic path and roadmap to improved internal processes. We conducted deep dive discussions with the full Caritas team as well as individual interviews to gain a deep understanding of strengths, weaknesses, opportunities, and risks as an organization, as well as insights into team structures and dynamics, relations with other Caritas and partner organizations, and overall and individual goals and objectives. We also undertook a mapping of all major players in the education ecosystem in the three countries of implementation, designing a visualized overview of themes and geographies covered in order to identify gaps and saturated areas for Caritas to assess in future program planning and development exercises.
A mural painted on the wall of a RHEP school in Jordan.
Although we could not lead these sessions in person due to Covid-19 restrictions, by using interactive remote learning tools such as Miro whiteboards and Prezi video presentation software, Catalystas was able to design interactive, engaging workshop environments that successfully kept discussions of up to 15 people highly engaged and actively contributing to the ideation and strategic development processes of the learning journey. Building on the foundations of continuing communicative, horizontal structures, Catalystas supported Caritas in a guided journey culminating in the delivery of a roadmap and strategic plan basis focused on empowerment of internal staff and external partners, capacity building with a key emphasis on cross cultural competency and knowledge exchange, and information management system development.
Ultimately, despite the challenges we faced – including limited access to program sites in Syria and increasingly difficult and changing contexts in Lebanon – Catalystas delivered a comprehensive final evaluation of the multi-country RHEP II program to the Caritas Austria team, along with a strategy document, mapping of ecosystem actors, and visualized roadmap to strategic planning and implementation. We look forward to keeping up with Caritas Austria and her sister organizations as they continue to provide children and families across the Middle East with essential education and support services, through their highly impactful and dynamic programming.
Catalystas Consulting, Lange Voorhout 43, 2514 EC Den Haag, The Netherlands
© Catalystas Consulting. All rights reserved. Privacy and Policy
Aviva is a development and political stability specialist with over eight years of experience working in and on the Middle East and North Africa (MENA), Sub-Saharan Africa, and South & Southeast Asia. She focuses on the intersection of peacebuilding, empowerment, and gender equity, with an emphasis on interweaving growth and sustainability throughout social, political, and economic development. Aviva has experience in program development, implementation, situational analysis, and strategic planning in various contexts, including highly volatile areas.
Holding a Master’s in Conflict Studies and Human Rights from Utrecht University in the Netherlands, she has worked with organizations large and small, including Search for Common Ground, SPARK, the Netherlands Entrepreneurship Development Agency (RVO), the Dutch Entrepreneurial Development Bank (FMO), and the Nobel Peace Prize-winning Organisation for the Prohibition of Chemical Weapons (OPCW).
A lover of the written word, Aviva is an expert in writing and communications and embraces the challenge of creating content for a variety of audiences – from grant proposals to awareness raising campaigns.
Judith is a passionate development and humanitarian professional with more than eight years of experience working for actors ranging from the Netherlands Ministry of Foreign Affairs to INGOs, where she has worked on program design and development, policy development, and program management. She is an expert in the fields of improved business development, access to the labor market through higher education, humanitarian aid, and justice and security governance.
Holding a MSc in Conflict Resolution and Governance at the University of Amsterdam as well as a BSc in Political Science and a BA in Middle Eastern Studies from Leiden University, her experience focuses on fragile regions in the Middle East, South East Asia, and Sub-Saharan Africa, where she has lived and worked over the past decade for various NGOs.
Judith has extensive experience in developing and initiating projects, including scoping potential partners, finding a program’s unique selling point, and grant-writing. When it comes to project implementation, Judith is highly passionate about working with those actors on the ground who can make a real difference.
Haley has over ten years experience working as a consultant across international development, management consulting, education, and regenerative agriculture. She has a Master’s in Public Administration from the Middlebury Institute of International Development in Monterey, California, and a Bachelor’s of the Arts with Honors from the University of Oregon, where she studied international relations and French.
In her work, she celebrates adaptive, innovative, and iterative methodologies, working with donors and clients to incorporate flexibility, responsiveness, and local leadership from a project’s design to its implementation and exit. Within the Monitoring, Evaluation, and Learning framework, she works with her clients to identify and apply learnings derived from positive deviance as opposed to focusing on what isn’t working. She has worked with Search for Common Ground and Mercy Corps in Cambodia, Kenya, Lebanon, and the District of Columbia, for donors such as USAID, UNHCR, and privately-funded philanthropists, and clients include Caritas Austria, Search for Common Ground, Freedom from Hunger, a public transportation company of a global capital, a multi-billion dollar financial services firm, and medical service providers throughout North America, as well as Big Sur Land Trust and Monterey County Rape Crisis Center.
Her commitment to intersectional feminism extends to her personal life, where she farms and homesteads with her family in Northern California, spending the assets of her time and attention on decolonizing and rewilding her self, environmental justice, and dismantling patriarchy and white supremacy.