What does success look like in a complex programme? Lessons from the Break Free! End-Term Evaluation
How do we assess whether a programme has contributed to meaningful change when outcomes are shaped by multiple actors and contextual factors?
This was one of the questions explored through the Break Free! (BF!) End-Term Evaluation (ETE), commissioned by Plan International Netherlands and conducted by hera across nine countries—Burkina Faso, Ethiopia, Kenya, Malawi, Mali, Mozambique, Niger, Sudan and Zambia—as well as at regional level.
Covering the full 2021–2025 programme period, the independent evaluation assessed programme performance against the OECD-DAC criteria of effectiveness, coherence and sustainability. It also examined the validity of the programme’s Theory of Change (ToC), achievement of results, adaptation to changing and fragile contexts, sustainability of outcomes, consortium collaboration, partnership equity and lessons learned.
The evaluation used a mixed-methods approach, combining contribution analysis, social network analysis, surveys, key informant interviews, focus group discussions, document review and an update of the programme’s Performance Management Framework (PMF).
One of the key methodological lessons from the evaluation was the value of Contribution Analysis when evaluating complex social change programmes.
Moving beyond “Did the programme cause this?”
For programmes working on adolescent sexual and reproductive health (SRH), gender equality and women’s and girls’ empowerment, establishing direct attribution can be challenging. Change is influenced by governments, communities, young people, schools, economic conditions, social norms and other contextual factors.
Contribution Analysis provides a way to approach this complexity by moving beyond the question of:
“Did the programme cause this?”
and instead asking:
“How did the programme contribute to change?”
For the Break Free! evaluation, this approach helped assess the programme’s contribution to observed results and test the pathways set out in its Theory of Change, while taking account of other factors influencing change.
The evaluation found strong evidence of progress in girls’ agency and youth leadership, alongside contributions to increased school participation and policy change. At the same time, it identified important challenges, including data gaps, conflict, poverty and deeply rooted social norms.
The evidence also showed the importance of collaboration. The strongest results emerged where communities, traditional leaders, parents, teachers and young people worked together from the start.
Evaluating change across different contexts
The evaluation methodology was strengthened through the incorporation of Contribution Analysis, Realist Evaluation, Conflict-Sensitive Approaches and Social Network Analysis.
These approaches supported a more comprehensive assessment of the programme, including its contribution to results, the influence of context, and the relationships between different actors.
Social Network Analysis was used to assess partnership networks, capacity strengthening, collaboration and equitable partnerships. This was particularly relevant to an evaluation that placed attention on Southern leadership and collaboration between consortium members and local organisations.
A conflict-sensitive approach was also integrated into the evaluation to ensure that the process did not exacerbate existing conflicts and supported equitable and inclusive engagement in fragile settings.
The evaluation itself remained adaptable. Following pilot testing in the first country, data collection instruments were refined based on feedback from the lead evaluator and core team.
This adaptability was important for conducting an evaluation across nine different country contexts while maintaining a consistent overall methodology.
What can the findings tell us about future programming?
The evaluation identified several areas that are important for future programmes.
Economic empowerment should be considered alongside efforts to address child marriage and teenage pregnancy, given the role that poverty continues to play.
There is also a need to invest more in Southern-led organisations, strengthening equitable approaches to partnership.
And while policy change is an important result, policy commitments need sufficient budgets and implementation mechanisms if they are to translate into change in practice.
The evaluation also highlighted the wider challenges facing progress on sexual and reproductive health and rights. Conflict and political instability can disrupt programme progress, while growing anti-rights movements and declining development funding threaten gains for girls and young people.
These findings point to the importance of sustained funding and strong accountability mechanisms to protect and build on progress.
Understanding contribution in complex programmes
The Break Free! evaluation reinforced an important lesson for evaluating complex social change: success is rarely the result of one factor alone.
When programmes operate across different countries and contexts, and when change involves governments, communities, families, schools, young people and wider social and economic conditions, evaluation needs to account for that complexity.
Contribution Analysis provided one way of doing this — helping the evaluation team move beyond proving attribution and instead examine the evidence of how the programme contributed to change.
For hera, the assignment also provided an opportunity to strengthen our experience in Contribution Analysis and the MFA/IOB evaluation quality standards, while building further experience in managing complex, multi-country evaluations across diverse and fragile contexts.
Ultimately, evaluating complex programmes is not only about determining whether results were achieved. It is also about understanding how change happened, what contributed to it, what constrained it, and what lessons can inform future programming.
If your organisation is looking to conduct programme evaluations, strengthen monitoring and learning processes, or generate evidence to guide future investments, the hera team would welcome the opportunity to collaborate. Contact us at hera@hera.eu.