By Dan James
This article was originally published in the February 2020 issue of the INTRAC newsletter. Subscribe today to receive new issues in your inbox.
Late last year the British Council celebrated the 10 year anniversary of its flagship Active Citizens programme. INTRAC has been involved with the programme in various ways for nearly half of that time, so it is an opportune moment to reflect on what we’ve learnt.
Active Citizens is a social leadership programme that promotes intercultural dialogue and social responsibility; it supports participants to develop leadership skills and facilitate community-led social development. Participants – often but not always young people – receive training in social leadership and are then supported to initiate social actions themselves.
The programme has achieved tremendous scale over the last 10 years, training over 280,000 people who have gone on to initiate over 10,000 social action projects across 77 countries. It is diverse, working with different partner organisations, groups of participants and contexts. Many social actions are small scale but some are much larger, such as the “Take a Child to School” initiative in Pakistan, which to date has enrolled over a quarter of a million out-of-school children.
Given this diversity, understanding the impact that the programme has is a significant challenge, and one that INTRAC has been grappling with over the last few years.
One approach we have used is to focus our evaluation efforts on the social actions that participants initiate. The first step is to identify as many of these actions as possible – through document review and key informant interviews. Then, we select a small number of these for case studies to explore the impact more thoroughly and how the programme has contributed. In M&E jargon, this is similar to “Outcome Harvesting”. This has the advantage of capturing the more significant short-term impacts of social action projects, while allowing for diversity. However, it struggles to capture the longer-term effects or put context around different impacts.
An alternative approach is to take a “thematic slice” of the impact “cake” and seek evidence about particular types of impact. We have recently tested this approach in the UK evaluation which focussed on how the programme contributes to community cohesion. While it inevitably misses some of the diversity of impact, it allows the evaluator to tell a more coherent story about what the programme has achieved in a particular thematic area. Having developed a conceptual framework for understanding community cohesion, we were able to build up a picture that encompassed some of the more subtle ways in which the programme contributes to cohesion in particular contexts.
There are no magic bullets to evaluating a programme of this scale and diversity, but continued engagement with the programme, and a willingness to use variety of evaluation approaches, has enabled evaluations to provide different perspectives on what the programme is achieving and how.