Improving Evaluation and Impact Assessment

Evaluation and Impact Assessment are complicated processes, especially when looking at intrinsically complex, intangible and often ill-defined processes such as organisational capacity building. While much progress has been made it is clear that this is a rapidly changing field, which is beset by definitional problems, methodological debates, contradictory criticisms, and uncertainty as to the primary purpose of such assessment processes. While advances have been made in this field there are still a number of issues that need to be addressed.

Characteristics of Organisational Capacity Building

It is important to understand the particular characteristics of organisational capacity building more clearly and whether, in order to be effective, this creates a need to develop more appropriate approaches tailored to these characteristics. This includes exploring whether it is possible, or even desirable, to make a direct link between cause and effect (i.e. between effective organisations and development change).

Any innovative, appropriate and accessible impact assessment approaches that are developed would also need to:
• Rise to this challenge of measuring what is important, not merely what is easy.
• Capture and assess the systemic, multi-dimensional and dynamic nature of organisational change with simplicity, clarity and flexibility.
• Use a variety of quantitative and qualitative processes.
• Involve a greater degree of participation by local communities and incorporate their stories and experiences.
• Balance ‘core’ principles – such as trust, equality, ownership, reliability, credibility and legitimacy – with flexibility to adapt to differing contexts.
• Recognise and respond to the needs and agendas of different stakeholders.
• Use methods to analyse and consolidate information from different sources which encompass some degree of consistency and comparability.

Linking to Organisational Learning

Impact assessments seem to be more effective where they are linked to improved practice within an organisational environment where learning and experimentation is prioritised. Only then can information collected and knowledge shared be used constructively for critical reflection and improved practice. However, currently there are few incentives for learning and openness. Without an explicit recognition of power imbalances, many participating organisations feel insecure about disclosing limitations or failure where this may be linked to funding considerations. The recent shift in emphasis from control and accountability towards learning offers room for optimism but more could be done to address this constraint.

Recognising the Value of Investing

It is vital to provide the necessary investment of time and resources, both human and financial. All the evidence suggests that successful impact assessment depends on a significant investment of funds. This needs to be recognised by participating organisations and their donors otherwise such processes will be poorly implemented, generate little information of operational value and suffer from limited credibility. Impact assessment should therefore be seen as an investment that can add value to the organisation’s ability to learn, rather than as an additional cost.