It is best to approach M&E as a mutual learning process for all involved, not merely as an information-gathering exercise. This encourages flexibility, openness and debate. The principles of accountability to vulnerable people outlined in Chapter 11 are very important here. Communities’ views should be central to evaluation, and communities (or beneficiaries) should be able to take an active part in the evaluation process. Participatory evaluation enables the voices of project stakeholders, particularly beneficiary communities and vulnerable groups, to be heard, draws on their local knowledge, stimulates dialogue and mutual learning and creates wider ‘ownership’ of the evaluation’s findings. However, many M&E systems are still top-down, designed to extract information from the field to give to headquarters staff and donors. Collecting data solely for external use can undermine the participatory process.
Beneficiary participation in M&E can take various forms. In some projects, it may be no more than providing information to review or evaluation teams, but this is too limiting. Beneficiaries should be involved in planning the assessment (including selecting indicators), providing information on what was and was not achieved, analysing and verifying the results and making decisions about future activities. Findings should always be fed back to communities. The needs of communities in this regard may differ from those of outside agencies, and the targets, indicators and priorities developed by communities may differ considerably from those of agency staff. Adopting participatory approaches does not prevent the use of more formal data collection methods: these can complement or validate information gathered in a participatory way. Methods should be selected according to their usefulness in helping to understand impact.
Participatory methods such as those described in Chapter 6 are valuable in allowing beneficiaries to express their views. Standard participatory learning and action exercises can yield valuable information. Since it is never possible to involve everyone, careful thought must be given to ensuring that those who are consulted are representative of the range of groups concerned, paying particular attention to the most marginalised as well as people who may have dropped out of the project. Some evaluations pick up the views of similar people who were not involved in the project as a kind of ‘control group’.
Participatory impact assessment focuses on a project’s impact on beneficiaries’ lives rather than measuring project performance. The importance of identifying who benefits from a DRR initiative, and who does not, cannot be overemphasised. Evaluators should never assume that benefits are spread evenly across a community. They should assess beneficiary communities’ socio-economic characteristics carefully, considering gender issues and people who are vulnerable due to other factors, such as poverty, ethnicity, age and disability, as well as the influence of local power relationships.
Beneficiaries are one group of stakeholders. Project staff are another. NGOs and other local institutions, local and national government officials, and, where appropriate, international donor agencies and other kinds of organisation (e.g. the private sector) should be consulted if they have been involved in the project, are affected by it or have some influence on its outcome. It can be difficult to reconcile the views of such diverse groups. This makes it all the more important to be clear from the start about what M&E is designed to look at. Meetings should be held to discuss and explain this. Where stakeholders have different priorities and perspectives, this should be made explicit at the start to avoid misunderstandings later.
Evaluations are often funded by donors or in some way linked to ensuring continued donor support. In such circumstances the ideal of M&E as mutual learning may be hard to sustain. Many of those involved will be tempted to overstate the positive features of their project and downplay the negative ones. They may be defensive about their work, fearing that evaluation teams are searching for faults and problems. Community members may only tell evaluators what they think they want to hear.