Full Circle Associates

I am in the midst of thinking about assessment, monitoring and evaluation for a number of projects. I always look for a balance of techniques and measures -- quantitative and qualitative. Evaluating online interactions (courses, teams, events) is still a bit of a crap shoot.

I have used After Action Review both online and offline, but I realized I had not used Most Significant Change online. I was reminded of this by a post on the Anecdote blog about MSC. Here is a bit of explanation from Shawn Callahan:

Anecdote: Evaluating the soft stuff: "In 1994 Rick Davies was faced with the job of assessing the impact of an aid project on 16,500 people in the Rajshahi zone of western of Bangladesh (6). The idea of getting everyone to agree on a set of indicators was quickly dismissed as there was just too much diversity and conflicting views. Instead Rick devised an evaluation method which relied on people retelling their stories of significant change they had witnessed as a result of the project. Furthermore, the storytellers explained why they thought their story was significant.

If Rick had left it there the project would have had a nice collection of stories but the key stakeholders' appreciation for the impact the project would have been minimal. Rick needed to engage the stakeholders, primarily the region's decision-makers and the ultimate project funders, in a process that would help them see (and maybe even feel) the change. His solution was to get groups of people at different levels of the project's hierarchy to select the stories which they thought was most significant and explain why they made that selection.

Each of the 4 project offices collected a number of stories and were asked to submit one story in each of the four areas of interest to the head office in Dhaka. The Dhaka head office staff then selected one story from the 16 submitted. The selected stories and reasons for selection were communicated back to the level below and the original storytellers. Over time the stakeholders began to understand the impact they were having and the project's beneficiaries began to understand what the stakeholders believed was important. People were learning from each other. The approach, called Most Significant Change, systematically developed an intuitive understanding of the project's impact that could be communicated in conjunction with the hard facts.

Rick's method was highly successful: participation in the project increased; the assumptions and world views surfaced, helping in one case resolve an intra-family conflict over contraceptive use; the stories were extensively used in publications, educational material and videos; and, the positive changes where identified and reinforced.

To date the application of Most Significant Change has been mostly confined to NGO programs and other not for profit organisations. But this is changing. Corporations are also recognising that issues such as culture change, communities of practice, learning initiatives generally and leadership development could benefit from an MSC approach. Anecdote is currently assisting one large IT and consulting company implement MSC to evaluate the impact of its culture change program."

We are going to be using stories in both the projects I'm thinking about. So now I have to refresh myself more on MSC and see if it might be a useful practice in our M&E.

I need to get my hands on the guide Shawn mentions as well!

Tags: evaluation, monitoring, most_significant_change, AAR