Evaluation in Complex Settings and Leadership

I was browsing around the site of a very interesting conference to be held in Australia, Show Me The Change, and I noticed a number of people I respect and follow were involved. No wonder – the site was engaging, inviting. If I were in Australia, I’d go. Here is a bit about the gathering:

A National Conference on ‘Evaluation of Behaviour Change’ for Sustainability

We all know that behaviour change is complex. How do we show what’s working and how do we evaluate it? You are invited to participate in Show Me The Change and explore what matters most to you. You can take part in the ongoing conversations here, on our Show Me The Change blog. We’d love to hear your ideas and your comments. If you’re a Twitter user, please use the hashtag #smtc for your posts there.

I can’t believe I’m actually enjoying wandering through a conference site. Thanks Johnnie, Viv, Bob, Anne, Andrew, Geoff, and Chris .

Then I remembered I had a blog draft noting something that Chris (as in Corrigan) had written waaaay back in September of 09. Time to dig it up.Why not mash up evaluation and leadership?  In truth, I think they have a lot to do with each other – at least participatory leadership does.

If you are interested in leadership, go take a look at the post — too much good stuff to just tease you with a quote!

Chris Corrigan » Describing participatory leadership
How do you explain participatory leadership in one sentence?

Are Status Updates Conversations?

I’ve had this link in my “drafts” box for too long, so I want to drag it out and blog it because it raises an important issue. What happens when we trivialize the concepts of “friends,” “conversation” and “community” when we apply them to things that are sort of like friends, conversation and community, but don’t quite cross the threshold. In this image from a recent Forrester blog post, they have added to their original “ladder of participation” (which I find useful, but I cringe at the linearity because I don’t think it is always sequential as shown). What they added was Tweets categorized as conversation. Take a look.

Forrester

Yes, you can have a conversation in Twitter, but I think most Twitter traffic is not conversation. It is a flow of snippets, of 140 character fragments which we can, if we wish, make sense of. We can construct a narrative, but we may not be constructing the intended narrative.

Conversation implies for me turn taking, listening and sensemaking. Status updates… not so much. What do you think?

Related: The Conversation Prism

A Gem from KM4Dev on Impact and Outcomes

There has been a great discussion on the KM4Dev mailing list the last 10 days or so about evaluation, impact and measurement. In the context of international development, this is critical. Why do something if it doesn’t make a difference. However, often we don’t do a very good job figuring out what does make a difference, let alone know why (causality.) Dave Snowden posted something that just rang the bell for me. I hastily copied it down to share here. The link to the web archive of the email discussion is at the bottom, if  you want to mine for the rest of the thread. Emphasis is mine.

The linear concept of input, leading to outputs, leading to outcomes which in turn leads to impact is I think at the heart of the problem, It implies (and I can see why people would want this) a causal chain that can be replicated.

However if the system is complex (in the sense of complex adaptive) then any input is a stimulus or modulator which influences but does not determine impact. That means we need to start measuring the sensitivity of a system to different stimuli, and the way in which some stimuli produce a disproportionate effect in that they catalyze other inputs. This is newly developing area which has not hit the development sector yet, but we are working on it in related fields, loosely termed modulator mapping. It also leads us to evolutionary representations (such as fitness landscapes) and measure based on stability of landscapes. In all those cases mathematics are simplified by representation and linked micro-narratives. There is no point in measuring anything if the results do not convince both donors and recipients alike to take action

All of that moves the “impact” agenda on. I didn’t confuse outputs and outcomes, I conflated them as the model means there is no real difference in what is measured in practice.

via Discussions.

I am now going to start paying attention to this idea of “measuring sensitivity of a system to different stimuli.” This relates closely to two projects I’m working on where I have been sensing this, but hadn’t had the words for it. Now I have a toehold. Onward!

Photo credit: