Trekking the world, one mountain at a time ¦ June Update

As part of Pamoja's 'Business for Good' strategy, we support good causes that are close to our hearts. One such amazing cause is the Christopher Angus Fund, which you can learn more about here.  Each month, Michael Angus, co-founder of the Fund, guest blogs to tell us about his progress to trek the world, one challenge at a time.


First day: setting off on Canal Trek, Lochrin Basin, Edinburgh

First day: setting off on Canal Trek, Lochrin Basin, Edinburgh

June 2017 has been, as predicted, all about getting the training going, in earnest, for the Rockies trek in September.

The month began with some self-inflicted isolation – in order to get my mind right basically; I headed off for three days into Scotland’s Westcoast wilderness, travelling back in time, literally and metaphorically, to walk in amongst the historical remnants of Scotland’s prehistoric past: along the Dalriada Way to Fort Dunadd and Kilmartin Glen.

There is something almost tangibly prescient about the past here – ancient bones emanate. It is where the original kings of Scotland were crowned, but before that, it is where settlers buried their chiefs in celebrated mounds, and practiced sun worship: the area abounds with stone circles; it is truly a magical place, a vast flat flood plain, stretching for six miles, almost perfectly aligned north to south – it must have provided the ideal landscape within which to study the skies…… Weather permitting of course - which seemed appalled by my presence, it must be said – maybe I imagined it. But I was the only figure in the landscape, so for whom other did nature wish to impress, by its awesome display of thunder and lightning at 11 in the morning on the second day? I felt honoured, to be honest - altogether, it was wonderfully primal……being in that the place, and nature performing at its powerful best.

FullSizeRender.jpg

It has in fact, been a month of being witness to impressive phenomena. Canals have been the predominant man-made feature of the month, and they have been pretty awesome in their own right; for these first three days, I stayed by the canal at Cairnbaan, and later in the month I completed the cross-country canal challenge that I set myself: to walk the Union Canal and the Forth + Clyde Canals, east to west, all the way from Edinburgh, to Glasgow and beyond, right to and back to the Westcoast. 65 miles in three days. Toughest thing I’ve set out to do; it’s no mean feat of human engineering and construction endeavour either.

Things did not go as well as intended. Seems my mind was not quite as right as I hoped - I took eight days to complete the trek, not three - due to an enforced injury, which was completely of my own making. I walked too far, too fast on the first day, injured my foot, which by the end of the second day was simply done. I had to let it heal, before going back to complete the last day five days later.

This has been the first challenge that I have not completed as intended – it’s provoked a lot of soul searching and reflection. I have a lot (personally) invested in this trekking campaign. But what I discovered, or rather what has been confirmed, is that I love trekking. I could never have predicted making such a statement – but the thrust of the planet, pushing back under one’s feet, even feet somewhat aching and bloody, is deeply comforting. It’s a healing thing to do, even if it harms – a contradiction, certainly, especially, as one’s mind wanders when one treks, and thoughts are not always entirely wholesome, the grief demons most certainly take advantage and invade – I suppose though, that really, they have nowhere to go – and the broad expanse of the landscape can accommodate their unhinged expulsion; whatever angers and rages I might feel, the natural world can summon breaths and downpours, and thunderous (literally) voices of its own, to both mirror and acknowledge my own dark heartache. The man-made world cannot match such ache – but one has to applaud the endeavour involved in the construction of such a thing as the Union and Forth + Clyde Canals – the will – which has created a place for water to rest. Within all the torpor of the natural world, the static and level calm of the canal infused more than anything else, by its silent balm, an unruffled ear to my unspoken aches and pains – both the physical and the mental.

In between these treks, I’ve continued to complete other shorter and regular training walks, locally and through the city; altogether, distance travelled 90.5 miles. Plans for the following month are to continue, with regular walks through the week, and longer treks at the weekends. It’s good to have a plan……and speaking of which, the longer-term plan has been moved on: this month, I officially registered to take the 8 day trek challenge in the Grand Canyon in October 2018: the fifth of six treks that I’m setting myself to complete in six years.

It’s all investment……

Thank you for reading.

Capturing complex change: is it really all about confidence?

Guest blog by Tom Aston

For those of us that sing the praises of social accountability (citizen-driven initiatives to hold those in power to account), making a claim about “impact” (or transformative change) is a challenge we face on a daily basis. And CARE’s not alone. The title of the first session at an NGO political economy analysis working group at which I’m presenting this week (“Building the Evidence-base for Social Accountability”) speaks to the same concern.

Some argue that we need more longitudinal studies. Others advocate the use of Randomised Control Trials (RCTs). And a recent study CARE conducted on the influence of Community Score Cards on reproductive health-related outcomes in Malawi shows that RCTs have a place, and do demonstrate that social accountability makes a difference.

But, once you consider that outcomes are behavioural changes of real (complicated) people, you quickly see, as Marina Apgar recently suggested, why we need to move “beyond measurement of linear pre-defined change and intervention-effect alone and [use] mixed-methods to help us understand emergent complex social change.” Social accountability outcomes (such as mayors changing budgets to benefit poorer areas, or even procuring a new ambulance) don’t fit neatly into boxes. They rely on our capacity to influence behaviour, and this is behaviour we can’t (fully) control. So, we need to better explain HOW change happened, not merely to assert that it did. 

Recognising this has led CARE to explore various theory-based methods such as Most Significant Change and Outcome Mapping. With a particular emphasis on the change process, we are now piloting Contribution Tracing with Pamoja Evaluation Services in Ghana and Bangladesh to help us better understand CARE’s contribution to social accountability outcomes.

Contribution Tracing is all about increasing your confidence in making claims about impact. Essentially, you make a “claim” about your intervention’s role in achieving an outcome that really happened (your contribution), and then find evidence to defend your claim.

To do this, like other theory-based methods, you need a hypothesis (a proposed explanation) about how you think change happened. You then review the connection between different steps (or components) in that process

You identify evidence that would help support (or undermine) your proposed explanation using the four tests of Process Tracing (Straws-in-the-wind, Hoops, Smoking Guns, Doubly Decisive).

What matters is not how much evidence you have, but how good that evidence is to help confirm that each part of your proposed explanation for your claim really exists (“probative value”).

In Contribution Tracing, you use Baysian (Confidence) Updating to assign a probability (how likely it is) that the various components of your contribution claim exist; and ultimately whether your claim holds true. You then update your confidence after gathering data precisely tailored to your claim (increasing or decreasing the probability using the four tests), compare this against rival explanations, and then put it up for “trial”, inviting others in to peer review your claim.

We’re right at the beginning of the journey, but to me, what our learning already suggests is that:

  • You can show your contribution, even when change processes are complex;
  • You can make credible impact claims, without a counterfactual;
  • You can tighten up your loose theory of change as you go along, and;
  • You may not need to gather as much data as you think you do to prove it.

But don’t take my word for it; listen to some reflections from staff on the experience so far. And watch this space for more to come.

A new learning partnership for inclusive governance

CARE International recently entered into a partnership with Pamoja Evaluation Services for an exciting new learning partnership, known as the Halcrow Project. The purpose of the learning partnership is to build on CARE’s substantial expertise in inclusive governance programming by better capturing its effects through a strengthened approach to monitoring and evaluation. Therefore, the Partnership will seek to produce credible and rigorous evaluations that can speak directly to CARE’s contribution to observed changes in contexts where inclusive governance work is taking place.

To help CARE realise its ambition to better capture the effects and contribution of its work in inclusive governance, Pamoja is supporting CARE to apply the cutting-edge evaluation approach of Contribution Tracing as an organising framework. Two of CARE's country offices in Ghana and Bangladesh are taking part in the project, due to the extensive inclusive governance work they deliver. This will provide us with an ideal learning opportunity for an evaluation that applies Contribution Tracing.

Debates in the evaluation practitioner and academic spaces, and our own in-depth literature review, strongly indicates that randomised approaches are, by themselves, not a solution to the challenges of measuring inclusive governance. While we have very many positive outcomes measured for inclusive governance work, there is very little documentation on impact. Therefore, we are facing an evidence challenge in which we can seek new opportunities to test and implement innovative approaches such as Contribution Tracing, to enhance this knowledge base and fill our evidence gaps around the impact of inclusive governance.

Contribution Tracing is based on the principles of Process Tracing and Bayesian (Confidence) Updating. This approach offers practitioners clear guidance on what data to collect. It has been designed to support the formulation and validation of a ‘contribution claim’ about the role played by an intervention to determine if outcomes of interest are realised. It then measures how much particular items of evidence increase or decrease confidence in relation to a specific contribution claim. This process will support CARE’s inclusive governance programmes to design a system which focuses on gathering the ‘right’ data, thereby using resources for monitoring and evaluation more efficiently.

Contribution Tracing forces evaluators to think – in great detail – about how and why a particular change has come about as a result of a project. Through this evaluative pilot process, we will identify what works and where gaps exist in the way the team has been monitoring and documenting change.

Under this exciting new initiative, a group of staff from Ghana and Bangladesh are working collaboratively with Pamoja until the end of this year. By challenging practices and assumptions, and using new ways of thinking about evidence - offered by Contribution Tracing - we aim to demonstrate CARE's unique contribution to transformation in the inclusive governance space. We are confident this approach will help CARE to unpack the nature of social accountability in this context and better articulate the role that CARE and its partners have played in delivering impact to citizens.

Advocacy evaluation: art of the possible

by Gavin Stedman-Bryce

I was asked by Médecins Sans Frontières (MSF) to attend a staff conference in Geneva earlier this year. I had been asked to speak about the importance of monitoring and evaluation of advocacy - a real passion of mine. Alas my diary was full the day of the conference but thankfully the organisers allowed me to attend via video.

This short film, featuring Tom Aston, a Governance Advisor from CARE UK International, touches on why advocacy M&E is not only increasingly important, but possible thanks to a range of innovative methods and tools. 

An evaluator's journey

I started calling myself an evaluator in 2008 when I established Pamoja; but for years after, during long flights, I would ponder how I ended up becoming an evaluator. On one such flight, pondering whether my in-flight meal was fit for human consumption, I finally found my answer. Tanzanian teachers!

As soon as I graduated, I leapt at the chance to volunteer in East Africa with VSO. When I first landed in rural Tanzania, I was only 23 years old, straight out of Glasgow’s East End. My naivety at how the world worked was at PhD levels of genius.

I recall getting really angry as I looked around at my new home, a place called Singida - a small town in the middle of Tanzania, days from anywhere. I was angry because nothing seemed to work at the school where I was volunteering as an A-Level biology teacher.

When I say nothing seemed to work, I am referring to the teachers who were never in class. I seemed to be the only teacher who would turn up. Where were my colleagues and why were they not taking their duties as a teacher seriously? Did they not care about the students?

This was my first hard lesson from mother Africa; everything has a reason. I learnt early in my career, that if you ask the ‘right’ questions of the ‘right’ people, in the ‘right’ way - and then listen carefully - an answer will usually present itself. In my school teacher case, it was that teachers were busy working in their fields to ensure a bumper maize crop to bring much needed money into their households. I later learned that the Ministry of Education hadn’t paid salaries to teachers in almost 12 months. That’s why they weren’t in school teaching their students.

I had found out it wasn’t laziness or greed, it was need. This memory reminded me of why I call myself an evaluator. I am a naturally curious person, always thinking about how things could be improved. For a time, I thought of myself as too critical. But I now believe that pessimists make great evaluators and I am definitely in the glass half-empty camp. At least that’s what I tell myself.

Pessimism aside, I firmly believe that as humans, we can always get better at what we do, if we take time to learn about what works (or doesn’t work), and share our knowledge.

Yes, I still get angry at the injustices in the world but as an evaluator, by inspiring others to learn and improve, I have found my unique way of turning frustration into positive action.

I am passionate about learning and improving and I want to inspire others to learn and improve too. It’s been a long time since I’ve been on the frontline in a development sense, but as an evaluator, I make my small contribution through inspiring good change. And I owe it all to some absentee biology teachers from Tanzania.   

 

What's your story? We'd love to hear about how you became an evaluator. Send your stories to info@pamoja.uk.com.