As the field gets ready to convene at ChangeFest18 it’s worth taking a look back on the learning throughout the year. Here is a reflection from the Complexity and Evaluation Conference held in May 2018.
A Reflection from Dr Sophia Harryba
The main messages from the Complexity and Evaluation Conference for me were to embrace failures as stepping stones to success and as learning opportunities, but also learning to know when to stop when evidence shows that efforts/initiatives are not working.
This is difficult for many, as we invest so much money and time, and ourselves into an ‘it’ which we thought would work, and it might have, but in the complexity space, there is a need to constantly evaluate and evolve.
When thinking about designing systems change, the speakers pointed out that we tend to focus on who will benefit, where the lens should be on who is being left out by any proposed change.
The challenges of evaluating systems change were discussed as:
- Lack of results in the short term. This is especially difficult in the current funding environment where we are constantly needing to ‘prove’ our programs ‘work’
- Systems are also difficult to measure – there are no standards, no certainty that what we measure will give use what we want, and it does not necessarily contribute or drive our decision making; and
- We tend to look to traditional solutions
Mark Cabaj then discussed the use of principle based evaluation and whether it was meaningful for evaluators to integrate principles in their work. I thought of principles as ‘values’, which should drive our work.
Kate McKegg outlined how her work in NZ was using principles based evaluation and encouraged us to delve deeply in to the value considerations chosen, and which ones are not – and the consequences of these decisions.
The three core questions from Kate’s work were:
o What is at stake? And for whom? What ought to be at stake?
o Who are the involved and affected stakeholders? Who ought to be the stakeholders?
o What are the key issues or problems related to what’s at stake?
Jess Dart offered a method to evaluate systems change by using Program Logic (PL) and Theory of Change. PL can enable visual representation of what outcomes we are trying to measure and which parts of the system we are aiming to change/evaluate. Jess also noted that when doing this work, you often aim to change one outcome but end up changing another, due to the dynamic and complex nature of systems change.
The conference was packed with activities and break out sessions to sharpen our evaluation skills and elevate our thinking from program to systems change.
As Mark Cabaj said- programs beat the odds but systems change the odds
I am working on seeing my failures as stepping stones and on sharing those with others on similar journeys.
About our guest blogger: Dr Harryba says her first love is being a mother of a 17 month old who keeps her on my toes. Her second love is serving vulnerable people and for the past 8 years this has been in the form of providing evidence to inform policy and practice on what works for whom and when.