Around 168 million children worldwide are involved in child labour, according to recent ILO estimates. More than half of them, 85 million, are doing hazardous work. These numbers have fallen in recent years, but trends vary across regions, countries and sectors and the figures remain alarmingly high. In that context, impact evaluation can help policymakers to best target their interventions. Part-time PhD fellow Julia Brümmer explains the background to her research.
You just presented your research, a meta study of impact evaluations on child labour, to the GPAC² community. What exactly is your research about?
My research focuses on impact evaluation designs in the context of child labour interventions. The idea is to look at the methodologies that have been used so far in evaluating the impact of child labour projects, programmes or policies in order to see what these methodologies allow us to know (and what not) about the impact of those interventions.
It seems that until now, mostly experimental and quasi-experimental approaches have been used to evaluate the impact of child labour interventions. These designs are stronger at quantifying impact than at explaining how and why things have worked (or not). They also have difficulties in dealing adequately with complex issues like child labour where various determining factors interact at household, community and national level.
Besides analysing the challenges faced by impact evaluations of child labour interventions, I am proposing to test some impact evaluation designs that may add value to the commonly used experimental and quasi-experimental approaches. These are likely to be theory-based evaluation designs focusing on mechanisms and how these work out in various contexts to produce various outcomes.
The evaluation aspect of your study is directly linked to your work as a project monitoring officer for the Lutheran World Federation and previously as an evaluation officer at the ILO in Geneva. Being involved in those evaluation projects made you question the evaluation approaches commonly used. Can you elaborate on what you found not to be functioning well in practice?
First of all, in my experience, evaluation is often seen as an exercise for accountability. Donors want to know whether it has been worth spending their funds on an intervention while implementers are eager to prove that they have merited the continuation of financial support. In such a situation, the learning aspect often gets neglected.
Secondly, I have seen large budgets spent on evaluations, which end up producing a report that is not very useful to anyone. In some cases, this is because evaluation methods found in text books are not well adapted to the complex – and often ‘messy’ – realities of development work. Moreover, these methods are not always suitable for answering the questions that are really important to those working on the ground. Generally, I believe that evaluation has a huge potential to better contribute to the relevance and effectiveness of development interventions, which is yet to be fully exploited.
In addition to your work, you are actively engaged in the European Evaluation Society, which organises biennial conferences on evaluation in theory, practice and research. The next conference will be held in Maastricht on 26-30 September 2016. How does the conference support PhD fellows and researchers interested in evaluation?
The conference of the European Evaluation Society will bring together more than 600 professionals and researchers in the field of evaluation, from all over Europe and beyond. This year’s conference theme is ‘Evaluation Futures in Europe and beyond – Connectivity, Innovation and Use’ and the early registration deadline is approaching fast, on 15 July 2016!
We will have a wide-ranging programme covering, among others, questions of ethics and professional standards in evaluation, the organisational architecture of evaluations – such as networks and partnerships – and the audiences and use of evaluation. One thematic strand touches upon evaluation research, including the role of various academic disciplines in relation to evaluation.
Students and researchers interested in evaluation will find an opportunity, at the conference, to engage with some of the world’s leading experts in this area. The regular conference is preceded by two days of training workshops on specific evaluation-related topics. Students and professionals who are new to the field of evaluation are taken special care of by the EES Thematic Working Group for Emerging Evaluators, which will be organising various activities throughout the conference.
As a PhD fellow in the GPAC2 programme, it helps to have a job that overlaps with your research. Can you tell us how your PhD benefits from your work experience, and how you manage to combine your full-time job with the activities done for your PhD?
I usually hear the question the other way around: colleagues often ask me how my work benefits from my PhD studies. In fact, I believe that both are stimulating each other. I have difficulties to imagine that I could be doing the same PhD research without having the ‘real-world’ exposure to development projects that I am getting through my job. At the same time, taking the time to reflect on how a development intervention actually works out – and how you can know about that, with which limitations etc. – allows me to look at my work from a fresh perspective.
Sometimes I wish that my PhD research and the things that I do for my job would be even more closely related. For instance, the kind of impact evaluation designs that I am looking at in my research are not likely to ever get used in the projects that I am overseeing in my work at the Lutheran World Federation. Those projects are mostly small in size and we neither have the human capacity nor the financial resources to implement sophisticated impact evaluations.
As for many other GPAC2 fellows, finding enough time to do both the PhD and the job well – plus all the other things one does in life – is a big challenge. It takes a strong motivation. Fortunately, I have that motivation and I tend to become more productive and efficient when I have plenty of things to do.