{"id":319,"date":"2024-05-16T16:29:45","date_gmt":"2024-05-16T15:29:45","guid":{"rendered":"https:\/\/blogs.bath.ac.uk\/cds\/?p=319"},"modified":"2024-05-16T16:29:45","modified_gmt":"2024-05-16T15:29:45","slug":"new-article-on-mixed-methods-evaluation","status":"publish","type":"post","link":"https:\/\/blogs.bath.ac.uk\/cds\/2024\/05\/16\/new-article-on-mixed-methods-evaluation\/","title":{"rendered":"New article on mixed-methods evaluation"},"content":{"rendered":"

By James Copestake on May 12, 2024 02:48 pm<\/em><\/p>\n

Any academic will tell you that one of the high points of their research is seeing work finally published; so I\u2019m delighted (and relieved!) to be able to share the news that another paper building on QuIP experiences has just been e-published \u2013 this time in the\u00a0Journal of Development Effectiveness<\/a>. This adds to the list of papers tackling knotty methodological issues that we\u2019ve published from QuIP work \u2013 others having addressed\u00a0confirmation bias<\/a>\u00a0(with blindfolding), opaque qualitative data analysis (with causal mapping) and\u00a0cherry picking<\/a>\u00a0(with a broadly Bayesian approach to purposive case\/source selection). The new paper addresses two problems at once \u2013 the problem of dealing with complexity, and the problem of how to pass on the baton of impact evidence effectively so that it might even be used in a useful way. I can\u2019t claim the new paper has comprehensively cracked either of these problems, but I think it does have useful things to say about them, by rethinking what we mean by mixed methods.<\/p>\n

The full title of the paper is \u201cMixed methods impact evaluation in international development practice: distinguishing between\u00a0quant-led<\/em>\u00a0and\u00a0qual-led<\/em>\u00a0models.<\/a>\u201d<\/p>\n

In brief, the\u00a0quant-led<\/em>\u00a0model is centred on \u2018variance-based\u2019 causal attribution, supported by qualitative contextualisation and design, and (sometimes) by \u2018process theory-based\u2019 attribution to help explain findings. It fits with a more positivist approach to social science, and a relatively replicable, technical, and linear view of development practice informed by answers to relatively stable and narrowly defined causal questions. While costly to produce it has the potential to come up with relatively easily understood and scientifically credible numbers for the magnitude of development impact that commissioners demand, even while leaving open the question of how relevant these findings are to other contexts.<\/p>\n

The\u00a0qual-led<\/em>\u00a0model combines quantitative monitoring with reliance on process theory-based attribution, combining multiple sources of evidence in an open-ended process of iteratively testing and updating theoretical understanding of causal mechanisms. It reflects an interpretive view of development that is more path-dependent, social, and complex. Findings tend to be less precise but can be broader in scope, informing reflection over their relevance to other contexts, picking up on unexpected causes and effects, and enriching understanding of underlying causal mechanisms.<\/p>\n

Somewhat ironically, perhaps, the paper contrasts the two models to subvert use of the dichotomy between \u2018quant\u2019 and \u2018qual\u2019 in lazier and more simplistic ways. It\u2019s easy to join the \u2018randomista\u2019 camp, and attack those who haven\u2019t as being less methodologically rigorous; or to join the \u2018complexity\u2019 camp and attack others as lacking real world sophistication, for example. The problem is that this process of self-justification and \u2018othering\u2019 ends up entrenching differences, and undermining promising possibilities for synthesis. In distinguishing between qual-led and quant-led approaches to mixed methods the paper avoids claims about the general superiority of one over the other; rather it explores the causal process through which the two traditions associated with each model have emerged, how they can each be strengthened and transcended. Humans are not algorithms, and most of our decisions are based on qualitative judgements, on an often complex tangling of quantitative and qualitative data collection and analysis. That\u2019s what the paper sets out to clarify, in the vain hope of fostering more constructive understanding, discussion and practice.<\/p>\n

And here\u2019s the rub! In the moment of satisfaction at seeing a paper published after many months of cogitation and doubt, the seeds of doubt are already sown about how it could have been better; whether adding another dichotomy to discussion will clarify and connect, or just add to the confusion and polarisation. Let\u2019s see! For the moment, what Albert Hirschman memorably called the capacity for \u2018self-subversion\u2019 can wait. Meanwhile, this is an opportunity to thank those of you who have contributed to the experimentation and debate that the paper tries to reflect.<\/p>\n

The post\u00a0New article on mixed-methods evaluation<\/a>\u00a0appeared first on\u00a0Bath SDR<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

By James Copestake on May 12, 2024 02:48 pm Any academic will tell you that one of the high points of their research is seeing work finally published; so I\u2019m delighted (and relieved!) to be able to share the news...<\/p>\n","protected":false},"author":1794,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-319","post","type-post","status-publish","format-standard","hentry","category-uncategorised"],"acf":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pd4Pxr-59","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/posts\/319","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/users\/1794"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/comments?post=319"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/posts\/319\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/media?parent=319"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/categories?post=319"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/cds\/wp-json\/wp\/v2\/tags?post=319"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}