Issue

Vol. 49 No. 1 (2024): Africa Development

3 - Randomised Trials As A Dead-End For African Development

https://doi.org/10.57054/ad.v49i1.5615
Seán Mfundza Muller
Johannesburg Institute for Advanced Study (JIAS), University of Johannesburg, South Africa

Corresponding Author(s) : Seán Mfundza Muller

academic@Seanmuller.Co.Za

Africa Development, Vol. 49 No. 1 (2024): Africa Development

Abstract

Randomised Control Trials (RCTs) have been presented as a method for identifying interventions that will improve the lives of residents of economically poorer countries. I argue that RCT-driven development policy is unlikely to achieve even its narrower objectives and, instead, is likely to hinder the attainment of long-term economic and social development objectives in African countries. The article examines two fundamental methodological problems: first, that proselytisers of this approach, despite claims to the contrary, have not addressed the challenge of extrapolating from experimental results to policy interventions; second, that while the use of RCTs is framed as an objective scientific approach to policymaking, it systematically smuggles in the ideological and other biases of the researchers involved. These claims are illustrated with three sets of examples: controversial studies that involved cutting off water to poor households in Nairobi and randomising exposure to a Christian missionary programme in the Philippines; studies on civil servant absenteeism in Kenya and India cited in the 2019 Nobel award; and two case studies from South Africa pertaining to labour market and educational interventions where RCTs distorted the policy process with negative consequences. The conclusion is that the RCT approach is a dead end for African development.

Keywords

African development development economics epistemic hierarchy evidence-based policy randomised control trials
[1]
Muller, S.M. 2024. 3 - Randomised Trials As A Dead-End For African Development. Africa Development. 49, 1 (Jul. 2024). DOI:https://doi.org/10.57054/ad.v49i1.5615.
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. Abramowicz, M., and Szafarz, A., 2020, ‘Ethics of Randomized Controlled Trials: Should Economists Care about Equipoise?’, in Bédécarrats, F., Guérin, I. and Roubaud, F., eds, Randomized Control Trials in the Field of Development: A Critical Perspective, Oxford: Oxford University Press.
  2. Angrist, J.D., and Pischke, J-S., 2009, Mostly Harmless Econometrics, Princeton: Princeton University Press.
  3. Angrist, J.D., and Pischke, J-S., 2010, ‘The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics’, Journal of Economic Perspectives, Vol. 24, No. 2, pp. 3–30.
  4. Baele, S.J, 2013, ‘The Ethics of New Development Economics: Is the Experimental Approach to Development Economics Morally Wrong?’, The Journal of Philosophical Economics, Vol. VII, No. 1, pp. 1–42.
  5. Banerjee, A., 2007, Making Aid Work, Cambridge (MA): MIT Press.
  6. Banerjee, A. and Duflo, E., 2006, ‘Addressing Absence’, Journal of Economic Perspectives, Vol. 20, No. 1, pp. 117–32.
  7. Banerjee, A., and Duflo, E., 2009, ‘The Experimental Approach to Development Economics’, Annual Review of Economics, Vol. 1, pp. 151–78.
  8. Banerjee, A., and Duflo, E., 2011, Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, New York: Public Affairs.
  9. Banerjee, A., Duflo E., and Glennerster, R., 2008, ‘Putting a Band Aid on a Corpse: Incentives for Nurses in the Indian Public Health Care System’, Journal of the European Economic Association, Vol. 6, No. 2–3, pp. 487–500.
  10. Bardhan, P., 2013, ‘Little, Big: Two Ideas about Fighting Global Poverty’, Boston Review, May.
  11. Bédécarrats, F., Guérin I., and Roubaud, F., 2019, ‘All that glitters is not gold the political economy of randomised evaluations in development’, Development and Change, Vol. 50, No. 3, pp. 735–762.
  12. Bold, T., Kimenyi, M., Mwabu, G., Ng’ang’a, A., and Sandefur, J., 2018, ‘Experimental Evidence on Scaling up Education Reforms in Kenya’, Journal of Public Economics, Vol. 168, December: pp. 1–20.
  13. Bryan, G., Choi, J., and Karlan, D., 2018, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, w24278, Cambridge, MA: National Bureau of Economic Research.
  14. Bryan, G., Choi, J., and Karlan, D., 2021, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, The Quarterly Journal of Economics, Vol. 136, No. 1, pp. 293–380.
  15. Burns, J., Edwards, L. and Pauw, K., 2010, ‘Revisiting wage subsidies: How pro- poor is a South African wage subsidy likely to be?’, Development Southern Africa, Vol. 30, No. 2, pp. 186–210.
  16. Cartwright, N., 2007, ‘Are RCTs the Gold Standard?’, BioSocieties, Vol. 2, No. 1, pp. 11–20.
  17. Center for International Development, 2008, CID South Africa Growth Initiative. http://www.cid.harvard.edu/southafrica/. Accessed 7 July 2017.
  18. Concato, J., Shah, N., and Horwitz, R., 2000, ‘Randomized Controlled Trials, Observational Studies and the Hierarchy of Research Designs’, New England Journal of Medicine, Vol. 342, No. 25, pp. 1887–92.
  19. Cook, T.D., and Campbell, D.T., 1979, Quasi-Experimentation: Design and Analysis Issues for Field Settings, New York: Houghton Mifflin.
  20. Coville, A., Galiani, S., and Gertler, P., 2020, ‘A Comment on the Ethical Issues Twitter Discussion’ in “Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums” (Version 2), 8 August 2020 (https://drive.google.com/file/d/1nVGR4qlhWt2EcNiy6d02By3e4kptVSer/view). Accessed 22 November 2021.
  21. Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2020, ‘Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums’, w27569, Cambridge, MA: National Bureau of Economic Research.
  22. Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2021, ‘Financing Municipal Water and Sanitation Services in Nairobi’s Informal Settlements’, 9725, Washington, DC: World Bank.
  23. Deaton, A., 2010, ‘Instruments Randomization and Learning about Development’, Journal of Economic Literature, Vol. 48, No. 2, pp. 424–55.
  24. Deaton, A., and Cartwright, N., 2018, ‘Understanding and Misunderstanding Randomized Controlled Trials’, Social Science & Medicine, Vol. 210, pp. 2–21.
  25. Duflo, E., Dupas, P., and Kremer, M., 2015, ‘School Governance, Teacher Incentives and Pupil-Teacher Ratios: Experimental Evidence from Kenyan Primary Schools’, Journal of Political Economy, Vol. 123, pp. 92–110.
  26. Duflo, E., Hanna, R., and Ryan, S.P., 2012, ‘Incentives Work Getting Teachers to Come to School’, American Economic Review, Vol. 102, No. 4, pp. 1241–78.
  27. Fleisch, B., 2008, Primary Education in Crisis: Why South African Schoolchildren Underachieve in Reading and Mathematics, Cape Town: UCT Press.
  28. Harrison, G.W., 2013, ‘Field Experiments and Methodological Intolerance’, Journal of Economic Methodology, Vol. 20, No. 2, pp. 103–17.
  29. Hoffmann, N., 2020, ‘Involuntary Experiments in Former Colonies: The Case for a Moratorium’, World Development, Vol. 127: 104805.
  30. Jedwab, R., Meier zu Selhausen, F., and Moradi, A., 2022, ‘The Economics of Missionary Expansion: Evidence from Africa and Implications for Development’, Journal of Economic Growth, Vol. 27, No. 2, pp. 149–92.
  31. Levinsohn, J., 2008, ‘Two Policies to Alleviate Unemployment in South Africa’, Center for International Development Working Paper 166.
  32. Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014a, ‘Wage Subsidies and Youth Employment in South Africa: Evidence from a Randomised Control Trial’, Stellenbosch Economic Working Papers 02/14.
  33. Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014b, A Youth Wage Subsidy Experiment for South Africa, 3ie Impact Evaluation Report 15 (http://www.3ieimpact.org/media/filer_public/2014/08/20/ie15-rankin-youth_wage_ subsidy.pdf).
  34. Manski, C.F., 2011, ‘Policy Analysis with Incredible Certitude’, Economic Journal, Vol. 121, No. 554, pp. F261–89.
  35. Muller, S.M., 2014, ‘Randomised Trials for Policy: A Review of the External Validity of Treatment Effects’, PhD dissertation, University of Cape Town.
  36. Muller, S.M., 2015, ‘Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations’, The World Bank Economic Review, Vol. 29, Suppl 1, pp. S217–25.
  37. Muller, S.M., 2017, ‘What Does an (South) African Economics Look Like?’, Proceedings of the 2017 Scholarship of Teaching and Learning (SoTL) in the South Conference, Johannesburg, 24–27 July, 2017, pp. 87–97.
  38. Muller, S.M., 2020, ‘The Implications of a Fundamental Contradiction in Advocating Randomized Trials for Policy’, World Development, Vol. 127, 104831.
  39. Muller, S.M., 2021a, ‘Evidence for a YETI? A Cautionary Tale from South Africa’s Youth Employment Tax Incentive’, Development and Change, Vol. 52, No. 6, pp. 1301–1342.
  40. Muller, S.M., 2021b, ‘Randomised Trials in Economics’, in Kincaid, H and Ross, D., eds, The Modern Guide to Philosophy of Economics, New York: Edward Elgar Publishing.
  41. Muller, S.M., 2022, ‘The Unacknowledged Normative Content of Randomised Control Trials in Economics and Its Dangers’, in Badei, S. and Grivaux, A., eds, 2022, The Positive and the Normative in Economic Thought, New York: Routledge.
  42. Muller, S.M., 2023, ‘Is economics credible? A critical appraisal of three examples from microeconomics’, Journal of Economic Methodology,Vol. 30, No.2, 157-175, DOI: 10.1080/1350178X.2023.2202682
  43. Nobel Media, 2019, Advanced Information.
  44. Rankin, N., 2012, ‘A Common Approach’, Financial Mail. http://www.financialmail.co.za/fm/2012/05/24/a-common-approach.
  45. Rankin, N., 2013, ‘Give Youth Wage Subsidy a Chance, Mail&Guardian. http://mg.co.za/article/2013-11-01-00-give-youth-wage-subsidy-a-chance.
  46. Ravallion, M., 2009, ‘Should the Randomistas Rule?’ Economists’ Voice, Vol. 6, No. 2, pp. 1–5.
  47. Ravallion, M., 2018, ‘Should the Randomistas (Continue to) Rule?’, Center for Global Development Working Paper 492. https://www.cgdev.org/publication/should-randomistas-continue-rule
  48. Robbins, L., 1932, An Essay on the Nature and Significance of Economic Science, London: Macmillan and Co.
  49. Rodney, W., 1972, How Europe Underdeveloped Africa, London: Bogle-L’Ouverture Publications.
  50. Rothwell, P.M., 2005, ‘Subgroup Analysis in Randomised Controlled Trials: Importance, Indications and Interpretation’, The Lancet, Vol. 365, pp. 176–86.
  51. Taylor, S., and Watson, P., 2015, The Impact of Study Guides on “Matric” Performance: Evidence from a Randomised Experiment, Stellenbosch Economic Working Papers.
  52. Unknown, 2011, POLICY INFLUENCE PLAN – Grant Number OW2.044 – University of the Witwatersrand. 3ie International Initiative for Impact Evaluation.
  53. Van der Berg, S., 2007, ‘Apartheid’s Enduring Legacy: Inequalities in Education’, Journal of African Economies, Vol. 16, No. 5, pp. 849–80.
  54. Van der Berg, S., Spaull, N., Wills, G., Gustafsson, M., and Kotzé, J., 2016, Identifying Binding Constraints in Education, Stellenbosch: RESEP.
  55. Veblen, T., 1898, ‘Why Is Economics Not an Evolutionary Science?’, The Quarterly Journal of Economics, Vol. 12, No. 4, pp. 373–397.
  56. Ziliak, S.T., and Teather-Posadas, E.R., 2016, ‘The Unprincipled Randomization Principle in Economics and Medicine’, in DeMartino, G. F. and McCloskey, D., eds, The Oxford Handbook of Professional Economic Ethics, Oxford: Oxford University Press.
Read More
Author biographies is not available.
Download
PDF
Statistic
Read Counter : 1926 Download : 210

Downloads

Download data is not yet available.

Similar Articles

1 2 3 4 5 6 7 8 9 10 > >> 

You may also start an advanced similarity search for this article.