3 - Randomised Trials As A Dead-End For African Development
Corresponding Author(s) : Seán Mfundza Muller
Afrique et développement,
Vol. 49 No 1 (2024): Afrique et développement
Résumé
Les essais contrôlés randomisés (ECR) ont été présentés comme une méthode permettant d'identifier les interventions qui améliorent la vie des populations des pays économiquement pauvres. Je soutiens qu’il est peu probable qu’une politique de développement axée sur les ECR atteignent ces objectifs, même les plus restreints d’entre eux, et qu’elle risque plutôt d’entraver la réalisation des objectifs de développement économique et social à long terme dans les pays africains. L'article examine deux problèmes méthodologiques fondamentaux : premièrement, malgré les affirmations contraires, les prosélytes de cette approche n'ont pas relevé le défi de l'extrapolation des résultats expérimentaux aux interventions politiques ; deuxièmement, même si le recours aux ECR est présenté comme une approche scientifique objective de l’élaboration de politiques, il intègre systématiquement les préjugés idéologiques et autres des chercheurs impliqués. Ces affirmations sont illustrées par trois séries d'exemples : des études controversées qui impliquaient de couper l'eau aux ménages pauvres de Nairobi, et de randomiser l'exposition à un programme missionnaire chrétien aux Philippines ; des études sur l’absentéisme des fonctionnaires au Kenya et en Inde citées dans le prix Nobel 2019 ; et deux études de cas de l'Afrique du Sud concernant des interventions sur le marché du travail et l'éducation dans lesquelles les ECR ont faussé le processus politique, avec des conséquences négatives. La conclusion est que l’approche ECR est une voie sans issue pour le développement de l’Afrique.
Mots-clés
Télécharger la référence bibliographique
Endnote/Zotero/Mendeley (RIS)BibTeX
- Abramowicz, M., and Szafarz, A., 2020, ‘Ethics of Randomized Controlled Trials: Should Economists Care about Equipoise?’, in Bédécarrats, F., Guérin, I. and Roubaud, F., eds, Randomized Control Trials in the Field of Development: A Critical Perspective, Oxford: Oxford University Press.
- Angrist, J.D., and Pischke, J-S., 2009, Mostly Harmless Econometrics, Princeton: Princeton University Press.
- Angrist, J.D., and Pischke, J-S., 2010, ‘The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics’, Journal of Economic Perspectives, Vol. 24, No. 2, pp. 3–30.
- Baele, S.J, 2013, ‘The Ethics of New Development Economics: Is the Experimental Approach to Development Economics Morally Wrong?’, The Journal of Philosophical Economics, Vol. VII, No. 1, pp. 1–42.
- Banerjee, A., 2007, Making Aid Work, Cambridge (MA): MIT Press.
- Banerjee, A. and Duflo, E., 2006, ‘Addressing Absence’, Journal of Economic Perspectives, Vol. 20, No. 1, pp. 117–32.
- Banerjee, A., and Duflo, E., 2009, ‘The Experimental Approach to Development Economics’, Annual Review of Economics, Vol. 1, pp. 151–78.
- Banerjee, A., and Duflo, E., 2011, Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, New York: Public Affairs.
- Banerjee, A., Duflo E., and Glennerster, R., 2008, ‘Putting a Band Aid on a Corpse: Incentives for Nurses in the Indian Public Health Care System’, Journal of the European Economic Association, Vol. 6, No. 2–3, pp. 487–500.
- Bardhan, P., 2013, ‘Little, Big: Two Ideas about Fighting Global Poverty’, Boston Review, May.
- Bédécarrats, F., Guérin I., and Roubaud, F., 2019, ‘All that glitters is not gold the political economy of randomised evaluations in development’, Development and Change, Vol. 50, No. 3, pp. 735–762.
- Bold, T., Kimenyi, M., Mwabu, G., Ng’ang’a, A., and Sandefur, J., 2018, ‘Experimental Evidence on Scaling up Education Reforms in Kenya’, Journal of Public Economics, Vol. 168, December: pp. 1–20.
- Bryan, G., Choi, J., and Karlan, D., 2018, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, w24278, Cambridge, MA: National Bureau of Economic Research.
- Bryan, G., Choi, J., and Karlan, D., 2021, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, The Quarterly Journal of Economics, Vol. 136, No. 1, pp. 293–380.
- Burns, J., Edwards, L. and Pauw, K., 2010, ‘Revisiting wage subsidies: How pro- poor is a South African wage subsidy likely to be?’, Development Southern Africa, Vol. 30, No. 2, pp. 186–210.
- Cartwright, N., 2007, ‘Are RCTs the Gold Standard?’, BioSocieties, Vol. 2, No. 1, pp. 11–20.
- Center for International Development, 2008, CID South Africa Growth Initiative. http://www.cid.harvard.edu/southafrica/. Accessed 7 July 2017.
- Concato, J., Shah, N., and Horwitz, R., 2000, ‘Randomized Controlled Trials, Observational Studies and the Hierarchy of Research Designs’, New England Journal of Medicine, Vol. 342, No. 25, pp. 1887–92.
- Cook, T.D., and Campbell, D.T., 1979, Quasi-Experimentation: Design and Analysis Issues for Field Settings, New York: Houghton Mifflin.
- Coville, A., Galiani, S., and Gertler, P., 2020, ‘A Comment on the Ethical Issues Twitter Discussion’ in “Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums” (Version 2), 8 August 2020 (https://drive.google.com/file/d/1nVGR4qlhWt2EcNiy6d02By3e4kptVSer/view). Accessed 22 November 2021.
- Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2020, ‘Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums’, w27569, Cambridge, MA: National Bureau of Economic Research.
- Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2021, ‘Financing Municipal Water and Sanitation Services in Nairobi’s Informal Settlements’, 9725, Washington, DC: World Bank.
- Deaton, A., 2010, ‘Instruments Randomization and Learning about Development’, Journal of Economic Literature, Vol. 48, No. 2, pp. 424–55.
- Deaton, A., and Cartwright, N., 2018, ‘Understanding and Misunderstanding Randomized Controlled Trials’, Social Science & Medicine, Vol. 210, pp. 2–21.
- Duflo, E., Dupas, P., and Kremer, M., 2015, ‘School Governance, Teacher Incentives and Pupil-Teacher Ratios: Experimental Evidence from Kenyan Primary Schools’, Journal of Political Economy, Vol. 123, pp. 92–110.
- Duflo, E., Hanna, R., and Ryan, S.P., 2012, ‘Incentives Work Getting Teachers to Come to School’, American Economic Review, Vol. 102, No. 4, pp. 1241–78.
- Fleisch, B., 2008, Primary Education in Crisis: Why South African Schoolchildren Underachieve in Reading and Mathematics, Cape Town: UCT Press.
- Harrison, G.W., 2013, ‘Field Experiments and Methodological Intolerance’, Journal of Economic Methodology, Vol. 20, No. 2, pp. 103–17.
- Hoffmann, N., 2020, ‘Involuntary Experiments in Former Colonies: The Case for a Moratorium’, World Development, Vol. 127: 104805.
- Jedwab, R., Meier zu Selhausen, F., and Moradi, A., 2022, ‘The Economics of Missionary Expansion: Evidence from Africa and Implications for Development’, Journal of Economic Growth, Vol. 27, No. 2, pp. 149–92.
- Levinsohn, J., 2008, ‘Two Policies to Alleviate Unemployment in South Africa’, Center for International Development Working Paper 166.
- Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014a, ‘Wage Subsidies and Youth Employment in South Africa: Evidence from a Randomised Control Trial’, Stellenbosch Economic Working Papers 02/14.
- Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014b, A Youth Wage Subsidy Experiment for South Africa, 3ie Impact Evaluation Report 15 (http://www.3ieimpact.org/media/filer_public/2014/08/20/ie15-rankin-youth_wage_ subsidy.pdf).
- Manski, C.F., 2011, ‘Policy Analysis with Incredible Certitude’, Economic Journal, Vol. 121, No. 554, pp. F261–89.
- Muller, S.M., 2014, ‘Randomised Trials for Policy: A Review of the External Validity of Treatment Effects’, PhD dissertation, University of Cape Town.
- Muller, S.M., 2015, ‘Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations’, The World Bank Economic Review, Vol. 29, Suppl 1, pp. S217–25.
- Muller, S.M., 2017, ‘What Does an (South) African Economics Look Like?’, Proceedings of the 2017 Scholarship of Teaching and Learning (SoTL) in the South Conference, Johannesburg, 24–27 July, 2017, pp. 87–97.
- Muller, S.M., 2020, ‘The Implications of a Fundamental Contradiction in Advocating Randomized Trials for Policy’, World Development, Vol. 127, 104831.
- Muller, S.M., 2021a, ‘Evidence for a YETI? A Cautionary Tale from South Africa’s Youth Employment Tax Incentive’, Development and Change, Vol. 52, No. 6, pp. 1301–1342.
- Muller, S.M., 2021b, ‘Randomised Trials in Economics’, in Kincaid, H and Ross, D., eds, The Modern Guide to Philosophy of Economics, New York: Edward Elgar Publishing.
- Muller, S.M., 2022, ‘The Unacknowledged Normative Content of Randomised Control Trials in Economics and Its Dangers’, in Badei, S. and Grivaux, A., eds, 2022, The Positive and the Normative in Economic Thought, New York: Routledge.
- Muller, S.M., 2023, ‘Is economics credible? A critical appraisal of three examples from microeconomics’, Journal of Economic Methodology,Vol. 30, No.2, 157-175, DOI: 10.1080/1350178X.2023.2202682
- Nobel Media, 2019, Advanced Information.
- Rankin, N., 2012, ‘A Common Approach’, Financial Mail. http://www.financialmail.co.za/fm/2012/05/24/a-common-approach.
- Rankin, N., 2013, ‘Give Youth Wage Subsidy a Chance, Mail&Guardian. http://mg.co.za/article/2013-11-01-00-give-youth-wage-subsidy-a-chance.
- Ravallion, M., 2009, ‘Should the Randomistas Rule?’ Economists’ Voice, Vol. 6, No. 2, pp. 1–5.
- Ravallion, M., 2018, ‘Should the Randomistas (Continue to) Rule?’, Center for Global Development Working Paper 492. https://www.cgdev.org/publication/should-randomistas-continue-rule
- Robbins, L., 1932, An Essay on the Nature and Significance of Economic Science, London: Macmillan and Co.
- Rodney, W., 1972, How Europe Underdeveloped Africa, London: Bogle-L’Ouverture Publications.
- Rothwell, P.M., 2005, ‘Subgroup Analysis in Randomised Controlled Trials: Importance, Indications and Interpretation’, The Lancet, Vol. 365, pp. 176–86.
- Taylor, S., and Watson, P., 2015, The Impact of Study Guides on “Matric” Performance: Evidence from a Randomised Experiment, Stellenbosch Economic Working Papers.
- Unknown, 2011, POLICY INFLUENCE PLAN – Grant Number OW2.044 – University of the Witwatersrand. 3ie International Initiative for Impact Evaluation.
- Van der Berg, S., 2007, ‘Apartheid’s Enduring Legacy: Inequalities in Education’, Journal of African Economies, Vol. 16, No. 5, pp. 849–80.
- Van der Berg, S., Spaull, N., Wills, G., Gustafsson, M., and Kotzé, J., 2016, Identifying Binding Constraints in Education, Stellenbosch: RESEP.
- Veblen, T., 1898, ‘Why Is Economics Not an Evolutionary Science?’, The Quarterly Journal of Economics, Vol. 12, No. 4, pp. 373–397.
- Ziliak, S.T., and Teather-Posadas, E.R., 2016, ‘The Unprincipled Randomization Principle in Economics and Medicine’, in DeMartino, G. F. and McCloskey, D., eds, The Oxford Handbook of Professional Economic Ethics, Oxford: Oxford University Press.
Les références
Abramowicz, M., and Szafarz, A., 2020, ‘Ethics of Randomized Controlled Trials: Should Economists Care about Equipoise?’, in Bédécarrats, F., Guérin, I. and Roubaud, F., eds, Randomized Control Trials in the Field of Development: A Critical Perspective, Oxford: Oxford University Press.
Angrist, J.D., and Pischke, J-S., 2009, Mostly Harmless Econometrics, Princeton: Princeton University Press.
Angrist, J.D., and Pischke, J-S., 2010, ‘The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics’, Journal of Economic Perspectives, Vol. 24, No. 2, pp. 3–30.
Baele, S.J, 2013, ‘The Ethics of New Development Economics: Is the Experimental Approach to Development Economics Morally Wrong?’, The Journal of Philosophical Economics, Vol. VII, No. 1, pp. 1–42.
Banerjee, A., 2007, Making Aid Work, Cambridge (MA): MIT Press.
Banerjee, A. and Duflo, E., 2006, ‘Addressing Absence’, Journal of Economic Perspectives, Vol. 20, No. 1, pp. 117–32.
Banerjee, A., and Duflo, E., 2009, ‘The Experimental Approach to Development Economics’, Annual Review of Economics, Vol. 1, pp. 151–78.
Banerjee, A., and Duflo, E., 2011, Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, New York: Public Affairs.
Banerjee, A., Duflo E., and Glennerster, R., 2008, ‘Putting a Band Aid on a Corpse: Incentives for Nurses in the Indian Public Health Care System’, Journal of the European Economic Association, Vol. 6, No. 2–3, pp. 487–500.
Bardhan, P., 2013, ‘Little, Big: Two Ideas about Fighting Global Poverty’, Boston Review, May.
Bédécarrats, F., Guérin I., and Roubaud, F., 2019, ‘All that glitters is not gold the political economy of randomised evaluations in development’, Development and Change, Vol. 50, No. 3, pp. 735–762.
Bold, T., Kimenyi, M., Mwabu, G., Ng’ang’a, A., and Sandefur, J., 2018, ‘Experimental Evidence on Scaling up Education Reforms in Kenya’, Journal of Public Economics, Vol. 168, December: pp. 1–20.
Bryan, G., Choi, J., and Karlan, D., 2018, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, w24278, Cambridge, MA: National Bureau of Economic Research.
Bryan, G., Choi, J., and Karlan, D., 2021, ‘Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes’, The Quarterly Journal of Economics, Vol. 136, No. 1, pp. 293–380.
Burns, J., Edwards, L. and Pauw, K., 2010, ‘Revisiting wage subsidies: How pro- poor is a South African wage subsidy likely to be?’, Development Southern Africa, Vol. 30, No. 2, pp. 186–210.
Cartwright, N., 2007, ‘Are RCTs the Gold Standard?’, BioSocieties, Vol. 2, No. 1, pp. 11–20.
Center for International Development, 2008, CID South Africa Growth Initiative. http://www.cid.harvard.edu/southafrica/. Accessed 7 July 2017.
Concato, J., Shah, N., and Horwitz, R., 2000, ‘Randomized Controlled Trials, Observational Studies and the Hierarchy of Research Designs’, New England Journal of Medicine, Vol. 342, No. 25, pp. 1887–92.
Cook, T.D., and Campbell, D.T., 1979, Quasi-Experimentation: Design and Analysis Issues for Field Settings, New York: Houghton Mifflin.
Coville, A., Galiani, S., and Gertler, P., 2020, ‘A Comment on the Ethical Issues Twitter Discussion’ in “Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums” (Version 2), 8 August 2020 (https://drive.google.com/file/d/1nVGR4qlhWt2EcNiy6d02By3e4kptVSer/view). Accessed 22 November 2021.
Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2020, ‘Enforcing Payment for Water and Sanitation Services in Nairobi’s Slums’, w27569, Cambridge, MA: National Bureau of Economic Research.
Coville, A., Galiani, S., Gertler, P., and Yoshida, S., 2021, ‘Financing Municipal Water and Sanitation Services in Nairobi’s Informal Settlements’, 9725, Washington, DC: World Bank.
Deaton, A., 2010, ‘Instruments Randomization and Learning about Development’, Journal of Economic Literature, Vol. 48, No. 2, pp. 424–55.
Deaton, A., and Cartwright, N., 2018, ‘Understanding and Misunderstanding Randomized Controlled Trials’, Social Science & Medicine, Vol. 210, pp. 2–21.
Duflo, E., Dupas, P., and Kremer, M., 2015, ‘School Governance, Teacher Incentives and Pupil-Teacher Ratios: Experimental Evidence from Kenyan Primary Schools’, Journal of Political Economy, Vol. 123, pp. 92–110.
Duflo, E., Hanna, R., and Ryan, S.P., 2012, ‘Incentives Work Getting Teachers to Come to School’, American Economic Review, Vol. 102, No. 4, pp. 1241–78.
Fleisch, B., 2008, Primary Education in Crisis: Why South African Schoolchildren Underachieve in Reading and Mathematics, Cape Town: UCT Press.
Harrison, G.W., 2013, ‘Field Experiments and Methodological Intolerance’, Journal of Economic Methodology, Vol. 20, No. 2, pp. 103–17.
Hoffmann, N., 2020, ‘Involuntary Experiments in Former Colonies: The Case for a Moratorium’, World Development, Vol. 127: 104805.
Jedwab, R., Meier zu Selhausen, F., and Moradi, A., 2022, ‘The Economics of Missionary Expansion: Evidence from Africa and Implications for Development’, Journal of Economic Growth, Vol. 27, No. 2, pp. 149–92.
Levinsohn, J., 2008, ‘Two Policies to Alleviate Unemployment in South Africa’, Center for International Development Working Paper 166.
Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014a, ‘Wage Subsidies and Youth Employment in South Africa: Evidence from a Randomised Control Trial’, Stellenbosch Economic Working Papers 02/14.
Levinsohn, J., Rankin, N., Roberts, G., and Schöer, V., 2014b, A Youth Wage Subsidy Experiment for South Africa, 3ie Impact Evaluation Report 15 (http://www.3ieimpact.org/media/filer_public/2014/08/20/ie15-rankin-youth_wage_ subsidy.pdf).
Manski, C.F., 2011, ‘Policy Analysis with Incredible Certitude’, Economic Journal, Vol. 121, No. 554, pp. F261–89.
Muller, S.M., 2014, ‘Randomised Trials for Policy: A Review of the External Validity of Treatment Effects’, PhD dissertation, University of Cape Town.
Muller, S.M., 2015, ‘Causal Interaction and External Validity: Obstacles to the Policy Relevance of Randomized Evaluations’, The World Bank Economic Review, Vol. 29, Suppl 1, pp. S217–25.
Muller, S.M., 2017, ‘What Does an (South) African Economics Look Like?’, Proceedings of the 2017 Scholarship of Teaching and Learning (SoTL) in the South Conference, Johannesburg, 24–27 July, 2017, pp. 87–97.
Muller, S.M., 2020, ‘The Implications of a Fundamental Contradiction in Advocating Randomized Trials for Policy’, World Development, Vol. 127, 104831.
Muller, S.M., 2021a, ‘Evidence for a YETI? A Cautionary Tale from South Africa’s Youth Employment Tax Incentive’, Development and Change, Vol. 52, No. 6, pp. 1301–1342.
Muller, S.M., 2021b, ‘Randomised Trials in Economics’, in Kincaid, H and Ross, D., eds, The Modern Guide to Philosophy of Economics, New York: Edward Elgar Publishing.
Muller, S.M., 2022, ‘The Unacknowledged Normative Content of Randomised Control Trials in Economics and Its Dangers’, in Badei, S. and Grivaux, A., eds, 2022, The Positive and the Normative in Economic Thought, New York: Routledge.
Muller, S.M., 2023, ‘Is economics credible? A critical appraisal of three examples from microeconomics’, Journal of Economic Methodology,Vol. 30, No.2, 157-175, DOI: 10.1080/1350178X.2023.2202682
Nobel Media, 2019, Advanced Information.
Rankin, N., 2012, ‘A Common Approach’, Financial Mail. http://www.financialmail.co.za/fm/2012/05/24/a-common-approach.
Rankin, N., 2013, ‘Give Youth Wage Subsidy a Chance, Mail&Guardian. http://mg.co.za/article/2013-11-01-00-give-youth-wage-subsidy-a-chance.
Ravallion, M., 2009, ‘Should the Randomistas Rule?’ Economists’ Voice, Vol. 6, No. 2, pp. 1–5.
Ravallion, M., 2018, ‘Should the Randomistas (Continue to) Rule?’, Center for Global Development Working Paper 492. https://www.cgdev.org/publication/should-randomistas-continue-rule
Robbins, L., 1932, An Essay on the Nature and Significance of Economic Science, London: Macmillan and Co.
Rodney, W., 1972, How Europe Underdeveloped Africa, London: Bogle-L’Ouverture Publications.
Rothwell, P.M., 2005, ‘Subgroup Analysis in Randomised Controlled Trials: Importance, Indications and Interpretation’, The Lancet, Vol. 365, pp. 176–86.
Taylor, S., and Watson, P., 2015, The Impact of Study Guides on “Matric” Performance: Evidence from a Randomised Experiment, Stellenbosch Economic Working Papers.
Unknown, 2011, POLICY INFLUENCE PLAN – Grant Number OW2.044 – University of the Witwatersrand. 3ie International Initiative for Impact Evaluation.
Van der Berg, S., 2007, ‘Apartheid’s Enduring Legacy: Inequalities in Education’, Journal of African Economies, Vol. 16, No. 5, pp. 849–80.
Van der Berg, S., Spaull, N., Wills, G., Gustafsson, M., and Kotzé, J., 2016, Identifying Binding Constraints in Education, Stellenbosch: RESEP.
Veblen, T., 1898, ‘Why Is Economics Not an Evolutionary Science?’, The Quarterly Journal of Economics, Vol. 12, No. 4, pp. 373–397.
Ziliak, S.T., and Teather-Posadas, E.R., 2016, ‘The Unprincipled Randomization Principle in Economics and Medicine’, in DeMartino, G. F. and McCloskey, D., eds, The Oxford Handbook of Professional Economic Ethics, Oxford: Oxford University Press.