|Generate QR code
Open Access
Subscription Access
“Randomistas”: A new development economics
https://doi.org/10.32609/0042-8736-2023-6-5-35
Abstract
The paper explores a curious phenomenon in the modern economic science — the intellectual confrontation between proponents of new development economics (“randomistas”) and its opponents. The general message of the new approach is that the technique of randomized controlled trials/experiments (RCT) must be considered the only truly scientific method, and only this technique should be used in studying the problems of developing countries. RCTs have been recognized as the “gold standard” in evaluating the effectiveness of anti-poverty programs; today this approach absolutely dominates in development economics. The paper discusses the main features of the RCT economics: an inclination to imitate biomedical research; atheoretical character; the idea of a hierarchy of methods; reorientation from large-scale macroeconomic and institutional reforms to targeted social and humanitarian interventions. The problems of internal and external validity of RCT are highlighted. A general conclusion is that conceptually counter-arguments of critics look more convincing: the idea of a hierarchy of methods is unscientific; no methodological “gold standard” exists in economic analysis; estimates obtained on the basis of RCT cannot be considered unbiased; RCTs are almost powerless in the face of the problem of external validity; policy recommendations derived from RCTs are of very limited practical value. However, the logic of critics has been trumped by the rhetoric of randomistas: most likely, randomized experiments will escalate, and their intellectual and political influence will grow.
About the Author
R. I. Kapeliushnikov
Primakov National Research Institute of World Economy and International Relations, RAS;
HSE University
Russian Federation
Russian Federation
Rostislav I. Kapeliushnikov
Moscow
References
1. Banerjee A., Duflo E. (2021). Poor economics. A radical rethinking of the way to fight global poverty. Moscow: Gaidar Institute Publ. (In Russian).
2. Enikolopov R. S. (2020). Evidence based development economics: Nobel Prize in Economic Sciences 2019. Voprosy Ekonomiki, No. 1, pp. 5—17. (In Russian). https://doi.org/10.32609/0042-8736-2020-1-5-17
3. Kapeliushnikov R. I. (2021). On the current state of economic science: Semi-sociological observations. In: Kapeliushnikov R. I. Economic essays. History of ideas, methodology, inequality, and growth. Moscow: HSE Publ., pp. 298—322. (In Russian).
4. Abramowicz M., Szafarz A. (2020). Ethics of RCTs: Should economists care about equipoise? In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 280—292. https://doi.org/10.1093/oso/9780198865360.003.0012
5. Aiken A. M., Davey C., Hargreaves J. R., Hayes R. J. (2015). Re-analysis of health and educational impacts of a school-based deworming programme in Western Kenya: A pure replication. International Journal of Epidemiology, Vol. 44, No. 5, pp. 1572—1580. https://doi.org/10.1093/ije/dyv127
6. Angrist J. D., Pischke J.-S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, Vol. 24, No. 2, pp. 3—30. https://doi.org/10.1257/jep.24.2.3
7. Banerjee A. V. (2005). “New development economics” and the challenge to theory.Economic and Political Weekly, Vol. 40, No. 40, pp. 4340—4344.
8. Banerjee A. V. (2006). Making aid work. Boston Review, Vol. 31, No. 4, pp. 7—9.
9. Banerjee A. V. (2009). Big answers for big questions: The presumption of growth policy.In: J. Cohen, W. Easterly (eds.). What works in development?: Thinking big and thinking small. Washington, DC: Brookings Institution Press, pp. 207—221.
10. Banerjee A. V. (2020). Field experiments and the practice of economics. American Economic Review, Vol. 110, No. 7, pp. 1937—1951. https://doi.org/10.1257/aer.110.7.1937
11. Banerjee A., Duflo E., Glennerster R. (2008). Putting a band-aid on a corpse: Incentives for nurses in the Indian public health care system. Journal of the European Economic Association, Vol. 6, No. 2—3, pp. 487—500. https://doi.org/10.1162/JEEA.2008.6.2-3.487
12. Banerjee A., Duflo E. (2019). Good economics for hard times: Better answers to our biggest problems. New York: Public Affairs.
13. Banerjee A., Duflo E. (2020). How poverty ends. The many paths to progress — and why they might not continue. Foreign Affairs, Vol. 99, No. 1, pp. 22—29.
14. Bédécarrats F., Guérin I., Roubaud F. (2019). All that glitters is not gold. The political economy of randomized evaluations in development. Development and Change, Vol. 50, No. 3, pp. 735—762. https://doi.org/10.1111/dech.12378
15. Bédécarrats F., Guérin I., Roubaud F. (2020). Editors introduction. Controversies around RCTs. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 1—28. https://doi.org/10.1093/oso/9780198865360.003.0001
16. Bernard T., Delarue J., Naudet J.-D. (2012). Impact evaluations: A tool for accountability? Lessons from experience at Agence Française de Développement. Journal of Development Effectiveness, Vol. 4, No. 2, pp. 314—327. https://doi.org/10.1080/19439342.2012.686047
17. Bold T., Kimenyi M., Mwabu G., Ng’ang’a A., Sandefur J. (2013). Scaling up what works: Experimental evidence on external validity in Kenyan education. CGD Working Paper, No. 321. Washington, DC: Center for Global Development.
18. Broadbent A., Vandenbroucke J. P., Pearce N. (2017). Formalism or pluralism? A reply to commentaries on “Causality and causal inference in epidemiology”. International Journal of Epidemiology, Vol. 45, No. 6, pp. 1844—1861. https://doi.org/10.1093/ije/dyw298
19. Cohen J., Dupas P. (2010). Free distribution or cost-sharing? Evidence from a randomized malaria prevention experiment. Quarterly Journal of Economics, Vol. 125, No. 1, pp. 1—45. https://doi.org/10.1162/qjec.2010.125.1.1
20. Cohen J., Easterly W. (2010). What works in development?: Thinking big and thinking small. Washington, DC: Brookings Institution Press.
21. Cook T. (2018). Twenty-six assumptions that have to be met if single random assignment experiments are to warrant “gold standard” status: A commentary on Deaton and Cartwright. Social Science and Medicine, Vol. 210, pp. 37—40. https://doi.org/10.1016/j.socscimed.2018.04.031
22. Davey C., Aiken A. M., Hayes R. J., Hargreaves J. R. (2015). Re-analysis of health and educational impacts of a school-based deworming programme in Western Kenya: A statistical replication of a cluster quasi-randomized stepped-wedge trial. International Journal of Epidemiology, Vol. 44, No. 5, pp. 1581—1592. https://doi.org/10.1093/ije/dyv128
23. de Cler A.-P. (2021). The rise and fall of a gold standard: The case of randomized controlled trials within the experimentation for Youth Fund. Master’s Thesis. Paris: SciencesPo, School of Public Affairs.
24. Deaton A. (2006). Evidence-based aid must not become the latest in a long string of development fads. Boston Review, Vol. 31, No. 4, pp. 13—14.
25. Deaton A. (2010). Instruments, randomization, and learning about development. Journal of Economic Literature, Vol. 48, No. 2, pp. 424—455. https://doi.org/10.1257/jel.48.2.424
26. Deaton A. (2020). Randomization in the tropics revisited: A theme and eleven variations. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 29—46. https://doi.org/10.1093/oso/9780198865360.003.0002
27. Deaton A., Cartwright N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science and Medicine, Vol. 210, pp. 2—21. https:// doi.org/10.1016/j.socscimed.2017.12.005
28. Duflo E. (2005). Evaluating the impact of development aid program: The role of randomized evaluations. In: Development aid: Why and how? Towards strategies for effectiveness. Paris: AFD-EUND. Notes and Documents, No. 22, pp. 207—245.
29. Duflo E. (2017). The economist as plumber. American Economic Review, Vol. 107, No. 5, pp. 1—26. https://doi.org/10.1257/aer.p20171153
30. Duflo E., Dupas P., Kremer M. (2015). School governance, teacher incentives, and pupil—teacher ratios: Experimental evidence from Kenyan primary schools. Journal of Public Economics, Vol. 123, pp. 92—110. https://doi.org/10.1016/j.jpubeco.2014.11.008
31. Duflo E., Hanna R., Ryan S. P. (2012). Incentives work: Getting teachers to come to school. American Economic Review, Vol. 102, No. 4, pp. 1241—1278. https://doi.org/10.1257/aer.102.4.1241
32. Duflo E., Kremer M. (2008). Use of randomization in the evaluation development effectiveness. In: W. R. Easterly (ed.). Reinventing foreign aid. Cambridge, MA: The MIT Press, pp. 93—120.
33. Dupas P. (2014). Short-run subsidies and long-run adoption of new health products: Evidence from a field experiment. Econometrica, Vol. 82, No. 1, pp. 197—228. https://doi.org/10.3982/ECTA9508
34. Easterly W. (2009). Development experiments: Ethical? Feasible? Useful? Development Research Institute, July 15. https://www.nyudri.org/aidwatcharchive/2009/07/development-experiments-ethical-feasible-useful
35. Favereau J. (2016). On the analogy between field experiments in economics and clinical trials in medicine. Journal of Economic Methodology, Vol. 23, No. 2, pp. 203—222. https://doi.org/10.1080/1350178X.2016.1157202
36. Favereau J., Nagatsu M. (2020). Holding back from theory: Limits and methodological alternatives of randomized field experiments in development economics. Journal of Economic Methodology, Vol. 27, No. 3, pp. 191—211. https://doi.org/10.1080/1350178X.2020.1717585
37. Heckman J. J. (2020). Epilogue: Randomization and social policy evaluation revisited. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 304—330. https://doi.org/10.1093/oso/9780198865360.003.0014
38. Imbens G. (2010). Better LATE than nothing: Some comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic Literature, Vol. 48, No. 2, pp. 399—423. https://doi.org/10.1257/jel.48.2.399
39. Ioannidis J. (2018). Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright. Social Science Medicine, Vol. 210, pp. 53—56. https://doi.org/10.1016/j.socscimed.2018.04.029
40. Jerven M. (2015). Africa: Why economists get it wrong (African arguments). London: ZedBooks.
41. Kenny C. P., Pritchett L. (2013). Promoting millennium development ideals: The risks of defining development down. Center for Global Development Working Paper, No. 338. https://doi.org/10.2139/ssrn.2364643
42. Labrousse A. (2020). The rhetorical superiority of poor economics. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 227—255. https://doi.org/10.1093/oso/9780198865360.003.0010
43. Lancet (2004). The World Bank is finally embracing science. The Lancet, Vol. 364, pp. 731—732. https://doi.org/10.1016/S0140-6736(04)16945-6
44. Leigh A. (2018). Randomistas: How radical researchers are changing our world. New Haven and London: Yale University Press. https://doi.org/10.12987/9780300240115
45. Miguel E., Kremer M. (2004). Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica, Vol. 72, No. 1, pp. 159—217. https://doi.org/10.1111/j.1468-0262.2004.00481.x
46. Ogden T. N. (2020). RCTs in development economics, their critics, and their evolution. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 126—151. https://doi.org/10.1093/oso/9780198865360.003.0006
47. Picciotto R. (2020). Are the “randomistas” evaluators? In: F. Bédécarrats, I. Guérin, Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 256—279. https://doi.org/10.1093/oso/9780198865360.003.0011
48. Pritchett L. (2014). An homage to the randomistas on the occasion of the J-PAL 10th anniversary: Development as a faith-based activity. Center for Global Development, March 10. https://www.cgdev.org/blog/homage-randomistas-occasion-j-pal-10th-anniversary-development-faith-based-activity
49. Pritchett L. (2020). Randomizing development: Method or madness? In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 79—107. https://doi.org/10.1093/oso/9780198865360.003.0004
50. Pritchett L., Sandefur J. (2013). Context matters for size: Why external validity claims and development practice don’t mix. Center for Global Development Working Paper, No. 336. https://doi.org/10.2139/ssrn.2364580
51. Ravallion M. (2020). Should the randomistas (continue to) rule. In: F. Bédécarrats, I. Guérin, F. Roubaud (eds.). Randomized control trials in the field of development: A critical perspective. Oxford: Oxford University Press, pp. 47—78. https://doi.org/10.3386/w27554
52. Rodrik D. (2009). The new development economics: We shall experiment, but how shall we learn? In: J. Cohen, W. Easterly (eds.). What works in development?: Thinking big and thinking small. Washington, DC: Brookings Institution, pp. 24—47.
53. Royal Swedish Academy of Sciences (2019). Press release: The Prize in Economic Sciences 2019. https://www.nobelprize.org/prizes/economic-sciences/2019/press-release/
54. Vass M. (2010). Prevention of functional decline in older people: The Danish randomised intervention trial on preventative home visits. Doctoral Dissertation. Copenhagen: University of Copenhagen, Faculty of Health Science.
55. Vivalt E. (2020). How much can we generalize from impact evaluations? Journal of the European Economic Association, Vol. 18, No. 4, pp. 3045—3089. https://doi.org/ 10.1093/jeea/jvaa019
56. Ziliak S. T., Teather-Posadas E. R. (2014). The unprincipled randomization principle in economics and medicine. Available at SSRN: https://ssrn.com/abstract=2396806
Supplementary files
Review
For citations:
Kapeliushnikov R.I. “Randomistas”: A new development economics. Voprosy Ekonomiki. 2023;(6):5-35. (In Russ.) https://doi.org/10.32609/0042-8736-2023-6-5-35
Views:
1012
Email this article 
