Ten years ago, Princeton University economist Angus Deaton used his Keynes Lecture to the British Academy to sound a note of caution about the new New Thing in development economics: the randomized controlled trial (RCT), or field experiment. He recounted the general frustration with the failure of traditional econometric methods to swiftly unlock the secrets of economic development (and with the inability of development agencies to learn more from their own experience). He surveyed the rising enthusiasm for RCTs as an alternative path to reliable knowledge without the traditional fuss.
He argued against the trend. “[E]xperiments have no special ability to produce more credible knowledge than other methods, and… actual experiments are frequently subject to practical problems that undermine any claims to statistical or epistemic superiority.” Citing a maxim of philosopher Nancy Cartwright, Deaton wrote,
Randomization is not a gold standard because “there is no gold standard,” Randomized controlled trials cannot automatically trump other evidence, they do not occupy any special place in some hierarchy of evidence, nor does it make sense to refer to them as “hard” while other methods are “soft”. These rhetorical devices are just that; a metaphor is not an argument.
More positively, Deaton continued,
I shall argue that the analysis of projects needs to be refocused towards the investigation of potentially generalizable mechanisms that explain why and in what contexts projects can be expected to work. The best of the experimental work in development economics already does so, because its practitioners are too talented to be bound by their own methodological prescriptions. Yet there would be much to be said for doing so more openly. I concur with the general message in [Ray] Pawson and [Nick] Tilley…, who argue that thirty years of project evaluation in sociology, education and criminology was largely unsuccessful because it focused on whether projects work instead of on why they work.
Nevertheless, the RCT movement continued to attract adherents, and researchers attracted more and for funding from the World Bank and like-minded foundations with an interest in ameliorating global poverty. Poor Economics A Radical Rethinking of the Way to Fight Global Poverty (Public Affairs, 2012), by Abhijit Banerjee and Esther Duflo, both of the Massachusetts Institute of Technology, imparted an impetus to the movement. Duflo’s 2017 Ely Lecture to the annual meetingof the American Economic Association, The Economist as Plumber, especially attracted interest. As economists seek to help governments design new policies and regulations, she argued,
[T]hey take on an added responsibility to engage with the details of policy making and, in doing so, to adopt the mindset of a plumber. Plumbers try to predict as well as possible what may work in the real world, mindful that tinkering and adjusting will be necessary since our models gives us very little theoretical guidance on what (and how) details will matter. Economists should seriously engage with plumbing, in the interest of both society and our discipline.
In 2015, awarding the Nobel Prize for economics, the Royal Swedish Academy of Sciences cited Deaton “for his analysis of consumption, poverty, and welfare.” Last month, they cited Banerjee, Duflo, and Michael Kremer, of Harvard University, “for their experimental approach to alleviating global poverty.”
Last week Deaton was back, with “Randomization in the tropics revisited: a theme and eleven variations,” a chapter prepared for Randomized Controlled Trials in the Field of Development: a Critical Perspective (Oxford, forthcoming), by Florent Bédécarrats, Isabelle Guérin and François Roubaud, Economic Principals tumbled to Deaton’s essay too late to do much more than read it and recommend it here. Many EP readers will wish to read it for themselves. Deaton is a remarkably clear and forceful writer.
This time the tenor was a little more personal.
Jean Drèze has provided an excellent discussion of the issues of going from evidence for policy. One of his examples is the provision of eggs to schoolchildren in India, a country where many children are inadequately nourished. An RCT could be used to establish that children provided with eggs come to school more often, learn more, and are better nourished. For many donors and RCT advocates, that would be enough to push for a “school eggs” policy. But policy depends on many other things; there is a powerful vegetarian lobby that will oppose it, there is a poultry industry that will lobby, and another group that will claim that their powdered eggs – or even their patented egg substitute – will do better still. Dealing with such questions is not the territory of the experimenters, but of politicians, and of the many others with expertise in policy administration. Social plumbing should be left to social plumbers, not experimental economists who have no special knowledge, and no legitimacy at all.
The argument – does it load the dice to call it technocrats vs. democrats? – promises to be long running, with attention soon to shift to Silicon Valley know-it-alls and the Gates Foundation. Banerjee and Duflo’s new book, Good Economics for Hard Times (Public Affairs) appears November 12. The next chapter will air December 8, when this year’s laureates are scheduled to give their lectures in Stockholm – live-streamed, for those who care to watch.
. xxx
New on the EP bookshelf:
The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution, by Gregory Zuckerman (Penguin),