The joys of cash benchmarking

when to just give people money

I.

Imagine you run a charity which gives cows to people in the developing world. Your new boyfriend is a statistician and you want to impress him, so you’ve decided to look into this Monitoring and Evaluation thing he keeps going on about. So you give a bunch of people cows and then follow up a year later, and sure enough the people you gave the cows to are richer, healthier, and happier. You proudly tell this to your boyfriend and expect to reap rewards in the form of admiration, cuddles, and a ‘yes’ to your marriage proposal.

Your boyfriend, however, isn’t impressed. What if everyone in the country is getting richer, because the country is industrializing? What if the poor people you gave cows to were having an unusually hard time, and then they got back on their feet, and your cows had nothing to do with it? Apparently, in order to get a sample that really says anything about the world, you need to randomly give cows to half the people, and then check whether the people you gave cows to are doing better than the people you didn’t give cows to (a “randomized controlled trial”).

This sounds like hard work, but your boyfriend is very handsome and charming and good at mathematics, so you find twice as many people and only give half of them cows. Sure enough, the people you gave cows to are richer, healthier, and happier. You return to your boyfriend with your findings while planning the flower arrangements for your June wedding.

And yet somehow your boyfriend is STILL not satisfied. He points out that cows cost money. You’d have to really fuck up giving people free stuff for it to not make them richer, and being richer probably makes people happier and healthier as well. What you should really do is split people in two groups, give half of them cows, and give the other half the amount of money it would have cost to give them cows.

Well, by this point you’re not really sure if your boyfriend’s beauty and charm and mathematical talent is worth all of this fuss, but you’ve already started calling the caterers, so you do another randomized controlled trial and give the control group cash.

Much to your shock, it turns out the group that got cash instead of cows is richer, healthier, and happier.

You slink off to your boyfriend and tell him the results. Fortunately, he believes that punishing researchers for getting negative results contributes to publication bias, so he’s pleased with you, he accepts your marriage proposal, and you both live happily ever after as wandering study design consultants.

The third trial you did is called “cash benchmarking” and I think it’s one of the most important concepts to understand to learn how to do good better.

II.

Your cow program isn’t particularly unusual for underperforming an equivalently sized cash transfer. In fact, many programs likely underperform cash—including programs which seem well-designed and were implemented by respected organizations.

For example, consider Huguka Dukore, a job training program which was implemented by the Education Development Center and which followed USAID’s guidance about workplace readiness and skills training in the developing world. Huguka Dukore is a year-long program in which students learn soft skills like interpersonal communication, traditional business skills like accounting, and either in-demand technical skills or entrepreneurship. Then they’re placed in an apprenticeship or internship with an experienced businessperson in their area.

Sounds great, and it is: Huguka Dukore increases the average number of hours worked per week, the value of the productive assets the household owns, and participants’ self-reported well-being. It increases savings, but also increases participants’ level of debt, for no net change in participants’ level of wealth.

However. A cash transfer of equivalent size causes a larger increase in the average number of hours worked, the value of productive assets, savings, and participants’ subjective well-being, and the gap is not small,¹ and cash transfers also increase income, household consumption, and household wealth. Indeed, the only outcome measure on which Huguka Dukore outperforms cash transfers is the test of whether you retained the information taught by Huguka Dukore.

The effects of both interventions fade out over time, so the results of cash transfers and Huguka Dukore look identical three and a half years later. However, shortly after the experiment, the covid-19 pandemic hit, which might have disrupted participants’ attempts to build long-term wealth.

Huguka Dukore is good. If you only compared it to controls, it would look awesome. But it is way worse than just giving participants the same amount of money.

To be clear, these underwhelming results aren’t about grift, overhead, or fraud. Huguka Dukore really did teach people business skills and get them apprenticeships. In our cow example, you really did give people the cows, you spent a reasonable amount of money in the process of giving them cows; and the cows really did leave people better off. It’s just that writing the recipients a check (well, in the modern era, sending the recipients a mobile money transfer) would have left them even better off.

We don’t have any idea what percentage of programs underperform cash. Cash benchmarking is a relatively new idea, and doing these studies takes a long time, and cash benchmarking studies were mostly funded by USAID which Elon Musk decided to smash for funsies. But I think it’s very reasonable to suppose that a high percentage of development aid—perhaps more than half—is worse than cash.

III.

Why do seemingly good programs often underperform cash?

You have likely had the experience of receiving bad birthday, Christmas, or Hanukkah presents, in all the many forms “bad” can take. You got makeup for your birthday because you’re female, even though you never wear makeup. You got a food processor you already own. You got a book that is secretly an unwanted piece of life advice: the Bible, He’s Just Not That Into You, The 40-Day Sugar Fast, Married Men Coming Out: The Ultimate Guide To Becoming The Man You Were Born To Be. You unwisely expressed a slight fondness for frogs a decade ago, and ever since then you’ve received nothing but frog mugs, frog shirts, frog paintings, frog sculptures, and whimsical frog tote bags.

You get bad presents for two primary reasons:

1. You know much more than other people about what you like, whether you currently own a food processor, and whether you are a Married Man who ought to Come Out and Become The Man He Was Born To Be.
2. You care a lot about what you look like, what you read, and whether every single surface in your house is covered with frog memorabilia; your friends and relatives don’t care nearly so much, because they are concerned with their own appearances, reading habits, and surfaces covered with memorabilia of sad dogs.

Both of these considerations also apply to charity.

Let’s return to our cow charity. Your recipients probably know much better than you whether they want to own a cow. They know what the pasture is like near their village, what the market for cow products looks like, whether they’re lactose-intolerant, and whether a cow kicked them in the head as a small child leaving them with crippling bovinophobia. Similarly, even though you’re of course very concerned about the well-being of the people you help, you simply don’t have the determination and motivation of a mother who has to choose the right farmed animal or her children will starve.

If you give people cows, you will probably give cows to people who have no particular use for a cow, and you’re not going to be especially motivated to filter those people out. If you give people money, they will buy what they have a use for, and they will be much more motivated to get the most out of every dollar.

IV.

If cash is so great, why don’t we shut down all other charities and just give everyone cash?

Consider the Against Malaria Foundation, the flagship effective altruist charity for global health and development. Sub-Saharan Africa isn’t suffering from some crippling insecticide-treated bednet shortage that can only be remediated by foreign donors. If you give people money, they could buy insecticide-treated bednets with it. Why give to the Against Malaria Foundation and force them to get a bednet whether they want one or not?

Well, one very consistent bias people have is undervaluing preventative health care. You’ve likely experienced this yourself. Have you ever procrastinated for months on getting your flu shot, only to come down with the flu and then lie in bed alternately vomiting and cursing yourself for your own laziness? In general, people don’t seek out preventative health care unless it is required (such as through a vaccine mandate) or made very easy for them (such as purified water coming automatically from your faucet)—even when they would very much like not to get sick. Since preventative health care happens long before you get the disease, sometimes you don’t get sick even if you don’t get preventative health care, and if preventative health care works it seems like nothing happens, you tend to think that you can skip it and you’ll be fine.²

Similarly, people in the developing world tend to undervalue insecticide-treated bed nets. In one study, a 55-cent increase in the price of a bednet decreased takeup by 23 percentage points (not 23%, 23 percentage points). It’s not that poor people in Africa don’t care whether they get malaria—they’re willing to pay more than 55 cents to treat their malaria once they already have it. But just as you irrationally avoid stopping by the pharmacy for your flu shot, many poor people in Africa irrationally don’t buy malaria nets.

In this case, my judgment is better than the Africans’ judgment about an issue affecting their own lives. You should be cautious before making a claim like that. But “people are weirdly averse to getting preventative health care” is a well-replicated finding across many different contexts, and I think it’s safe to assume that it applies to the Against Malaria Foundation’s recipients. Most, but not all, of GiveWell’s top charities beat cash benchmarking precisely because they compensate for people’s bias against preventative healthcare.³

Why else might a charity outperform cash?

Some charities have positive externalities: that is, they benefit people other than the recipient. For example, antiretroviral drugs not only keep people with HIV alive but also keep them from transmitting HIV to others. So people pursuing their individual self-interest will spend less than the socially optimal amount on antiretroviral drugs. Similar positive externalities are another reason for many of GiveWell’s top charities to beat cash benchmarking.

More often, charities do things that individuals can’t pay for. For example, charities and governments can often get discounts on medication because they buy a huge volume of medications, reliably, for years or decades. They might even be able to convince corporations to give away medicine in exchange for good publicity. The late lamented American anti-HIV program PEPFAR was incredibly cost-effective in part because the U.S. government negotiated discounts on medications.

Other things charities do aren’t market goods at all. For example, USAID used to do a lot of “health system strengthening” work: improving the collection of mortality and morbidity statistics; tracking the spread of diseases; making sure that medicines get to the right hospitals instead of one having too many antibiotics and one having none at all. Individual people can’t pay to have their own individual diseases tracked; this only makes sense as a collective project.

Similarly, the organization Human Rights Watch puts pressure on dictators and human rights abusers to treat their people somewhat better. In principle, you can pay money to keep human rights abusers from torturing you, but it’s rather expensive and out of the reach of most victims. It’s more effective (at least sometimes) to produce credible reports about what’s going on, organize protests and press conferences, and hold meetings with human rights abusers and their international allies—and these are things an individual victim’s money can’t possibly buy. Likewise, other forms of political activism have to be funded through donations.

Finally, some beneficiaries of charity can’t buy things at all. For example, if you give an egg-laying chicken a dollar bill, she will peck it in confusion and then look at you and demand corn. But egg-laying chickens are still being horrifically mistreated in factory farms and we ought to do something to help them. Similarly, future people don’t exist yet, so they can’t spend money. So we often take reckless risks with the well-being and even existence of future people: climate change, nuclear proliferation, unsafe development of AI and bioengineering.

I’m not saying that any of these charities definitely outperform cash. But they all plausibly do. You can probably think of dozens of other charities that also plausibly outperform cash. The lesson here isn’t “nothing outperforms cash, be a nihilist” or even “you have to do cash-benchmarked randomized controlled trials of all your charities or handsome statisticians won’t love you.” The lesson is that, unless you have a plausible story about why a charity outperforms just giving the beneficiaries some money, you shouldn’t do it.

V.

For most of my readers, who neither have the capacity to run cash-benchmarked randomized controlled trials nor handsome statisticians to court with them, cash benchmarking is most useful as a thought experiment.

The default thing to do with your altruistic dollars should be:

1. Find the poorest person you can.
2. Give them some money.

If you do anything with your altruistic dollars other than find the poorest person you can and give them money, you should have an explanation for why this is better than finding the poorest person you can and giving them money.

Some effective altruist grantmakers already more-or-less use this framework.

GiveWell aims to make grants that are eight times better than its benchmark. Its benchmark is the cost to double the consumption (basically, the amount of goods and services someone uses) of someone at the $2.15 poverty line. GiveWell converts the health benefits of its programs to consumption by making assumptions about how much money a year of healthy life is “worth.”⁴

Interestingly, cash transfers are actually three to four times better than GiveWell’s benchmark, because they have benefits other than increasing consumption for the individual recipient. For example, when you have more money to spend you buy things from people around you and make them richer, and you also buy health care for your kids and thus they’re less likely to die. GiveWell uses only a subset of the benefits of cash transfers as its benchmark, because it wants changes in its benchmark over time to reflect changes in GiveWell’s resources and the available opportunities, not changes in our best assessment of the benefits of cash transfers.

Coefficient Giving uses the “Coefficient Giving dollar”, which is equivalent to the benefit of giving $1 to someone making $50,000 a year (the U.S. GDP when the heuristic was adopted). Its current bar is 2,100x, which means that only funds opportunities that are 2,100 times better than giving a dollar to someone making $50,000 a year. Coefficient Giving primarily uses the Coefficient Giving dollar in its health and wellbeing work, rather than its work on progress, animals, or global catastrophic risk, which are significantly harder to put numbers on. Nevertheless, as far as I know, across grant areas Coefficient Giving’s grantmakers think about how many “Coefficient Giving dollars” programs get, even if they do it in a less rigorous and mathematical way.

Some people prefer to use other ways of comparing different donation opportunities, such as disability-adjusted life years or lives saved. But I prefer thinking about how much an opportunity improves on cash.

Partially, I prefer it because cash benchmarking makes it easier to compare very different programs. You can’t calculate lives saved for a program that doesn’t save lives, but with some simplifying assumptions you can figure out how much better any program is than cash.

But primarily I think cash benchmarking puts the default in the right place. If I am “helping” someone, and I would help them more by handing them a check equivalent to the cost of my “help,” I’m not really helping them. I might be giving myself warm fuzzies, or exercising my desire to control other people, or providing a full-employment program for development economists, but to the extent my program is worse than cash, I’m making my beneficiaries’ lives worse.

So before I donate, I try to ask myself: “is this better than cash transfers? How much better? Why is it better? How do I know it’s better?” It is clarifying. I recommend you do the same.

1. For example, people’s happiness increases twice as much if they get the cash transfer compared to Huguka Dukore.

2. The book Poor Economics has an excellent discussion of why people tend to undervalue preventative health care which goes much more in depth on the research.

3. To be clear, they all outperform cash, but for different reasons.

4. GiveWell doesn’t consider other effects of its programs outside health and wealth, because it thinks those two criteria capture the vast majority of the benefits of its programs.