Discovering new technologies is the only way to get long-term economic growth. Rote expansions of existing technologies and machines inevitably hit a ceiling: replacing and repairing the existing infrastructure and capital becomes so expensive that there is no income left over for building extra copies. The only way out of this is to come up with new technologies which create more income for the same investment, thus restarting the feedback loop between income growth and investment.
So R&D is extremely valuable. But most of the gains from R&D accrue to external parties. William Nordhaus estimates that firms recover maybe 2% of the value they create by developing new technologies. The rest of the value goes to other firms who copy their ideas and customers who get new products at lower prices. Firms don’t care much about the benefits that accrue to others so they invest much less in R&D than the rest of us would like them to.
Governments, on the other hand, collect much more of the benefits from new technologies. They get to tax the entire economy so when benefits spillover across firms and consumers, they still come out ahead. They don’t collect on international spillovers but for large economies at the frontier of technological growth, like the US, they internalize a large chunk of the value from R&D, much more than 2%.
All of this is a setup for a classic externalities problem. There’s some big benefit to society that private decision makers don’t internalize, so we should rely on governments to subsidize R&D closer to its socially optimal level.
But in fact, the private sector spends ~4x more than the public sector on R&D: $463 vs $138 billion dollars a year. One explanation for this might be that the extra $138 billion is all that was needed to bump up private spending to the social optimum, but this doesn’t seem to hold up in the data. One piece of evidence that we are still far off the socially optimal spending on R&D comes form a simple accounting of the average returns to R&D by Larry Summers and Benjamin Jones.
They model the returns to R&D spending like this: imagine stopping all R&D spending for a single year. You’d save several hundred billion dollars upfront, but there would be no economic growth,[1] so we’d miss out on a ~2% increase in per capita GDP. The upfront savings only happen once, but next year when we start R&D up again we have 2% less to invest so we grow less, and next year we’re still behind, and so on ad infinitum.
These repeated long term costs of missing R&D add up to outweigh the upfront benefits from the money we’d save under most reasonable views of the value of economic growth and the contribution of R&D to that growth. Summers and Jones suggest that each dollar spent on R&D creates $14 dollars of value on average!
So this leaves us with a question: Why aren’t governments taking this free lunch? Why are they letting the weakly incentivized private firms outspend them on the world’s most important positive externality?
This puzzle is explained by the distinction between spatial and temporal externalities. The argument we made above about how the government collects on spillovers between firms and customers because it taxes the entire economy is true, but only if those spillovers happen fast. No decision maker in government today benefits from R&D spillovers that accrue 20 years later. In fact, they are often made worse off since R&D spending has immediate costs and only future benefits. Perhaps governments as a single abstract entity internalize the country wide benefits of R&D that accrue decades in the future, but no actual decision maker working in government stands to gain.
Market actors, on the other hand, are better incentivized to care about temporal externalities. The owner of a firm investing in R&D doesn’t account for all the benefits their technology might bring to non-paying consumers and firms, but they do care about the benefits that R&D will bring to the firm long into the future, even after their death. One part of this is that owners don’t face term limits that incentivize pump-and-dump attempts to garner voter support. But even if the owner of a company knows they are retiring soon, they still have good reason to care for the long term value of their firm. This is because when they go to retire and sell the company, they are paid the present discounted value, which takes into account the company’s future prospects. In many industries R&D is a major determinant of these future prospects.
There is more going on in government’s decision of how much R&D to fund than their greater care for spatial externalities over temporal ones. This story doesn’t explain why they spend $50 billion on long-term basic research without short term benefit, for example, but it does explain why private firms are doing more to provide for this positive externality than governments are.
- ^
The qualitative conclusion that R&D spending has large average returns holds under significant relaxations of this assumption.
My two cents:
You may be interested in Arnaud Dyevre's job market paper which looks at how much the source of R&D matters in terms of spillover effects. He finds that public sector R&D tends to have much higher spillover effects than private sector.
Another incentive for private firms to do R&D is to make their product at least 30% more valuable than the competition to compel customers to switch. Applies to b2b mostly and long-term products and contracts.
And another reason for why governments don't do more R&D is that they cannot do it effectively, outside of the context of concrete problems and without resources (infrastructure, tools, know-how, data, etc.) available to a concrete firm.