Robert Waldmann--who knows about biochemistry and medicine--wields the bludgeon:

Robert's Stochastic thoughts: Dean Baker asks:

Just for the record, the U.S. government already spends $30 billion a year on biomedical research.... Why shouldn't we believe that if we doubled this appropriation, to replace the $25 billion that the drug industry claims to spend on drug research?...

Brad DeLong applauds Baker and (but) replies:

The answer is that we don't trust the NIH to be able to set up procedures that cover all the bases in drug research. Low-probability but high-payoff projects are likely to be underfunded by the government--but properly funded by private companies willing to roll the dice.

I think Brad is totally wrong. The idea is that the private sector is willing to take risks for huge potential profits. This is very true of startups where decision makers are effectively protected by bankruptcy law. [But] the patent holders in the pharmaceutical industry are huge corporations which will not benefit from the services of a bankruptcy court. Decisions are made by people who will do very well if they just keep on keeping on. A sporty business compared to retailing, but not one that requires particular daring.

In contrast, the US Federal Government is willing to shoot for the moon (literally) spending huge amounts of money for purely speculative possible advantages. It is just a fact that the most daring research and development is publicly financed mostly by the US DOD. A closer look at what big Pharma and the NIH do with their money makes this very clear. Big pharma takes chances with drugs which might be valueless because of side effects or something. They bet with a worse than 50 % of a big payoff. The NIH bets with a miniscule chance of a huge payoff, as is typical of public sector research.

The Federal government has very deep pockets. It should have huge risk tolerance compared to well anything else.... It's just not true that public research is more cautious and short term. Quite the opposite. The private firms often come in when the basic concept is proven and details need to be fixed, roughly the development stage not the research stage. A pharmaceutical company issue is making a drug so that we don't digest it in our stomach, piss it away immediately or metabolise it (pharmacokinetics). While outstanding original path breaking research has been done by drug companies, they generally come in at the relatively low risk close to market phase. In fact, the advantage of the big drug companies is the advantage of a large firm whih can coordinate a huge team. Thus agents developed by say biotech startups are brought to market by big pharma.

Oddly NIH employees interact with big pharma roughly the same was biotech start ups do. The NIH is huge, but to avoid bureauschlerosis it is organised in an immense number of tiny fiefdoms with senior investigators who can't be fired and do what they want (think full professors but there are many more of them proportionally). The NIH budget is spent on an absolutely immense number of tiny organisations (think of an NSF grant and that's like an NIH grant except there are many fewer NSF grants). Often the principal investigators have no reason to fear risk, can shoot for the moon and do.

So why is it that the federal bureaucracy is better than the private sector at daring original high risk research and worse at anything requiring organisation ? Well I think one big factor is ideology. The NIH is in the USA and for it to develop drugs and bring them to market would be socialism. Can't have that even if it works (see general debate about health care). Another is that too much freedom is licence.... NIH funded researchers study things which are interesting scientifically. Congress tries to push them towards more applied work (failing to calculate the return so far on pure and applied work which shows that more bang has consistently been obtained for the pure research buck). The boring work of improving a molecule and testing and testing and testing it (after applying and applying and applying to the FDA for permission to do so) is something few people will do without pressure indirectly from shareholders via ruthless managers.

You could ask how many principal investigators at the NIH have ever received FDA approval for an IND (Investigative New Drug), a necessary step in the drug development process. Now if you suggest that they and they alone get to divy up the 20 billion a year, then I'm all for it. Conflict of interest disclaimer. I only know of one such person, and he is my father.