Why Scientists Often Have To Repeat Their Studies

Harvard sleep expert, Dr Charles Czeisler, has spent about $3 million over the years showing that doctors who don’t get enough sleep make mistakes on the job. Yet long shifts for interns and residents are a staple of hospital culture, and, as anyone’s welfare in hospital might be at stake, one might have thought it important to rectify excessive hours.

But it has taken Czeisler the best part of three decades getting the medical establishment to acknowledge it, and still the rules governing doctors’ working hours remain hard to change. When he gave evidence that workers on rotating shifts at a chemical plant suffered from disrupted sleep, the medical establishment said doctors were different. Czeisler’s data “was dismissed out of hand”. They kept using the same argument even when tests had refuted it. When he published results showing that physicians’ 24 hour plus shifts contributed to car accidents and attention lapses at work, some said it might be true—but not for them!

In 2008, the Institute of Medicine issued guidelines calling for limiting interns’ and residents’ shifts to 16 consecutive hours. Eventually, authorities did cut back to 16 hours, but only for interns. Czeisler had studied interns, so the establishment claimed they had seen no evidence for residents! Now Czeisler is having to research whether residents’ performance also is affected by lack of sleep. “I can’t believe we have to do this extra study.”

Science cannot accept a single study as definitive proof of its findings. Some error could have been made or some bias have been inadvertantly built in, and any such mistakes need independent repetition of the study to discount error. Repeating a previous study which confirms it multiplies the reliability of both studies. Moreover, this case on the working hours of hospital doctors shows another reason why some research has to be repeated—a refusal to act on well established scientific work for political or economic reasons, or simply reasons of will.

Daniele Fanelli, an expert on bias at the University of Edinburgh in Scotland, points this out. “People want to draw attention to problems” rather than aiming to find something new, especially when important policy decisions are being delayed by procrastination or lack of political will. Experts have to prove some things again and again to get decision makers to act. Some might object that it is not a scientist’s job to persuade decision makers, but it is the duty of all of us to do it, surely, especially when the proof is there that lack of action is costing lives.

“There are some subjects where it seems you can never publish enough”, says Ronald J Iannotti, a psychologist at the National Institutes of Health. “Think about the number of studies that had to be published for people to realize smoking is bad for you.” Almost 50 years after cancer and lung disease were first linked to smoking, work continues to be published because the extent of the problem is still challenged, not least by those who make money out of selling tobacco products. A detailed analysis in the Canadian Medical Association Journal has had painstakingly to lay out that secondhand smoke in cars is bad for children. Many people will say that is too obvious to merit funding, but cigarette vendors, and those still addicted to smoking evidently still need reminding that harming the health of kids is not excusable—it is wrong.

The Ig Nobel Prizes are spoof awards to mock improbable research. One winner was a study that found nose picking was common among teens. Some might consider the research is not only pointless but in bad taste(!), yet it can hardly be said to be obviously so, and finding that it is common has health consequences. Staphylococcus aureus is a bacterium that is getting highly dangerous through its growing resistance to antibiotics (MRSA).

Iannotti says, even if initial findings seem self evident “you still need to establish the facts. That’s how science moves forward—incrementally”. Plainly not every study is equally worthwhile, and some studies approved for funding might be bad decisions, but the danger is that an over zealous aim to cut back on wasteful research will succeed only in cutting out useful research.

It would be far more useful to cut back on the excessive rewards given to bankers for not doing much at all, and to stop giving them even bigger rewards for wrecking the national economy. It is far more costly and ridiculous to reward useless bankers than it is to hand out funds for occasionally poorly thought out scientific studies. Bankers reward themselves with millions of dollars each a year. Many useful studies cost buttons by comparison, but no one seems to object to us giving megabucks to greedy bankers for doing little of merit.