Why Scientists Often Have To Repeat Their Studies

Harvard sleep expert, Dr Charles Czeisler, has spent about $3 million over the years showing that doctors who don’t get enough sleep make mistakes on the job. Yet long shifts for interns and residents are a staple of hospital culture, and, as anyone’s welfare in hospital might be at stake, one might have thought it important to rectify excessive hours.

But it has taken Czeisler the best part of three decades getting the medical establishment to acknowledge it, and still the rules governing doctors’ working hours remain hard to change. When he gave evidence that workers on rotating shifts at a chemical plant suffered from disrupted sleep, the medical establishment said doctors were different. Czeisler’s data “was dismissed out of hand”. They kept using the same argument even when tests had refuted it. When he published results showing that physicians’ 24 hour plus shifts contributed to car accidents and attention lapses at work, some said it might be true—but not for them!

In 2008, the Institute of Medicine issued guidelines calling for limiting interns’ and residents’ shifts to 16 consecutive hours. Eventually, authorities did cut back to 16 hours, but only for interns. Czeisler had studied interns, so the establishment claimed they had seen no evidence for residents! Now Czeisler is having to research whether residents’ performance also is affected by lack of sleep. “I can’t believe we have to do this extra study.”

Science cannot accept a single study as definitive proof of its findings. Some error could have been made or some bias have been inadvertantly built in, and any such mistakes need independent repetition of the study to discount error. Repeating a previous study which confirms it multiplies the reliability of both studies. Moreover, this case on the working hours of hospital doctors shows another reason why some research has to be repeated—a refusal to act on well established scientific work for political or economic reasons, or simply reasons of will.

Daniele Fanelli, an expert on bias at the University of Edinburgh in Scotland, points this out. “People want to draw attention to problems” rather than aiming to find something new, especially when important policy decisions are being delayed by procrastination or lack of political will. Experts have to prove some things again and again to get decision makers to act. Some might object that it is not a scientist’s job to persuade decision makers, but it is the duty of all of us to do it, surely, especially when the proof is there that lack of action is costing lives.

“There are some subjects where it seems you can never publish enough”, says Ronald J Iannotti, a psychologist at the National Institutes of Health. “Think about the number of studies that had to be published for people to realize smoking is bad for you.” Almost 50 years after cancer and lung disease were first linked to smoking, work continues to be published because the extent of the problem is still challenged, not least by those who make money out of selling tobacco products. A detailed analysis in the Canadian Medical Association Journal has had painstakingly to lay out that secondhand smoke in cars is bad for children. Many people will say that is too obvious to merit funding, but cigarette vendors, and those still addicted to smoking evidently still need reminding that harming the health of kids is not excusable—it is wrong.

The Ig Nobel Prizes are spoof awards to mock improbable research. One winner was a study that found nose picking was common among teens. Some might consider the research is not only pointless but in bad taste(!), yet it can hardly be said to be obviously so, and finding that it is common has health consequences. Staphylococcus aureus is a bacterium that is getting highly dangerous through its growing resistance to antibiotics (MRSA).

Iannotti says, even if initial findings seem self evident “you still need to establish the facts. That’s how science moves forward—incrementally”. Plainly not every study is equally worthwhile, and some studies approved for funding might be bad decisions, but the danger is that an over zealous aim to cut back on wasteful research will succeed only in cutting out useful research.

It would be far more useful to cut back on the excessive rewards given to bankers for not doing much at all, and to stop giving them even bigger rewards for wrecking the national economy. It is far more costly and ridiculous to reward useless bankers than it is to hand out funds for occasionally poorly thought out scientific studies. Bankers reward themselves with millions of dollars each a year. Many useful studies cost buttons by comparison, but no one seems to object to us giving megabucks to greedy bankers for doing little of merit.

Loans Give Hard Up Students a Buzz Until Pay Back Time Looms

Debt can be a good thing for young people—it can help them achieve goals that they couldn’t otherwise, like a college education…Young people seem to view debt mostly in just positive terms rather than as a potential burden.

Rachel Dwyer, assistant professor of sociology, Ohio State University

Professor Dwyer seems to be encouraging young people to take on more debt to feel empowered! A nationwide study she conducted with Randy Hodson, professor of sociology at Ohio State, and Laura McCloud, an Ohio State graduate now at Pacific Lutheran University, found many young adults actually feel empowered by their credit card and education debts rather than feeling stressed by them. Ms Dwyer did add that the results offer some worrying signs about how many young people view debt:

Debt can be a positive resource for young adults, but it comes with some significant dangers.

The more credit card and college loan debt held by young adults aged 18 to 27, the higher their self-esteem and the more they felt like they were in control of their lives. The effect was strongest among those in the lowest economic class. Only the oldest of those studied—those aged 28 to 34—began showing signs of stress about the money they owed.

Researchers examined data on two types of debt:

1. loans taken out to pay for college
2. total credit-card debt.

They looked at how both forms of debt were related to people’s self-esteem and sense of mastery—their belief that they were in control of their life, and that they had the ability to achieve their goals. Dwyer said:

We thought educational debt might be seen as a positive because it is an investment in their future, while credit card debt could be viewed more negatively

How debt affected young people depended on what other financial resources they had available:

• Those in the bottom 25 percent in total family income got the largest boost from holding debt—the more debt they held, both education and credit card, the bigger the positive impact on their self-esteem and mastery
• Those in the middle class didn’t see any impact on their self-esteem and mastery by holding educational debt, perhaps because it is so common among their peers that it is seen as normal, but they did see boosts from holding credit-card debt—the more debt, the more positive effects
• Those who came from the most affluent families received no boost at all from holding debt. Debt is not an issue for them. They have the most resources and options available to them.
• The oldest people in the study, those over age 28, were just starting to feel the stress of their debt.

Having education debt is still associated with higher self-esteem and mastery, compared to those who don’t have any such debt. That suggests they still see some benefits to investing in a college degree. But the amount of education debt mattered—having higher levels of debt actually reduced their sense of self-esteem and mastery. Dwyer said:

By age 28, they may be realizing that they overestimated how much money they were going to earn in their jobs. When they took out the loans, they may have thought they would pay off their debts easily, and it is turning out that it is not as easy as they had hoped. We found that the positive effects may wear off over time, but they still have to pay the bills. The question is whether they will be able to.

The study involved 3,079 young adults who participated in the National Longitudinal Survey of Youth 1979—Young Adults sample. The NLSY interviews the same nationally representative group of Americans every two years. It is conducted by Ohio State’s Center for Human Resource Research on behalf of the US Bureau of Labor Statistics.

The results suggest that debt can be an important resource for young adults that allows them to make investments that improve their self-concept. But the results may also have troubling implications for the future of young people. Dwyer summed up:

Debt may make young people feel better about themselves in the short-term, but that doesn’t mean it won’t have negative consequences in the long term.

Some young people from all social classes see education as important enough to get into debt for, but those from poorer backgrounds get the biggest buzz from borrowing money, and the rich kids get little or none. It seems hardly surprising. Just being able to get the money will make many such kids feel that their education is already bringing benefits. As the debt mounts and the benefits begin to seem less clear and further off, their enthusiasm wears thin.

Poorer students must stay realistic about their future. They will have to pay back their loans and borrowings, so they should not take on excessive debt, and must not try to compete with middle class and rich kids at university. Rich kids have no worries whatever happens. They are assured of a substantial allowance and nepotistic job opportunities from daddy and mummy so can get no buzz from borrowing a the odd few thousand dollars.

Kindergarten Math Skills Predict Later Academic Achievement Best

The transition from a home to a school environment is seen by many kindergarten teachers as associated with learning difficulties like an inability to follow directions, trouble working independently or in groups, and a lack of academic skills. A National Research Council report reckons social and emotional aptitude is as important as language and cognition in young children’s scholastic achievement, while another NRC report emphasizes the importance of early acquisition of linguistic skills, and the National Council of Teachers of Mathematics considers good math instruction helps 3 to 6 year olds.

To shed light on what constitutes school readiness and which K-5 skills and behaviors predict later academic success, UC Irvine distinguished professor of education, Greg Duncan and colleagues identified six population based data sets—involving 16,387 children—that included measures of reading and math competency, attention skills, prosocial behavior, and antisocial and internalizing behavior taken around the time of school entry, as well as measures of reading and math competency taken later in the primary or middle school years. Duncan says:

We found that only three of the school entry measures predicted subsequent academic success—early reading, early math and attention skills, with early math skills being most consistently predictive. Early behavior problems and social skills were not associated with later reading and math achievement. These patterns generally held both across studies and within each of the six data sets examined.

A student’s school entry ability to pay attention and stay on task is modestly predictive of later achievement, while early problem behavior and other dimensions of social and mental health issues are not at all correlated. If school readiness is defined as having the skills and behaviors that best predict subsequent academic success, concrete numeracy and literacy skills are decidedly more important than socio-emotional behaviors.

Prof Greg Duncan

Early math skills appear to be the strongest predictor of subsequent scholastic success, more so than early reading skills. While this analysis provides a clear pointer to the relative role of school entry skills and behaviors, Duncan and Katherine Magnuson of the University of Wisconsin-Madison recently extended the work with a second study using two large data sets (2,843 children) and the same achievement, attention and behavior measures.

They determined that K-5 students with persistently low math skills were much less likely to graduate from high school or attend college. The math results were striking. Children with persistent math problems in elementary school were 13 percentage points less likely to graduate from high school and 29 percentage points less likely to attend college.

Surprisingly, chronic reading problems were not predictive of these outcomes, after accounting for the fact that children who struggle with reading tend to also struggle with math. In contrast to the first study’s findings, persistent antisocial behavior was correlated to dropping out of high school and not attending college. But chronic difficulty paying attention and internalizing behavior were not predictive of this. Duncan proposes further work:

The next level of research should focus on why math skills—which combine conceptual and procedural competencies—are the most powerful predictor of subsequent achievement and attainment. Experimental evaluations of early math programs that focus on particular skills and track children’s reading and math performance throughout elementary school could help identify missing causal links between early skills and later success.