Research Built on a House of Cards

It’s about time someone said this:

Diet research built on a ‘house of cards’?

The next time a headline about diet and disease catches your eye, check the fine print of the study.

If it is based on a food questionnaire — and there’s a good chance it will be — then the conclusions should be handled with caution.

That’s because people don’t tell the truth, not even to scientists.

Scientists know this. Research has proven it. It’s been an awkward problem in nutrition science for more than 40 years.

And that’s long enough, according to an international group of nutrition researchers. They’ve launched a campaign to end the use of one of the most common research tools in nutrition science.

“All of these studies, if they are based on self-report estimates of energy intake, really don’t contain scientifically meaningful information,” said David Allison, a prominent obesity researcher at the University of Alabama at Birmingham and lead author on a report signed by 45 scientists from around the world.

This matters, the expert group warned, because the flawed data could result in public health policies that are not science-based, but built on a “house of cards,” Allison said.

We’re talking about hundreds if not thousands of papers published every year.

Is a high-fat diet linked to breast cancer?  Do fruit and vegetables protect against Alzheimer’s disease?  How much of obesity can be explained by eating too many calories?

To get those answers, scientists need to gather data about who ate what, and when. So they ask research subjects to self-report, to think back, sometimes over months and years, and fill in a detailed food questionnaire about what they ate.

Other studies ask people to keep food diaries. And sometimes the researchers do random spot checks, calling subjects and asking them to remember everything they’ve eaten for the last 24 hours.

But because humans are very bad at admitting or remembering what they ate, all the self-reported methods have shown evidence of bias.

I’ve been saying something like this for ages. And it applies to tobacco ‘research’ as well, of course. Nothing is measured accurately. The numbers are all fuzzy. All ‘research’ done using questionnaires should be regarded as unscientific. It should never be used as the underpinning for research conclusions or policy decisions.

He admits the group is taking a controversial stand that has sparked mixed reaction from the nutrition science community.

“It varies a lot with who you ask,” Allison said. “Much of the scientific community applauds us and says it’s a refreshing point of view.”

“And as you might expect, some individuals who have based much of their research career and program on the use of these methods say we can’t abandon them because in part we don’t have anything better.”

Marian Neuhouser of the Fred Hutchinson Cancer Research Centre in Seattle, who has done some of the research that has exposed the bias in self-reported data, agrees the data is flawed. But she says if researchers perform a backup biomarker study they can correct for some of those biases.

Biomarker studies using urinalysis are accepted methods of accurately measuring energy intake. But that requires research subjects to be brought into a lab for a urine test. Researchers say that’s too expensive and impractical for large-population studies.

Considering the obesity epidemic and the level of chronic disease, surprisingly little work is being done to improve methods, said Ross Prentice, a professor of biostatistics at the Hutchinson Centre.

Technical solutions being considered include chin-mounted “chew monitors,” or wrist monitors that measure hand-to-mouth movements.  Some have suggested using smart phones to take photographs of food.

Allison prefers a biologically based approach, perhaps a test of breath or urine for products of digestion, which does not depend on the honour system and the fallibility of human memory.

I can see a problem with wrist monitors that measure hand-to-mouth movements: Smokers would come out way ahead of everyone.

And chew monitors would call out people who like to chew gum.

And let’s suppose that biomarker studies using urinalysis accurately measure energy intake. They won’t, however, measure energy expenditure. And if people are only eating what they need to balance intake with expenditure, then a lumberjack cutting down trees with a hand axe in the depths of winter will be expending far more energy than someone sitting quietly in a warm room, and will need to have an energy intake to match it. You only know if someone is over-eating or under-eating if you also know their energy expenditure as well.

Basically, the whole field is wall-to-wall bad science, and probably always will be. And the best thing that could happen would be for the funding for all of it to be terminated. Because if you can’t do good research, you’d best not do any at all.

If you haven’t got a telescope with sufficient resolution to be able to see the canals on Mars, you shouldn’t be trying to draw maps of them. And, who knows, there might not be any canals at all on Mars.

For a time in the late 19th and early 20th centuries, it was erroneously believed that there were canals on Mars. These were a network of long straight lines in the equatorial regions from 60° N. to 60° S. Lat. on the planet Mars. They were first described by the Italian astronomer Giovanni Schiaparelli during the opposition of 1877, and confirmed by later observers. Schiaparelli called these canali, which was translated into English as “canals”. The Irish astronomer Charles E. Burton made some of the earliest drawings of straight-line features on Mars, although his drawings did not match Schiaparelli’s. By the early 20th century, improved astronomical observations revealed the “canals” to be an optical illusion, and modern high resolution mapping of the Martian surface by spacecraft shows no such features.

Sciaparelli's map of Martian canals

Sciaparelli’s map of Martian canals

Just as a matter of interest, I got hold of another more recent NASA/JPL map of Mars (click to enlarge). I found Tharsis. Chryse, Eden, Arabia, and Hellas in both of them.


More at Science Daily.

About Frank Davis

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

20 Responses to Research Built on a House of Cards

  1. cherie79 says:

    Best just to ignore it all.

  2. They’d do better to stop trying to dictate to the greater population how to live their lives, and make the best of the life they, themselves, are living.

    I know…I know…there ain’t no money in that. A girl can dream, no?

  3. mikef317 says:

    Another epidemiological problem is the diagnostic accuracy of doctors. The link below is about historical changes effecting death certificate data, but the same problems apply to non-fatal illnesses.

    The phrase “get a second opinion” is probably familiar to most readers. To my knowledge, studies of disease take a single doctor’s diagnosis as proof that a patient has the disease. Unfortunately, if the diagnosis is wrong, the study’s results will be equally wrong.

    Since epidemiology typically draws momentous conclusions from small statistical differences between small groups of people, even a small number of diagnostic errors can seriously distort the results. Alvan R. Feinstein, The Intellectual Morbidity of Vital Statistics.

  4. Furor Teutonicus says:

    I always find a visit to a new doctor (specialist) interesting.

    “How much do you smoke?”

    Frankly, I don’t bloody KNOW! I smoke a pipe, and of the five I have, not ONE of them is the same size, and most times I only smoke about half of the baccy that is in it any way. Some days I can smoke ten or fifteen, other days one or two, or, when busy, NONE.

    BUT(!) The quack always insists on an “Amount per day.” O.K, so X amount of baccy per month. (Note, If I only smoke half the pipe, the rest is thrown away, so….)

    They then divide this into how manycigarettes could be made with the amount.

    They ignore the fact that pipe baccy is pure and fags have all kinds of other shite in them. That when someone smokes a fag, they normally smoke ALL of it. But that counts as a scientific estimation of “an average smoker.”


    “How much PER DAY, do you drink?”

    Well, I do not drink EVERY day, for a start. I CAN drink 10 to 15 liters per day, then none for a week, then two bottles over dinner, and then none for two weeks. Or whatever the pattern may be.

    BUT, they then say “O.K, so about how many a month?” (When I have a string of 12 to 18 hour shifts, it can be NONE for two or three months. But then comes “But when you DO drink in the month, how much. The months I do not are simply ignored.)

    Whatever I say, they divide that to come out with a DAILY figure(!)

    So now my medical record looks like I smoke so many fags a day, and can not get through a day without beer.

    THESE are the figures these “scientists” are using!!!

  5. Frank Davis says:

    James Lovelock’s reaction to first reading about the CRU emails in late 2009 was one of a true scientist:

    “I was utterly disgusted. My second thought was that it was inevitable. It was bound to happen. Science, not so very long ago, pre-1960s, was largely vocational. Back when I was young, I didn’t want to do anything else other than be a scientist. They’re not like that nowadays. They don’t give a damn. They go to these massive, mass-produced universities and churn them out. They say: “Science is a good career. You can get a job for life doing government work.” That’s no way to do science.

    I have seen this happen before, of course. We should have been warned by the CFC/ozone affair because the corruption of science in that was so bad that something like 80% of the measurements being made during that time were either faked, or incompetently done.

    Fudging the data in any way whatsoever is quite literally a sin against the holy ghost of science. I’m not religious, but I put it that way because I feel so strongly. It’s the one thing you do not ever do. You’ve got to have standards.”

  6. The intended purpose, the bottom line of Pfizer.

    Pfizer Supports Complete Smoking Ban in Public Areas and Improved Access to Smoking Cessation…

    (BUSINESS WIRE)–Pfizer Inc (NYSE:PFE) (LSE:PFZ) welcomes the European Commission’s Green Paper on smoke free environments1, which was put for public consultation earlier this year. Today in Brussels, a week before World No Tobacco Day (May 31st), the world’s leading life sciences company submitted…

  7. Vince Harden Seems the governing body has reversed their guidelines (for awhile at any rate) because they looked like bigoted fools.Now if the rest of this smoker bashing was reversed for the same reason-we’d all be better off.

    Social workers given updated guidance on e-cigarette use by foster carers or adopters
    British Association of Adoption and Fostering now advises social workers to distinguish between e-cigarettes and tobacco cigarettes

    • nisakiman says:

      British Association of Adoption and Fostering now advises social workers to distinguish between e-cigarettes and tobacco cigarettes

      And why?

      Given that the WHO’s own study into SHS (or ETS, whatever) found only one statistically significant result, that being that exposure to SHS in childhood had beneficial effects in later life. And yet still they adhere to the PC orthodoxy, despite the facts (according to the WHO research) being at odds with their ideological stance.

      Is there any hope? I do wonder sometimes. No, not sometimes: often.

  8. garyk30 says:

    This study was mentioned the other day.
    It was ‘peer-reviewed’ and published.
    Apparently, none of the people involved is capable of logical thought or has a tiny bit of commonsense.

    Tobacco smoking and all-cause mortality in a large Australian cohort study: findings from a mature epidemic with current low smoking prevalence
    Compared to never-smokers, the adjusted RR (95% CI) of mortality was 2.96 in current smokers..

    Cause of death information was not available at the time of analysis.

    In Australia, up to two-thirds of deaths in current smokers can be attributed to smoking


    This is what passes for ‘Science’ in Tobacco Control.

    They do not say that 2/3rds of smokers die from the diseases said to be ’caused’ by smoking because they have no idea what caused the deaths.

    Such information was not available.

    2/3rds(67%) of deaths attributed to smoking would mean that:

    1. If 3 smokers die from Ebola, 2 of those deaths are due to smoking and not the Ebola virus.

    2. If 3 smokers die in highway accidents, 2 of those deaths are due to smoking and not due to the injuries from the accident.

    3. If 3 smokers are killed in shark attacks, 2 of those deaths are actually due to smoking and not due to sharks.

    4. If 3 smokers die from lung cancer, only 2 of those deaths are due to smoking.

    This sort of stuff is beyond stuff!!!

  9. garyk30 says:

    Funny, I do not see smoking or cigarettes mentioned here.

    Table 1: Leading underlying causes of death in Australia by sex, 2011
    Rank– Males
    Cause of death #.. %.. Cause of death.. #.. %
    1 Coronary heart disease (I20–I25)– 11,733– 15.6
    2 Lung cancer (C33, C34)– 4,959– 6.6
    3 Cerebrovascular diseases (I60–I69)– 4,427– 5.9
    4 Prostate cancer (C61)– 3,294– 4.4
    5 Chronic obstructive pulmonary disease (J40–J44)– 3,278– 4.4
    6 Dementia and Alzheimer disease (F00–F03,G30)– 3,268– 4.3
    7 Colorectal cancer (C18–C21) –2,248– 3.0
    8 Diabetes (E10–E14)– 2,178– 2.9
    9 Cancer, unknown, ill-defined (C26, C39, C76–C80)– 1,920– 2.6
    10 Suicide (X60–X84)– 1,726– 2.3
    11 Heart failure and complications and ill–defined heart diseases (I50–I51)– 1,464– 1.9 I
    12 Pancreatic cancer (C25)– 1,218– 1.6
    13 Kidney failure (N17–N19)–1,208– 1.6
    14 Influenza and pneumonia (J09–J18)– 1,136– 1.5
    15 Cirrhosis of the liver (K70–K76)– 1,087– 1.4
    16 Melanoma (C43)=-= 1,071– 1.4
    17 Land transport accidents (V01–V89)– 1,003– 1.3
    18 Liver cancer (C22)– 980– 1.3
    19 Leukaemia (C91–C95)– 933– 1.2
    20 Oesophageal cancer (C15)– 903– 1.2

    • garyk30 says:

      ‘Causes of death’ are not the same as the ‘manner of death’.

      Basically, there are 3 ‘manners of death’.

      They are;
      1. Heart Failure
      2. Reperatory Failure
      3. Brain Failure

      These are the real reasons why a person is considered ‘dead’.

      Altho, you can be ‘brain dead’ and still be ‘alive’.(See Parliment or Congress)

      Death Certificates merely show/document the injuries or diseases that led to those 3 manners of death.

  10. beobrigitte says:

    The moment real science died was when it could be bought by companies/groups that dictate the results to be obtained.
    With respect to tobacco – for as long as the tobacco industry could fund research, too, there was a kind of balance. Then tobacco control&friends eliminated the competition.

    As it stands, e.g. food companies are not (yet) where the tobacco companies are.

    That’s because people don’t tell the truth, not even to scientists.
    Especially NOT to scientists!!! Selfpreservation prevents people from telling ANYONE how many cakes/burgers/chocolate etc. they eat or how much alcohol they drink or how much they smoke. It opens them up for being reprimanted.

    This matters, the expert group warned, because the flawed data could result in public health policies that are not science-based, but built on a “house of cards,” Allison said.

    “We’re talking about hundreds if not thousands of papers published every year.“
    Indeed we are talking papers, papers and papers again. The more produced, the quicker it’s ‘up the ladder’ of social standing and income.
    I believe tobacco control pays well. It first attacked the tobacco industry – this appears to be happening to the food industry now. The Telegraph (in print) today ran the story that ‘ready-to-eat-lettuce’ is swirled around in ‘stagnant’ water to which chlorine is being added to kill off bacteria….
    I guess the food industry will get the REAL treatment soon enough. And once the food industry has been taken out, the real herbal tea/celery sticks gourmets etc. have a free run to dictate laws.
    I am waiting to read about the breweries misconducts….

    The thing with papers and ‘peer review’ is that a computer can generate nonsense papers. Many of them were published, one in ‘Nature’.

    WHY SHOULD anyone believe what scientists publish? Science died the day it could be bought…..

    • garyk30 says:

      Science would be more interesting if the studies showed stuff like this.

      Women’s Farts Smell Worse, and Five More Facts You Need to Know About Flatulence
      Flatulence is a fact of life. Americans collectively break wind to the smelly tune of up to 6.3 billion times each day. That’s a lot of hot air. For such a ubiquitous activity, it’s amazing how taboo it is. Face palms and pinched noses mark the passing of gas in most social settings. Science, however, has no ingrained distaste for flatulence.

      Here are six facts we’ve learned about farting.

      1. There are three main fart smells. Hydrogen sulfide produces the signature “rotten eggs” note, methanethiol produces hints of “decomposing vegetables,” and dimethyl sulfide adds a hint of “sweetness.”

      2. The average fart is roughly 100 milliliters in volume and lasts approximately two seconds. More interesting than the statistic itself is how it was calculated. Basically, it involved subjects farting into specially designed, airtight, gas-collecting underwear.

      3. There’s a way to make your farts (mostly) odorless. Marketed as the only “internal deodorant,” the over-the-counter drug Devrom, with its active ingredient bismuth subgallate, reduces almost 100% of the odor caused by sulfur gasses, the primary contributors to smelly farts

      4. Women’s farts smell worse. In studies conducted by eminent flatulence researcher Michael Levitt, women’s farts consistently sported significantly greater concentrations of hydrogen sulfide. Odor judges have confirmed that — at similar volumes — this translates to a noticeably worse odor compared to men’s farts.

      5. Red meat kicks up a stink. Sulfur compounds contribute the most to flatus malodor, but compounds called thiols also royally reek. Methanethiol is one of the worst. Naturally found in blood, and, in turn, red meat, it can be released via the digestive process and eventually off-gassed via the anus.

      6. Holding in your farts won’t kill you, but it won’t be comfortable either. As Tara fromD-News explained, “When we hold farts in, the gas retreats back into our body and gets absorbed into the intestinal walls where it eventually mixes in with our blood. At best, that can cause bloating, abdominal pain, and constipation but if you do it repeatedly it can lead to adistended bowel.

      Primary source: Gulp: Adventures on the Alimentary Canal, Mary Roach, 2013

No need to log in

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.