The CATCH Debate Recalled

A year ago today, I left Devon. The day also marked the end of the CATCH (Colloquium About The Cigarette Hypothesis) debates on my old blog, which had been running for about a month after Chris Snowdon, who is firmly of the belief that smoking causes lung cancer, had challenged me to a debate about it.

At the time, with the move from Devon imminent, I wasn’t able to devote as much as my time to the debate as I would have liked, and furthermore as the debate developed I found myself more in the role of editor than debater. But Rich White took up the baton, and very forcibly argued the case against smoking as a cause for lung cancer.

At the end of the debate, however, I felt that it had been a 2-2 draw. Chris Snowdon refused to change his mind, and so also did the sceptical Rich White. Much of the debate centred around the Doll and Hill studies of the 1950s, and while Chris (perhaps) retreated somewhat on the value of these studies, he pointed out – quite rightly – that there were several further studies which had drawn the same conclusion on tobacco causation which had yet to be discussed. And so, in many ways, the debate was incomplete in that it failed to look at all the evidence.

Over the past year, I’ve continued thinking about the matter, and have found my scepticism deepening. And if I were to engage in the argument again, I think that I would adopt a far more radical position than I did a year ago: I would now argue that the research conducted by Doll and Hill, and numerous other researchers, wasn’t science at all.

For in my view true science rests upon accurate measurement. For example, the gas law that states that the pressure of a gas varies linearly with its temperature (over a wide range of temperatures and pressures) is one that was empirically derived from making accurate measurements of both the temperature and pressure of gases. Scientists – real ones – take very great pains to measure quantities like pressure, temperature, time, length, mass, electrical potential, etc, as accurately as they possibly can. And indeed it is only because such quantities were accurately measured that it became possible to discover physical laws like the gas laws. Science, in many ways, is accurate measurement.

And in addition, real scientists make estimates of the accuracy of their measurements, and give the length of something as, say, 1.3 cm +/- 0.1 cm. That is to say that the length fell in the range of 1.2 cm to 1.4 cm. Scientists quantify inexactitude, because they are aware that all measurements are inexact, even the most accurate ones.

And furthermore, real science rests upon a set of well-defined units – the kilogram, the metre, the second, etc. -. There are (or for a long time, there were) actual lumps of metal kept in controlled conditions which were the standard against which all other measures of mass and length were ultimately to be compared. The same applied to clocks, such as those kept at Greenwich in England.

It is an aspect of science which is seldom emphasised: the tremendous care that scientists take in defining the units, and measuring the numbers of those units in the experiments they conduct.

But when we consider the epidemiological studies of researchers like Doll and Hill, we find ourselves in a world in which nothing is measured accurately, and in which even the units being used are ill-defined.

Take for example the unit of the ‘cigarette’, according to which some people smoke 10 or 20 of them a day. What is the standard cigarette, kept in controlled conditions in the vaults of some government laboratory in France or Switzerland? The answer is that there isn’t one. A cigarette is whatever looks like a cigarette. It may be king-sized or filter-tipped, hand-rolled or manufactured, and made of a variety of different tobaccos, and of a variety of strengths. By an estimate of mine a while back, I think that it would require two or three of my thin hand-rolled cigarettes to fill a king-sized manufactured cigarette. So we are immediately dealing with a unit – the cigarette – which may vary in size back a factor of 3 at least, and quite possibly more. That’s like having a set of rulers on which the inches on some are 3 times longer than the inches on others.

And then what does it mean to ‘smoke’ a cigarette? This can range all the way from sitting with the burning cigarette sitting on an ashtray, or between one’s fingers (like my current cigarette), to something that is greedily inhaled deep into one’s lungs, with hardly a microgramme of any of the smoke going anywhere else. And in between there are all sorts of different ways of smoking cigarettes which range from not inhaling the smoke at all, to inhaling it mixed with air, or inhaling directly from the cigarette. So ‘smoking’ a cigarette can range from inhaling much the same amounts of smoke as a ‘passive smoker’, or inhaling ten or a hundred times as much.

Another ill-defined unit is the ‘smoker’, whereby studies refer to ‘smokers’ and ‘non-smokers’ (and perhaps ‘ex-smokers’). One might think that a non-smoker was someone who has never smoked any cigarettes. But in reality, in many studies, a smoker is defined as someone who has smoked 100 cigarettes or more in their lifetime, and a non-smoker is defined as anyone who has smoked less. All of which raises a number of questions. Why 100? Is it because it’s a nice round number? Why not 34 cigarettes? Or 276? And anyway, can anybody actually recollect precisely how many cigarettes they have smoked in their lifetime.

This latter problem is sometimes called ‘recall bias’. But it’s more than a matter of bias. There can only be bias if estimates of cigarette consumption could be compared with accurate measurements of actual cigarette consumption. And since there are no accurate measurements of anyone’s actual cigarette consumption, how can it ever be possible to decide how biased recollections may or not be? The truth of the matter is that most people simply don’t know how many cigarettes they have smoked, much like they have no idea whatsoever how many cups of coffee they have drunk, or books they have read, because they weren’t keeping a record.

And here’s another proclivity of real scientists: they keep accurate records. When they measure the length of some object, or find its weight, they carefully record it, usually in words and numbers written on paper. And they do this because they know that if they don’t, they’ll rapidly forget the numbers they’ve measured. They don’t rely on their memories for the results of their experiments, even of the previous day. They record the results they get as and when they get them. And yet tobacco research regularly relies on the recollections of smokers from 10 or 20 or even 50 years earlier. And there is no way of knowing whether such recollections are biased in one direction or the other, underestimating or overestimating.

And then, when these ‘smokers’ have ‘smoked’ some ill-remembered number of ‘cigarettes’ over their lifetime, and they die of lung cancer, nobody is actually sure whether they a) really died of lung cancer, and b) whether the cancer originated in their lungs in the first place or spread from elsewhere.

But in tobacco research, all these various numbers are then treated as if they had been as accurately measured as any lump of plutonium in a nuclear laboratory, and then subjected to statistical manipulations which end up showing that smokers are 13 times more likely to get lung cancer than non-smokers. Or maybe 20 times more likely. Or 3 times more likely. The mathematical techniques used to produce such numbers cannot, however, improve in the least upon the quality of the data on which they are based.

And so to my charge against all these studies: they are not science. They may look like science, with all their charts and tables, and all their relative risks and confidence intervals, but the raw data on which their results all rest is made up of numbers which are guesses and vague recollections using ill-defined and arbitrary units, without any way of knowing how accurate any of these might be. In such circumstances, there is no need to even bother to read any of the studies based on such methods in order to know that their conclusions are devoid of value.

One might even say that even astrology is more scientific than this. At least astrologers base their calculations on accurate measurements of the movements of the planets in the solar system.

I imagine that, faced with such an attack on their science, tobacco epidemiologists might well concede that many (and indeed perhaps all) of the raw data they use is highly suspect, but would say: “That’s the best we can manage. Can you do any better?” To which my answer would be that, in the absence of accurate data, it is quite simply impossible to carry out genuine scientific research of any sort at all, even with the best will in the world, and that if theirs was the best that can be managed, then it wasn’t worth doing in the first place.

The most that I would concede of such ‘research’ is that it vaguely suggests that smoking may have some connection with lung cancer, much in the way that visiting West Africa may have a connection with contracting malaria. Certainly no intrusive advice to anyone to stop smoking (or stop visiting West Africa) is called for, and certainly not divisive and socially and economically destructive laws like smoking (or travel) bans. Such interventions can only be justified to the extent that they are based on near-certainties. And in the case of tobacco research, there is nothing nothing that even begins to approach any certainty.

And yet, it’s a strange thing that from out of this most inexact science (so inexact as to not even merit the title of Science) there has sprung what is perhaps one of the deepest convictions of the age – believed by almost everybody (including Chris Snowdon) – that smoking causes lung cancer. This belief is, it might be said, regarded as a fact of life, more certain than the fact that the earth goes round the sun.

How is it that such an uncertain hypothesis came to be widely regarded as absolutely certain? Perhaps simply by being asserted over and over again for 50 years or more. Or perhaps because the principal sceptics of this dogma died out before its advocates. Sir Ronald Fisher, the ‘father of statistics’, who contested the cigarette hypothesis throughout the 1950s, died in 1962. By contrast, the advocates of the hypothesis, Bradford Hill and Richard Doll, lived respectively until 1991 and 2005. It might have been a different story if Doll and Hill had died in 1962, and Fisher had lived until 2005, and had 40 years to lay his opponents’ arguments to rest.