‘Mounting Evidence’

I have often considered this idea before.

Suppose that Doll’s Doctors Study and his Hospital Study had shown no significant difference between LC prevalence in smokers and non-smokers. What would have happened?

But we must first bear in mind that the Hospital Study was designed to be a ‘proof of concept’ study to justify the much bigger and much more expensive Doctors Study. If the Hospital Study had failed, then the Doctors Study would never have happened. So we must ask, simply, what would have happened if the Hospital Study had produced a ‘Null’ result?

Out of any given number of studies about smoking, there are some which indicate a Null result of even a positive (protective) result. For example, the early questionnaires about smoking in the Doctors Study included inhaling. The results from that question showed that inhaling was protective. Doll dropped that question from later questionnaires. Was Doll dishonest?  Probably not, because he was honest in his own way. He KNEW from his German visits pre-war that smoking was dangerous. All that was needed was to show sufficient evidence that that was true. Thus, scientific rigour was not required. All that was required was comparisons between heavy smokers, moderate smoker, light smokers and non-smokers. QED. The ‘inhaling benefit’ was an anomaly.

But consider this example.

Island A has a population which smokes. It also has lots of mosquitoes. Island B has a population which does not smoke, and has no mosquitoes. It is easy to see how the prevalence of malaria on island A could be put down to smoking, if no one knew that mosquitoes carried the malaria parasite. For years and years, that was accepted as the truth, apart from the fact that it was not smoke in the air which was the culprit. It was the air itself. ‘Bad air’ = ‘malaria’.

Neither Doll’s Hospital Study nor his Doctors Study allowed for other contributing factors.

Many other studies of smoking ensued, and all of them built up ‘A BODY OF EVIDENCE’. But they all did the same thing. They all compared smokers with non-smokers.

What was missing?

It was, and always has been, the lack of comparisons between sets of non-smokers. Some non-smokers die before others. Even the Doctors study had a ‘line’ showing the incidence of deaths at various ages of non-smokers. Some died in their fifties, but not many; more died in their sixties, and even more died in the seventies. What was the frequency of a particular cause of death?

What I am getting at is that all studies seem to start off in a sort of ‘primitive’ state. Both Doll’s studies were primitive since they did they not take into consideration other contributing factors. Air quality was insignificant, genetic predisposition was insignificant, work conditions were insignificant, wartime conditions were insignificant.

If such other considerations are always ignored, then you get a BODY OF EVIDENCE which is skewed. In criminal law, such evidence would be considered to be ‘circumstantial’.

A says “I saw X in the corridor of the room where the murder was committed, therefore X must have committed the murder”.

B says, “I saw X there too”. C says “I saw X there too”.

Conclusive proof that X committed the murder.

But along comes D. “I saw X in the cloakroom at precisely the time when the scream was heard. X could not have committed the murder”.

Does the evidence of D override that of A, B and C? It should do, since it is precise.

The craziness is that Null results are ignored, and yet they are the results which matter most. The ‘body of evidence’ is an average which does not take account of the plenitude of junk science. ‘Tell me what results you want, and I will supply the evidence”.



7 Responses to “‘Mounting Evidence’”

  1. Some French bloke Says:

    The ‘body of evidence’ is an average which does not take account of the plenitude of junk science.

    Funnily the same argument, with about the same wording, is also used by the other side of the debate:
    his analysis of the causal association between lung cancer and smoking was flawed by an unwillingness to examine the entire body of data available and prematurely drawn conclusions. (Paul D. Stolley)

    Here, the “his” does not refers to Doll (or Hill), but to R.A. Fischer, and his late 1950s refutation of the former’s hasty conclusion…


    • junican Says:

      Well observed. And true in the sense that all such data are insufficient. What Stolley was actually saying is that Fisher did not cover ‘the entire body of data’ in his criticism. But how could he be expected to and did he need to? In my example of of the islands, the discovery of the real cause of malaria blew the whole ‘smoking causes malaria’ out of the water. There was no need for an in depth study of studies ‘proving’ that smoking caused malaria on island A.

    • Some French bloke Says:

      Or, picking an idea about Karl ‘Falsification’ Popper from a comment Peter Carter posted on Frank Davis’ blog yesterday:

      Some genuinely testable theories, when found to be false, are still upheld by their admirers – for example by introducing ad hoc some auxiliary assumption, or by re-interpreting the theory ad hoc in such a way that it escapes refutation.

      So, who broke the methodological rule that we must accept falsification in favour of “auxiliary assumptions” aimed at upholding the dogma at all costs? Well, Doll broke that rule by ignoring, throughout his career, all of the valid points made by Fisher, Alice Stewart, Philip R. J. Burch and others, while *they* certainly didn’t ignore, and in fact quite competently addressed, Doll’s et al’s points.

      Two cases in point;

      • First, considering the general drop in smoking prevalence in the West, national and international statistics now provide us with a surfeit of evidence that smoking and lung cancer trends are totally unrelated. If you look at them with the preconception that smoking has been driving the trend, then the actual results (country to country, men vs women) are all over the place, i.e. the assumption has to be wrong!

      • Second, concerning the inhaling question. As Stolley mentions in his ‘When genius errs: R.A. Fisher and the lung cancer controversy’ article of 1991, when Doll & Hill reintroduced the inhaling factor in their Doctors questionnaire (1957), the same results ensued, i.e. a slight protection for inhalers was shown… but then Stolley feebly argues that since that apparent protection was smaller than the one that was downplayed (if not suppressed) in the London Hospitals Study, therefore Fisher was wrong!

      • junican Says:

        I did some investigations into LC as cause of death a little while ago. As a proportion of deaths, the incidence of LC was less than it was decades ago. But what other causes replaced LC?
        The incidence of ‘mortality or ‘morbidity’ from SHS disappears when you look at the stats.

      • Some French bloke Says:

        the incidence of LC was less than it was decades ago

        In the UK, the likelihood of dying from LC may have dropped by 43% for men since 1979, however, taking into account the steady increase in risk for women*, the general population’s likelihood of dying from LC is the same now as it was back in 1964!

        * Crude rates per 100,000: 15.92 (1964), 31.50 (1979), 49.31 (2013).

  2. Philip Neal Says:

    Indeed. In controversy with Burch and elsewhere, Doll quite explicitly talked of a “mass” of confirming evidence which ought to be “weighed” against a few embarrassing paradoxes and he did not follow sound scientific method. It ought to be called the Mound of Evidence Fallacy.

    • junican Says:

      I agree. Doll’s ‘mass’ was just like the ‘mass’ of evidence which declared that ‘bad air’ was the cause of malaria.

Comments are closed.

%d bloggers like this: