• An Examination of the Double-Blind

Sponsored Links

  • FileName: 1973-v02n03-p107.pdf [read-online]
    • Abstract: An Examination of the Double-BlindMethod as It Has Been Applied toMegavitamin TherapyA Hoffer, M.D., Ph.D., F.A.O.P.1Introduction of the placebo group were well. However, two-

Download the ebook

An Examination of the Double-Blind
Method as It Has Been Applied to
Megavitamin Therapy
A Hoffer, M.D., Ph.D., F.A.O.P.1
Introduction of the placebo group were well. However, two-
Under my direction, the first double-blind thirds of the vitamin groups were well, there being
experiments were conducted in psychiatry. With no difference between nicotinamide and nicotinic
the first one we examined the efficacy of a yeast acid. After that, three more double-blind
nucleotide preparation which had been claimed to experiments were completed in Saskatchewan. In
be effective in treating chronic schizophrenic each case, the results when vitamin B3 was used
patients. None of the patients improved. With the we're superior to standard treatment only.
second experiment we compared the efficacy of Since then, as a result of the work of
nicotinic acid, nicotinamide, and placebo in physicians, now known as Orthomolecular
combination with the standard treatment of that psychiatrists, many significant improvements have
day (electroconvulsive therapy, psychotherapy, been made. The treatment today contains many
and sedatives). The clinician in charge of each additional variables and produces a greater
case decided whether or not to use ECT. Half from number of recoveries. Orthomolecular therapy
each of the three groups (usually the sickest includes megadoses of vitamins as a main
patients) received ECT. Nicotinamide was used as component but also includes attention to
a hidden control group to compensate for the nutritional therapy, to tranquilizers,
vasodilation (flush) produced by nicotinic acid antidepressants, and to ECT. The best
when the medication is started. None of the comprehensive outline is in Orthomolecular
nursing or medical staff were aware of the Psychiatry (Hawkins and Pauling, 1973).
nicotinamide group. The 30 patients were admitted 107
from the community to a psychiatric ward and had
not spent long periods of time in any institution.
They were mainly acute and subacute. On the
average they were in hospital about two months
while going through the experimental program.
1.1201 C. N. Towers, Saskatoon, Saskatchewan, Canada,
S7K 2L5
They were re-evaluated every three months
after discharge and a final evaluation was made
one year after discharge. The follow-up worker
made his evaluations blind. Three grams per day
of nicotinic acid or nicotinamide was used for 33
days while in hospital.
At the end of one year after discharge, one-third
Only one other psychiatrist, Dr. H. Osmond, has experiments are applied to groups of patients
had as much experience as I in conducting double- whose duration of illness varies from several
blind experiments, not in the number of trials, but months to many years (Click and Margolis,
in the number of years that trials have been 1962).
conducted. Over the years we have become more
and more aware of the inherent defects of this
method, and in a series of papers we have drawn (3) The three main components of the patient-
these difficulties to the attention of the medical doctor relationship are the status of patient and
world (Hoffer and Osmond, 1961, 1963; Hoffer, doctor tied together by the relationship. The
1967). I have concluded that the double-blind patient is impelled to have some faith and trust
method has so many imperfections that its use is that the physician will be able to help him. The
limited, that it leads to a large number of serious greater the threat to the person, the greater is the
errors, i.e., it would do so if clinicians took it drive to have faith. The physician, on the basis of
seriously, and that its main function is to make it his education and experience, will have a certain
easier for government agencies to turn down new degree of confidence that he can help the patient,
drugs. It seems to be most appreciated by groups either to achieve a cure or to alleviate the
who have access to unlimited funds and limited discomfort. There is no generally accepted term
access to clinical curiosity and creativity. I have for this complex relationship. Terms like placebo
finally concluded that the observations and con- effect, psychotherapy, etc., have been used but do
clusions made from double-blind experiments not quite get at the essential relationship. Placebo
have as little relevance to the therapy of patients as effect is generally described as a positive
do observations on monkeys in a cage to their response in the patient due to his expectations,
behavior in their native habitat. The beast seems to his faith in the doctor or in his medication.
be the same, but there the resemblance ends. Negative placebo effects have been given less
The defects of the double-blind methodology
Hoffer and Osmond (1961) presented the term
are these:
obecalp to describe another aspect of the
(1) The basic assumptions behind the use of relationship. Inasmuch as a positive response to
statistical analyses are ignored (Hogben, 1957). an inert substance is a placebo response, so a
The design assumes that the comparison groups negative response to an active chemical is an
will be equivalent, will be homogeneous, and will obecalp reaction (placebo backward). This we
be invariant. Over time a proper randomization defined as a reaction in which a compound of
may approximate equivalence but since known potency produces no response in test
psychiatric populations are very heterogeneous, it subjects due to factors such as fear of medication,
is impossible to obtain equivalent groups. This can either in the subject or in the doctor and others in-
be overcome by very large samples, but these are volved in giving the drug, lack of faith, negative
very rare in medical research. Nor can we assume suggestion, and so on.
the conditions are invariant. There is a natural
historical drift in the nature and intensity of
diseases due to factors only dimly understood.
Therefore, according to Hogben, statistical
analyses based upon probability theory is invalid.
(2) Little account is taken of the disease model
being followed in most double-blind experiments.
Diseases may be short-lived, like pneumonia, or
may last most of one's life, like diabetes or
schizophrenia. The same variability of models is
inherent in any psychiatric population. Since
chronic patients usually require chronic treatment,
it is inappropriate to use short-term treatment for
chronic patients. But chronic double-blind
experiments are very difficult to control so they
are seldom used. We therefore find that short
There are two phases in the placebo and However, unless there is a sustained placebo
obecalp reaction, the initial reaction when effect, the recovery will not be maintained since
treatment is just underway, and the sustained the patient will discontinue medication.
reaction. Of these the sustained reaction is more Fortunately, most psychiatric patients when well
rational and more important. During the initial do not wish to relapse into their previous sick
phases, the patient is dependent upon the condition. Many schizophrenics who have been
transaction with his doctor and has no way of well for many years on the megavitamin therapy
directly experiencing whether he is being helped. will discontinue medication but will in most cases
However, once treatment has started he begins to resume medication if they become aware of a
experience changes in himself, good or bad, from resurgence of symptoms.
which he draws conclusions which enhance his
In JAMA 224, 1584, 1973, Borda's report on
placebo or obecalp reaction. If a patient finds that
patient evaluation of tranquilizers was reviewed.
his depression or his hallucinations become less
Borda studied over 15,000 patients in 10
troublesome, this increases his faith and trust in
hospitals of whom 25 percent had received at
the medication and so enhances placebo response.
least one tranquilizer. There was a marked
A positive feedback develops which increases the
discrepancy between the patients' and their
level of placebo effect as more and more
doctors' evaluation of efficacy. About 66 percent
improvement is experienced. This is the optimum
of the physicians reported good results but only
placebo effect which will remain high as long as
21 percent of the patients reported feeling better.
improvement is sustained. The physician is also
With this major disagreement, it is simple to un-
affected in the same way for, as he observes his
derstand why patients are so reluctant to take
patient's improvement, his faith in the treatment
them, and why there is such a development in
that he is giving goes up, which in turn is evident
slow-release, long-acting tranquilizers. Ap-
to his patient.
parently tranquilizers, either because they are
Generally there must be a sufficient placebo ineffective or because of undesirable side effects,
effect for the patient to start the treatment. This the do not sustain the placebo effect. A major
doctor achieves by his manner of confidence in problem in treating chronic schizophrenics is in
giving the diagnosis to his patient, in his keeping them on their tranquilizers. There must
explanations of it, and in describing the therefore be enough initial and sustained placebo
importance of the treatment and the outcome or reaction to ensure the medication will be taken.
prognosis. If the patient remains unconvinced, he
will not even start the therapy unless he has a In the normal situation the patient's personal
family who can persuade him to cooperate, and physician attempts to set placebo effect at a very
supervise the medication. If this is impossible, the high level. However, in a double-blind
patient will have to be hospitalized for treatment. experiment, if it remains truly blind, it is
The patient's initial faith may be zero, provided the impossible for the physician to do so. Since he
family can ensure the program will be followed. knows that the patient may receive an inert
Thus, patients have been given nicotinic acid or substance, he cannot honestly advise the patient
nicotinamide disguised in their food because they that this will be helpful to him. The patient will
were violently opposed to taking any medication, detect his physician's reluctance to be a real
and they have recovered. In the same way, a physician to him. This is even more apt to occur
pellagrin given nicotinamide in his food will also in situations where double-blind experiments are
recover whether or not he knows this is being run, i.e., in institutions, university wards, and so
done. on, for in many cases the patient does
not have a physician or does not know who he is. I have always displaced weaker drugs even before
have seen large numbers of patients after double-blinds became fashionable. If penicillin
discharge from these institutions who did not hadn't been developed we would still be using
know who had been their doctor. They had been sulfonamides. Finally, the fact that double-blinds
seen by medical students, internes, and residents, generally show active drugs to be inactive may
but very infrequently by the physician under be interpreted as showing that the technique is no
whose name they had been admitted and who was good since it fails to detect therapeutic activity in
finally responsible for treatment. By setting the compounds known to be active. Double-blinds
placebo effect at a minimum, the double-blind enhance obecalp reactions.
destroys one of the essential components of the
treatment. This is like measuring the rate of a Hogben and Wrighton, 1952, summarize their
chemical reaction, say an enzyme, which works point of view in this way: (1) Hitherto it has been
best at 37°C, by setting the reaction at 0°C. There customary to assess the claims of therapeutic and
may be a reaction but it will not help the patient prophylactic measures in statistical terms by
much. This is the main reason why double-blind recourse to tests which invoke a unique and so-
results have little relevance to the real therapeutic called null hypothesis, namely that the
effects of chemicals as commonly used by procedures compared are equally efficacious. (2)
physicians. This procedure has no bearing on the operational
intention of the trial, viz., to find out how much
advantage accrues from substituting one
(4) Perhaps the most serious criticism of the treatment for another. (3) Within its more
double-blind method is that it has not been restricted domain, the credentials of any
established empirically. There is no data which significance test which takes within its scope
shows that the double-blind method does control only one hypothesis have now to meet the
those factors which theoretically it was designed criticism that it takes into account only one sort
to control. We are therefore swept up in a of error, viz., that of rejecting the hypothesis
technique which has received overwhelming when it is true. (4) A procedure which justifies
approval by institutions and universities but which assertions of so limited and conditional a scope
has never been subjected to the experimental test may be a useful self-disciplinary convention; but
of whether it works. The discussion that I have its claims to rank as an instrument of statistical
presented so far indicates that when such a test is inference are no longer acceptable.
finally performed, it will show the double-blind
In spite of the mounting concern over the
not to be a useful method.
heavy reliance placed on results of double-blind
In science, a new technique is not used until it experiments by men like Glick and Margolis
has been calibrated, i.e., compared to a current (1962); Freyhan (1963) in his discussion of their
method. If it is more accurate, and more sensitive, paper; Bellak and Chasson (1964); Chasson
it will supercede the older method. If it has not (1957, 1959, 1960, 1961);
these advantages, it may still take over if it is more
economical, quicker, and so on. But in the double-
blind we find a method which is not established by 110
experiment, not proven to be better, more difficult
to run, and much more expensive, which has
displaced usual clinical trials. Credit must be given
to those research workers who have carried the
field with enthusiasm and dedication for they had
little else to bolster their position. They assumed
that, because chemical treatments replaced each
other, e.g., sulfonamides by penicillin, etc., and
because double-blind experiments generally
showed drugs considered active to be no better
than placebo, this proved (1) double-blinds were
superior, and (2) caused active drugs to replace
inactive ones. This is a major fallacy. Better drugs
Plutchik, Platman, and Fieve (1969); Baird Wittenborn (1973), and DeLiz (1973), all bearing
(1964, 1968); Cromie, (1963); Dalen (1969) upon the same double-blind therapeutic trial.
(who questions the power of statistical tests. They
Patients newly admitted to a mental hospital
are always used in double-blind experiments
but ill at least 4.8 years on the average were
where the null hypothesis is used. This, according
randomized into two groups. The randomization
to Popper, is only a test of the null hypothesis for
was imperfect since the nicotinic acid group had
only that hypothesis is in danger of being refuted);
been HI 4.8 years on the average and the control
Lasagna (1972); Feinstein (1970, 1971, 1972),
group had been ill for three years. The
Cotzias (1972). Many investigators will use
experimental group were given 3 grams per day
no other technique and depend upon
of nicotinic acid with or without tranquilizers,
these questionable methods to resolve therapeutic
depending upon the usual indications for
controversy. They seem to be unaware that
giving tranquilizers. No ECT was used. The
clinicians (doctors who work directly with
experiment was described as double-blind but no
patients) generally remain unconvinced by double-
evidence is given that it was double-blind, nor
blind experiments. Double-blind experiments
were there suggestions that the code had been
generally tend to prove active drugs to be
broken. Out of an initial group of 140, 75
inactive, i.e., yield obecalp responses. On the
completed two years of treatment. Out of 36
other hand, clinical trials in the older fashion may
patients who dropped out of the experimental
too often show inactive or slightly active drugs to
group, 20 or 58 percent were uncooperative. Out
be too active. I consider the latter error safer for
of 29 who dropped from the control group, 21 or
medicine. If every drug used today had been
72 percent were uncooperative. Out of 83
forced to be double-blinded at its inception I doubt
who started on nicotinic acid, 24 percent
that we would have insulin, thyroid, aspirin, an-
dropped out while out of 57 on placebo, 37
tihistamines, and other very valuable drugs. It has
percent were dropped because they did not
been claimed that the need to impress government
cooperate. This suggests that more placebo
agencies in the U.S.A. has substantially reduced
patients were suffering from negative placebo
the flow of new drugs onto the market. The
effects and might be explained by the fact that
government agencies will only accept double-
some of them were aware of the fact that they
blind studies.
were on placebo (DeLiz, 1973). Their general
The megadose vitamin B3 controversy is caught conclusion was that there was no therapeutic ef-
in the middle of this methodology controversy. It fect from the use of nicotinic acid.
was established on the basis of the first double-
In a more recent report, Wittenborn (1973)
blind experiments in psychiatry on acute and
concluded that patients with certain descriptive
subacute schizophrenics treated with ECT,
indices did respond well to nicotinic acid. Each
sedatives, and psychotherapy. The groups were
patient had been examined carefully before
randomized. One group received placebo. So far
treatment was started. From 111
no one has attempted to reproduce this experiment
using similar patients and similar treatment. We
also reported that nicotinic acid alone did not
benefit chronic patients such as are found in a
mental hospital (see O'Reilly, 1955). This we
reiterated in many of our reports.
However, in a series of studies over a five-year
period, investigators seemed unaware of these
conclusions and used subacute and chronic
patients in mental hospitals without ECT. They
concluded that nicotinic acid was not therapeutic
for schizophrenia. Hoffer (1971) reviewed the
reasons for their failure to obtain positive results.
In order to illustrate the inherent defects of the
double-blind design, I will review several reports
by Wittenborn, Weber, and Brown (1973),
social data so obtained 12 factors (items) reveals that persons whose premorbid history
discriminated between placebo and nicotinic acid suggested a participatory life style tend to return
as the study progressed. He concluded, "These to a participatory pattern of living after a year or
items appear to have a common implication for the more treatment with high levels of niacin. No
description of the premorbid personality of such reconstructive trend was indicated for the
patients who responded relatively well to the high control patients, however."
dosage of niacin. Thus niacin was most effective
"There is a conceivable relationship between
for those patients for whom some features of
the fact that in the present sample patients with a
definite interpersonal participation was found in
high predictive score responded well to niacin
the premorbid background."
and the fact that Hoffer and Osmond had claimed
The 12 items were combined into a predictor that niacin was more effective in relatively acute
scale ranging from 0.00 to 1.00. The 75 patients patients than in chronic patients. It is probable
who scored over 0.50 indicated easily evident that patients who, in the present sample, had a
pathology. The average scores indicated the high positive predictor score would have been
experimental group had more pathology than the classified by Hoffer and Osmond as acute
control group (consistent with the finding that they schizophrenics. Perhaps in this way the
had been sick longer). Patients with high predictor differential effect observed by them could be in
scores generally responded better to therapy. part explained."
When high-score patients (over 0.6) were "Why should a pretreatment disposition which
compared, the group on niacin responded better has a favorable significance for patients treated
than the placebo group. For example, at 24 months with niacin have an unfavorable significance for
the niacin group had half the depression of the patients not treated with niacin? One possible
placebo group. For schizophrenic excitement, the explanation for the paradoxical worsening in the
incidence of significant pathology was more than control group draws upon observations that many
twice as great in the placebo group. Paranoia and patients with a favorable premorbid history are
hebephrenia were twice as prevalent in the placebo harmed by phenothiazine treatment in the sense
group. During the discussion after presenting this that their remission is burdened."
paper, Wittenborn reported that two-thirds of the Here is illustrated one of the main defects of
niacin group were well compared to one-third of the double-blind method — its inapplicability to
the placebo group. He concluded, "A high positive heterogeneous groups. Since we had many times
predictor score was associated with a clinical pointed out that early cases responded better and
significant advantage for those patients who were without need for as many other chemotherapies,
treated with the high level of niacin medication." there was no
"The items of the follow-up inquiry which 112
showed the required consistency of relationship
with the predictor score in the experimental group
appear to be mutually consistent and describe the
kind of person who participates in ordinary
interpersonal interactions and who is approaching
at least some of the tasks and challenges of his life
constructively. The respective correlations based
on the control sample displayed in Table 3 do not
show a comparable or even an inverse set of
relationships with the predictor scores. Thus it
would appear that among the patients treated with
high-dosage niacin those patients with a high
positive predictor score have resumed the
constructive quality of adjustment which the high
positive predictor score implied for the premorbid
status of these patients.
"The present post hoc treatment of the data
need to use a mixture of early and well-developed Fortunately, double-blinds are inherently
cases. As a result, the double-blind by Wittenborn unconvincing to clinicians and to their patients
et al. yielded no significant difference. However, and will not long stand in the way of good
when the groups were purified (made more clinical observations made by Orthomolecular
homogeneous) by using indications which sorted physicians, by their patients, their families, and
the early from late cases, the differences in the by others who work with them.
early cases became highly significant. The chronic
cases should have been given ECT in combination In a personal communication, Dr. DeLiz
with the vitamin if it had been desired to repeat reported that when relatives of patients on
our original double-blind experiments. placebo complained, they were assured that this
was untrue and that the patients were having
However, this is not the only problem. Recently fantasies. This would alienate patient from
Dr. DeLiz submitted a critique of the double-blind family. DeLiz wrote, "It is rather easy to see how
experiment run by Wittenborn. This is published the secondary symptoms, real reactions to a
in this Journal. The most serious charge is that the contaminated social and psychological
experiment was not double-blind. Patients on environment as regards to niacin, were pervasive
placebo discovered this and felt they were being and might in their turn distort the totality of the
deceived (as they were), and at least one and psychiatric, social, and psychological rating
perhaps others tried to get niacin on their own. If procedures."
the patients knew, so must have some of the staff.
Since the preponderant feeling in most institutions It is evident that the double-blind method does
is violently opposed to megavitamins, it is easy to not solve the problem of proper trials and in fact
understand that there would be immense negative is probably worse than usual clinical observations
pressure against niacin and positive pressure in made by interested clinicians. Orthomolecular
favor of placebo. Obecalp would be strongly psychiatrists who value the welfare of their
favored. patients will be wise not to expose them to
The findings from the second study are experiments of this kind. It is possible to run
therefore even more significant, emerging from a comparison experiments where one program is
study where a double-blind design was not compared against another. The procedure is
adhered to and where obecalp reactions were simple, relatively inexpensive, and the results
favored by the staff. It is apparent that when would be decisive and convincing. But so far no
treatment was matched to the right patients, i.e. one in any University or research setting has
nicotinic acid and tranquilizers to acute patients shown much interest in this kind of study. We
(phase one), the superiority of nicotinic acid over will therefore have to depend on our clinical
placebo was amply evident in a setting favoring observations bolstered by psychological tests
obecalp responses. Two of the psychiatrists such as MMPI, HOD, and EWI, and by clinical
involved in the study eventually began to practice tests of hair, blood, and urine (kryptopyrrole, for
Orthomolecular psychiatry after leaving the study. example). 113
The main opposition to the Orthomolecular
approach has come from these so-called double-
blind experiments conducted by psychiatrists who
accept only double-blinds as scientific evidence.
The debate is not a debate between the physicians
who have used similar treatment methods and
have obtained conflicting results. This is the
normal kind of scientific debate. It is between two
sets of methodologies. It is clear that each group
can reproduce each other's work. The
Orthomolecular psychiatrist using any method as
well as double-blind gets similar results. The
double-blind methodologists also confirm each
other since they all use essentially the same
FREYHAN, F.: Discussion of Paper by Click and Margolis.
BAIRD, K. A.: Assessment of Drug Trials. Can. Med. Ass. J. 90,1279
only, 1964. CLICK, B. S. and MARCOLIS, R: A Study of the Influence '.'of
Experimental Design on Clinical Outcome in Drug Research.
BAIRD, K. A.: Medicines Domination by the Control Worshippers. Am. J. Psychiatry 118, 1087-10%, 1962.
Can. Doctor, 27 only, 1968.
HAWKINS, D. R. and PAULING, L: Orthomolecular Psychiatry.
BELLAK, L. and CHASSAN, J. B.: An Approach to the Evaluation W. H. Freeman and Co., San Francisco, 1973.
of Drug Effect During Psychotherapy: A Double Blind Study of a
Single Case. J. Nerv. Mental Dis. 139, 20-30,1964. HOFFER, A.: A Theoretical Examination of Double Blind Design.
Can. Med. Ass. J. 97, 123-127,1967.
CHASSAN, J. B.: On the Unreliability of Reliability and Some Other
Consequences of the Assumption of Probabilistic Patient States. HOFFER, A.: Megavitamin Therapy for Schizophrenia. Can.
Psychiatry 20, 163-171, 1957. Psychiat. Ass. J. 16, 499-504, 1971.
CHASSAN, J. B: On the Development of Clinical Statistical HOFFER, A. and OSMOND, H.: Double Blind Clinical Trials, j.
Symptoms for Psychiatry. Biometrics 15, 396-404, 1959. Neuropsychiatry 2, 221-227,1961.
CHASSAN, J. B.: Statistical Inference and the Single Case in Clinical HOFFER, A. and OSMOND, H.: Some Problems of Stochastic
Design. Psychiatry 23, 173-184, 1960. Psychiatry. J. Neuropsychiatry 5, 97-111, 1963.
CHASSAN, J. B.: Stochastic Models of the Single Case as the Basis HOGBEN, L: Statistical Theory: The Relationship of Probability,
of Clinical Research Design. Behavioral Science 6, 42-50,1961. Credibility and Error. George, Allen and Unwin, Ltd., London,
COTZIAS, G. C: Limitations of Controlled Double Blind Studies of
Drugs. New Eng. J. Med. 287, 937 only, 1972. HOGBEN, L. and WRICHTON, R.: Quoted in Hoffer and Osmond
CROMIE, B. W.: The Feet of Clay of the Double Blind Trial. The
Lancet 2, 994-997,1963. LASAGNA, L.: The Nature of Evidence: Triangle 11, 145-
DALEN, PER.: Causal Explanations in Psychiatry: A Critique of
O'REILLY, P. O: Nicotinic Acid Therapy and the Chronic
Some Current Concepts. Brit . J. Psychiatry 115, 129-137, 1969.
Schizophrenic. Dis. Nerv. System 16, 67-72, 1955.
DeLIZ, A. J: A Note of Criticism Concerning Wittenborn's Paper on
PLUTCHIK, R., PLATMAN, M.B., and FIEVE, R. R.: Three
an Experimental Double-Blind Research Design Dealing with the
Alternatives to the Double Blind. Arch. Gen. Psychiatry 20,
Action of Nicotinic Acid on Schizophrenia. J. Orthomolecular
428-432, 1969.
Psychiatry 2: 3, 1973.
WITTENBORN, J. R.: The Selective Efficacy of Niacin in the
FEINSTEIN, A. R.: What Kind of Basic Science for Clinical
Treatment of Schizophrenia. N.I.M.H. Conference, Washington,
Medicine? New Eng. J. Med. 283, 847-852, 1970.
FEINSTEIN, A. R.: Clinical Biostatistics IX. How Do We Measure
WITTENBORN, J. R., WEBER, E. S. P., and BROWN, M.: Niacin
Safety and Efficacy? Clin. Pharm. and Ther. 12, 544-558, 1971.
in the Long-Term Treatment of Schizophrenia, Arch. Gen.
FEINSTEIN, A. R.: The Need for Humanized Science in Evaluating Psychiatry 28, 308-315, 1973.
Medication. The Lancet 2, 421-423, 1972.

Use: 0.1386