THERE IS NOW CONFIRMATION THAT THE CNBSS BREAST CANCER SCREENING TRIALS WERE COMPROMISED AND SHOULD NOT BE USED TO DETERMINE BREAST CANCER SCREENING GUIDELINES
PREFACE
If investigators ran a breast cancer treatment trial in which women who were treated with the chemotherapy were compared to women who were not:
and
1. they chose to study an obsolete, outdated chemotherapeutic agent (corresponding to the obsolete mammography used in the CNBSS)
and
2. they first determined which women had advanced cancers prior to assigning them to one group or the other
and
3. they then assigned the women in notebooks so that the women with more advanced cancers could be assigned, out of random order, in larger numbers to the treatment arm compared to the controls
and
4. they found that the therapy did not result in lives saved compared to the untreated control women
and
5. the investigators claimed “there was no benefit from any form of chemotherapy”
the results would have never passed peer review and would have never been published!
This would be the correct action for a corrupted treatment trial.
How is it that you can run a breast cancer screening trial where you claim to compare women being screened every year using obsolete mammography and a clinical breast examination to women who receive the “usual care” – find a lump and see your doctor, where :
1. You recruit volunteers so that they do not represent the general population
and
2. You allow and recruit women into the trial with clinical evidence of breast cancer who cannot benefit from screening
and
3. You first perform a clinical breast examination on the women and identify those with clinical evidence of breast cancer and those with evidence of advanced breast cancer (large lymph nodes in their underarms)
and
4. You assign women on open lists so that you can put any woman into whichever group you choose
and
5. You place more women with advanced cancers, out of random order, into the screening arm
and
6. You use outdated, poor quality mammographic techniques that miss cancers, to screen the women in the screening arms of the trial
and
7. You have more women die from breast cancer in your screening arm in the first 7-10 years of your trial and ultimately find no benefit from screening
and
8. Contrary to the results from all the other screening trials, you conclude that there is no benefit from screening for any women ages 40-60
and
9. Your studies get published and, despite major violations of the rules for Randomized, Controlled Trials, are claimed to be “gold standard studies” and have been used by guidelines panels to deny women, particularly in their 40’s, easy access to screening
Why is the science-based approach correct in treatment trials, but has not been applied to the breast cancer screening trials known as the Canadian National Breast Screening Studies (CNBSS)?
INTRODUCTION
In our recent paper “The randomized trial of mammography screening that was not—A cautionary tale” published in the Journal of Medical Screening” (1) we examine the concerns that have been raised over the years about the Canadian National Breast Screening Studies (CNBSS) and the consequences of the unblinded allocation process that was used in those trials. Although the investigators prevented the coordinators from being interviewed over the years, there has now been eyewitness testimony that women with clinically evident cancers (who could not benefit from screening) were recruited into these trials of screening, and many were placed in the screening arms (to receive outdated mammography) out of random order imbalancing the trials and “loading” the sides against screening and invalidating the results.
*************
FUNDAMENTALS
A randomized, controlled trial (RCT) is the most rigorous, scientific method for determining the efficacy of a medical intervention. RCT are the gold standard for evaluating new approaches to treatment. The RCT’s of breast cancer screening, with the exception of the National Breast Screening Studies of Canada, have proven that early detection saves lives for women ages 40-74 which are the ages of the women who participated.
There were two RCT’s performed in Canada in the 1980’s that have been called the Canadian National Breast Screening Studies (CNBSS). CNBSS1 evaluated screening for women ages 40-49 and CNBSS2, using a different protocol, did the same for women ages 50-59. The results from these Canadian trials have turned out to be major “outliers” since, unlike the other trials, they did not show a benefit for screening women at any age. Their results have been used by guidelines panels to deny women access to screening, particularly those ages 40-49. For over three decades there have been major concerns about the design, execution, and the results from the CNBSS that have called into question their negative results. A simple solution would have been to have an impartial third party interview the coordinators who assigned the women and given them protection from retribution, to find out what actually took place. I actually urged this when two reviewers were asked to review the trials (2). However, the investigators for the CNBSS have always prevented the coordinators from being interviewed. Eyewitnesses have just recently come forward to provide testimony that major violations did, in fact, occur in these trials and these explain the published imbalances. The trials were corrupted rendering their results unreliable.
The reason that RCT’s are so important and why strict adherence to “rules” is critical is that their validity relies on taking a large population of women and “randomly” dividing them to create two identical groups. Random allocation is absolutely critical. RCT require that if nothing else was done, the same number of women would develop breast cancer, and the same number of women would die from breast cancer in both groups. Essentially, every woman in the screening group has a twin in the control group. To accomplish this, random allocation is fundamental and absolutely critical.
To try to ensure random allocation, nothing can be known about the participants prior to assignment (blinding) that could inadvertently or intentionally be used to imbalance the groups. The ideal RCT for screening identifies a population of women, and, before anything else is known about them, randomly divides them. Following allocation, the women in one of the groups are invited to participate in the screening program. The other women serve as the unscreened controls. Over time, deaths among the screened women are compared to deaths among the control women who were not being screened. If the trial is done properly, and there are fewer deaths among the screened women, and the difference in deaths between the two groups is “statistically significant”, then this is “proof” that screening and early detection saves lives.
THE CNBSS VIOLATED THE FUNDAMENTAL RULES FOR RCT’s
In order for the results of RCT’s to be valid, blinded allocation is essential. Unfortunately, this fundamental rule was ignored by the CNBSS. Contrary to the rules, women who volunteered to be in the CNBSS had a clinical breast examination prior to being assigned to one group or the other. Women who had evidence of breast cancer were identified. Despite the fact that they could not benefit from screening they were allowed to participate. Unfortunately, the results of the CBE were conveyed to the coordinators who were to assign the women. These violations were compounded by the fact that assignment to one group or the other was on open lists. The coordinators could place any woman, in either group, out of random order. This would never be allowed in RCT’s of treatment or any other RCT because it provides an untraceable potential to compromise random assignment.
The data published by the CNBSS investigators have always suggested that the violations of the rules for RCT’s, documented by a review of the trials (3), had, indeed, resulted in an imbalance of the participants and a “loading” of the screening side with women who had more cancers and more advanced cancers. This has been denied by the investigators, but they have persistently refused to allow the coordinators of these trials to be interviewed to find out if non-random allocation took place. Our paper now confirms, with eyewitness testimony by a technologist who worked in the trial, that women with clinical evidence of cancer, who could not benefit from screening, were recruited to be in the trials and that some of these women with clinical evidence of cancer were assigned, out of random order, to the screening arms. This explains why, unlike any of the other screening trials, there were more women with cancer in their axillary lymph nodes (advanced cancers), and, consequently, there were more deaths among the screened women in at least the first 7 years in CNBSS1 (4).
The Canadian trials are the only ones that have failed to show a benefit from mammography screening. Their negative results have been used by panels such as the United States Preventive Services Task Force (USPSTF), that set screening guidelines, to downplay the value of screening.
HISTORICAL CONCERNS
From the beginning of the CNBSS (performed in the 1980s) concerns were raised about the poor quality of the mammograms in the trial. The equipment was far from state-of-the-art. There was no training for the radiologists and no training for the technologists, and all evaluations of the quality of the mammograms have shown them to be poor to unacceptable for much of the trials. The poor quality is reflected in the fact that only 32% of the cancers detected in the screening program were detected only by mammography. This was worse than in a screening trial undertaken ten years earlier called the Breast Cancer Detection Demonstration Project (BCDDP), using even older equipment, in the 1970’s, in which 42% of the cancers were detected by mammography alone. It is fairly telling that none of the radiologists who participated in the CNBSS trials has come forward to defend the quality of the mammography. Even the trials reference physicist was concerned that
"..in my work as reference physicist to the NBSS, [I] identified many concerns regarding the quality of mammography carried out in some of the NBSS screening centers. That quality [in the NBSS] was far below state of the art, even for that time (early 1980's).” (5).
Problems with the quality of mammography resulted not only from inadequate equipment in some cases, but also from inappropriate imaging technique and lack of availability of specialized training for technologists and radiologists. A review undertaken by the trial leaders, in which I was invited to participate, confirmed that the quality of the mammography was poor to unacceptable for much of the trials (6,7). The fact that the mammography being used was obsolete, even for that period, has been ignored by guidelines Panels because they have intentionally excluded experts in mammography. I have no doubt that inexpert panel members, incorrectly assumed that “a mammogram is a mammogram”.
THE DATA HAVE SHOWN THE PRIMARY PROBLEM
Regardless, the problems with the CNBSS are even more fundamental. The foundation of RCT’s lies in the random allocation of women to one group or the other. If this is compromised, and the distribution of cancers is not random but weighted toward one group of the other, then the trial results will be invalid. As noted earlier, nothing can be known about the women participating in the trials prior to their assignment, that could unintentionally, or intentionally be used to bias the assignment of more women who will, ultimately, die from breast cancer to one group or the other. If we look at the extreme, if all of the women with incurable cancer were assigned to the screening arm, the trial would be doomed to failure before it even began. Random allocation is fundamental and critical for valid RCT’s.
The CNBSS violated the most critical rules for RCT’s. – “blinded allocation”. If you place more women who are destined to die in one arm than the other there will be more women who will die in that arm, and the RCT results will be invalid.
As the trials published more and more results over the years concerns were raised that there had been violations of random allocation that skewed the results. Among the women ages 40-49 there were 24 women with advanced cancers at the time of the first screen. 19 of the 24 women with advanced breast cancers were assigned to the screening arm and only 5 were assigned to the control arm (a statistically significant difference). The investigators falsely claimed that mammogrpahy finds “more of everything”, ignoring the fact that 17 out of the 19 cancers were evident on the clinical examination.
Other published data strongly suggested an allocation imbalance.
1. Unlike the other screening trials there were almost twice as many women with lymph node positive cancers (cancer in the lymph nodes in the underarm) assigned to the screening arms (8).
2. The 5-year survival rate for women in the unscreened control arm of CNBSS1 was greater than 90% at a time when the five-year survival in Canada was only 75%. This was likely due to the fact that the women with advanced cancers, who would have died early in the control arm, were, instead, assigned out of random order to the screening arm.
3. In the first 7 years of CNBSS1 there were more deaths among women in the screening arm than the control arm again likely due to the fact that more women with advanced cancers were assigned out of random order to the screening arm.
4. As noted earlier, only 32% of the cancers detected in the screening arms were detected by mammography alone. Not only does this support the fact that the mammography, in a trial of mammography screening, was completely inadequate, but it also supports the fact that many clinically evident cancers were likely allocated out of random order to the screening arms increasing the number of cancers evident clinically.
5. Concerns about allocation imbalances have been raised for years. In 1995 I wrote to Brian MacMahon who was recruited to review the allocation process (3):
“I would strongly urge that you interview those involved in examining the women, as well as those involved in assigning them to the study or control group. I suspect, however, that, if the process was compromised, individuals will be reluctant to admit to actions that may have such important consequences. Either they have to be assured of complete confidentiality (to avoid any retribution) or they must be interviewed under some form of oath.”
Unfortunately, this was never done. The CNBSS investigators never allowed the coordinators and nurses to be interviewed with protection from any retaliation. This strongly suggests that there were major issues in the allocation process that they did not want to be known.
AN EYEWITNESS HAS NOW CONFIRMED THAT WOMEN WERE ALLOCATED OUT OF RANDOM ORDER
A technologist who participated in the CNBSS has just recently come forward and provided eyewitnesses testimony that women were, in fact, assigned out of random order and women with likely cancers were assigned in larger numbers to the study group.
“If the patients in fact had any sort of abnormal finding- they were put into the Mammo arm of the study. If the Nurse added a patient- then they would quite often remove another patient that had been on the list to have a Mammo later that day. Unless of course that person also had a finding.
When I questioned this the Nurses informed me that it was fine for them to add people & that the patient needed a Mammo due to findings. Who was I to question- as they had already been doing this for a long time before my arrival as was pointed out to me.”
“The Nurses were very much of the thought that the study was to confirm the importance of the physical component of the study versus the Mammo arm of the study.”
“Therefore the majority of these women I did Mammograms on- already had a pre-existing HX of multiple cysts, aspirations or something felt in past/ present that day.”
The investigators have argued that the demographic information was similar in both groups supporting random assignment. However, in trials totaling over 80,000 women, shifting even as many as 100 or more women with cancers from the control arm to the screening arm would have no effect on the demographics.
HOW WAS THIS ALLOWED?
I have no doubt that nonrandom allocation was not the intent of the investigators, but the study design protocols allowed it to happen. I suspect that the coordinators and nurses who had no previous experience with RCT’s had good intentions and did not realize they were compromising the trial. Nevertheless, the protocols permitted nonrandom allocation; the data have always suggested that this happened; and now we have firsthand testimony that this took place.
The fact is that the CNBSS protocols violated the fundamental guidelines for RCT’s. The CBE prior to assignment first identified women with clinically evident advanced cancers and then, as the published data clearly show, women were assigned out of random order causing the imbalances noted. The fact that women with clinically evident cancers were recruited to participate in the CNBSS and many were assigned out or random order has now, also, been verified by eyewitnesses. These violations have hopelessly corrupted the results. It is now clear that the CNBSS trials were fundamentally compromised, and the trials’ results are unreliable. They should not be used to determine screening guidelines.
REFERENCES
1. Yaffe MJ, Seely JM, Gordon PB, Appavoo S, Kopans DB. The randomized trial of mammography screening that was not-A cautionary tale. J Med Screen. 2021 Nov 23:9691413211059461. doi: 10.1177/09691413211059461. Epub ahead of print. PMID: 34812692.
2. Kopans DB. NBSS: opportunity to compromise the process CMAJ 1997;157:247
3. Bailar JC, MacMahon B, Randomization in the Canadian National Breast Screening Study: A Review for Evidence of Subversion. Can Med Assoc J 1997;156:193-199.
4. Miller AB, Baines CJ, To T, Wall C. Canadian National Breast Screening Study: 1. Breast cancer detection and death rates among women aged 40 to 49 years. CMAJ. 1992 Nov 15;147(10):1459-76. Erratum in: Can Med Assoc J 1993 Mar 1;148(5):718 Miller AB, Baines CJ, To T, Wall C. Canadian National Breast Screening Study: 1. Breast cancer detection and death rates among women aged 40 to 49 years. CMAJ. 1992 Nov 15;147(10):1459-76. Erratum in: Can Med Assoc J 1993 Mar 1;148(5):718
5. Yaffe MJ. Correction: Canada Study. Letter to the Editor JNCI 1993;85:94
6. Baines CJ, Miller AB, Kopans DB, Moskowitz M, Sanders DE, Sickles EA, To T,Wall C. Canadian National Breast Screening Study: assessment of technical quality by external review. AJR Am J Roentgenol. 1990 Oct;155(4):743-7.
7. Kopans DB. The Canadian Screening Program: A Different Perspective. AJR 1990 155:748-749
8. Miller AB, et al. Canadian National Breast Screening Study: 1. Breast cancer detection and death rates among women aged 40 to 49 years. CMAJ. 1992 Nov 15;147(10):1459-76. Erratum in: Can Med Assoc J 1993 Mar 1;148(5):718
