The Wall Street Journal’s “numbers guy” Carl Bialik examines the new mammogram recommendations recently issued by the U.S. Preventive Services Task Force–specifically the section in the report which states that 1,900 women in their 40s would need to be screened annually to save one life, compared with 1,300 women in their 50s.  Bialik finds that the research incorporates wide margins of error. 

The 1-out-of-1,900 figure was derived from comparing breast-cancer death rates in women who were being screened with those who weren’t.  

The task force amassed results from various trials, using a statistical technique for combining research studies called meta-analysis, to arrive at the 1-in-1,900 figure. But that figure comes with massive uncertainty. Because the women in these studies represented a sample of all women, the results have a margin of error, as with political polls. 

Yet the uncertainty here wouldn’t pass muster at Gallup. Researchers said there was a 95% chance that to avert one cancer death, somewhere between 900 and 6,000 women in their 40s would have to be screened, though somewhere around 1,900 was their best estimate. That range is so broad that it is possible far more lives are saved with annual screening among 40-year-olds than the studies indicated.  

Bialik also looked at the studies and trials the panel included in their results: 

The panel didn’t factor in any so-called observational studies, where instead of randomly assigning women to receive or not receive screening, women were tracked after making their own decisions about mammograms. Such studies are considered inferior because, without randomization, other factors could affect improved survival for women who are screened, such as better overall health and health care. 

Robert A. Smith, director of cancer screening for the American Cancer Society, says failing to include results from observational studies was a missed opportunity. He noted a 2003 study he co-wrote in the Lancet medical journal with a longer follow-up period that found that only 726 women in their 40s must be screened to save one life. 

Read his entire examination of the “1-in-1900” number here.