Search, rate, commentBehind biomedical publishing’s new forum for commenting on journal articles

What would you do if you saw an error in a medical journal article? If you’re biostatistician Rob Tibshirani, PhD, you’d try to correct it. And then, because there’s no easy way to do that, you’d get frustrated and think, “Wouldn’t it be nice if there were a website where you could let people know about mistakes in journal articles?”

Photograph: Norbert Von Der Groeben

Rob Tibshirani helped launch a commenting system on PubMed, one of the world’s largest indexes of scientific literature.

Tibshirani, a Stanford professor of health research and policy, had this thought most recently after noticing what seemed to be a serious drawback with a new method, published in Science, for analyzing large data sets. So he asked Pat Brown, PhD, professor of biochemistry, what he thought: Was a website for commenting on journal articles feasible? Brown was a good person to ask. He co-founded Public Library of Science, an open-access publishing venture, in 2000 and started one of the first open-access journals, PLOS Biology.

“Pat told me PLOS had tried it, but didn’t have enough traffic to make it work,” Tibshirani says. “He suggested we try it at a place with a lot of traffic: PubMed.”

PubMed, the massive literature index run by the National Institutes of Health’s National Center for Biotechnology Information, is one of the Internet’s busiest biomedical websites, with millions of users a day.

Over Skype, Tibshirani and Brown pitched the idea to David Lipman, MD, the NCBI’s director. Lipman, in fact, had been thinking about doing something like this for nearly a decade. “But I was concerned how we could manage it, and until recently I didn’t think it would get the support of NIH leadership,” he says.

Now he thought it was worth a try. So Tibshirani became the community organizer, recruiting 300 people to discuss the ground rules and test an early version of the site, and Lipman proposed it to the leaders of the NIH. The result is PubMed Commons, which opened to a wide audience in the fall of 2013. The site, http://www.ncbi.nlm.nih.gov/pubmedcommons/, allows all authors of articles indexed in PubMed to make comments on any article indexed there. The comments, which numbered roughly 600 as of January, can be read by anyone with an Internet connection.

It’s a step forward for what’s been termed “post-publication peer review,” which many scientists argue will speed scientific progress. The development team hopes it will become not only a place for criticism but for discussion — for questions about techniques, for suggestions and for praise.

In some ways, it’s surprising that the NIH leaders have let comments go forward, Tibshirani says. He knew there was concern that comments could have financial ramifications, be used to attack competitors or harm the reputation of NIH-funded research.

To keep commenters responsible and to make any potential conflicts of interest transparent, no anonymous comments are allowed, and commenting is restricted to the scientific community, at least during the pilot period.

During Tibshirani’s discussions with researchers as the site was being developed, he found that 10 to 20 percent of them worried about its repercussions, or just didn’t see the point. “They say you could trash someone’s reputation,” Tibshirani says.

But is that so bad?

“My thought is if someone publishes something really wrong, maybe their reputation shouldn’t be trashed but it should be at least tarnished. Should the facts not come up? I think yes, they should. It’s in everyone’s interest.”

— Rosanne Spector