New York Times
June 29, 2013
Breaking the Seal on Drug Research
By KATIE THOMAS
PETER DOSHI walked across the campus of Johns Hopkins University in a rumpled polo shirt and stonewashed jeans, a backpack slung over one shoulder. An unremarkable presence on a campus filled with backpack-toters, he is 32, and not sure where he’ll be working come August, when his postdoctoral fellowship ends. And yet, even without a medical degree, he is one of the most influential voices in medical research today.
Dr. Doshi’s renown comes not from solving the puzzles of cancer or discovering the next blockbuster drug, but from pushing the world’s biggest pharmaceutical companies to open their records to outsiders in an effort to better understand the benefits and potential harms of the drugs that billions of people take every day. Together with a band of far-flung researchers and activists, he is trying to unearth data from clinical trials — complex studies that last for years and often involve thousands of patients across many countries — and make it public.
The current system, the activists say, is one in which the meager details of clinical trials published in medical journals, often by authors with financial ties to the companies whose drugs they are writing about, is insufficient to the point of being misleading.
There is an underdog feel to this fight, with postdocs and academics flinging stones at well-fortified corporations. But they are making headway. Last fall, after prodding by Dr. Doshi and others, the drug giant GlaxoSmithKline announced that it would share detailed data from all global clinical trials conducted since 2007, a pledge it later expanded to all products dating to 2000. Though that data has not yet been produced, it would amount to more than 1,000 clinical trials involving more than 90 drugs, a remarkable first for a major drug maker.
The European Medicines Agency, which oversees drug approvals for the European Union, is considering a policy to make trial data public whenever a drug is approved. And on June 17, the medical world saw how valuable such transparency could be, as outside researchers published a review of a spinal treatment from the device maker Medtronic. The review, which concluded that the treatment was no better than an older one, relied on detailed data the company provided to the researchers.
For years, researchers have talked about the problem of publication bias, or selectively publishing results of trials. Concern about such bias gathered force in the 1990s and early 2000s, when researchers documented how, time and again, positive results were published while negative ones were not. Taken together, studies have shown that results of only about half of clinical trials make their way into medical journals.
Problems with data about high-profile drugs have led to scandals over the past decade, like one involving contentions that the number of heart attacks was underreported in research about the painkiller Vioxx. Another involved accusations of misleading data about links between the antidepressant Paxil and the risk of suicide among teenagers.
To those who have followed this issue for years, the moves toward openness are unfolding with surprising speed.
“This problem has been very well documented for at least three decades now in medicine, with no substantive fix,” said Dr. Ben Goldacre, a British author and an ally of Dr. Doshi. “Things have changed almost unimaginably fast over the past six months.”
Much of that change is happening because of what Dr. Goldacre calls an “accident of history.” In 2009, Dr. Doshi and his colleagues set out to answer a simple question about the anti-flu drug Tamiflu: Does it work? Resolving that question has been far harder than they ever envisioned, and, four years later, there is still no definitive answer. But the quest to determine Tamiflu’s efficacy transformed Dr. Doshi and others into activists for transparency — and turned the tables on drug makers. Until recently, the idea that companies should routinely hand over detailed data about their clinical trials might have sounded far-fetched. Now, the onus is on the industry to explain why it shouldn’t.
IN summer 2009, Dr. Doshi received a call from Dr. Tom Jefferson, a British epidemiologist based in Rome. That year, the swine flu pandemic was spreading worldwide, and Dr. Jefferson had been hired by the British and Australian governments to update an earlier review of Tamiflu, a drug produced by the Swiss company Roche, aimed at reducing the flu’s severity and preventing more serious complications. He asked if Dr. Doshi wanted to help.
Determining Tamiflu’s efficacy had significant economic as well as health consequences. Around the world, private companies and governments — including that of the United States — were stockpiling Tamiflu in case of influenza outbreaks, and their spending accounted for almost 60 percent of the drug’s $3 billion in sales in 2009.
The review of Tamiflu was being conducted under the auspices of the Cochrane Collaboration, a well-regarded network of independent researchers, including Dr. Jefferson, who evaluate medical treatments’ effectiveness by analyzing all available research.
At the time, Dr. Doshi knew little about clinical trials or even much about the drug industry. But he knew Dr. Jefferson. Dr. Doshi, after receiving undergraduate and master’s degrees in anthropology and East Asian studies from Brown and Harvard, had shifted focus and was pursuing a doctorate at M.I.T., studying the intersection of medicine and politics. He met Dr. Jefferson, a prominent skeptic of the flu vaccine, after researching whether the Centers for Disease Control was exaggerating the deadliness of the disease.
“We were both lone wolves in the field of influenza,” Dr. Doshi recalled.
Dr. Jefferson had conducted a Cochrane review of Tamiflu’s effectiveness a few years earlier, concluding that the drug reduced the risk of complications from the flu. He assured Dr. Doshi and other researchers on his team that the update would be fairly simple.
But just as their work was getting under way, a simple comment arrived on the Cochrane Web site that changed the course of the research and would ultimately fuel a worldwide effort to force drug companies to be more transparent.
The author of that comment, Dr. Keiji Hayashi, had no connection to the Cochrane group; he was a pediatrician in Japan who had prescribed Tamiflu to children in his practice, but had come to question its efficacy. He was curious about one of the main studies on which Dr. Jefferson had relied in his previous analysis. Called the Kaiser study, it pooled the results of 10 clinical trials. But Dr. Hayashi noticed that the results of only two of those trials had been fully published in medical journals. Given that details of eight trials were unknown, how could the researchers be certain of their conclusion that Tamiflu reduced risk of complications from flu?
“We should appraise the eight trials rigidly,” Dr. Hayashi wrote.
Reviews by the Cochrane group are known for being among the most thoroughly researched medical analyses available. But in trying to answer the pediatrician’s question, Dr. Jefferson realized that there was a flaw: they relied too heavily on the assumption that the articles published in journals accurately represented the results of all clinical trials that had been conducted.
As he tried to track down the authors of the Kaiser study and the two published trials, Dr. Jefferson said he hit dead ends: One author said he had moved offices and no longer had the files; another said he had never seen the primary trial data, instead relying on a summary analysis provided by Roche. All the authors suggested that he contact the company.
“We took it on faith — on trust,” Dr. Jefferson, 59, said recently in a phone interview. Dr. Hayashi’s question had tested that faith. Dr. Jefferson began typing each new discovery in a private journal he called Hayashi’s Problem, which, he said, “charted my transformation from Dr. Jekyll to Mr. Hyde.”
Dr. Doshi said that medicine “relies on hierarchies of trust.” He added: “A patient is not going to be in a position to review the entire evidence base themselves. But they trust that there is a watchdog out there.”
As they dug into the Tamiflu research, Dr. Doshi said, he realized that such a watchdog didn’t exist. Instead, he said, “we have partial watchdogs who see part of the full picture.” It became his mission to see the full picture.
Having struck out with the authors of the disputed Kaiser paper and the two other published trials, Dr. Jefferson approached Roche itself, asking for the underlying data from the missing trials. But when he declined to sign a confidentiality agreement, Roche decided not to cooperate with the researchers.
Without more complete data about the clinical trials, the Cochrane group decided that it could not include the disputed study that summarized those results. In December 2009, the team reported that Tamiflu could not be shown to reduce complications like pneumonia or hospitalizations.
The British Medical Journal, which printed the team’s conclusions, also published its own investigation, showing that Roche had hired ghost writers to author some of the articles involving Tamiflu, and that those writers had said they were under pressure to highlight positive messages about the drug. Roche responded that hiring such writers was common industry practice at the time of the articles, and it rejected the idea that they had been pressured to write positively about the drug.
The articles in the British journal created a sensation, and the Cochrane Collaboration’s efforts became a cause célèbre. “Everyone knows about publication bias,” said Dr. Steven Woloshin, a professor of medicine at the Dartmouth Institute for Health Policy and Clinical Practice and an advocate of more widespread sharing of clinical trial data. “But they just had so much energy and they brought so much attention to it.”
The group’s efforts seemed to make a difference: After the articles in the British journal, Roche turned over partial copies of study reports, amounting to a little more than 3,000 pages. Then, in 2011, the European Medicines Agency turned over more than 22,000 pages of documents for 19 trial reports to Dr. Jefferson and his team.
The door had been opened. As they read through the records, the researchers discovered the importance of documents called clinical study reports, which are thousands of pages long and contain details as varied as descriptions of trial protocol and design and the ingredients of the placebo pills.
“We used to know that there was a published paper and there were data behind it,” said Dr. Fiona Godlee, the editor of the British Medical Journal. “But people haven’t talked about these things, like clinical study reports, that are now being talked about a great deal.” Last fall, the journal said it would publish the results of clinical trials only if drug companies and researchers agreed to provide data upon request.
In April, Roche said it would make available to the Cochrane researchers clinical study reports for all Roche-sponsored trials of Tamiflu. Dr. Jefferson, Dr. Doshi and their colleagues hope to complete another update to their review of the drug by year-end.
Some said it was a shame that it took this long for the company to relent. “All these years later, and we still don’t know if Tamiflu is effective,” said Dr. Harlan Krumholz, the Yale cardiologist who oversaw the review of Medtronic’s bone treatment. “It’s perplexing to have a billion-dollar drug, and you’re still not willing to share everything you’ve got to know whether this thing is effective and safe.”
THOUGH the Tamiflu question is not yet resolved, the Cochrane researchers have succeeded in a bigger way: by helping to change the conversation around companies’ responsibility to reveal drug trial data.
Drug companies do not always credit the Cochrane Collaboration. In February, Roche followed Glaxo’s lead and announced that it, too, would release detailed clinical data to outside researchers, upon request. But Daniel O’Day, chief operating officer of pharmaceuticals at Roche, denied that its pledge had been motivated by the Tamiflu experience. He said Roche has provided data to “thousands” of researchers.
Mr. O’Day said “there were probably errors on both sides” in how the Cochrane researchers and Roche communicated with each other, and said the relationship deteriorated after Dr. Jefferson refused to sign a confidentiality agreement. He said the company was trying to rebuild its relationship with the Cochrane researchers, but that it stood by the safety and efficacy of Tamiflu.
In 2010, Roche commissioned researchers at the Harvard School of Public Health to conduct a re-analysis of Tamiflu clinical data, which largely confirmed the positive conclusions of the Kaiser study.
Mr. O’Day asserted that the company’s transparency pledge had arisen from “the call from society in general for greater transparency of the clinical trials that we have.” But others say the Cochrane researchers are largely responsible for that call for transparency.
Andrew Witty, Glaxo’s C.E.O., said in an interview that his promise to provide detailed clinical data had grown out of a companywide effort, initiated soon after he became chief in 2008, that would “really ensure that we were more in step with where I thought, frankly, society and the world was moving.”
Glaxo, moreover, was in need of an image rehabilitation. Last year, it pleaded guilty to criminal charges and agreed to pay $3 billion in fines after the United States Justice Department accused the company, based in London, of failing to report safety data about its diabetes drug Avandia, and of publishing misleading information about Paxil, the antidepressant, in a medical journal. The settlement, which also included civil penalties over marketing of other drugs, was the largest ever involving a pharmaceutical company.
“We don’t see any reason for this information to be held out of the public domain,” Mr. Witty said, “provided that the people who are interrogating the information are legitimate researchers with a legitimate question to ask.”
In a twist, Roche now finds itself on the same side as the Cochrane researchers — and against many in its own industry — in a debate over what kind of data the European Medicines Agency should be making public. On Monday, the agency released a draft policy, expected to take effect next year, in which it would release clinical trial data whenever it approved a new drug. While Roche and Glaxo have supported the policy, the Pharmaceutical Research and Manufacturers of America, a major industry group, and other drug companies have opposed it.
John J. Castellani, chief executive of PhRMA, said the industry had championed open-source efforts to develop better methods for testing cancer drugs, for example. But proposals like those from the Cochrane team and the European agency go too far, he said.
“If you dump onto the sidewalk all the data and you include commercially protected information,” he said, “then you’re essentially giving to competitors what we invested billions of dollars in.”
Others warned that such a policy could discourage drug companies from investing in Europe. “If you, on the other hand, say, ‘You guys are bad actors, we want to cut your prices, we want to take your confidential data and share it with any one of your competitors,’ you don’t get the same feeling of encouragement,” Christopher A. Viehbacher, C.E.O. of the French pharmaceutical company Sanofi, told reporters in Brussels on Monday, according to Reuters.
Industry officials and regulators in the United States say the public already has access to vast amounts of information about clinical trials. The basic results of all clinical trials must now be registered in a federal clearinghouse, for example, and the Food and Drug Administration publishes staff reviews and other documents when it approves a new drug. The F.D.A. has said that it is monitoring the developments in Europe but that federal laws in the United States restrict what types of information can be released, particularly data that could reveal personal or commercially confidential information.
Cochrane group members point to the Medtronic study as an example of the value of a neutral perspective.
In 2011, Medtronic awarded a $2.5 million grant to Yale and asked it to oversee a detailed review of trial data for Infuse, a bioengineered material in spinal fusions to treat back pain. The company was facing claims that it had published misleading information about the treatment, and it turned over its data in an effort to address those criticisms. Two teams that examined the data came to similar conclusions: Infuse appeared to be no better than an older treatment, and may pose added risks.
EARLIER this month, Dr. Doshi opened what he hopes will be a new chapter in his quest for greater understanding of clinical trials. He and several other researchers published what amounted to an ultimatum to drug companies: publish your data, or we’ll do it for you.
Under the plan, researchers would publish articles summarizing trial results in cases where the underlying data has already been released. In isolated cases, such information has been made public through litigation and Freedom of Information Act requests.
“It’s really neat to see a larger opportunity for a larger impact,” he said. “Tamiflu just happened to be the lever that opened that door.”