“At what point shall we expect the approach of danger? By what means shall we fortify against it? Shall we expect some transatlantic military giant, to step the Ocean, and crush us at a blow? Never! All the armies of Europe, Asia and Africa combined, with all the treasure of the earth (our own excepted) in their military chest; with a Buonaparte for a commander, could not by force, take a drink from the Ohio, or make a track on the Blue Ridge, in a trial of a thousand years. At what point, then, is the approach of danger to be expected? I answer, if it ever reach us it must spring up amongst us. It cannot come from abroad. If destruction be our lot, we must ourselves be its author and finisher. As a nation of freemen, we must live through all time, or die by suicide.” Abraham Lincoln.Researchers from the U.S. Army and Thailand announced last month they had found the first vaccine that provided some protection against HIV. But a second analysis of the $105 million study, not disclosed publicly, suggests the results may have been a fluke, according to AIDS scientists who have seen it.
The second analysis, which is considered a vital component of any vaccine study, shows the results weren’t statistically significant, these scientists said. In other words, it indicates that the results could have been due to chance and that the vaccine may not be effective.
The additional data were available to the researchers on Sept. 24 when they announced the trial results, but they chose not to disclose them, said Jerome Kim, a scientist with the U.S. Army who was involved in the study. News of the second analysis was first reported on the Web site of Science magazine, but the story didn’t provide specific data. Full details of the trial are to be aired at an AIDS meeting in Paris that starts Oct. 19.
The incomplete disclosure raises the question of whether the Army, the Thai government and the U.S. National Institutes of Health — which helped fund the study — rushed to give a positive spin to what may turn out to be another inconclusive AIDS-vaccine effort.
“We thought very hard about how to provide the clearest, most honest message,” Dr. Kim said. “We stand by the fact that this is a vaccine with a modest protective effect.” He called the trial results “complex.”
Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases, the part of the NIH that oversees AIDS research, declined to comment.
The study was criticized as pointless by some AIDS scientists when it was launched three years ago because it combined a failed vaccine with one widely thought to have little chance of success. It was the largest HIV vaccine trial ever conducted, with more than 16,000 participants in Thailand.
Some AIDS researchers and activists who have learned of the additional data still think the vaccine shows promise and should be investigated further. But they worry that not disclosing the study transparently will cause people to conclude the vaccine trial was a failure and undermine support for more research.
“I would have preferred to have seen both results straight up. It might spring back on them, and that would be unfortunate,” said Mitchell Warren, executive director of the AIDS Vaccine Advocacy Coalition in New York.
About 33 million people were living with HIV world-wide in 2007, according to the latest United Nations statistics. About two million people died from AIDS that year. There have been more than 100 HIV vaccine trials since 1987. None had succeeded until the latest Thai trial.
The results announced last month were based on a “modified intent to treat analysis,” which includes virtually everyone who enrolled in the study, regardless of whether they ended up getting the full course of the vaccine. It is a good stand-in for the real world, where people don’t always follow instructions properly. By this measure, the vaccine tested in Thailand reduced by 31% the chance of infection with HIV, the AIDS-causing immunodeficiency virus.
But the result was derived from a small number of actual HIV cases: New infections occurred in 51 of the 8,197 people who got the vaccine, compared with 74 of the 8,198 volunteers who got placebo shots. Statistical calculations showed there was a 3.9% probability that chance accounted for the difference. In drug and vaccine trials, anything above a 5% probability of a chance result is deemed statistically insignificant.
The second analysis is called “per protocol” and adheres strictly to how the trial was designed by only including the study participants who got the full regimen of vaccine shots at the right time. Because it excludes study participants who didn’t get the full vaccine regimen, it usually provides corroboration to the looser “intent to treat” findings.
Two AIDS scientists, who have seen the “per protocol” analysis, said it indicates there is a 16% chance the study results were a fluke — a far greater probability than is considered statistically acceptable. This analysis included 86 people who received either the vaccine or a placebo and were infected. The “per protocol” analysis also showed that the supposed effectiveness was lower, at 26.2%. Dr. Kim, of the U.S. Army, declined to comment on the data. It isn’t clear why the vaccine was seemingly ineffective among participants who followed the guidelines to the letter.
These anomalous results sparked discussion last week at a meeting of the Center for HIV-AIDS Vaccine Immunology in Durham, N.C. The group is made up of a team of universities and academic medical centers established by the NIH to help vaccine design and development.
“I think in general it’s best to lay out as much data as possible,” said Barton Haynes, director of the center and an HIV vaccine expert at Duke University, who was at the meeting. “This is a very difficult situation for everyone, and we’ll have to wait until all the data are released so we can drill down into it.”
When drug or vaccine trials results are disclosed, it is common for investigators to simultaneously provide “per protocol” and “intent to treat” data. For example, when Merck & Co. announced the details of its failed HIV vaccine trial in 2007, the Whitehouse Station, N.J., company provided both sets of statistics at the same time.
In September, the AIDS Vaccine Advocacy Coalition published a report in anticipation of the Thai results that noted: “The safest route is to report both PP [per protocol] and ITT [intent to treat] and to analyze the difference.”
In January 2004, a group of 22 scientists in article in the journal Science noted that one component of the Thai vaccine, a primer dose made by Sanofi Pasteur, a division of Sanofi-Aventis SA of France, was poor at triggering an immune response. They also pointed out that trials of the second component of the Thai vaccine, a booster component now licensed to Global Solutions for Infectious Diseases, of South San Francisco, Calif., had been proven “to be completely incapable of preventing or ameliorating HIV-1 infection.”
They added: “One price for repetitive failure could be crucial erosion by the public and politicians in our capability of developing an effective AIDS vaccine collectively.”
