• methodology
• health policy
• real world evidence

COVID-19 has transformed our world. None of us alive have ever seen anything quite like it in scale, except the very few who still remember World War 2. Like that conflict, the pandemic has elicited responses from governments unparalleled in scope and speed. Massive restrictions on people’s liberties have been accompanied by massive efforts to prevent the complete collapse of the economy. Extraordinary things have happened as a result. Right-wing governments have funded workers’ wages while they are idle. Indians in some cities have discovered that their grandparents were telling the truth when they said you used to be able to see the Himalayas. People in atomized cosmopolitan neighborhoods are finding out who their neighbors are, and looking after them.

Of course medical research has been hugely affected, this being a medical crisis. Governments have swept aside sedate procedures which normally add months or years to the life cycle of research projects, and offered huge sums to speed any projects which offer hope of a solution. Many researchers too are galvanized by the urgent need to do something and eager to offer their talents to what all deem a worthwhile cause. Some of the results have been truly dramatic. The RECOVERY trial of drugs which might ameliorate the effects of the virus went from protocol to first patient enrolled in 9 days,1 and recruited over 5000 patients in 4 weeks, and the Oxford vaccine trial launched last week2 are among the most impressive examples. These achievements required not just money, but the willing cooperation of many people who would not normally have countenanced such haste.

But the crisis has, as crises do, brought out the worst as well as the best in us. The remarkable research achievements listed above have been paralleled by a tsunami of proposals and projects whose chance of improving the lives of patients, like their prospects of contributing to the sum of medical knowledge, is near to zero. It would be invidious (and legally risky) to name names, but many proposals we have seen are quite clearly doomed to fail because of obvious flaws in design or in basic logic, or because they are asking a question of absolutely no importance. Do we really need detailed studies of the effect of the virus on practice in every conceivable specialty? Qualitative research with affected medical personnel has a valuable place, and will help us to understand the responses we have seen to the epidemic, from the heroic to the disgraceful, but some of the proposed studies of reactions to it are not research—journalism, perhaps, history possibly, or in rare cases art, but not science. Trials and observational studies whose focus is too poorly defined to make a valid conclusion possible, or which rely on data which clearly will not be possible to collect are exercises in futility. Much of this is classic research waste, as was recently pointed out in the BMJ.3 It is also critical that we don’t make premature or exaggerated claims. The Adaptive COVID-19 Treatment Trial of remdesivir was highlighted before any details were available even on preprint servers, and inferences made publicly about mortality reduction, although the trial did not show this.4 5 On the same day another randomized clinical trial was published showing no trend towards improved survival with remdesivir,6 and meta-analysis of these two trials shows no difference in mortality outcomes (p value 0.22) (see figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Relative risk of mortality in the trials of remdesivir.

Why is this happening? And what should we do about it? While hope for cure is biasing many policy leaders, the lure of easy money is also hard to resist, and scientists are as susceptible as anyone else to a bonanza. This is undoubtedly one major underlying cause of the COVID-associated flood of junk science. An opportunity to do something large and impressive is presenting itself to many researchers, because so much goodwill and effort is being put into the drive to do something constructive by researchers, patients, clinicians and even the much-maligned bureaucrats who run our systems. But bad science will leave its footprint everywhere: in our memory, in public trust and in overcoming fear: it will not help us either in the short or the long run. The laws of scientific inference and statistics have not been affected by the virus, and studies whose design guarantees they cannot produce a valid result still will not do so during the crisis. And of course, applying for funding in the full knowledge that it will not be used for the purposes stated, or that it cannot yield the knowledge ostensibly sought, is still unethical. The crisis has shown that the normal processes of peer review and prioritization, both in funding and in publication, can be radically accelerated, but should be robust to protect the conduct of meaningful clinical research. Those responsible for managing these activities have a responsibility to ensure that funds are not wasted. Many processes already have an executive filter for clearly inappropriate submissions before peer review is even activated, and this needs to be exercised whether or not the terms COVID or coronavirus appear in the title or abstract. We hope the research world, like the rest of society, will keep some of the helpful adaptations it has made to cope with the crisis. However, it is also essential that the rigor of the scientific process is maintained if it is to continue to have value and meaning. This is not just the responsibility of the regulators, funders and publishers. The scientific community also needs to reflect and conduct itself according to the high standards of integrity it claims to espouse.