Bad science can do lasting damage. A study falsely connecting COVID-19 to 5G wireless technology, for instance, was quickly retracted for its sloppy methods and unearned conclusions, though not before causing a wave of conspiracy theories. This past March, a controversial 2017 study suggesting sepsis could be treated with vitamin C—a sign of hope in the battle against the leading cause of death in hospitals—was alleged to be based on fraudulent data, not the first time its findings had been challenged.

Yet other studies go beyond bad data and are wholly fraudulent from their inception. A curious subset of these are scientific papers that are generated by computer programs. Custom algorithms can contribute sections or even spin up whole papers by splicing together likely-sounding phrases or linguistically twisting already-published results. Such papers get printed, even by reputable publishers, and while many of these papers have been retracted, some are still being cited in the scientific literature years after their retraction. The idea of a computer-generated paper was originally pioneered as a prank—intended to expose duplicitous practices in the publishing world—but the technology is now sometimes used to pad resumes of those desperate to publish. The end result is a veritable trash island of papers that clog research databases and, occasionally, get published and cited.

While this has been a problem for some time, the COVID-19 pandemic made garbage research especially difficult—and especially vital—to ferret out.  The onset of the health crisis unleashed a flood of desperately needed studies, with one analysis finding that between February and May 2020, the number of papers submitted to health and medicine journals published by Elsevier, a leading publisher of scientific research, 63% higher than a year earlier. The growing volume of published studies across the globe—some 2.9 million science and engineering studies in 2020 alone—calls for an effort to clean house.

Several ingenious tools are now helping to do that. A recent tool, developed by French computer scientist Guillaume Cabanac at the University of Toulouse and his colleagues, can scan over 100 million papers in a database, zeroing in on “tortured phrases.” These are oddball paraphrases of scientific terms that can be the tip-off of an effort to copy existing research.

Among the substituted phrases his group has found are “bosom peril” (for breast cancer) and “counterfeit consciousness” (for artificial intelligence). “These are the fingerprints that make us take a closer look,” says Cabanac. Such substitutions might happen when a plagiarist, eager to add a scientific publication to a curriculum vitae, uses an online program like spinbot.com to copy large blocks of text from an existing paper—or a paper in its entirety—and swap in seemingly synonymous phrases to sidestep software that scans for plagiarism.

Unfortunately for science, this tactic has had some success. By April 2022, the tool had turned up these kinds of awkward replacements in more than 6,000 publications—most from large publishing companies that produce top-tier journals, raising important questions about the rigor of the peer review process.

Adam Day, a data scientist at SAGE Publishing in London, also focuses on identifying scientific misconduct, which he notes is a long-standing, endemic problem in academic research. His tools for identifying questionable research practices include programs that search for signs of intentional manipulation of the peer-review process, or that can look for multiple submissions of the same paper to different journals.

A purer breed of nonsense paper also exists, and many papers that fall under that category were created by a computer program that was meant as a prank. In 2005, a trio of MIT computer science graduate students invented SCIgen, which generates computer science papers complete with author names, scientific jargon, figures and convincing formatting. The content makes no scientific sense. Their intent, beyond having fun, was to expose fraudulent journals and conferences by showing that some would accept almost anything, including, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,” one of the first SCIgen creations.

But SCIgen soon took on a life of its own, spawning other paper generators—such as Mathgen, for math—which have been used to generate many more nonsense papers. And more recent AI tools like GPT-3 have accelerated the ability to generate realistic-sounding science, sometimes used by dishonest journals, says Cabanac, to suggest they have a long, robust publishing history—which might attract future authors—or by paper mills that peddle research to CV-padding scientists who are eager to get their names in print.

In 2015, computer scientist Cyril Labbé at Joseph Fourier University in Grenoble, France, released SciDetect, a free, open-source program that can recognize SCIgen-created papers by their built-in phrases and structures. Its use led to the retraction of hundreds of papers. In 2020, Cabanac collaborated with Labbé to develop a script to look for computer-generated papers and, in 2021, adapted the script to screen for tortured phrases. This led to the identification of many more papers—including hybrid papers that were only partially generated by software—in fields that included physics, health sciences and biology.

When Alexander Magazinov, an engineer in Moscow, heard about these fraud-detecting tools, he emailed the researchers with a list of suspicious phrases he’d recently encountered in computer science papers. Cabanac then programmed the software to focus on phrases Magazinov had flagged, a bogus-language lexicon that “snowballed” as each tortured phrase turned up more, Cabanac says.

Cabanac says that although finding a tortured phrase invites questioning, it doesn’t automatically signify bad science. An author may have wanted to include background information from another source and intended to paraphrase a paragraph, for example, but accidentally left it in place. For that reason, Cabanac’s first move is often to ask an author directly: Why is this phrase here?

Unfortunately, cleaning up “trash island” is not the most appealing pursuit for most career scientists since it doesn’t lead to peer recognition in itself. Yet thankfully some continue to take on the task. Dutch microbiologist Elisabeth Bik, for instance, has developed expertise in identifying duplicate photographic images and plagiarized text. Prominent journals are beginning to take the problem more seriously and have implemented better screening processes for submissions—in part because of Cabanac’s work. “We must all be aware of the problem,” says Cabanac, “and stay vigilant.”