During the first peak of the coronavirus pandemic, it became apparent that the processing of the standard COVID-19 test using nasal swab samples, while considered a gold standard, was slow, taking over 24 hours to give results in most cases.

As demand for quicker test results increased with supplies running low, scientists and engineers rushed to fill the gap at scale for rapid diagnostic testing that would replace the standard real-time reverse transcription polymerase chain reaction (rt-PCR) test.

Such technologies included rapid antigen tests, antibody tests, as well as tests using the loop-mediated isothermal amplification method (LAMP) and mass spectrometry. The UK is currently deploying “lateral flow” antigen tests, which deliver results on the spot, are cheap and easily mass produced. However they have important limitations.

With so many different types of test now on the market, how do we sort the wheat from the chaff and work out what form of testing is best for which healthcare or community setting? Our research group is testing the tests to find out.

Not all tests are equal

The outcome of any COVID test can vary according to what kind of sample it is testing (nose or throat), how much virus is in the patient’s body, and the environment the test is taken in. But the type of test itself can also affect the outcome, and no test is completely perfect. Some people test positive when they don’t have COVID-19 (false positives) and some test negative when they do (false negatives). The numbers of false results will vary with the prevalence of disease.

To measure how often a test is likely to give a false result, we analyse a test’s performance across a number of samples with or without the virus present. This allows us to estimate how often the test will correctly identify a positive sample in those which do contain the virus – known as test sensitivity – and how often the test correctly picks up samples without virus – test specificity.

Why false results matter

The consequences of false results must be taken into consideration when deciding how and where tests can be safely used. In a low-prevalence setting (for example, when you are testing everyone in a local area rather than just people you think might be sick), someone who receives a false negative result could go on to unknowingly transmit the virus. A false positive could result in unnecessary self-isolation for a whole household with subsequent impact on work, education and finances.

A highly sensitive test will reduce false negative results in a low prevalence setting. In this situation, it is very important that the test also shows a high specificity to avoid hundreds of thousands of false positives.

In hospitals, testing is commonly used to decide which patients can be safely treated alongside other patients who do not have COVID-19. Diagnostic tests which had acceptable characteristics in a low prevalence setting may return too many false negative results. This would inadvertently allow more positive cases through into wards with non-COVID patients, thereby potentially infecting higher risk patients who are already unwell, and the staff caring for them.

Evaluating new tests

It’s clear that a single test result affects not just the person tested, but many other people and organisations. Therefore, it is important to evaluate a new test in the precise context in which it will be used.

The COVID-19 National Diagnostic Research and Evaluation platform (CONDOR) was set up in June 2020 by a national group of experts in the evaluation of diagnostic tests. Since then, we have been evaluating commercial COVID-19 tests to provide evidence of the suitability of each test for use in different situations, such as laboratories, hospital emergency departments, GP surgeries, testing centres and care homes. This ensures that the right type of test is used in each different clinical situation.

Setting up this study in the middle of a pandemic felt a bit like boarding a moving train. Researchers paused hundreds of pre-pandemic studies as dozens of new studies were required, all with the same sense of urgency. But, thanks to the National Institute of Health Research’s infrastructure, we have been able to set up more than 100 study sites across the UK and, so far, to evaluate diagnostic tests from 22 different companies.

To date, our preliminary findings have helped to inform the use of lateral flow tests for mass testing in Liverpool and NHS staff testing. Our group asked people who had tested positive on a COVID-19 PCR test to return to a drive-through testing centre, where they provided new samples. The results of the new lateral flow tests were compared to the “reference standard” PCR results. We found that just over three-quarters of people with COVID-19 will test positive using the lateral flow tests. Knowing that helps us to understand how to best use this type of test.

Our group has also looked at testing in care homes, showing that it is possible to use a rapid PCR test which returns results in 90 minutes and is safe for staff and patients.

We have also reviewed existing data to look at the accuracy of a five-minute finger prick test. These showed that this type of test accurately identifies more than nine out of ten patients with COVID-19.

Nobody knew that COVID-19 was going to be the next pandemic, but we did know that another pandemic was likely. Yet pre-pandemic studies did not exist for diagnostic tests. This is a lesson we should not forget. The infrastructure we now have in place to evaluate tests should be retained as a “sleeping study” so that, in future pandemics, the challenges could be met during the first wave, not the second or third.

Our thanks to the CONDOR patient and public representatives, Graham Prestwich and Val Tate, for reviewing and contributing to this article.