How do the public interpret COVID-19 swab test results? Comparing the impact of official information about results and reliability used in the UK, US and New Zealand: a randomised, controlled trial

Objectives

To assess the effects of different official information on public interpretation of a personal COVID-19 PCR (‘swab’) test result.

Design

A 5x×2 factorial, randomised, between-subjects experiment, comparing four wordings of information about the test result and a control arm of no additional information; for both positive and negative test results.

Setting

Online experiment using recruitment platform Respondi.

Participants

UK participants (n=1,744, after a pilot of n=1,657) collected by quota sampling to be proportional to the UK national population on age and sex.

Interventions

Participants were given a hypothetical COVID-19 swab test result for ‘John’ who was presented as having a 50% chance of having COVID-19 based on symptoms alone. Participants were randomised to receive either a positive or negative result for ‘John’, then randomised again to receive either no more information, or text information on the interpretation of COVID-19 test results copied from the public websites of the UK’s National Health Service, the US’s Centers for Disease Control, New Zealand’s Ministry of Health, or a modified version of the UK’s wording incorporating uncertainty. Information identifying the source of the wording was removed.

Main outcome measures

Participants were asked “What is your best guess as to the percent chance that John actually had COVID-19 at the time of his test, given his result?”; questions about their feelings of trustworthiness in the result, their perceptions of the quality of the underlying evidence, and what action they felt ‘John’ should take in the light of his result.

Results

Of those presented with a positive COVID-19 test result for ‘John’, the mean estimate of the probability that he had the virus was 73%; for those presented with a negative result, 38%. There was no main effect of information (wording) on these means. However, those participants given the official information on the UK website, which did not mention any uncertainty around the test result, were significantly more likely to give a categorical (100% or 0%) answer (for positive result, p <.001; negative, p =.006). When asked how much they agreed that ‘John’ should self-isolate, those who were told his test was positive agreed to a greater extent (mean 86 on a 0-100 scale), but many of those who were told he had a negative result still agreed (mean 51). There was also an interaction between wording and test result ( p < 0.001), with those seeing the New Zealand wording about the uncertainties of the test result significantly more likely to agree that he should continue to self-isolate after a negative test than those who saw the UK wording ( p =.01), the experimental wording ( p =.02) or no wording at all ( p =.003). Participants rated positive test results more trustworthy and higher quality of evidence than negative results.

Conclusions

The UK public perceive positive test results for COVID-19 as more reliable and trustworthy than negative results without being given any information about the reliability of the tests. When additionally given the UK’s current official wording about the interpretation of the test results, people became more likely to interpret the results as definitive. The public’s assessment of the face value of both the positive and negative test results was generally conservative. The proportion of participants who felt that a symptomatic individual who tests negative definitely should not self-isolate was highest among those reading the UK wording (17.4%) and lowest among those reading the New Zealand wording (3.8%) and US wording (5.1%).

Pre-registration and data repository

pre-registration of pilot at <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://doi.org/10.17605/OSF.IO/8N62F">osf.io/8n62f</ext-link> , pre-registration of main experiment at <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://osf.io/7rcj4">osf.io/7rcj4</ext-link> , data and code in <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://osf.io/pvhba/">https://osf.io/pvhba/</ext-link> .

What is already known on this topic

• Different countries have had different approaches to conveying the meaning of a COVID-19 swab test result, particularly regarding the uncertainties inherent in the result due to limitations of specificity and sensitivity.

• Previous research has suggested that people’s trust and understanding is not affected by conveying quantified uncertainties numerically, but that perceptions of the quality of the underlying evidence can affect trust.

• It is not known whether the different wordings around COVID-19 test uncertainties are likely to affect people’s trust in, or behavioural response to, the results they receive.

What this study adds

• This study provides the first empirical evidence to our knowledge of the responses the public have to COVID-19 swab test results.

• It suggests that the public have a higher degree of trust and confidence in positive swab test results than negative when they are not given any other information accompanying the result. The experimental wording that we created for this study appeared to boost their trust in and assessment of quality of positive test results, but did not change their lower ratings of negative results.

• The wording used by the UK’s National Health Service, which does not include any cues of uncertainty in the result, was more likely to lead people to definitive (100% or 0%) answers to questions about the meaning of the result.

• The wording used by New Zealand’s Ministry of Health, which is more explicit about the reliability of the tests, appears to lead people to be more cautious about recommending that a test participant with a negative test (but still symptomatic) no longer needs to self-isolate.