Evaluating two decision aids for Australian men supporting informed decisions about prostate cancer screening: A randomised controlled trial

Summary

Background

Australian clinicians are advised to ‘offer evidence-based decisional support to men considering whether or not to have a PSA test’. This randomised trial compared the performance and acceptability of two new decision aids (DAs) to aid men in making informed choices about PSA screening.

Methods

~3000 Australian men 45–60 years with varying educational attainment were recruited via an online panel and randomised to view one of two online decision aids (one full length, one abbreviated) and completed a questionnaire. The primary outcome was informed choice about PSA screening.

Findings

Significantly more men in the long DA group (38%) made an informed choice than men who received the shorter DA (33%) (95% CI 1.1% to 8.2%; p = 0.008). On knowledge, the long DA group scored, on average, 0.45 points higher than the short DA group (95% CI 0.14 to 0.76; p = 0.004) and 5% more of the participants achieved an adequate knowledge score (95% CI 1.9% to 8.8%; p = 0.002). Men allocated the long DA were less likely to intend to have a PSA test in the future (53%) than men in the short DA group (59%). Both DAs rated highly on acceptability.

Conclusions

Both DAs were useful and acceptable to men regardless of education level and both supported informed decision making. The long version resulted in higher knowledge, and a higher proportion of men able to make an informed choice, but the differences were small. Long DAs may be useful for men whose informational needs are not satisfied by a short DA.

Keywords:

Cancer screening – Screening guidelines – Health screening – Educational attainment – Questionnaires – Decision making – Psychometrics – prostate cancer

The balance between the benefits and harms of screening for prostate cancer remains controversial. There is no population screening programme for prostate cancer in Australia however Australia has high ad hoc screening rates [1,2] with patient request a common driver. [3,4] In 2017 the United States Preventive Services Task Force (USPSTF) assigned prostate cancer screening a “C” recommendation for men 55–69 years, concluding that the potential benefits and adverse impacts of prostate-specific antigen (PSA)-based screening are closely balanced in that age group.[5] Current international guidelines emphasise that the decision should be an individual one, based on personal values and preferences.[5–7]

In response, there is much focus on developing and disseminating health care information to assist men to make informed choices. Making an informed, evidence-based decision is especially important in screening of asymptomatic people ‘because there is no medical urgency for intervention or treatment and therefore choices are made in a preference-sensitive decision setting’. [8]

To enable a person to make an informed choice they need to be given adequate, high-quality, relevant, unbiased information on all possible consequences of the options. [9] Decision aids are evidence-based tools designed to support participation in decision making and improve the quality of people’s health care decisions.[10] Providing information within a decision aid can increase informed choice about breast cancer screening.[11]

Randomised trials have consistently shown that men who use decision aids are better informed and less conflicted in prostate screening decisions when compared to usual care.[12] A 2012 Cochrane review concluded, however: ‘little is known about the degree of detail that decision aids need in order to have positive effects on attributes of the decision or decision-making process’.[13] To our knowledge, no study has compared the performance of a full-length with an abbreviated decision aid for men deciding whether or not to have a PSA screening test.

A person’s health literacy and educational status can affect their ability to use health information and services, so developers of decision aids need to ensure that tools are accessible to groups with lower and higher literacy and education.[14,15] Another important consideration of this research is therefore to assess how acceptable and effective these interventions may be for men with low levels of education and health literacy.

This study had three research questions:

• Which decision aid is better at supporting informed decision making about prostate cancer screening in a community sample–a long or short decision aid?

• Do the two decision aids differ on cognitive and psychological variables or measures of acceptability?

• What is the impact of educational background on the performance of the decision aids?

Methods

In 2012, the Prostate Cancer Foundation of Australia (PCFA) partnered with Cancer Council Australia (CCA) to develop a clinical practice guideline for prostate cancer screening in Australia [http://wiki.cancer.org.au/australiawiki/index.php?oldid=134877]. To support implementation of the guideline, development of an evidence-based decision aid for men considering having a PSA test, compliant with international standards for best practice, was recommended. We developed two decision aids, one long and one short (essentially an abbreviated version of the long form) to meet this need and conducted the evaluation described here.

Purpose

To collect information from Australian men about the usefulness and acceptability of two decision aids, one long (10 pages) and one short (2 pages). See Fig 1.

Design

Randomised controlled trial with participants randomised to view one of two online decision aids, either (a) a full-length decision aid, or (b) an abbreviated version of the decision aid. Participants completed a baseline questionnaire and answered further questions after viewing the online decision aid.

Participants

Community sample of Australian males aged 45–60 years, recruited via an international survey sampling company frequently used in research studies. Quota sampling was used to ensure inclusion of men in relevant age groups and to obtain strong representation of men with lower educational attainment (i.e. school-level qualifications only). Eligibility criteria: (1) belonged to the survey sampling panel, (2) accepted an invitation to participate in the online questionnaire, (3) aged 45–60 years, and (4) did not have a prostate cancer diagnosis.

Outcomes

The primary outcome was informed choice.[19] It comprises 3 constructs combining (1) adequate knowledge of possible outcomes of screening, and consistency between (2) a man’s attitude towards the screening test (positive or negative), and (3) intention to have a PSA test, to determine the proportion of men who made an informed (or uninformed) choice. We assessed both conceptual and numerical knowledge with a competency-based approach, [20] drawing on our team’s previous work.[16] Questions were asked to assess participants’ understanding of screening outcomes (mortality benefit, false positives, overdiagnosis) and awareness of the approximate numbers affected. Secondary outcomes were divided into two broad categories: (1) cognitive and psychological measures, including anticipated decisional regret, risk perceptions, and cancer worry, and (2) the use and acceptability of both decision aids. The acceptability of a DA refers to ratings regarding the comprehensibility of its components, its length, amount of information, balance in presentation of information about options, and overall suitability for decision making.

Procedure

Outcomes were assessed via an online questionnaire that was developed using internationally accepted, validated scales and items in previous published studies that evaluated decision aids.[11,16,21] All men completed the same questionnaire. Standard socio-demographic data was obtained from participants and included personal history of cancer, family history of prostate cancer, and prostate cancer screening history.

Analysis

Indicators of informed choice (knowledge, attitudes, intention) were scored according to a previously developed and tested marking scheme [11,16], amended minimally for our purposes (S1 Appendix). For published scales, responses were scored as per author coding instructions. For all analyses we compared either the proportion of men (categorical variables) or the mean (continuous variables) in the long and short decision aid arms. We stratified the primary outcome (informed choice) as well as some secondary outcomes (use and acceptability of DA) by education. For this purpose, we dichotomized highest educational attainment into lower (non-tertiary education) and higher levels (tertiary education).

A sample size of 3000 men (2000 45-54y, 1000 55-60y) was calculated to enable percentage estimates with confidence intervals of +/-4% (or less) within each randomised group, with stratification by education, and to enable us to detect a difference of approximately 7% between the randomised groups (in each stratum of education) in the proportion who found the aid acceptable and comprehensible, assuming acceptability proportions of 0.7 or more in each group. Our previous work with the development of a similar DA for breast cancer screening suggested the proportions finding the aids acceptable and understandable were likely to be in this range.[11] Analysis was by intention to treat. Categorical outcomes were analysed using a χ² test, continuous outcomes with a two-sample t test (α of 5%, two-sided).

A sensitivity analysis was performed to account for missing data. We conducted multiple imputation by creating 20 imputed datasets using chained equations and pooling the resulting effect measures. With this process we imputed missing values for men who were randomised but did not answer all attitudes questions (n = 173, 5.8%). The sensitivity analysis provided similar results to the main analysis and conclusions were unchanged. Ethics approval was received from the Human Research Ethics Committee of the University of Sydney (2018/165). The trial was registered on the Australian New Zealand Clinical Trials Registry (ANZCTR: ACTRN12618001718235). All interested participants were directed to an online Participant Information Statement; subsequent completion and submission of the questionnaire was considered evidence of consent. Participants by virtue of being on the survey sampling database have already consented to being involved in online research.

Results

From 27 June to 26 July 2018, 5093 men aged 45 to 60 years were contacted by a panel survey sampling company, with 4885 men invited to the main study (208 men participated in a pilot study) (See Fig 2). A total of 4398 men consented; 676 were excluded because they did not meet the required age range or dropped out prior to randomisation. The remaining 3722 men were allocated at random to view one of the two DAs.

Of the 3722 participants (1851 allocated to the long DA and 1871 to the brief DA) a total of 2993 (80%) men completed all questions in the survey and were included in the analysis of the primary outcome. 173 men did not provide answers to all attitudes questions and were excluded from the analysis of the primary outcome but were included in relevant secondary analyses.

Baseline demographics were similar across both groups (Table 1). Overall, 55% of participants indicated that they had heard of the PSA test before and 38% of participants had previously had a PSA test (up to 51% in 55-60-year-old age group). Of these men, the majority (65%) stated that they had had a PSA test because their doctor had suggested it as part of a routine check-up. Around one-quarter of men who had received a PSA screening test indicated that their doctor ‘just conducted the blood test’ in a consultation.

Which decision aid is better at supporting informed decision making about prostate cancer screening in a community sample—A long or short decision aid?

Primary outcome: Informed choice

A significantly higher proportion of men allocated the long DA (38%) were assessed to have made an informed choice about PSA screening than men who received the brief DA (33%) (4.7% more; 95% CI 1.1% to 8.2%; p = 0.008) (Table 2).

A similarly small but significant difference in the two groups was observed in the knowledge score where participants in the long DA group scored, on average, 0.45 points higher than in the short DA group (95% CI 0.14 to 0.76; p value 0.004) and 5% more of the participants achieved an adequate knowledge score (95% CI 1.9% to 8.8%; p value 0.002). Men in the long DA group had significantly better understanding of the ‘best description for overdiagnosis’ (34%) than the short (24%) (9.3% difference; 95% CI 6.2% to 12.5%; p<0.001).

Attitudes were positive overall; participants in the short DA group reported a slightly more positive attitude towards PSA screening (1.07 points difference; 95% CI 0.49 to 1.65; p value <0.001). 6% more men from the short DA group (59%) intended to have a PSA test in the future compared with participants in the long DA group (53%) (95% CI 2.4% to 9.4%; p value 0.001). We further categorized men’s choices according to knowledge, attitudes, and intentions. 843 (28%) had inadequate knowledge but positive attitudes and intentions towards PSA screening. This was seen more frequently in men in the short (495/1543; 32%) than long group (348/1450; 24%) (p<0.001) (Table 3).

A sensitivity analysis for the primary outcome was conducted using alternative criteria to define adequate knowledge (i.e. defined adequate knowledge using conceptual items only), because there is little consensus in the literature regarding what level of knowledge constitutes being informed. In this sub-analysis men had to score at least four correct out of the six main conceptual items (S2 Appendix). With this knowledge threshold, an informed choice was made by 6662 men (46%) allocated to the long DA and 670 men (43%) who viewed the short (p = 0.219).

We also conducted a per protocol analysis including only those men who said they read all or most of the DA. 50% of men who read all or most of the short DA and 57% of men who read all or most of the long DA achieved adequate knowledge (7.4% more; 95% CI 3.5% to 11.4%; p value 0.001). 39% of men in the short and 45% in the long condition who read all or most of the information reached an informed choice [5.9 (2.0 to 9.9<0.003)].

Do the two decision aids differ on cognitive and psychological variables and measures of acceptability?

Secondary outcomes: Cognitive and psychological variables

The groups differed on three items when distributions of the cognitive and psychological measures were compared: worry, anticipated regret, and perceived risk (Table 4). Men in the long DA group were less likely than men in the short DA group to feel that they might later regret not having a PSA screening test (p = 0.002). Men in the long DA group generally perceived their risk of developing prostate cancer as lower than men in the short DA group (p = 0.039). These differences were significant, but small.

Secondary outcomes: Use and acceptability of decision aids

Table 5 shows how men used and evaluated the decision aids. Participants generally spent less time reading the short DA (p<0.001) and more men in the short DA group indicated that they had read most or all of it (3% difference, p<0.031). More men in the long DA group found the DA too long (14% difference, p<0.001).

Overall, participants in the longer DA group perceived it to be slightly more credible than those in the shorter DA group (0.06 points, p = 0.037). However, even though statistically significant, the actual difference was minimal. No significant differences were observed about how much of the information was new to participants, how balanced, clear and helpful they found the DA, or if they would recommend it to other men.

What is the impact of educational background on the performance of the decision aids?

Outcomes stratified by education

Regression models showed no statistically significant evidence that the effect of the intervention differed by education level, with few exceptions (S3 and S4 Appendices). The only exceptions were “time spent on reading the decision aid” (p value = 0.016), and “information in decision aid was new” (p value = 0.040).

Discussion

It is of utmost importance that men are given adequate information and a genuine opportunity to make an informed decision about whether or not to have a PSA test. In the present study, an abbreviated version of a decision aid increased knowledge and enabled informed choices about PSA screening almost as well as a full-length decision aid. The effect of the intervention was no different among men with lower or higher levels of education. These findings make an important contribution: few studies have compared a short and a long DA, and even fewer among men with a known, broad spread of educational attainment. A sub-analysis comparing detailed and simpler decision aids was included in a 2011 Cochrane review and similarly reported only a marginal improvement in knowledge for the detailed versions.[13] This is noteworthy because if a short DA presenting minimum, but enough, information can demonstrate effectiveness and rate favorably on measures of usefulness and acceptability as in this study, then this is likely to be universally preferred–at least in the first instance with options for further information available for those who desire it.

There are benefits to a shorter DA that works well enough and offers a solution to concerns about feasibility. We recognise and acknowledge some losses in terms of knowledge and in-depth understanding (it did appear that on the more difficult knowledge questions, participants in both educational strata performed better if they viewed the long version), but the short DA is practical and realistic and, importantly, does not disadvantage men with lower education.[14,15,21] Our study was conducted in Australia, where the clinical context and uptake of screening using the PSA test is similar to that in the USA and Canada.[23] Our findings are therefore applicable and relevant to those jurisdictions.

The percentage of men who achieved informed choice in this study is consistent with other DAs promoting informed choice internationally, including in the context of breast (15–24% of women)[11] and bowel screening (34%).[21] Although the absolute rate of adequate knowledge and informed choice overall seems low, we observed that more men reached adequate knowledge and were able to make an informed choice when they read the DA all the way through. We highlight that this was achieved with a single reading of the DA without support from any other source, e.g. a physician in a consultation.

Around 50% of men in both conditions indicated that they had seen the term ‘overdiagnosis’ before but, overall, understanding of the overdiagnosis information was particularly low, suggesting a need for targeted community education and engagement specifically around overdiagnosis.[24] Importantly, however, men exposed to the long version of the DA appeared to understand overdiagnosis better than men receiving the short form. This finding offers useful insight on methods to explain overdiagnosis, which is generally considered a difficult concept to convey. Identifying effective strategies to improve public knowledge about the downsides of cancer screening, particularly overdiagnosis, remains a priority for future research.

The effect of DAs on screening intentions and behaviour has not been consistent; in the prostate screening context, a recent Cochrane review reported that decision aids reduced the number of people choosing PSA screening when compared to usual care (RR 0.88; 95% CI 0.80–0.98; 10 studies; N = 3996). [10] In our study, the majority of men indicated positive attitudes toward screening after viewing the DA, slightly more in the short DA group, and around half of the men overall still intended to have a PSA test in the future.

The majority of participants interpreted the DA as neither recommending for nor against PSA screening, however a large minority believed the aid recommended screening. This is similar to other studies (e.g. [25]). Our findings that men remained positive and found the DA to be favoring screening highlight that it may take more than a single exposure to information like this for men to grasp the complexity of a message that is probably counter to their usual understanding and practices, including men who have had a PSA test previously.

Strengths and limitations

We conducted a rigorous evaluation of the performance and acceptability of a short and long DA outside of the clinical setting with a large sample, including a large proportion of men with lower educational attainment. Comparing the impact among high and low education samples is rarely done and is important to ensure equity in SDM.[15]

The two DAs were evaluated by men registered with a survey sampling company who may or may not have been engaged and motivated by issues of PSA screening. It is therefore possible that our findings are an underestimate of the outcomes that would be achieved if the DAs were implemented in practice. It is likely that the information would have more salience for men receiving the DA in general practice or searching for it online, resulting in higher motivation to engage with the issues contained in the DAs.

Conclusion

Both decision aids were useful and acceptable to men regardless of education level and both supported informed decision making. The long version resulted in small but significantly higher levels of knowledge, particularly around the unfamiliar topic of overdiagnosis. We suggest that the best approach at this time is to widely disseminate a printed version of the short decision aid for doctors to distribute in primary care, with the longer version made available online for those patients who, either then or later, may want to consider the question further. The longer online version could be promoted in the short-printed version and made available publicly as part of a broader strategy to disseminate information about the benefits and harms, including overdiagnosis and overtreatment, of screening for prostate cancer.

Supporting information

S1 Appendix [docx]

S2 Appendix [docx]
Conceptual knowledge items only scoring.

S3 Appendix [docx]
Analysis of primary outcomes—By education.

S4 Appendix [docx]
Analysis of secondary outcomes—Use & acceptability of decision aid by education.

S5 Appendix [docx]

S6 Appendix [docx]

S1 Questions [docx]