Article Text
PDF
Abstract
Introduction A clinical prediction rule (CPR) using psychosocial questions was previously derived to target sexual healthcare in general practice by identifying women at risk of unintended pregnancy (UIP) and sexually transmitted infections (STIs). This psychosocial CPR may help target resources within contraception and sexual health (CASH) services. This study investigated how well it predicted recent self-reported risk of UIP and STI acquisition among women attending a CASH clinic.
Methods Female patients aged 16–44 years attending a CASH clinic in South-East England were offered a questionnaire on arrival. This comprised psychosocial questions, and others addressing three sexual risks: (1) two or more male sexual partners in the last year (2+P), (2) risk of STI acquisition through most recent partner and (3) risk of UIP in the last 6 months. A CPR score was calculated for each participant and cross-tabulated against self-report of each sexual risk to estimate CPR sensitivity and specificity.
Results The psychosocial questions predicting 2+P had sensitivity 83.2% (95% CI 79.3% to 86.5%) and specificity 56.1% (95% CI 51.3%−60.6%). Those predicting combined 2+P and/or risk of STI acquisition through most recent partner had a sensitivity of 89.1% (95% CI 85.7%−91.8%) and specificity of 43.7% (95% CI 39.0%−48.5%). Questions predicting risk of UIP in the last 6 months had a sensitivity of 82.5% (95% CI 78.6%−86.0%) and specificity of 48.3% (95% CI 43.4%−53.1%).
Conclusions The CPR demonstrated good sensitivity but low specificity, so may be suited to triaging or stratifying which interventions to offer CASH patients and by which mode (eg, online vs face-to-face). Further investigation of causal links between psychosocial factors and sexual risk is warranted to support development of psychosocial interventions for this patient group.
- family planning service provision
- genitourinary medicine
- needs assessment
- service delivery
- surveys
- sexually transmitted infections
Data availability statement
Data are available upon request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- family planning service provision
- genitourinary medicine
- needs assessment
- service delivery
- surveys
- sexually transmitted infections
Key messages
Clinical prediction rules (CPRs) may help to identify intervention needs and target resources within sexual health services.
Psychosocial questions can be used in CPRs to identify recent sexual risk experiences in women of reproductive age.
A psychosocial CPR to identify sexual risk among women of reproductive age has higher sensitivity but lower specificity in contraception and sexual health than in general practice.
Introduction
Clinical prediction rules (CPRs) identify risk of adverse outcomes in individuals using patient characteristics data, and can inform decisions about clinical intervention. Most sexual health CPRs comprise sexual behavioural and sociodemographic factors and focus on sexually transmitted infection (STI) risk.1 2 Previously we developed a CPR to identify women at risk of STIs and/or unintended pregnancy (UIP) attending general practice (GP),3 using psychosocial questions identified from preliminary studies.4 5 Psychosocial factors are increasingly explored in sexual health research, reflecting the need to address social determinants of sexual health,6 and may prove more acceptable than sexual behaviour questions in GP7 8 for which the CPR was originally developed.
The CPR was generated for women only as psychosocial determinants of sexual risk and morbidity may vary considerably between the sexes,9 for example, alcohol use predicts number of sexual partners among female STI clinic attenders but not male.10 The nature of contraception also indicates the benefit of a sex-specific CPR, while sexual healthcare engagement and delivery preferences also vary by sex and/or gender.11 12
CPRs may offer opportunities for efficiency amid cuts to specialist sexual healthcare of up to 20% in Britain13 and elsewhere.14 As an example, previous research has investigated a digitally-delivered CPR to triage STI testing in specialist sexual health settings for high-risk subpopulations.15 In Britain, specialist contraceptive advice and supply, and STI testing, treatment and partner notification, are increasingly co-delivered in contraception and sexual health (CASH) clinics.6 Although women attending specialist sexual health services experience higher rates of sexual risk than those attending GP for sexual healthcare,16 17 this may not be true for those women attending CASH clinics only for contraceptive advice and supply. We hypothesised that our psychosocial CPR may support differential targeting of women attending CASH clinics who may not require both contraception and STI testing, and set out to assess its performance using an existing dataset, before investigating digital delivery.
The study aimed to investigate how well our psychosocial CPR – developed for use in GP – performed in predicting self-reported recent risk of UIP and STI acquisition among women of reproductive age attending a specialist sexual health service.
Methods
Patient and public involvement
We conducted patient and public involvement (PPI) at a women’s centre to explore the purposes, format and acceptability of a psychosocial CPR. Opportunistic PPI consultation took place in GP waiting rooms to explore the feasibility and acceptability of the recruitment and data collection plans described below. We also held consultations at a youth forum with women aged 16–17 years. As participation was anonymous, we distributed lay summary leaflets of findings to all recruitment sites instead of to individual participants.
Study sample
We undertook a cross-sectional survey of women attending an urban specialist sexual health and contraception clinic in a hospital outpatient setting in South-East England (referred to as ‘SHAC’ as it was a genitourinary medicine clinic expanded to include contraception services). We aimed to recruit a convenience sample of n=500 women between April and August 2016, sufficient to enable multivariable statistical modelling.18 Using a nomogram for sensitivity and specificity sample size calculations, a sample size of n=500 would be adequate to estimate a sensitivity of 80% with a 95% CI to within ±5% given a prevalence of 50%.19 The sample comprised female attenders aged 16–44 years regardless of attendance reason, recruited as part of a larger study to develop a CPR to target sexual healthcare to women attending GPs, but eventually not needed for that derivation.3
Recruitment and data collection
Recruitment took place two mornings per week, when women were attending for walk-in and booked appointments for dedicated contraception services, STI testing and treatment. Women were offered an envelope by reception staff, or by a researcher immediately after they booked in, with the exception of visibly distressed women, those known to have insufficient English language skills, or those clearly outside the eligible age range. Due to resource limitations, it was not possible to record the numbers of women not offered or refusing a questionnaire, or the reasons why. Each envelope contained a pen, participant information sheet (PIS) and a questionnaire, comprising potential CPR items and the outcomes of interest. The questionnaire was designed to take 5 min to complete, while awaiting an appointment. To maintain brevity we did not capture additional demographic data. Participants were instructed to complete the questionnaire anonymously, sitting alone in the waiting area if possible. Consent was implied by questionnaire completion. Three initial questionnaire items were used to screen out those not identifying as female, outside the eligible age range and/or who had completed the questionnaire previously. The questionnaire text instructed these patients to seal and return their questionnaire without completing it further. The questionnaire and PIS instructed participants to seal their questionnaire in the envelope before returning to staff to be securely passed to the researchers.
Data management and storage
Data were stored at Brighton and Sussex Medical School for the study duration. Data were double-entered into a statistical package by an external company and transferred to Stata 1320 for analysis. Accuracy checks were performed on a random 10% sample and anomalies checked and addressed individually by researchers.
Measurement of sexual risk and psychosocial predictors
Three sets of CPR psychosocial questions were investigated, each set having been previously derived using clinical prediction modelling3 to predict one of the following outcomes:
Psychosocial questions and response options, indicating which clinical prediction rule set each contributed to
Self-report of 2+ male sexual partners in the last year (hereafter ‘2+P’)
Self-report of 2+P and/or risk of STI through most recent partner (abbreviated hereafter as ‘combined risk’)
Self-report of risk of UIP in the last 6 months (abbreviated hereafter as ‘risk of UIP’).
The first outcome was chosen as indicative of possible need for sexual health advice and STI testing, and measured using the item ‘In the last year, how many men have you had sexual intercourse with (by sexual intercourse, we mean a man’s penis in a woman’s vagina, mouth or anus)?’.
The second outcome was chosen as experiences such as multiple partnerships and condom use only partly predict poor sexual health outcomes.21 This was measured using a composite variable comprising positive responses to the items ‘The man I most recently had sex with didn’t always use condoms for vaginal sex with previous partners’, and ‘The man I most recently had sex with had at least one sexual partner in the last year before me’ and ‘Thinking about condom use with your most recent male sexual partner … condoms were sometimes used for vaginal sex OR condoms were never used for vaginal sex’.
The third outcome was chosen as indicative of possible ongoing need for contraceptive advice and supply. This was measured by combining responses to an item about contraception use (‘Thinking about your contraception use in the last 6 months, please tick one statement which most applies to you’) with another about pregnancy intention (‘Overall, in the last 6 months how much have you wanted to avoid getting pregnant?’), each adapted from previous studies.22 23 Women responding ‘Contraception was used but the method failed’ or ‘Contraception was never used’ or Contraception was used, but not on every occasion’ and that they had wanted to avoid getting pregnant ‘very much’ or ‘quite a lot’ were categorised as having been at risk of UIP in the last 6 months.
Data analysis
For each CPR set, a score was generated for each participant using their psychosocial question responses (table 1). Each participant’s score was cross-tabulated against their self-report of that outcome to assess the sensitivity and specificity of that item set in predicting 2+P, combined risk and risk of UIP. Sensitivity and specificity were calculated for a range of CPR scores, using the original cut-off value specified from the GP sample3 and an alternative cut-off value that better balanced sensitivity and specificity. Participants were excluded if there were missing data for either the outcome of interest or any of the exposures comprising that CPR item set (on the basis that this would constitute an incomplete and therefore invalid CPR score). Positive and negative predictive values for each CPR set were also calculated on this basis using the same cross-tabulation.
Results
The final sample comprised n=532 respondents of which 44.5% were aged 16–24 years. Logistically it was not possible to assess what proportion of eligible women attending the clinic during recruitment this figure represents. However, of 589 questionnaires handed out, 553 (94%) were returned, of which n=537 (97%) were eligible. A further five participants were excluded from analysis because sexual risk items were not completed.
Among the final sample, 62.3% (95% CI 58.0% to 66.5%) (n=324) self-reported multiple male sexual partners in the last year (n=12 missing data); 76.6% (95% CI 72.7% to 80.3%) (n=387) self-reported combined risk of multiple male sexual partners and/or risk of STI through most recent partner (n=27 missing data); and 34.7% (95% CI 30.6% to 39.1%) (n=173) self-reported risk of UIP in the last 6 months (n=34 missing data). Table 2 presents the distribution of item responses.
Distribution of responses to clinical prediction rule psychosocial items by outcome
Table 3 comprises a cross-tabulation of CPR scores against participants’ self-report of each outcome, displaying the sensitivity and specificity offered by using each score as a cut-off value categorising a participant as ‘at risk’ or ‘not’.
Sensitivity and specificity of the clinical prediction rule in identifying different sexual risk experiences
For 2+P, excluding participants with missing outcome data or CPR items gave a sample size of n=445, of which 66.7% (297/445) reported multiple partnerships in the last year. For combined risk, excluding participants with missing outcome data or CPR items gave a sample size of n=437, of which 80.0% (350/437) reported combined risk in the last year. For UIP risk, excluding participants with missing outcome data or CPR items gave a sample size of n=429, of which 37.3% (160/429) reported unintended pregnancy risk in the last 6 months. Thus, missing exposure data was greatest for items contributing to the CPR for unintended pregnancy risk, but sample sizes for all CPRs were compromised by missing data.
Using the original cut-off score of 9 (on a scale of 0–18), the CPR for ‘2+ sexual partners in the last year’ had a positive predictive value (PPV) of 79.2% (247/312) and negative predictive value (NPV) of 62.4% (83/133). Alternatively, a cut-off score of 10+ had a PPV of 82.9% (228/275) and NPV of 59.4% (101/170). The CPR for ‘combined risk from multiple partnerships or most recent partner’, using the original cut-off score of 6+ (on a scale of 0–17), had a PPV of 86.4% (312/361) and NPV of 50.0% (38/76). Alternatively, a cut-off score of 7+ gave a PPV of 87.8% (296/337) and NPV of 46.0% (46/100). Using the original cut-off score of 3+ (on a scale of 0–11), the CPR for ‘unintended pregnancy risk in the last 6 months’ had a PPV of 48.7% (132/271) and NPV of 82.3% (130/158). Alternatively, a cut-off score of 4+ gave a PPV of 59.9% (103/172) and NPV of 77.8% (200/257). For each outcome the c-statistic and Receiver Operating Curve is provided in the online supplementary file.
Supplemental material
Discussion
Findings indicate that the psychosocial variables demonstrated good sensitivity in identifying those at recent risk of UIP or of STI acquisition in our dataset, but that the CPR does not constitute a useful alternative to direct questions about recent sexual risk experiences in this setting.
Overall, the CPR scores and original cut-off values yielded higher sensitivity but lower specificity for women attending this SHAC service when compared with women attending GP3 for whom the CPR was originally developed. This may indicate higher prevalence of psychosocial risk factors among SHAC-attending women compared with GP attenders. Certainly our previous comparative analysis between the GP and SHAC samples indicated that after adjusting for the younger age of the SHAC participants, 2+P in the last year was more strongly associated with current smoking in the GP cohort, but more strongly associated with illicit drug use ever in the SHAC cohort.24 Similarly, it is important to note that the CPR was not designed to identify women needing contraceptive advice and supply for reasons other than inconsistent or failed use, such as switching methods due to side effects or lifestyle and/or attending for long-acting reversible contraception removal.25
The CPR performance was broadly comparable with tools developed using specialist sexual health setting data, such as a chlamydia infection tool for heterosexual women (sensitivity 70.0%, specificity 62.3%)26 and an acute HIV infection tool for men who have sex with men (sensitivity 83.3%, specificity 52.5%).27 Nonetheless, the low NPVs would not provide clinical confidence in using the CPR to gate-keep interventions without full sexual history-taking. This reflects the difficulty of applying a CPR developed in primary care in higher prevalence settings, suggesting that external CPR validation should consider the impact of setting prevalence of NPV and PPV when determining the best cut-off score.
To ensure the questionnaire was brief and suitable for self-completion, the only sociodemographic data captured were age and sex (inclusion criteria) and housing tenure (a proxy of socioeconomic status5). Capture of more sociodemographic data would have enabled assessment of the transferability of these findings to other settings; however, the study was not designed to generate generalizable findings, but rather to identify if validation of the CPR is worthy of pursuit.
Convenience sampling may have resulted in under- or over-sampling those reporting sexual risk experiences and/or adjunct psychosocial issues. Nonetheless, those who chose not to participate based on sexual risk and/or other covariates would also likely decline the CPR in practice so our findings may still anticipate CPR clinical performance. The CPR question sets are not yet validated in primary care; nonetheless, this study offers some validity by demonstrating their discriminatory potential in women of the same age attending a SHAC.
The validity of self-reported risk could not be evaluated because it was not logistically possible to link answers with clinic data. Oral and anal intercourse were not captured to ensure questionnaire brevity, therefore the combined variable for STI acquisition risk only captured risk from vaginal intercourse. Several psychosocial items were unvalidated (due to necessary adaptations to enable brief questionnaire self-completion). Recall bias may also have affected item responses.
The CPR was developed as a paper-and-pencil self-completion tool, meeting delivery preferences identified during public consultation, and addressing a recognised need for brief sexual health assessment using self-scoring.28 However, digital delivery may enable more complete and accurate reporting. Limited resources meant that it was not possible to estimate response rates, investigate reasons for non-participation or capture the number of women who refused a questionnaire or were not offered one due to the exclusion criteria. Thus, it is not possible to deduce the acceptability of the CPR questions in this and other CASH settings, although PPI indicated all items were acceptable.
Increasing digital and/or shared delivery of contraception and STI interventions in CASH clinics opens the possibility for CPRs to triage patients to different interventions. Self-completion CPRs such as this one can be incorporated into electronic or face-to-face book-in processes, or within specific care pathways, to identify if additional intervention is required.
Finally, our findings contribute to the broader study of association between social factors and sexual health, indicating the value of further research to investigate the role of psychosocial factors as causal factors in sexual risk, to inform psychosocial interventions aiming to reduce risk for women accessing sexual healthcare.
Data availability statement
Data are available upon request.
Ethics statements
Ethics approval
Ethical approval for this study was granted by the National Research Ethics Service (Ref 16/LO/0206).
Acknowledgments
The authors wish to acknowledge all those who have helped with and contributed to the study. This includes attenders at Brighton Women’s Centre and at Newhaven Youth Forum, the NIHR Clinical Research Network Kent, Surrey and Sussex, the site Principal Investigators including Dr Paul Deffley, and the staff and patients of the participating site.
References
Footnotes
Twitter @natalieedelman
Contributors NE led the study conception, design, implementation, analysis and publication. JC, RV and CM contributed to conception, design and publication. JW and AG acted as Principal Investigators at study sites, contributing also to study planning and data acquisition, and consulting on the publication and cut-off score choices. CJ and SB advised on the sample size, plan of analysis and interpretation of findings, as well as this publication, and SB oversaw the analysis itself.
Funding This is a summary of independent research funded by the National Institute for Health Research (NIHR)’s Doctoral Research Fellowship programme (DRF-2013-06-004).
Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. The Methods section contains further details.
Provenance and peer review Not commissioned; externally peer reviewed.