BMJ Rapid Recommendations and patient involvement

In accordance with the BMJ Rapid Recommendations process,17 a guideline panel provided critical input and guidance during the review process, which included identifying populations, subgroups and outcomes of interest. The OMERACT (Outcome Measures in Rheumatology) draft core domain set for clinical trials of shoulder disorders18 formed the starting point for consideration of outcomes of interest.

The guideline panel included content experts, front-line clinicians, methodologists and four patients with lived experience of SAPS . Patients received personal training and support to contribute throughout the guideline development process. They were full members of the guideline panel, contributed to the selection and prioritisation of outcomes, and provided critical feedback to the BMJ Rapid Recommendations manuscript.

The guideline panel also requested a linked systematic review to identify the most credible estimates of minimally important differences (MIDs) for pain, function and quality of life (Hao Q et al., in submission). We used the most credible MID estimates to inform judgements regarding estimates of effect in our systematic review, and to determine the importance of observed differences in effects, reflecting patients’ values and preferences.

Patient involvement in this systematic review

In addition to the input from patients in the guideline panel, we involved a second group of patients to help prioritise the outcomes for this systematic review. We invited 33 patients seeking care for SAPS in an orthopaedic outpatient clinic (Helsinki University Hospital) and a primary care unit (Occupational Health Helsinki) to answer a questionnaire addressing their shoulder condition. We queried, with open questions, what was the most bothersome symptom, and asked respondents to name the three most important activities affected by their shoulder problem.

Thirty patients answered our questionnaire. Pain was the most bothersome symptom for 26 patients (86%). The most commonly affected activities were disturbed sleep, difficulties elevating the arm and participation in sports.

Eligibility criteria

We included randomised controlled trials comparing subacromial decompression surgery for SAPS with any other treatment(s) in adults. Because SAPS is a clinical diagnosis, we did not set strict diagnostic criteria. Instead, we accepted studies’ own criteria for SAPS diagnoses with no requirement for imaging. We excluded trials in which any enrolled patient had a symptom duration of less than 3 months. Subacromial decompression surgery was defined as open or arthroscopic bursectomy and/or acromioplasty. As the anticipated effects and outcomes of relevance are similar following open and arthroscopic acromioplasty,19 20 we combined data from these two surgical approaches.

We excluded trials with mixed diagnoses across participants, including calcific tendinitis, full-thickness rotator cuff tears and secondary shoulder pain (post-traumatic, laxity, thrower’s shoulder and so on, diagnosed by imaging or at surgery) unless it was possible to extract data from patients with SAPS separately.

For harms, we anticipated trial data would not represent the best body of evidence due to few events and incomplete reporting. In our protocol we stated that we would include studies regardless of design. However, we realised that well-performed observational studies designed to detect harms, including consecutive samples of patients undergoing surgery, outside the context of strictly controlled trials, provide much higher certainty evidence of rare harms than designs at higher risk of bias. Therefore, we included only prospective cohort studies on harms after shoulder arthroscopy for a variety of diagnoses including SAPS from registries designed to evaluate harms. We did not specify a minimum symptom duration and there were no exclusion criteria. Among the identified studies we selected studies that provided estimates of harms with the highest certainty. In this assessment we emphasised the risk of bias and issues related to indirectness (eg, type of patients, surgery and time frame for recording harms).

Literature search

For trials regarding efficacy, we searched MEDLINE, Embase, PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Physiotherapy Evidence Database (PEDro), ClinicalTrials.gov and WHO International Clinical Trials Registry Platform. We used the Cochrane Highly Sensitive Search Strategy for identifying randomised trials in MEDLINE—sensitivity-maximising and precision-maximising version (2008 revision) in MEDLINE and PubMed—and adapted for other sources where needed.

For harms, we searched MEDLINE, Embase, PubMed, CENTRAL, Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Health Technology Assessment, CINAHL and PEDro.

All searches were performed from database inception to 23 July 2018 (search strategies for all databases are in online supplementary appendix), excluding animal studies and publication types unlikely to contain relevant information (news, comments, letters to the editor and editorials) but without other search limits.

Study selection

Two independent reviewers evaluated eligibility and extracted data using records imported to the Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia; www.covidence.org). When necessary, we obtained full-text articles to determine eligibility for inclusion, including English-language translations. Disagreements were resolved by discussion.

Outcomes

The guideline panel identified outcomes of importance to patients; we used outcomes from the trials that most closely corresponded to those chosen by the patients (table 1). Because it was the most bothersome symptom for 86% of the patients who completed our survey, we considered pain as the most important outcome in this systematic review.

We defined serious harms as death, bleeding (uncontrolled or requiring transfusion), cardiac arrest requiring cardiopulmonary resuscitation, myocardial infarction, cerebrovascular accident, acute renal failure, unplanned intubation, requiring ventilator for >48 hours, deep infection (surgical site or organ/space), sepsis, septic shock, pneumonia, wound dehiscence, pulmonary embolism, deep vein thrombosis or peripheral nerve injury.

Data extraction

Two independent reviewers extracted outcome data for all available follow-up assessments. If authors did not report relevant numeric outcome data in the text, we contacted the authors or, when available, extracted the data from figures and graphs. We also extracted the following data: trial characteristics, patient demographic variables, diagnosis, treatment and data about trial methodology. Online supplementary appendix table 1 presents a full list of extracted data items.

We used a priori-defined decision rules for data extraction:

1. When triallists reported final values and change from baseline values for the same outcome, we extracted the final values.

2. When triallists reported unadjusted and adjusted values for the same outcome, we extracted the unadjusted values.

3. When triallists reported data based on the intention-to-treat (ITT) sample and another sample (eg, per-protocol, as-treated), we extracted ITT-analysed data.

Risk of bias assessment

Two independent reviewers assessed the risk of bias in included trials using the Cochrane Risk of Bias Tool21 and assessed all the publications including protocols. We categorised outcomes as patient-reported or clinician-assessed, and classified outcome measures with mixed patient-reported and clinician-assessed items as patient-reported. When a trial had multiple follow-up time points, we assessed these individually, but the overall risk of bias was judged prioritising follow-up of 6 months to 1 year.

We used the Quality In Prognosis Studies tool22 to assess the risk of bias in observational studies contributing data to the estimate of serious harms.

Data management

When trials used different outcome measures to evaluate the same construct, we chose the most common outcome measure as the index and transformed mean differences (MD) and SD of other outcome measures to the index instrument, and pooled the data using MD as the summary estimate.23 24 When large variations in SDs led to problematic weights in the meta-analysis, we pooled standardised mean differences (SMD). In this case, the SMDs were back-transformed to the scale of the index outcome measure. If SDs or 95% CIs were unavailable, we imputed the SD using the most representative study.

For trials not reporting the index instrument, we followed a prespecified outcome hierarchy when deciding which data would be pooled (online supplementary appendix table 2). For Farfaras et al,25 we combined15 the open and arthroscopic subacromial decompression surgery arms in all analyses.

Data synthesis and analysis

In the meta-analysis, we synthesised continuous outcome data using MDs and SDs. We estimated the absolute values in the control group by applying the median of the means, and in the intervention group by summing the pooled MDs and SDs to the medians of the means. For dichotomous outcome data, we used relative risk for synthesis. We calculated the absolute estimates of effect by applying relative risk estimates from the meta-analysis to a baseline risk, for which we used the pooled mean of the control groups of the included trials. We used RevMan V.5.3 (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014) for the meta-analyses, with the generic inverse-variance random-effects model for continuous outcomes, and Mantel-Haenszel method and random effects for dichotomous outcomes. We assessed statistical heterogeneity with the χ² and I² statistics.

We conducted meta-analyses, guided by considerations of bias. For the primary comparison, we pooled data from comparisons at low risk of bias. For the secondary comparison, we pooled data from comparisons irrespective of risk of bias.

We assessed outcomes at 3 months, 6 months, 1 year, 2 years (for which we pooled data up to 3 years if no 2-year data were available), 5 years (we prioritised time points closest to 5 years) and >10 years following randomisation.

For harms, we calculated incidence proportions using a generalised linear model, using a binomial distribution and an identity link function.

Certainty of evidence

For all outcomes, we assessed the certainty of evidence of benefits of surgery compared with other treatments and harms of surgery using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. We appraised the potential limitations due to risk of bias, inconsistency, imprecision, indirectness and publication bias. We created summary of findings tables using MAGICapp (www.magicapp.org) to provide evidence summaries with relative and absolute effects across all outcomes and associated certainty.

Benefits

The search yielded 842 records. Of these, we considered the full texts of 15 trials for inclusion. After full-text review, 9 trials (18 publications including 2 trial protocols) fulfilled the eligibility criteria (see flow chart in online supplementary appendix figure 1).12 13 25 28–42

Study characteristics

The 9 eligible trials included 1014 patients. The mean age at baseline ranged from 43 to 59 years (table 2 and online supplementary appendix table 3). Two trials12 13 compared subacromial decompression surgery with placebo surgery. Both trials also included a non-surgical arm. Five trials25 30 33 35 41 compared subacromial decompression surgery with exercise therapy; one had a placebo laser arm with no active treatment.30

One trial34 compared acromioplasty plus bursectomy with bursectomy alone. One trial42 compared acromioplasty with bursectomy plus radiofrequency-device microtenotomy. We excluded both trials from the meta-analysis because they compared different approaches to subacromial decompression surgery.

Risk of bias assessment

For the comparisons of subacromial decompression surgery versus placebo surgery, the risk of bias was low for all outcomes (online supplementary appendix table 4). Due to detection bias (all studies), selection bias,25 attrition bias25 35 and selective reporting,33 41 for the comparisons of subacromial decompression surgery versus non-surgical treatment, the risk of bias was high for all outcomes except rotator cuff tears (online supplementary appendix table 4).

Primary comparisons

Two trials12 13 were at low risk of bias and sufficiently clinically and methodologically homogeneous to allow pooling for the comparison of subacromial decompression surgery plus postoperative rehabilitation versus placebo surgery plus postoperative rehabilitation. Table 3 and online supplementary appendix table 5 present the GRADE summary of findings for outcomes for this comparison.

For pain (VAS, 0–10, lower better, MID=1.5 units), we found high certainty evidence of no benefit of surgery at 6 months (2 trials, 299 participants: MD 0.07, 95% CI −0.51 to 0.64, I²=0%, p=0.82) or at 1 year (2 trials, 284 participants: MD −0.26, 95% CI −0.84 to 0.33, I²=0%, p=0.39) (figure 1, figure 2). One trial reported pain at 3 months and 2 years (online supplementary appendix table 5).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Forest plots of pain at (A) 6 months and (B) 1 year. Dashed vertical line denotes minimally important difference (1.5 units, lower better). ASAD, arthroscopic subacromial decompression surgery,IV, inverse-variance.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Change in pain over time for all available time points. Control group (diagnostic arthroscopy and exercise therapy) values are medians of the mean pain values in the trials. Intervention group values are control groups values + mean differences with 95% CIs from the meta-analysis (VAS, visual analogue scale, MID=1.5 units). MID, minimally important difference; Mod, moderate; NRS, numeric rating scale.

For function (Constant Score, 0–100, higher better, MID=8.3 points), we found high certainty evidence of no benefit of surgery at 6 months (2 trials, 286 participants: MD −3.7, 95% CI −8.7 to 1.3, I²=30%, p=0.08) or at 1–2 years (2 trials, 274 participants: MD 2.8, 95% CI −1.4 to 6.9, I²=0%, p=0.19) (figure 3, figure 4).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Forest plots of function at (A) 6 months and (B) 1–2 years (Beard et al,12 1 year; Paavola et al,13 2 years). Dashed vertical line denotes minimally important difference (8.3 points, higher better). ASAD, arthroscopic subacromial decompression surgery; IV, inverse-variance.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Change in function over time for all available time points. Control group (diagnostic arthroscopy and exercise therapy) values are medians of the mean function values in the trials. Intervention group values are control groups values + mean differences with 95% CIs from the meta-analysis. Constant Score MID=8.3 points. Data pooled from 1-year and 2-year time points are presented at 18 months. MID, minimally important difference.

For global perceived effect, we found moderate certainty evidence of no benefit of surgery at 6 months (2 trials, 293 participants: relative risk (RR) 1.04, 95% CI 0.81 to 1.34) or at 1 year (2 trials, 290 participants: RR 1.10, 95% CI 0.94 to 1.30) (online supplementary appendix figure 2).

For health-related quality of life (EQ-5D, −0.59 to 1, MID 0.07 units), we found high certainty evidence of no benefit of surgery at 6 months (2 trials, 292 participants: SMD −0.05, 95% CI −0.27 to 0.18, which back-transformed to −0.01 points, 95% CI −0.08 to 0.05 in EQ-5D) or at 1 year (2 trials, 285 participants: SMD −0.09, 95% CI −0.39 to 0.21, which back-transformed to −0.03 points, 95% CI −0.11 to 0.06 in EQ-5D) (online supplementary appendix figure 3).

One trial reported return to work and return to sport/leisure activities, providing low certainty evidence of no benefit of surgery in participation in work or sport/leisure activities at all time points (online supplementary appendix table 5).

Studies included in the primary comparison provided no data regarding the prevalence of full-thickness rotator cuff tears at follow-up.

Secondary comparisons

Comparisons in six trials25 30 33 35 40 41 were at high risk of bias and sufficiently clinically and methodologically homogeneous to allow pooling for the comparison of subacromial decompression surgery plus postoperative rehabilitation versus exercise therapy alone. Online supplementary appendix table 6 presents the GRADE summary of findings for this comparison.

For pain (VAS, 0–10, MID 1.5 units), we found moderate certainty evidence of no patient-important benefit of surgery at 3 months (4 trials, 361 participants: MD −0.55, 95% CI −1.24 to 0.14) and 6 months (4 trials, 399 participants:  MD −0.56, 95% CI −1.09 to −0.02), and at 2 years (3 trials, 352 participants:  MD −0.44, 95% CI −1.37 to 0.48) (table 3, online supplementary appendix figure 4) and 5 years (2 trials, 188 participants:  MD 0.36, 95% CI −1.17 to 1.89). At 1 year (3 trials, 316 participants:  MD −1.01, 95% CI −1.60 to −0.42) we found low certainty evidence of a small benefit, which is likely unimportant for patients (figure 2, online supplementary appendix figure 4). One trial reported pain at 10 years.

For function (transformed to Constant Score, 0–100), we found low certainty evidence of no patient-important benefit at 3 (3 trials, 257 participants:  MD 6.1, 95% CI −5.6 to 17.8) and 6 (4 trials, 398 participants:  MD 3.7, 95% CI −2.3 to 9.6) months, and at 1 (3 trials, 259 participants:  MD 3.2, 95% CI −8.1 to 14.6), 2 (5 trials, 467 participants:  MD 4.9, 95% CI 0.8 to 9.1) and 5 (2 trials, 157 participants:  MD 7.6, 95% CI 0.2 to 15.1) years. At 10 years, we found low certainty evidence of a mean patient-important benefit (2 trials, 156 participants:  MD 9.5, 95% CI 1.9 to 17.2) (figure 4, online supplementary appendix figure 5). The CI included values that were lower than the MID.

We found moderate certainty evidence of no benefit in the number of patients working at 6 months (2 trials, 187 participants:  RR 1.05, 95% CI 0.81 to 1.36) and 2 years (2 trials, 183 participants:  RR 0.87, 95% CI 0.70 to 1.07). Only single-study data were available for working status at 3 months, 1 year and 10 years. One study reported return to sports/leisure activities, with no difference between groups (online supplementary appendix table 6, online supplementary appendix figure 6).

We found low certainty evidence of no benefit of surgery in the incidence of full-thickness supraspinatus tendon tears (diagnosed with MRI36) at 5 years (1 trial, 90 participants:  RR 1.00, 95% CI 0.40 to 2.52). 31 (online supplementary appendix table 6).

Harms

Both placebo-controlled trials reported harms,15 16 while two of the trials with non-surgical comparisons reported that there were no harms33 38 and three did not mention harms.25 33 41 Overall the included trials provide low certainty evidence regarding harms due to small sample size and incomplete reporting. The two placebo-controlled trials reported five participants experiencing frozen shoulder in the subacromial decompression group and three in the placebo surgery group (table 3). Four participants experienced frozen shoulder in the non-surgical groups. One patient who received placebo surgery experienced temporary swelling related to a brachial plexus block.

The search for studies addressing harms yielded 2363 records, of which 140 were assessed in full text for eligibility. We excluded 138 of these publications (online supplementary appendix figure 1). The data of the four remaining studies came from the same large prospective registry designed to record harms.43 Of the four studies, one assessed harms in patients over 65 years and was excluded due to unrepresentative patient population. Of the remaining three, one was excluded as it enrolled a subset of patients in the two other studies, leaving two studies that were included. The studies addressed 30-day harms after a variety of shoulder arthroscopic procedures including subacromial decompression surgery. We found no eligible studies reporting data separately for subacromial decompression surgeries.

We based our estimates of harms on two studies from the same registry, reporting harms over two slightly overlapping time periods.44 45 The studies included 15 01544 and 10 22545 patients, both at low risk of bias (online supplementary appendix table 7). The incidence of serious harms following mixed shoulder arthroscopic procedures during 2006–2011 was 0.5% (0.4%–0.7%)45 and during 2011–2013 was 0.6% (0.5%–0.7%).44 Online supplementary appendix table 8 presents the details of harms. Longer duration of surgery (>1.5 hours) was associated with harms (OR in one study 1.80, 95% CI 1.29 to 2.5043; adjusted OR in the other 1.93, 95% CI 1.21 to 3.0745). One report noted that harms from ‘major’ or ‘repair’ procedures were similar to harms for ‘minor’ or ‘non-repair’ procedures (adjusted OR 0.77, 95% CI 0.48 to 1.22).45 Because these studies reported results from a variety of arthroscopic shoulder procedures, many of which take longer to perform, we rated down the evidence to moderate certainty.

Strengths and limitations of the systematic review

Our systematic review includes evidence from placebo surgery-controlled trials, is based on a comprehensive search for observational studies on harms and directly involved patients. Our results extend previous research by including high certainty evidence from placebo-controlled trials and by considering, through establishing best estimates of MIDs (H Qea, in review), results in light of what patients would consider important. Our findings are consistent with previous systematic reviews20 48 that found very limited evidence of any benefit of subacromial decompression surgery versus exercise therapy for treating SAPS. We made an a priori decision to extend our analysis of potential harms to include observational data because RCTs will be underpowered for serious rare harms.

We invited patients to guide us in prioritising outcomes for our systematic review, an important step in ensuring that research aiming to guide shared decision-making addresses the issues most important to patients.49

Inferences and implications

On average, patients with SAPS reported reduced pain, and improved physical function and quality of life following both surgical and non-surgical treatment. However, at up to 5 years, irrespective of treatment, patients continued to report pain of an average of 1.5–3 on a scale of 0–10 points on a visual analogue scale. Clinicians working in primary care who are treating patients with SAPS should be aware that some patients experience prolonged symptoms and consider care strategies to support coping.50–52

A placebo control helps answer the research question ‘is there a benefit of subacromial decompression surgery?’ because it minimises the risk of detection and performance biases. Both sources of bias contribute to overestimation of treatment effects by up to 20%.53 The largely consistent findings of the unblinded studies leave little doubt of the inference that subacromial decompression surgery provides no important benefit to patients.

The current evidence provides no support for subacromial decompression surgery as an intervention providing important benefit for patients with SAPS. High-quality evidence indicates that surgery versus placebo surgery confers no important benefit on pain and function—the outcomes most important to patients. Considering the body of evidence, further head-to-head comparisons of subacromial decompression surgery compared with placebo surgery or non-surgical management with the same population are unlikely to change the results. Policymakers, funders and clinicians should consider these results in their funding and clinical decisions regarding the management of patients with shoulder pain.

Unanswered questions and future research

Our review was designed to assess the benefits and harms of subacromial decompression surgery for managing SAPS. To date, no trial has demonstrated benefit of surgery for any clinical subgroup. In the future, subgroup claims should be supported by data from well-conducted trials at low risk of bias and the use of established criteria for credibility of subgroup effects,54 ideally enhanced by individual participant data meta-analysis.55 For harms we welcome well-performed observational studies that specifically report harms after subacromial decompression surgery separately from harms following other types of shoulder surgery.

Although the finding of no important benefit of surgery is robust, the root cause of subacromial pain and the underlying pathological process remain uncertain, as does the possible best treatment—if such exists—for SAPS. Network meta-analysis might provide information on this question and provide hypotheses to be tested in future methodologically sound trials. Future triallists investigating any treatment for SAPS should adopt a common set of outcomes,18 and the outcome measures should be standardised.56