You are here

  • Home
  • Archive
  • Volume 54, Issue 22
  • Improving concussion education: consensus from the NCAA-Department of Defense Mind Matters Research & Education Grand Challenge

Article Text

Article menu

Download PDFPDF

Consensus statement
Improving concussion education: consensus from the NCAA-Department of Defense Mind Matters Research & Education Grand Challenge
Free
  1. Emily Kroshus1,2,
  2. Kenneth L Cameron3,
  3. J Douglas Coatsworth4,
  4. Christopher D'Lauro5,
  5. Eungjae Kim6,
  6. Katherine Lee7,
  7. Johna K Register-Mihalik8,
  8. Jeffery J Milroy9,
  9. E Paul Roetert10,
  10. Julianne D Schmidt11,
  11. Ross D Silverman12,
  12. Dee Warmath13,
  13. Heidi A Wayment14,
  14. Brian Hainline10
  1. 1 Department of Pediatrics, University of Washington, Seattle, Washington, USA
  2. 2 Department of Pediatrics, Seattle Children's Research Institute, Seattle, Washington, USA
  3. 3 Orthopaedics, Keller Army Community Hospital, West Point, New York, USA
  4. 4 Human Development and Family Studies, Colorado State University, Fort Collins, Colorado, USA
  5. 5 Behavioral Science and Leadership, US Air Force Academy, Colorado Springs, Colorado, USA
  6. 6 Emory University, Atlanta, Georgia, USA
  7. 7 Health Readiness and Policy Oversight, Health Affairs, Department of Defense, Falls Church, Virginia, USA
  8. 8 Exercise and Sport Science, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
  9. 9 Public Health Education, University of North Carolina at Greensboro, Greensboro, North Carolina, USA
  10. 10 National Collegiate Athletic Association, Indianapolis, Indiana, USA
  11. 11 Kinesiology, University of Georgia, Athens, Georgia, USA
  12. 12 Richard M Fairbanks School of Public Health, Indiana University, Indianapolis, Indiana, USA
  13. 13 Family and Consumer Sciences, University of Georgia, Athens, Georgia, USA
  14. 14 Psychological Sciences, Northern Arizona University, Flagstaff, Arizona, USA
  1. Correspondence to Dr Emily Kroshus, Department of Pediatrics, University of Washington, Seattle, Washington, USA; ekroshus{at}u.washington.edu

Abstract

Early disclosure of possible concussive symptoms has the potential to improve concussion-related clinical outcomes. The objective of the present consensus process was to provide useful and feasible recommendations for collegiate athletic departments and military service academy leaders about how to increase concussion symptom disclosure in their setting. Consensus was obtained using a modified Delphi process. Participants in the consensus process were grant awardees from the National Collegiate Athletic Association and Department of Defense Mind Matters Research & Education Grand Challenge and a multidisciplinary group of stakeholders from collegiate athletics and military service academies. The process included a combination of in-person meetings and anonymous online voting on iteratively modified recommendations for approaches to improve concussion symptom disclosure. Recommendations were rated in terms of their utility and feasibility in collegiate athletic and military service academy settings with a priori thresholds for retaining, discarding and revising statements. A total of 17 recommendations met thresholds for utility and feasibility and are grouped for discussion in five domains: (1) content of concussion education for athletes and military service academy cadets, (2) dissemination and implementation of concussion education for athletes and military service academy cadets, (3) other stakeholder concussion education, (4) team and unit-level processes and (5) organisational processes. Collectively, these recommendations provide a path forward for athletics departments and military service academies in terms of the behavioural health supports and institutional processes that are needed to increase early and honest disclosure of concussion symptoms and ultimately to improve clinical care outcomes.

  • concussion
  • consensus
  • implementation
  • education
  • knowledge translation

https://doi.org/10.1136/bjsports-2020-102185

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Early and honest disclosure of possible concussive symptoms may improve concussion-related clinical outcomes. Emerging data suggest that earlier evaluation for concussion is associated with sooner return to full activity.1 2 Collegiate athletes with undiagnosed concussions are at elevated risk of future concussions.3 Early clinical care diminishes the risk of undiagnosed catastrophic injury, and limits the potential for additional brain trauma during a window of cerebral vulnerability.4 In addition, early care provides the potential for targeted management and treatment with the goal of reducing the likelihood of developing persistent postconcussion symptoms. As there is no error-free objective measure of concussion diagnosis5 and many issues experienced by patients post concussion are not visible to an external observer6, early clinical evaluation requires at least some element of symptom self-disclosure by the injured individual. This may involve directly seeking medical care, or telling a trusted adult or peer about symptoms to facilitate the care-seeking process. Despite the benefits of early care seeking, estimates suggest that in some sports more than half of adolescent and young adult athletes have continued to play while experiencing potential concussive symptoms, with many suspected concussions never receiving medical attention.7

    The design and implementation of effective efforts to increase symptom disclosure have not yet been comprehensively addressed through leading consensus documents such as the 2016 Berlin Consensus Statement on Concussion.5 Addressing this important behavioural health issue and gap in the literature, the National Collegiate Athletic Association (NCAA) and Department of Defense established the NCAA-Department of Defence Mind Matters Research & Education Grand Challenge (Grand Challenge). Collegiate athletes and military service members are often of similar age and have been identified as ‘at-risk’ groups for sustaining concussions, due to their participation in activities with elevated risk of contact or collision.8 In addition, both groups have goal-focused social environments that may encourage individuals to ‘participate through’ concussion symptoms.9 The partnership between the NCAA and Department of Defence recognises the highly aligned nature of the concussion disclosure-related challenges for these two groups. For both the Research and Education Challenge, a Request for Proposals for competitive grant awards was issued and research teams were selected and funded. Eight proposals were funded under the Research Challenge, whose projects ran from 2016 to 2019. Six proposals were funded under the initial Education Challenge, which ran from 2015 to 2016, with one group (Chestnut Hill College) receiving continued funding from 2017 to 2019. More detail about the Research and Education Challenge and funding recipients is available at: http://www.ncaa.org/sport-science-institute/topics/mind-matters-challenge. At the conclusion of the period of funding, the NCAA and Department of Defense sought to synthesise findings relevant to increasing concussion disclosure and inform guidance to member organisations.

    Despite a growing body of research on predictors of disclosure behaviour,7 ,10 ,11–17 literature in this area is not yet at a stage where a systematic review or meta-analysis can indicate clear steps forward for collegiate athletic departments, service academies, or the Department of Defense. One key reason is that predictor variables are inconsistently operationalised and measured. A second key reason is that most studies in this area lack a behavioural outcome, with behavioural intentions often used as a proxy for behaviour.7 In the few studies that have prospectively measured disclosure intentions and behaviour, the association is statistically significant but explains less than half of the variance in behaviour.18 Third, whether intentions or actual behaviour are being measured, what counts as disclosure is defined in varying ways across studies. Some ways that lack of disclosure has been operationalised include failing to seek care immediately once symptoms are onset, never seeking care or playing through potentially concussive symptoms.18 19 Finally, determining which statistically significant associations are practically meaningful, and translating them into actionable recommendations for institutions requires considering local implementation-related constructs such as feasibility and thus involves some subjectivity. In light of these limitations to the current literature and the importance of considering implementation feasibility, a formalised consensus process using the Delphi method was determined to be a useful strategy for developing recommendations.20–22 Thus, in June 2019, NCAA and Department of Defense convened a meeting of experts and key stakeholders to use the Delphi method to identify feasible and useful recommendations that institutions (athletics departments, military service academy leaders and the Department of Defense) can implement to increase concussion disclosure. The goal of this paper is to describe this consensus process, discuss the resultant recommendations in the context of implementation in collegiate athletics and military settings, and identify areas for future research and programme development.

    Methods

    Sample and procedure

    The consensus process was guided by the Delphi method, an unbiased and efficient approach for aggregating opinions of content experts and key stakeholders.21 Participants were attendees at the June 2019 Mind Matters Task Force meeting and included academic researchers funded by Grand Challenge grants, individuals who develop and implement concussion-related policy and programming (eg, athletic administrators across levels of competition in the NCAA, military health officials) and stakeholders affected by the recommendations (eg, coaches and athletes across levels of competition in the NCAA). Mind Matters grant recipients were the eight Research Challenge teams and the one Education Challenge Team that received continued funding for programme development and evaluation. Other stakeholders were recruited using a key informant-led approach: stakeholders from the NCAA membership were chosen by the representative divisional governance and student–athletes were chosen by the national Student-Athlete Advisory Committee. A list of all panel participants is provided in the online supplementary. A two-phase modification of the Delphi process was used, inclusive of an exploration and evaluation phase,21 as described below. This process was reviewed and classified as exempt by the University of Washington Institutional Review Board.

    Supplemental material

    Exploration phase

    Participants in the consensus process attended a meeting hosted by the NCAA and Department of Defense in June of 2019. In preparation for this meeting, the nine grant funded research teams shared presentations of the core findings of their research programme related to understanding and increasing concussion disclosure among athletes and/or military service members. Copies of each presentation were made available prior to the meeting for participant reference and review, and attendees also were provided with the 21 peer-reviewed published papers resultant from the Grand Challenge-funded projects to date. Opportunities for group discussion about these findings were provided.

    Prior to the meeting, each of the nine research teams provided one or more statements summarising what they considered to be the most important takeaways from their individual projects’ findings as well as the collection of projects that were funded. An internal working group reviewed these 26 statements and noted that there were areas of overlap across groups and that not all statements were worded as actions that institutions could take to increase concussion disclosure. Thus, a thematic analysis was conducted and statements were grouped initially into seven emergent themes (eg, cognitive targets of education, multilevel problem, organisational values).23 Statements within each theme were reviewed and synthesised/combined where necessary, retaining original terminology where possible, to create 12 provisional recommendations. The goal of these synthesised recommendations was to reflect the content of the research group submissions while (1) reducing redundancy across groups, and (2) making the recommendations actionable for institutions. The original statements, thematic analysis documentation and provisional recommendations were shared with participants.

    Evaluation phase

    At the in-person meeting, participants reviewed the original statements and provisional recommendations, with the goal of ensuring that the provisional recommendations were (1) clearly worded and (2) reflected the science informing the original statements. As a result of this process, edits were made to most statements, and 12 new statements were added, separating double-barrelled concepts within statements so that they could be voted on independently and adding key content that was not included in the original submitted statements. Participants were then provided with a link to an anonymous online survey and asked to indicate the utility and feasibility of each recommendation. Utility was defined as whether, if implemented, the recommendations would meaningfully improve concussion disclosure. Feasibility was defined as whether the recommendation could reasonably be implemented by institutions (inclusive of collegiate athletic departments of varying resource levels and military service academies). Consistent with prior models of Delphi process voting,20 24 each dimension was rated on a 9-point scale, where higher scores indicated greater utility/feasibility. After each recommendation, participants were provided with an open-ended space for comment and encouraged to describe their rationale if they scored an item 6 or less.

    Subsequently, the mean and standard deviation (SD) for utility and feasibility were calculated for each recommendation. For a recommendation to be included in the final set, it required mean utility and feasibility scores of 7 or more.24 25 Recommendations scoring 3 or below were dropped from consideration while those with mean scores of 4, 5 or 6 on either dimension were considered to lack consensus and were revisited. Participants met in person and were provided with the mean scores for each item, along with the aggregated open-ended feedback. For each recommendation lacking consensus, participants worked in small groups to review the aggregated open-ended feedback, to identify emergent themes to be addressed and to generate potential wording modifications to statements. A facilitated full-group discussion about each statement followed, during which small group discussion points were shared and core components of statement revisions were determined. After this in-person meeting, a small working group (led by the lead author along with representatives with expertise in health education from the NCAA and Department of Defense) synthesised the in-person and written comments and feedback to finalise a second iteration of recommendation on statements that lacked consensus in the first round of voting.

    The remainder of the consensus process occurred asynchronously following the same process. Participants were emailed a second survey link containing these updated statements, with voting again anonymous and occurring online. As in the first round, descriptive statistics were tabulated and reviewed by the working group, with edits made to statements not reaching consensus based on themes emergent from the open-ended comments. In most cases, feedback indicated relatively minor wording changes to make the recommendations more concrete or inclusive; those changes were prioritised. This continued through a total of three rounds of voting, open-ended feedback review and statement editing and included combining several statements for final dissemination given conceptual overlap. Statements not reaching consensus after three rounds of voting were not included in the final set of recommendations.

    Results and discussion

    A total of 17 statements met identified thresholds for utility and feasibility after 3 rounds of voting (see table 1 list of final recommendations and online supplementary table 1 for scores). Statements that were not included in the final set were eliminated because consensus could not be reached on feasibility. The final list of statements is presented below, separated into five conceptual domains: (1) content of concussion education for athletes and military service academy cadets, (2) dissemination and implementaiton of concussion education, (3) other stakeholder concussion education, (4) team-level and unit-level processes, and (5) organisational processes. Each domain is briefly discussed, with a focus on providing context for the recommendation and proposing how the recommendations can be made actionable by organisations.

    Supplemental material

    View this table:
    Table 1

    Recommendations for institutions

    Domain 1: content of concussion education for athletes and military service members

    Provide athletes/service members with education that addresses:

    • Recommendation 1: the potential dilemma individuals face when deciding to disclose a concussion (eg, tradeoffs, concerns about what might happen next, knowing how to report, etc)

    • Recommendation 2: short-term benefits of early concussion symptom disclosure (eg, athletic, academic, occupational).

    • Recommendation 3: what is known about possible long-term manifestations of concussion and head injury.

    • Recommendation 4: concussion-related misperceptions (eg, knowledge gaps).

    • Recommendation 5: site-specific information regarding institutional concussion resources and policies (eg, steps to take if an individual suspects they have a concussion).

    There was agreement about the importance of addressing the disclosure decision explicitly, acknowledging the individual will be considering the tradeoffs between potential benefits and consequences of disclosure at a given time in their setting.26 Information can then be provided about the potential benefits of early, and honest care seeking. This information should emphasise short-term benefits to the individual (eg, ‘seeking care for a concussion will help you feel better sooner’) as opposed to fear tactics. While there are a growing number of approaches to concussion education,27 28 few, if any, address all of the content domains listed in recommendations 1-5. Critically, all of these content domains do not need to be addressed in a single educational programme, session or set of materials. Rather, they could be addressed through a combination of approaches including team discussions and other daily interactions. Because of the localised nature of some of the recommendations (institutional policies and resources, and misperceptions), providing concussion education consistent with these recommendations will require the engagement of institutional stakeholders to tailor content to their setting. Institutional stakeholder teams should use this list to review their planned approach to education to ensure that, in sum across all modalities and time points of education, all relevant content areas are being addressed. This may mean supplementing a resource such as the NCAA’s Concussion Fact Sheet for Athletes (which addresses short-term benefits of disclosure, what is known about possible long-term manifestations of concussion and head injury) with a discussion led by medical personnel, in conjunction with coaches or leaders in the military chain of command. Such a discussion could focus on the disclosure decision, providing site-specific information on the actions to take to report symptoms.29 As more educational approaches and tools for student–athletes and military service members are developed and evaluated, guidance should be provided to institutions about which resources address recommended content and have demonstrated efficacy in improving disclosure behaviours.

    Domain 2: dissemination and implementation of concussion education

    • Recommendation 6: actively collaborate with organisational stakeholders (including coaches/commanders, primary healthcare providers, athletes/service members, military chain of command) to select concussion education approaches that are engaging, interactive and foster discussion.

    • Recommendation 7: share messaging about concussion symptom disclosure on a regular basis and in a variety of ways (eg, formal education, informal conversation, posters).

    • Recommendation 8: integrate messaging about the importance of complete concussion symptom disclosure throughout the recovery process.

    Concussion education, no matter the content, will not be effective in changing behaviour if the athlete or military service academy cadet is not attentive,16 30 underscoring the importance of selecting engaging, interactive and population-relevant educational modalities.17 31 Critically, lack of attention to dissemination and implementation strategies can result in learning environments that undermine educational messaging.32 The three listed recommendations related to dissemination and implementation of educational messaging (recommendations 6, 7 and 8) represent both a guide and a paradigm shift, in which dissemination and implementation are viewed as central to the ongoing concussion education process. This includes engaging student–athletes/service members and other key stakeholders in the education planning process and sharing information throughout the year, in multiple modalities, from multiple sources, using formal and informal methods (eg, programmatic education, conversation, signage or other static reminders).33 Educational messaging can target smaller groups of individuals according to their different learning needs and preferences,11 34 motivational goals7 and setting (eg, sport, military). Institutions will also have different resources and barriers to sharing information. Thus, it is essential that concussion education dissemination and implementation is an ongoing process of stakeholder engagement with local tailoring rather than a static one-size fits all approach.

    Domain 3: concussion education for other stakeholders

    • Recommendation 9: provide coaches and military leaders with evidence-based concussion education that is aimed at supporting athletes/service members in concussion symptom disclosure.

    • Recommendation 10: provide sports medicine/front-line medical staff with strategies about how to engage coaches/leaders in the military chain of command in supporting athletes/service members in concussion symptom disclosure.

    • Recommendation 11: provide easily accessible information to parents/guardians about how to support athlete/service member concussion symptom disclosure.

    • Recommendation 12: provide easily accessible information to other key site-specific stakeholders (eg, student-life administrators, faculty athletic representatives, leadership, military chain of command) about how to support athlete/service member concussion symptom disclosure.

    Explicit verbal communication supportive of concussion disclosure from key stakeholders, inclusive of coaches and parents as well as military leadership is an important strategy for shaping athlete/service member beliefs about whether disclosure is valued in their setting.16 35–38 The present recommendations indicate that institutions can help encourage these disclosure-supportive interactions through the educational materials they proactively share with coaches/leaders or make available for parents/guardians. While the present consensus process did not specify content of these educational materials, similar content as provided to athletes may be beneficial to motivate action among non-athlete stakeholders. This would include information about the benefits of early concussion symptom disclosure (eg, athletic/military readiness, health, academic, occupational) as well as stakeholder-tailored content about how they can help create an interpersonal context that encourages concussion disclosure. One specific strategy is explicit verbal communication about the importance of concussion disclosure—by medical staff to coaches/leaders, by coaches/leaders to their team,16 and by parents/guardians to their child.36 The NCAA’s Concussion Fact Sheet for Coaches has improved intentions to engage in disclosure-supportive communication with their team,39 and is a feasibly disseminated resource for coaches. An in-person discussion between sports medicine staff and coaches has the potential to augment the impact of this handout or other coach education resources (e.g., providing opportunity for coach questions to be answered and strengthening trust between coaching and medical staff). Emerging evidence suggests that in military settings individuals such as those along the chain of command play a critical role in shaping a disclosure supportive environment,19 and may be similarly appropriate targets for education to encourage communication with military service members about concussion disclosure. More research is needed about disclosure-supportive communication in military settings; however, stakeholder-engaged adaptation of the NCAA’s Concussion Fact Sheet for Coaches to the military setting may be an appropriate starting point for developing effective educational content.

    Domain 4: team-level and unit-level processes

    • Recommendation 13: provide athletes/service members with education that addresses the role they can play in encouraging peers to disclose possible concussion symptoms (eg, share evidence-based bystander education programming).

    • Recommendation 14: provide opportunity for team members and coaches/leaders in the military chain of command to discuss and establish team values that are supportive of concussion symptom disclosure.

    Team-level and unit-level, in addition to individual-level, activities are needed to establish and maintain disclosure-supportive cultures. Current research suggests that many collegiate student–athletes overestimate their teammates’ willingness to play through a concussion.40 41 Providing explicit opportunities to discuss and clarify individual and team values is an important strategy for correcting these misperceived norms. Respected team leaders (eg, captains, coaches, military chain of command) are natural facilitators for such a process.7 Explicit messaging that places teammates as responsible and safety-supportive bystanders is another important strategy for creating disclosure supportive team cultures.34 Teammates can help encourage care seeking directly by noticing potentially symptomatic peers. They can also express support for adherence to concussion evaluation and recovery protocols, limiting pressure on peers to try to return to play as quickly as possible. The NCAA’s Fact Sheet for Athletes provides content that specifically addresses the teammate-as-prosocial bystander role. Content about the importance of bystander engagement to facilitate concussion disclosure (whether from the NCAA Fact Sheet or other sources) could be shared in a team setting, paired with a discussion about barriers to speaking up about a teammates suspected concussion, and opportunity for establishing explicit team values about looking out for each other. Importantly, coaches should be present at such discussions given the central role they play in shaping team culture.16

    Domain 5: organisational processes

    • Recommendation 15: actively collaborate with organisational stakeholders (including coaches/military leaders, primary healthcare providers, athletes/service members) to identify and address organisational barriers to concussion symptom disclosure.

    • Recommendation 16: evaluate the effectiveness of institutionally selected concussion education approaches in changing athlete/service member concussion symptom disclosure behaviour.

    • Recommendation 17: communicate in a deliberate manner institutional values that emphasise safety and its importance in athletic performance/military readiness.

    These statements recognise the importance of organisational (eg, athletic department, military unit, military service) processes on disclosure behaviour. Organisations should consider how their stated and unstated values might support or undermine disclosure-related initiatives.42 Organisational stakeholders, including administrators and military leaders, should discuss ways that unstated institutional values (such as playing through pain, etc) that are counter to concussion safety might be internalised by athletes/service members in their setting. In the military, this has been a recent topic of discussion for other mental health issues, with organisational and leadership support identified as important facilitators of a culture that reduces stigma related to care seeking.43 With respect to concussion, the present results suggest that deliberate messaging about safety-supportive organisational values, and alignment of these values with other core organisational values (eg, athletic performance, military readiness) is recommended. This can help create a culture where messaging about concussion safety is viewed as credible by key stakeholders.37 Central to all organisation-level recommendations is collaboration, stakeholder engagement and local tailoring. Working with student–athletes/service members can help institutions select approaches to intervention that address setting-specific needs and resources. Thus, it is recommended that institutions have an ongoing process for gauging the effectiveness of their selected approach(es) to concussion education in changing disclosure behaviour. Ideally, this would be in the form of valid and reliable common data elements related to concussion disclosure that would serve the military, athletics and the general public. Such data collection could also have the benefit of informing guidance at a national level by helping build a sufficient evidence base for systematic quantitative comparisons of the effectiveness of different educational approaches and elements. At present, such disclosure-related common data elements do not exist; their development and validation are an important area for future research.

    Conclusion

    Results of the consensus process suggest the importance of applying a social-ecological approach to address the problem of how to increase concussion disclosure.14–16 This means considering determinants of behaviour at multiple levels. Recommendations were inclusive of the beliefs of the individual at risk of concussion (athlete or service member), key interpersonal relationships supportive of individual behaviour, and processes at the team, unit and organisational level. Adopting this multilevel, multistakeholder approach to increasing concussion disclosure is a departure from viewing concussion education as something that is delivered only to athletes or military service academy cadets at a single time point. While there is a need for ongoing concussion education programme development and evaluation work (eg, including content specified and implementation strategies), the recommendations make clear that it is unlikely that a single educational approach delivered at a single point in time is going to be effective for all individuals in all settings.16 Thus, institutions are encouraged to view concussion disclosure as a local cultural issue, and the present recommendations as a guide for establishing and maintaining ongoing processes of concussion education that can create and sustain local cultures supportive of concussion disclosure. Although presentation of these recommendations has emphasised their application for college athletes and military service academy cadets, they can also be considered within the context of youth sports.

    These recommendations underscore the critical and active role of organisations in successful dissemination and implementation of concussion disclosure initiatives. Statements addressed local adaptation, within-setting communication, organisational values, stakeholder engaged planning and evaluation—all components of dominant implementation science frameworks such as the Consolidated Framework for Implementation Research.44 The provided recommendations also align with processes advocated in evidence-based delivery systems in prevention science for changing ‘cultures’ that support unhealthy behaviours and for selecting, adapting and delivering tested and effective programmes.45 Implementation and organisational culture are infrequently addressed in sports medicine research,16 46 limiting the potential impact and sustainability of efforts in a variety of injury-related domains. The process we described here provides a model for making implementation and organisational culture a focus of future consensus efforts in the sports medicine domain. Specifically, the consensus process (1) engaged in diverse group of stakeholders from institutions with varying resource levels, organisational structures and needs, and (2) considered both feasibility and utility as key criterions for recommendation inclusion. Importantly, because feasibility was a key determinant for recommendation inclusion, statements that participants agreed would be useful but not feasible were not retained. As disclosure-supportive interventions and resources are developed, the present consensus process should be revisited and guidance updated.

    To optimise the use of the present recommendations, organisations should establish a multidisciplinary implementation team. This team should review the list of recommendations and conduct an inventory of current practices and gaps. Because of the importance of tailoring educational approaches to local needs, this multidisciplinary team should work with athletes/service members and other stakeholders to identify their specific needs (recommendation 15), evaluate the effectiveness of approaches chosen (recommendation 16), and adapt approaches accordingly. Further work is needed to provide structured support for local implementation teams in this process of assessment and planning. Such implementation support could also include access to educational interventions that are consistent with the present guidelines as such resources are developed. Policy-level changes by governing bodies to align with the present guidelines may also be a useful strategy for encouraging organisation-level implementation, perhaps most efficiently increasing organisational motivation to change by increasing the salience or consequences of non-compliance. Ensuring that such policies are communicated clearly to organisational members and are also readily available to others who may be influenced by them (eg, parents) is critical for their influence on organisational culture and behaviour.

    As the field of research related to concussion disclosure and implementation of disclosure supportive interventions matures, there is a need for systematic reviews and meta-analyses to identify key individual, interpersonal and institutional intervention targets. Such efforts would be facilitated by improved methods of eliciting data about disclosure that are more efficient and comparable than current one-off surveys developed by independent research teams. The results of the present consensus process provide an important starting point for this research, and for including evidence-based approaches to increasing concussion disclosure in conversations about concussion harm reduction in sport and military settings. As more research in this area is conducted, the present recommendations should be revisited and updated as needed.

    Limitations

    The present consensus process was focused on the needs of NCAA member institutions and US Service Academies and literature informing the consensus process was largely from research conducted in these same settings. Results may not be generalisable to sport or military settings beyond the NCAA and US Service Academies.

    What is already known?

    • Concussion education is increasingly mandated for participants in activities with elevated risk of concussion, with the goal of ensuring concussions are disclosed to medical personnel for appropriate evaluation and care.

    • There are no data to indicate that existing approaches to concussion education are associated with lasting changes in concussion disclosure behavior.

    What are the new findings?

    • Key additions to the content of concussion education should include short-term benefits of early symptom disclosure and the potential dilemma individuals face when deciding to disclose a concussion.

    • Improving concussion disclosure requires more than improving the content of concussion education. This includes attending to how concussion education is disseminated and implemented and ensuring that team/unit and organisational processes positively reinforce rather than undermine concussion education messaging.

    Acknowledgments

    The authors thank all participants in the NCAA Mind Matters Task Force.

    References

    1. ↵
      2. Asken BM ,
      3. Bauer RM ,
      4. Guskiewicz KM , et al
      . Immediate removal from activity after sport-related concussion is associated with shorter clinical recovery and less severe symptoms in collegiate Student-Athletes. Am J Sports Med 2018;46:1465–74.doi:10.1177/0363546518757984 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29558195
      OpenUrlPubMed
    2. ↵
      2. Elbin RJ ,
      3. Sufrinko A ,
      4. Schatz P , et al
      . Removal from play after concussion and recovery time. Pediatrics 2016;138:e20160910. doi:10.1542/peds.2016-0910 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27573089
      OpenUrlAbstract/FREE Full Text
    3. ↵
      2. Schmidt JD ,
      3. Rizzone K ,
      4. Hoffman NL , et al
      . Age at first concussion influences the number of subsequent concussions. Pediatr Neurol 2018;81:19–24.doi:10.1016/j.pediatrneurol.2017.12.017 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29523494
      OpenUrlPubMed
    4. ↵
      2. Greco T ,
      3. Ferguson L ,
      4. Giza C , et al
      . Mechanisms underlying vulnerabilities after repeat mild traumatic brain injuries. Exp Neurol 2019;317:206–13.doi:10.1016/j.expneurol.2019.01.012 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30853388
      OpenUrlPubMed
    5. ↵
      2. McCrory P ,
      3. Meeuwisse W ,
      4. Dvořák J , et al
      . Consensus statement on concussion in sport-the 5thinternational conference on concussion in sport held in Berlin, October 2016. Br J Sports Med 2017;51:838–47.doi:10.1136/bjsports-2017-097699 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28446457
      OpenUrlFREE Full Text
    6. ↵
      2. Eisenberg MA ,
      3. Meehan WP ,
      4. Mannix R
      . Duration and course of post-concussive symptoms. Pediatrics 2014;133:999–1006.doi:10.1542/peds.2014-0158 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24819569
      OpenUrlAbstract/FREE Full Text
    7. ↵
      2. Wayment HA ,
      3. Huffman AH ,
      4. Lane TS , et al
      . Relationship of athletic and academic identity to concussion reporting intentions. Musculoskelet Sci Pract 2019;42:186–92.doi:10.1016/j.msksp.2019.04.003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31014920
      OpenUrlPubMed
    8. ↵
      2. Van Pelt KL ,
      3. Allred D ,
      4. Cameron KL , et al
      . A cohort study to identify and evaluate concussion risk factors across multiple injury settings: findings from the care consortium. Inj Epidemiol 2019;6:1.doi:10.1186/s40621-018-0178-3 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30637568
      OpenUrlPubMed
    9. ↵
      2. D'Lauro C ,
      3. Johnson BR ,
      4. McGinty G , et al
      . Reconsidering Return-to-Play times: a broader perspective on concussion recovery. Orthop J Sports Med 2018;6:2325967118760854. doi:10.1177/2325967118760854 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29568786
      OpenUrlPubMed
    10. ↵
      2. Foster CA ,
      3. D'Lauro C ,
      4. Johnson BR
      . Pilots and athletes: different concerns, similar concussion non-disclosure. PLoS One 2019;14:e0215030. doi:10.1371/journal.pone.0215030 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31042725
      OpenUrlPubMed
    11. ↵
      2. Weber ML ,
      3. Suggs DW ,
      4. Bierema L , et al
      . Collegiate student-athlete sex, years of sport eligibility completed, and sport contact level influence on concussion reporting intentions and behaviours. Brain Inj 2019;33:592–7.doi:10.1080/02699052.2019.1568573 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30704294
      OpenUrlPubMed
    12. ↵
      2. Wigfield A ,
      3. Eccles JS
      . Expectancy-Value theory of achievement motivation. Contemp Educ Psychol 2000;25:68–81.doi:10.1006/ceps.1999.1015 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10620382
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      2. Glanz K ,
      3. Rimer BK ,
      4. Viswanath K , et al
      . Research, and practice. John Wiley & Sons, 2015.
    14. ↵
      2. Register-Mihalik J ,
      3. Baugh C ,
      4. Kroshus E , et al
      . A multifactorial approach to sport-related concussion prevention and education: application of the Socioecological framework. J Athl Train 2017;52:195–205.doi:10.4085/1062-6050-51.12.02 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28387550
      OpenUrlPubMed
    15. ↵
      2. Corman SR ,
      3. Adame BJ ,
      4. Tsai J-Y , et al
      . Socioecological influences on concussion reporting by NCAA division 1 athletes in high-risk sports. PLoS One 2019;14:e0215424. doi:10.1371/journal.pone.0215424 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31067230
      OpenUrlPubMed
    16. ↵
      2. Kroshus E ,
      3. Chrisman SPD
      . A new game plan for concussion education. Health Educ Behav 2019;46:916–21.doi:10.1177/1090198119859414 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31296053
      OpenUrlPubMed
    17. ↵
      2. Lininger MR ,
      3. Wayment HA ,
      4. Craig DI , et al
      . Improving Concussion-Reporting behavior in national collegiate athletic association division I football players: evidence for the applicability of the Socioecological model for athletic trainers. J Athl Train 2019;54:21–9.doi:10.4085/1062-6050-47-18 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30721092
      OpenUrlPubMed
    18. ↵
      2. Kroshus E ,
      3. Baugh CM ,
      4. Daneshvar DH , et al
      . Concussion reporting intention: a valuable metric for predicting reporting behavior and evaluating concussion education. Clin J Sport Med 2015;25:243.doi:10.1097/JSM.0000000000000137 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25051193
      OpenUrlCrossRefPubMed
    19. ↵
      2. Register-Mihalik JK ,
      3. Cameron KL ,
      4. Kay MC , et al
      . Determinants of intention to disclose concussion symptoms in a population of U.S. military cadets. J Sci Med Sport 2019;22:509–15.doi:10.1016/j.jsams.2018.11.003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30551922
      OpenUrlPubMed
    20. ↵
      2. Jorm AF
      . Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry 2015;49:887–97.doi:10.1177/0004867415600891 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26296368
      OpenUrlCrossRefPubMed
    21. ↵
      2. Adler M ,
      3. Ziglio E
      . Gazing into the Oracle: the Delphi method and its application to social policy and public health. Jessica Kingsley Publishers, 1996.
    22. ↵
      2. Dalkey N
      . An experimental study of group opinion. Santa Monica, CA: Rand Corp, 1969: 408–26. https://apps.dtic.mil/docs/citations/AD0690498 doi:10.1016/S0016-3287(69)80025-X
    23. ↵
      2. Braun V ,
      3. Clarke V
      . Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101.doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    24. ↵
      2. Rivara FP ,
      3. Ennis SK ,
      4. Mangione-Smith R , et al
      . Quality of care indicators for the rehabilitation of children with traumatic brain injury. Arch Phys Med Rehabil 2012;93:381–5.doi:10.1016/j.apmr.2011.08.015 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22280892
      OpenUrlCrossRefPubMed
    25. ↵
      2. Brook RH
      . The RAND/UCLA appropriateness method: 1995. Available: https://www.rand.org/pubs/reprints/RP395.html [Accessed 21 Oct 2019].
    26. ↵
      2. Foster CA ,
      3. D’Lauro C ,
      4. Johnson BR
      . A social dilemma model of information self-disclosure, applied to the concussion domain. Journal of Concussion 2019;3:205970021988287.doi:10.1177/2059700219882876
      OpenUrl
    27. ↵
      2. Kroshus E ,
      3. Baugh CM
      . Concussion education in U.S. collegiate sport: what is happening and what do athletes want? Health Educ Behav 2016;43:182–90.doi:10.1177/1090198115599380 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26293460
      OpenUrlCrossRefPubMed
    28. ↵
      2. Mrazik M ,
      3. Dennison CR ,
      4. Brooks BL , et al
      . A qualitative review of sports concussion education: prime time for evidence-based knowledge translation. Br J Sports Med 2015;49:1548–53.doi:10.1136/bjsports-2015-094848 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26307498
      OpenUrlAbstract/FREE Full Text
    29. ↵
      2. Warmath D ,
      3. Winterstein AP
      . Reporting skill: the missing ingredient in concussion reporting intention assessment. Sports Health 2019;11:416–24.doi:10.1177/1941738119856609 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31268836
      OpenUrlPubMed
    30. ↵
      2. Kroshus E ,
      3. Daneshvar DH ,
      4. Baugh CM , et al
      . NCAA concussion education in ice hockey: an ineffective mandate. Br J Sports Med 2014;48:135–40.doi:10.1136/bjsports-2013-092498 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23956336
      OpenUrlAbstract/FREE Full Text
    31. ↵
      2. Schmidt JD ,
      3. Weber ML ,
      4. Suggs DW , et al
      . Improving concussion reporting across national College athletic association divisions using a Theory-Based, data-driven, multimedia concussion education intervention. J Neurotrauma 2020;37:593–9.doi:10.1089/neu.2019.6637 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31597536
      OpenUrlPubMed
    32. ↵
      2. Kroshus E ,
      3. Baugh CM ,
      4. Hawrilenko M , et al
      . Pilot randomized evaluation of publically available concussion education materials: evidence of a possible negative effect. Health Educ Behav 2015;42:153–62.doi:10.1177/1090198114543011 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25128434
      OpenUrlCrossRefPubMed
    33. ↵
      2. Craig DI ,
      3. Lininger MR ,
      4. Wayment HA , et al
      . Investigation of strategies to improve concussion reporting in American football. Res Sports Med 2020;28:181–93.doi:10.1080/15438627.2019.1586706 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30859851
      OpenUrlPubMed
    34. ↵
      2. Kroshus E ,
      3. Garnett BR ,
      4. Baugh CM , et al
      . Engaging Teammates in the promotion of concussion help seeking. Health Educ Behav 2016;43:442–51.doi:10.1177/1090198115602676 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27405801
      OpenUrlCrossRefPubMed
    35. ↵
      2. Kroshus E ,
      3. Baugh CM ,
      4. Hawrilenko MJ , et al
      . Determinants of coach communication about concussion safety in US collegiate sport. Ann Behav Med 2015;49:532–41.doi:10.1007/s12160-014-9683-y pmid:http://www.ncbi.nlm.nih.gov/pubmed/25712480
      OpenUrlCrossRefPubMed
    36. ↵
      2. Kroshus E ,
      3. Chrisman SPD ,
      4. Harmon KG , et al
      . What do parents need to know about concussion? developing consensus using the Delphi method. Clin J Sport Med 2019.
    37. ↵
      2. Wayment HA ,
      3. Huffman AH
      . The Indirect Influence of Organizational Safety Climate on Football Players’ Concussion Reporting Intentions. Health Educ Behav 2019;1090198119879216.
    38. ↵
      2. Wayment HA ,
      3. Huffman AH
      . Psychosocial experiences of concussed collegiate athletes: the role of emotional support in the recovery process. J Am Coll Health 2019;0:1–6.doi:10.1080/07448481.2019.1577863 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30908165
    39. ↵
      2. Kroshus E ,
      3. Buth D ,
      4. Parsons JT , et al
      . Randomized Evaluation of the National Collegiate Athletic Association’s Concussion Education Fact Sheet for Coaches. Health Educ Behav 2019;1090198119864621.
    40. ↵
      2. Kroshus E ,
      3. Kubzansky LD ,
      4. Goldman RE , et al
      . Norms, athletic identity, and concussion symptom under-reporting among male collegiate ice hockey players: a prospective cohort study. Ann Behav Med 2015;49:95–103.doi:10.1007/s12160-014-9636-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25236670
      OpenUrlCrossRefPubMed
    41. ↵
      2. Kroshus E ,
      3. Garnett BR ,
      4. Baugh CM , et al
      . Social norms theory and concussion education. Health Educ Res 2015;30:cyv047–13.doi:10.1093/her/cyv047 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26471918
      OpenUrlPubMed
    42. ↵
      2. Ruston SW ,
      3. Kamrath JK ,
      4. Zanin AC , et al
      . Performance versus safety: understanding the logics of cultural narratives influencing concussion reporting behaviors. Communication & Sport 2019;7:529–48.doi:10.1177/2167479518786709
      OpenUrl
    43. ↵
      2. Hom MA ,
      3. Stanley IH ,
      4. Schneider ME , et al
      . A systematic review of help-seeking and mental health service utilization among military service members. Clin Psychol Rev 2017;53:59–78.doi:10.1016/j.cpr.2017.01.008 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28214634
      OpenUrlPubMed
    44. ↵
      2. Damschroder LJ ,
      3. Aron DC ,
      4. Keith RE , et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50.doi:10.1186/1748-5908-4-50 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19664226
      OpenUrlCrossRefPubMed
    45. ↵
      2. Brown EC ,
      3. Hawkins JD ,
      4. Arthur MW , et al
      . Prevention Service System Transformation Using Communities That Care.. J Community Psychol 2011;39:183–201.doi:10.1002/jcop.20426 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23606774
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      2. Finch CF
      . Implementation and dissemination research: the time has come! Br J Sports Med 2011;45:763–4.doi:10.1136/bjsports-2011-090252 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21743105
      OpenUrlFREE Full Text

    Footnotes

    • Twitter @ekroshus, @CogNerd, @ugaconcussion

    • Presented at This article has been co-published in the British Journal of Sports Medicine and the Journal of Special Operations Medicine.

    • Contributors All authors have made substantial contributions to either conception and design or analysis and interpretation of data, drafting or revision of the article, and have seen and given final approval of the submission.

    • Funding This publication was made possible, in part, with support from the NCAA-DoD Grand Alliance Mind Matters Challenge, funded, in part by the National Collegiate Athletic Association and the Department of Defense.

    • Disclaimer Opinions, interpretations, conclusions and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense.

    • Competing interests EK reports funding from the National Collegiate Athletic Association during the conduct of the study and research funding to institution in past 3 years from the Pac-12 Athletic Conference. KC reports grants from National Collegiate Athletic Association and US Department of Defense during the conduct of the study, book royalties from Springer, and membership on the NATA Foundation Board of Directors. JDS reports grants from National Collegiate Athletic Association and US Department of Defense during the conduct of the study. CD’L reports grants from the National Collegiate Athletic Association and US Department of Defense during the conduct of the study. JKR-M reports grants from the National Collegiate Athletic Association and US Department of Defense during the conduct of the study, research funding to institution in past 3 years from: National Football League, National Athletic Trainers Association Foundation, Centers for Disease Control, National Operating Committee on Standards for Athletic Equipment, and current membership on USA Football's Football Development Council. RS reports funding from the National Collegiate Athletic Association during conduct of the study.

    • Patient consent for publication Not required.

    • Ethics approval University of Washington Institutional Review Board.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available upon request