Neuropsychology is a relatively young discipline that focuses on understanding brain-behaviour relations. The discipline flourished after World War II due to the need for assessment of soldiers with traumatic brain injuries. Since radiological techniques were not available for imaging structural damage, neuropsychological assessment became an important addition to neurophysiological and neurological assessment in localising brain injury for neurosurgeons and for assessing neurocognitive functioning in order to develop rehabilitation and disposition plans. Because neuropsychological techniques and procedures provided a unique opportunity to assess brain functioning there was a natural progression to the use of neuropsychological assessment procedures in the diagnosis and assessment of mild traumatic brain injury (MTBI), which also proves to be opaque to traditional imaging techniques. From there, the progression to working with sports-related MTBI and concussion was inevitable. Sports provided a unique “laboratory” in which the natural progression of MTBI could be studied and neuropsychological assessment provided quantification of an injury that thus far had defied quantification.

The aim of this paper is to examine the literature and provide a framework that describes the role of the clinical neuropsychologist in the administration and interpretation of neuropsychological data with sports-related brain injury.

METHODS

Pertinent literature was reviewed by the authors’ experience and the use of Pubmed, Medline and Psych Info databases using key words: neuropsychology+testing, neuropsychology+practice and neuropsychology+concussion. The position papers of the major professional organisations of neuropsychology were reviewed (that is, the National Academy of Neuropsychology, Division 40 (Neuropsychology) of the American Psychological Association and the American Academy of Clinical Neuropsychology). A classic text in neuropsychology by Lezak1 was reviewed as was a recently edited volume by Echemendia2 on sports neuropsychology.

RESULTS

The use of neuropsychological assessment paradigms in sports-related MTBI was pioneered by Dr Jeff Barth and his colleagues at the University of Virginia.3 This group first implemented the baseline post-injury evaluation paradigm that is widely used today. The underlying assumption of this model is that there is significant individual variability in cognitive functioning in addition to marked differences in socioeconomic, cultural, ethnic and linguistic backgrounds. This variability in cognitive functioning may lead to significant error in the interpretation of post-injury test scores when compared with group-derived average levels of functioning. By testing each athlete individually prior to a brain injury, any post-injury comparison will likely contain less error because the individual is being compared with his or her own pre-injury levels of functioning. While pre-injury baselines help to reduce error due to individual differences, they also introduce additional complexity in the interpretation of test data because of the measurement error associated with a second test administration. In other words, the baseline post-injury assessment model involves measurement error, broadly defined, which surrounds both the test and retest scores.

The sport setting is unique because players and teams have limited amounts of time available for non-sports related activities. Historically, neuropsychological evaluations typically lasted 6–8 hours or more. The sports domain required quick, efficient evaluations that typically did not exceed 30 minutes. Barth et al’s efforts paved the way for a brief series of traditional neuropsychological tests that proved to be sensitive to detecting neurocognitive dysfunction following sports-related MTBI. At times, neuropsychological test data may also be used to provide information about localisation of cognitive dysfunction that is correlated with the mechanism of injury thereby providing the clinician with a better understanding of the athlete’s injury. Over the years, a significant body of literature has developed supporting the use of neuropsychological tests (both traditional and computer-based instruments) in the identification and management of sports-related MTBI.3^–21 While a robust literature has developed that supports the use of neuropsychological assessment, concerns have been raised about issues of reliability and validity.22^–25

NEUROPSYCHOLOGICAL TRAINING

Neuropsychological training models vary significantly across international borders with some countries having rudimentary guidelines while others have well-developed curricula and training requirements. A detailed examination of this variability is beyond the scope of this paper. Instead, we will briefly focus on the US approach, which has been viewed by some as the most comprehensive. According to the National Academy of Neuropsychology, clinical neuropsychologists use specialised knowledge in brain-behaviour relations in the “assessment, diagnosis, treatment, and/or rehabilitation of patients across the lifespan with neurological, medical, neurodevelopmental and psychiatric conditions, as well as other cognitive and learning disorders. The clinical neuropsychologist uses psychological, neurological, cognitive, behavioral, and physiological principles, techniques and tests to evaluate patients’ neurocognitive, behavioral, and emotional strengths and weaknesses and their relationship to normal and abnormal central nervous system functioning.”1 Within this model, neuropsychologists receive extensive and specialised training and they are licensed as independent practitioners of clinical neuropsychology and/or psychology. A doctoral degree is required, as is a full-time clinical internship. Two years of full-time experience and specialised training is required, at least one of which is at the post-doctoral level. Increasingly, neuropsychologists are following widely accepted recommendations that 2 years of formal postdoctoral residency is required for the specialty practice of neuropsychology.26 A total of 7–9 years of training following the baccalaureate is typical in order to achieve independent practice. Core curriculum requirements include rigorous training in statistics, psychometric theory, research methodology, test administration and interpretation of test data, as well as training in neuroanatomy, psychotropic medications, psychopathology and clinical interventions.

TESTING VERSUS INTERPRETATION

There are several distinctions that are critical to understanding neuropsychological assessment. First, there is an important difference between test administration and the interpretation of test data. In order to ensure the viability of test data, tests must be administered in a standard way across all test administrations irrespective of who conducts the testing. Neuropsychologists are trained to rigidly adhere to standard test administration protocols since failure to comply with standardised procedures has been associated with erroneous findings.26 However, tests vary with respect to ease of administration and the training/qualifications needed for appropriate test administration. Typically, the test instruments used in the sport setting have well-established administration protocols that are relatively uncomplicated and require little to no expertise on the part of the administrator. Therefore, we deem it appropriate for a neuropsychologist to designate test administration to non-neuropsychologists provided that the individual is properly trained by the neuropsychologist in the administration of said tests. The use of non-neuropsychologists for testing is consistent with the positions of several neuropsychological organisations, including the National Academy of Neuropsychology,27 the neuropsychology division of the American Psychological Association28 and the American Academy of Clinical Neuropsychology.29

Test administration is fundamentally different, of course, than test interpretation. The interpretation of test results requires an understanding of the complex interactions among test data, psychometric properties, sources of error, patient symptoms, extra-test variables as well as intra-individual variables. It is well known that neuropsychological test scores are affected by a wide range of factors that may be unrelated to the brain injury that is being evaluated. Psychological factors (anxiety, depression, fear), physiological factors (sleep, fatigue, pain, medication, nutrition, hormonal), cultural factors (education, language, exposure to testing) and pre-morbid characteristics (learning disorders, attention-deficit/hyperactivity disorder, developmental disorders, personality disorders) all may play important roles in the interpretation of any derived score or configuration of scores.

In addition to significant variability that may be associated with non-test factors, there are a number of test-specific psychometric factors that must be appreciated for appropriate interpretation of test data. All neuropsychological (as well as medical) tests are imperfect and contain various sources of measurement error. One must have a thorough understanding of test validity (the extent to which a test measures the construct that it is intended to measure) and reliability (the extent to which a test score remains stable across time, test settings and evaluators). For example, an understanding of test-retest reliability is critical in the evaluation of test data. The sports paradigm calls for repeated testing with the same measure. The vast majority of neuropsychological tests (including most computerised tests) exhibit “practice effects,” which occur when an initial exposure to a test (for example, baseline test) leads to better performance on a later test (for example, post-injury) simply due to learning the test content (content practice effect) or having been exposed to the procedure (procedural practice effect). Content practice effect can be mitigated through the use of multiple alternate test forms while little can be done to mitigate procedural practice effects. While these practice effects can be partially accounted for through the use of statistical techniques, such as the use of reliable change indices or regression formulas,30^–32 these complex statistical methods have limitations that must be incorporated into proper interpretation of test scores. Additionally, although practice effects are often viewed as a source of error that needs to be managed or controlled, there are studies that suggest practice effects are diagnostically useful. For example, Echemendia and his colleagues8 found that the absence of a practice effect significantly differentiated concussed athletes from uninjured controls, presumably because the injured athletes were unable to learn from the prior exposure. Therefore, by removing or failing to analyse practice effects, the clinician is discounting valuable information about the injured athletes’ neurocognitive functioning.

Test scores can also vary significantly as a function of motivation and effort. Bailey, Echemendia and Arnett33 reported that differential motivation from baseline to post-injury testing may lead to erroneous interpretation of test data; detailed examination of test and subtest score variability may identify this sub-optimum test performance and improve the accuracy of test score interpretation. Variability in test scores has also been reported due to cultural and linguistic factors. Recent data on professional ice hockey players34 suggest that cultural differences, identified by language of origin, have significant influence on the test scores of professional hockey players on both traditional neuropsychological measures and computerised tests. These differences emerged when players were tested in English as well as their own language.

In short, we have presented a small sampling of the myriad variables that should be considered in the valid interpretation of neuropsychological test data. It is important to acknowledge that tests vary with respect to their susceptibility to various sources of error. Those tests that assess multiple domains of functioning (for example, learning, memory, problem solving and sequencing) are subject to greater sources of error than those instruments that measure single domains of functioning (for example, simple reaction time). Nevertheless, even rather simple, unimodal, tests are subject to extra-test variability in test performance; the interpretation of which cannot be adequately reduced to blind or rigid interpretation of statistical formulas. An appreciation of the complexity of the return to play (RTP) decision helped steer the Concussion in Sport group away from group-based guidelines for RTP to an individualised approach to RTP in Vienna35 and later reaffirmed in Praque36 and now Zurich. Echemendia and Cantu37 underscored the complexity and dynamic relations that exist among variables in the RTP decision.

LEGAL CONSIDERATIONS: SCOPE OF PRACTICE

Thus far, we have focused on clinical and professional reasons for having neuropsychologists interpret neuropsychological test data. Often overlooked are the possible medico-legal issues that may come into play if non-neuropsychologists interpret neuropsychological test data. There are wide ranging differences in licensing laws across disciplines, states, provinces and certainly international borders. While it is not our intention to provide interpretation of laws or guidelines, we do wish to call attention to scenarios that may prove difficult in the legal arena. In some instances, scope of practice issues may be more salient than others. For example, under a scope of practice scenario it may be less defensible for a certified athletic trainer to have interpreted neuropsychological test scores than it is for physicians to have interpreted such scores. Even if the certified athletic trainer has attended “weekend workshops” on interpretation of specific tests, it may prove to be an uncomfortable situation if challenged to justify that he or she was not practicing outside the scope of their expertise. Similarly, while most state laws allow physicians to use and interpret psychological/neuropsychological tests, a physician may also be placed in an uncomfortable position when asked to justify his or her training in neuropsychological assessment when compared with that of a duly licensed or certified clinical neuropsychologist. The situation may be akin to an orthopaedically trained sports medicine physician relying on his or her own interpretation of a brain MRI rather than considering the interpretation of a neuroradiologist.

PRACTICAL MATTERS

We have argued thus far that the interpretation of neuropsychological test data in the sports context is best accomplished by a clinical neuropsychologist who is serving as a consultant to the team physician. An obvious limitation to this position is that there are not enough appropriately trained neuropsychologists to satisfy the demand for services or that consultation with neuropsychologists may be cost prohibitive. It is certainly true that there are not enough neuropsychologists to individually test the number of athletes who sustain sports-related brain injuries. However, we have argued that a clinical neuropsychologist is not required to administer the vast majority of tests that are used in the sports context. Paraprofessionals can administer tests as long as they have been properly trained by the supervising neuropsychologist. The advent of computerised test platforms and easy access to the internet, in addition to telemedicine capability, makes access to a qualified neuropsychologist far easier than it was in the past. It is not uncommon for a neuropsychologist to live in a distant city or country and be able to provide consultation on the interpretation of test data of an athlete. This approach requires close cooperation and coordination among the team physician, team athletic trainer and the neuropsychologist but it allows for cost-efficient and practical methods of seeking appropriate consultation.

CONCLUSIONS

Over the past 10 years there has been growing excitement and widespread use of neuropsychological methods for the evaluation of sports-related brain injury. Neuropsychological tests provided “objective” data that could be used to identify a concussion where no objective tools existed before. Concurrently, computerised-based testing platforms allowed for easy and standardised modes of test administration. However, the ease and expediency of computerised test administration can result in the mistaken belief that the person interpreting the tests no longer needs (i) training and supervision in test administration; (ii) expertise in reliability, validity and diagnostic accuracy; and (iii) applying clinical judgment to situations involving complex factors related to the interpretation of test results. Although the administration of some neuropsychological tests may be relatively uncomplicated, the interpretation of neuropsychological test data is far from “easy” nor can it be easily reduced to a “yes-no” statistical formula in most cases. The RTP decision is complex and dynamic. The team physician requires neuropsychological input from a variety of sources, including comprehensive and competent interpretation of neuropsychological assessment data. The interpretation of neuropsychological tests requires comprehensive knowledge of the tests, their characteristics given a specific population (for example, team, sport), the athlete and his or her specific situation, psychological variables and many others. For these reasons, we conclude that neuropsychological tests may be administered under the guidance of a neuropsychologist but that the interpretation of neuropsychological test data is best managed by a clinical neuropsychologist.