NORMS FOR AVIATION PSYCHOLOGY

Size: px

Start display at page:

Download "NORMS FOR AVIATION PSYCHOLOGY"

Jasper Gilmore
5 years ago
Views:

1 WHICH NORMS SHOULD I BE USING? 18 September 2016 Gary Kay & Keith Atkins 4th Annual Aerospace Psychology Seminar Denver, Colorado CORRECTING AND ADJUSTING TEST RESULTS SCENARIO; A THOUGHT EXPERIMENT Pilot is administered a test battery. He correctly responds to 12 out of 20 math problems. The math problems are known to assess a 10 th grade level of math equivalence. Assume that aviation task analysis has shown that 10 th grade level math equivalence is essential for competent flight operations. Pilot is 80 years old and completed 13 years of formal education. Score places him at the 32 nd percentile compared to Heaton Norms (corrected for age and years of education). Score places him at the 12 th percentile compared to Non-Age corrected norms adjusted for education. Score places him at the 5 th percentile compared to Non-Age AND Non- Education corrected norms. How do you determine AEROMEDICAL SIGNIFICANCE? Do you use the same norms to evaluate for NEUROCOGNITIVE IMPAIRMENT? 1

2 NORMS FOR AVIATION PSYCHOLOGY Classic Halstead-Reitan Neuropsychology Approach Compare examinee s performance to population norms Norms not adjusted for age or education. Norms had a wide age distribution How norms were obtained unspecified: Hospital employees Cases where testing showed no abnormalities At one major NP clinic: Clinical cases with Normal EEG & CT Performance falling below the 5 th percentile was considered impaired. Were these population norms suitable for evaluating aviators? SUITABILITY OF POPULATION NORMS FOR EVALUATING AVIATORS Classic population norms were often obtained from populations with the following characteristics: IQ average ~ 100 Education average ~ 12 years Race - almost entirely Caucasian; and Age average in the 40 s Test scores were not corrected for the effects of age, education, or ethnicity. 2

3 AVAILABILITY OF AVIATOR NORMS Mid 1980 s: Only pilot norms on neuropsychological tests were obtained from Repatriated POWs (US Air Force Pilots shot down over Viet Nam; these norms had originally been marked Classified) Scores for this group were at a minimum 1 SD higher than population norms RECOGNITION OF ROLE OF AGE AND EDUCATION ON NEUROPSYCHOLOGICAL TEST PERFORMANCE In 1980 s many studies were being published in neuropsychology demonstrating the role of sex, age, years of education, and IQ on neuropsychological test performance. Norms broken down by sex and age were becoming available for many of the tests included in the Halstead-Reitan Battery. (but were not co-normed) The Heaton Comprehensive Norms became popular in the 1990 s, originally in a printed workbook and later as a computer-based tool. Norms that correct for age and education; controlling for the impact of these factors on performance. Obviously, a better test score resulting from Age-Correcting does not indicate that the task was performed better. [i.e., the test score doesn t change.] 3

4 HEATON NORMS Age, education, gender, and race each explain some percentage of the variance in test performance. In the absence of baseline data the use of norms which correct for these factors provide a reasonable estimate of the deviation between the airman s current performance and the airman s expected level of performance (considering their age and education). Age/Education corrected data are useful for making inferences regarding neurocognitive impairment (but require consideration of base rate factors). Scores corrected for age and education don t indicate whether an individual s level of performance is aeromedically significant. OCCUPATIONAL NORMS Very little has been done to develop Occupational Norms for neuropsychological measures; exceptions Doug Powell s work with physicians for Harvard Our work with CogScreen Airline Pilots Helicopter Pilots Various military populations Other examples? 4

5 PILOT NORMS VS. HEATON NORMS NAN Poster presentation (1995) Adjusting scores for age and years of education (via Heaton Comprehensive Norms) eliminated the difference between pilot and non-pilot performance on conventional test measures. The average aviator at the time had 16 years of education and an average IQ of approximately 125 Does this eliminate the need for Aviator Norms? No FAA NEUROPSYCHOLOGY CONSULTANTS RECOMMENDATION: OBTAIN PILOT NORMS ON TEST After CogScreen was shown to be sensitive to brain dysfunction (the FAA Phase B study). Expert panel requested commercial pilot norms. Norms (n=400) were obtained at a Major Airline. To support construct validity research, approximately 25% of the sample volunteered to take the WAIS-R and a number of other common neuropsychological tests. These Aviator Norms are published in Aeromedical Psychology. 5

6 PUBLISHED AVIATOR NORMS These are for the WAIS-R PUBLISHED AVIATOR NORMS Category Test Score: Score is for the 120-item version 6

7 PUBLISHED AVIATOR NORMS PASAT: Scores are for the 49-item (Levin/Galveston version; also reported by Heaton) FAA NEUROPSYCHOLOGY CONSULTANTS RECOMMENDATION: OBTAIN PILOT NORMS ON TEST FAA Consultants questioned whether the norms obtained for Major Airline pilots were representative of commercial pilots at other airlines. Additional airline pilot groups were tested: Medium-Sized domestic carrier (n= 120)* Small-Sized (discount) domestic carrier (n= 61)* Regional Airline (n~100)* *Pilots at these airlines were only administered CogScreen and Brief IQ measures 7

8 AIRLINE GROUPS Criteria for Large, Medium, Small were not operationalized LARGE: One of the 3 largest US Carriers at the time; domestic and international (truly global) operations. High percentage of pilots were former military. MEDIUM: A major airline with primarily domestic operations. Carrier operated in a limited number of international markets. A more heterogeneous group of pilots; airline had undergone several mergers of smaller airlines. SMALL: Still considered a major carrier, almost entirely limited to US Operations. Limited number of fleets (i.e., operating 3 or 4 different aircraft). Lower paying. Even more heterogeneous group of pilots. REGIONAL: Entry airline job with considerably lower pay. Very few with prior military. Data collection was prior to regional jet era, i.e., aircraft were turboprops. 12 DIFFERENCES IN COGSCREEN PERFORMANCE AS A FUNCTION OF CARRIER SIZE/TYPE Shifting Attn. Discovery Condition Number of Rules Completed 10 Number of Rules Completed Large M edium Sm all Eagle Regional Airline These differences can probably be accounted for on the basis of years of education. They are also correlated with salary differences. 8

9 DIFFERENCES IN COGSCREEN PERFORMANCE AS A FUNCTION OF CARRIER SIZE/TYPE WHAT CAN YOU TELL FROM PILOT NORMS? Indicates the range (and distribution) of test performance found among individuals who are currently performing the JOB OF PILOT. Useful for determining when performance on an essential mental ability falls outside the range of normal for individuals who perform the same job. Pilot norms allow you to psychometrically determine the probability that an individual s performance is highly atypical of aviators and therefore legitimately can be considered to reflect a potentially aeromedically significant deficit. Useful for determining when a pilot s performance has recovered to the expected level for a pilot. 9

10 AVAILABLE PILOT NORMS FOR AVIATION NEUROPSYCHOLOGY Conventional Neuropsychological Tests Norms covering the following tests: WAIS-R (Commercial n=456) WAIS-IV Coding (GA Pilots n=207) TRAIL MAKING TEST (Commercial n=132) (GA Pilots n=207) TACTUAL PERFORMANCE TEST (Commercial n=132) HALSTEAD CATEGORY TEST (Commercial n=132) SEASHORE RHYTHM TEST (Commercial n=132) SPEECH SOUNDS PERCEPTION TEST (Commercial n=132) FINGER TAPPING TEST (Commercial n=132) REY COMPLEX FIGURE TEST (Commercial n=132) CALIFORNIA VERBAL LEARNING TEST (Commercial n=132) PACED AUDITORY SERIAL ADDITION TEST (Commercial n=132) (GA Pilots n=207) WISCONSIN CARD SORTING TEST (Commercial n=132) AVAILABLE PILOT NORMS FOR AVIATION NEUROPSYCHOLOGY CogScreen-AE(2) *Major Airline Pilots (by age group; up to 55+) *Large Carriers (all ages) *Medium Carriers (all ages) *Small Carriers (all ages) *Regional Carriers (all ages) **General Aviation Pilots (all ages and by age group) Aeroflot Pilots **USAF (student pilots) [not publicly available: Indian Air Force; Colombian Air Force, Israeli Air Force) **Requesting use of airline pilot applicant results Commercial helicopter pilots *included in CogScreen-AE **included in CogScreen-AE2 10

11 WHICH OF THESE NORMS SHOULD I BE USING FOR COGSCREEN? WHICH COGSCREEN NORMS? For commercial pilots over 50: age-corrected norm for Major Carrier (regardless of size of carrier). The normative sample contains very few pilots over age 60 (norms were collected when the Age 60 Rule was still the law) For commercial pilots over the age of 60. Use age-corrected norm for Major Carrier. Enter (raw) scores for test scores with T<40 into GA Norms Calculator for corresponding age. For commercial pilots under the age of 50 use norms consistent with type of carrier (Large, Medium, Small, or Regional). Use Regional Pilot Norms for: Student pilots Rotary pilots Enlisted military aircrew ATCS For recreational pilots over 50 use age-corrected norm for Major Carrier. Enter (raw) scores for test scores with T<40 into GA Norms Calculator for corresponding age. For recreational pilots under 50 use Regional and follow-up with GA Pilot Norms for scores with T<40 11

12 COGSCREEN-AE2 GA PILOTS For CogScreen-AE2 Use age-based GA norm to determine if pilot is abnormal compared to peers; likely a sign of neurocognitive disorder beyond aging Use non age-based GA norm to determine if pilot s performance falls outside the range of actively flying GA pilots CogScreen-AE2 will allow you to simultaneously display both age and non-age corrected performance on the same figure. COGSCREEN-AE2 NEW FIGURE Provides Comparison for Age and Non- Age Correction 12

13 COGSCREEN-AE2 BASE RATE 5 TH VS 15 TH PERCENTILE CUT-OFFS IN BASE-RATE ANALYSIS Scores falling at or below the 15 th percentile Considered to reflect performance below expectation Often reflect a relative weakness Scores falling at or below the 5 th percentile Considered to reflect a deficiency Base Rate analysis lets us know the frequency with which an individual has scored at or below the 5 th and 15 th percentile. When you have multiple outcome measures, this helps to control for normal variability. You still need to address ALL DEFICIENCIES even if the base rate is WNL. 13

14 WHICH NORMS FOR CONVENTIONAL TESTS? Default to Pilot Norms (Kay, 2002; GA Norms 2014) instead of Heaton and Internal Norms These are preferred over Heaton Revised Norms and Internal Test Norms for the following measures: CVLT-2, CPT-2, PASAT, WCST, Category Test, RCFT, TPT, SSRT, SSPT, Trails, Tapping ADDITIONAL NORM GROUPS: CLINICAL GROUPS & OCCUPATIONAL GROUPS CLINICAL GROUPS ADHD; UNTREATED AND ON STIMULANT MEDICATION ALCOHOL RECOVERY AT VARIOUS STAGES SSRI USERS GRANTED A SPECIAL ISSUANCE TBI? OCCUPATIONAL GROUPS MILITARY CONTROLLERS COMMERCIAL HELICOPTER PILOTS PHYSICIANS? 14

15 COGSCREEN-AE2 CLINICAL GROUP COMPARISON General Aviator Norms Presentation of Normative Data Collected at Sun-n-Fun and at Oshkosh in 2013 Special Thanks to: Chris Front, David Mirich, Carlos Porges, Keith Atkins, Jon VanDoren, Kevin O Brien, & Bob Elliott & CogScreen Staff: Melissa Kay, AJ Thurston & Jessica Rentschlar 15

16 16

17 Age N=208 Mean=58.2 Median=61.5 S.D.=14.8 Minimum=18 Maximum=86 17

18 Trail Making Test Paced Auditory Serial Addition Test (PASAT; Levin Version; 49 items) PASAT SCORES ARE > 1SD BELOW PUBLISHED AVIATOR NORMS; BUT ABOVE 5 TH PERCENTILE 18

LRPV N=207 Mean=0.670 Median=0.886 S.D.=0.377 85 th %ile = 0.

19 LRPV N=207 Mean=0.670 Median=0.886 S.D.= th %ile = th %ile = 1.00 Age x LRPV r=0.527 Distribution of LRPV Scores Impact of Age: GA Pilots 2014 PROBABILITY OF LRPV 0.80 Age Range n Percent % % % % % % % % 38 19

20 Impact of Age on Speed Variables Impact of Age on Dual Tracking Error (DTTDABS x Age; r=0.647) 20

21 Impact of Age on SAT Instruction Condition Speed (SATINRTC x Age; r=0.517) Impact of Age on Accuracy Variables 21

22 Impact of Age on Dual Task Test Previous Number Dual Accuracy (DTTPDACC x Age; r=0.535) Impact of Age on Process Variables 22

23 Impact of Age on SAT Discovery Condition Rules Completed (SATDIRUL x Age; r=-.372) Impact of Age on Multitasking Variables 23

24 SUMMARY OF EFFECTS OF AGE ON COGSCREEN Tracking ability (> age 60); especially under multitasking conditions Working memory (> age 70); especially under multitasking conditions Novel problem solving (> age 65); deductive reasoning (troubleshooting) under time pressure Memory FINDINGS NOT UNIQUE TO COGSCREEN 24

25 Should we correct for age? Are there age-corrected runways? Age and Flight Safety Normal aging degrades aviation-related abilities. Demonstrated by CogScreen Performance on flight simulator tasks has been shown to be related to pilot age in healthy aviators. Expertise appears to mediate this effect. Airline Pilots: Results from cockpit data recorder show a significant increase in the frequency of flight performance violations (per hours of flight) as a function of age. 25

26 FAIR TREATMENT FOR EXPERIENCED PILOTS ACT (DEC 2007): THE AGE 65 LAW Eliminated the Age 60 rule. [Pilot] shall not be subject to different medical standards or different, greater or more frequent medical examinations, on account of age unless the Secretary determines (based on data received or studies published after the date of enactment of this section) that different medical standards, or different, greater, or more frequent medical examinations, are needed to ensure an adequate level of safety in flight. 3 rd Class VFR Rated 400 hours No incidents Cardiac arrhythmia 9 years earlier Has not flown since the cardiac event CASE EXAMPLE: 79 YEAR OLD GA PILOT LRPV =

27 CASE EXAMPLE: 79 YEAR OLD GA PILOT LRPV = 1.00 TAYLOR FACTOR SCORES* T ATTRIBUTE IDENTIFICATION 31 COORDINATION 63 MEMORY 23 SPEED/WORKING MEM 34 TRACKING 39 *based on all ages and types of carriers COMPARISON OF BASE RATES WITH DIFFERENT NORMS ASC, Visual Monitoring single/dual, Tracking single/ dual, Previous Number single/dual, Math, all Pathfinder, all SAT, Discovery FMS/Rule Shifts, Boundary Hits single/dual 27

28 CORE BATTERY FSIQ = 104 AMI = 95 Category = 13 th percentile GAI = 102 VMI = 108 Trails A = 2 nd percentile VCI =98 IMI = 97 Trails B = 6 th percentile PRI = 107 DMI = 102 PASAT 1 = <1 st percentile WMI = 102 PASAT 2 = 12 th percentile PSI = 105 PASAT 1-4 = 108 SUMMARY Selection of norms is complex and is a function of the assessment question being addressed: Determination of AEROMEDICALLY SIGNIFICANT NEUROCOGNITIVE DYSFUNCTION is best established by comparing test scores to pilot norms (not age corrected). Determination of NEUROCOGNITIVE IMPAIRMENT is best established by comparison to norms that best reflect the expectation for that aviator, e.g., norms that account for variance due to age, education and where available - occupation. There are no age-corrected runways. 28

5 th Annual Psychology Seminar September 22-24, 2017 Denver, Colorado

5 th Annual Psychology Seminar September 22-24, 2017 Denver, Colorado Commercial Pilots General Aviation Pilots Student Pilots & ATCS Age Educ N Major Carrier 42.8 16.1 584 Large 44.6 16.1 403 Medium 43.0