Evidence for change in flight crew training developed by Thomas Leoff IAAPS chairman APATS Bangkok 17.09.2013 1
Follow Me. APATS Bangkok 17.09.2013 2
Evolution in Aviation APATS Bangkok 17.09.2013 3
Where do we come from? Today s training system is based on the seventies ICAO s SARPs on training stem from that time with the exception of the MPL The operation philosophies of aircraft like the B707 which where a core part of that huge progress in aviation at that time are still a cornerstone in today s training APATS Bangkok 17.09.2013 4
The seventies. APATS Bangkok 17.09.2013 5
are the scenario for today's legal requirements for training programs They are: Based on traditional developments Engineering driven Knowledge based Based on numbers like requirements of flight hours of instruction or landings, legs etc. APATS Bangkok 17.09.2013 6
The major deficiencies are: Syllabus structures from the 70ies and 80ies No feedback from the real world OPS Human Factors training outdated and not related to actual needs No feedback system to control the efficiency of the training in place APATS Bangkok 17.09.2013 7
International activities by IATA, ICAO and EASA There are 3 international activities which are relevant for the corrective action: IATA s ITQI initiative ICAO s developments regarding EBT EASA s planned Rulemaking Activity for Evidence Based Training APATS Bangkok 17.09.2013 8
IATA s Data Report on EBT (soon to be published) APATS Bangkok 17.09.2013 9
The 4 Ws of change in training: What training, testing & legal requirements therefore Why? number of new pilots in the next 20 years, introduction of SMS for ATOs, pilot qualification as key factor for safety improvements and the frequent technical improvements in the system Who? ICAO, EASA, NAAs &Stakeholders When? ASAP! APATS Bangkok 17.09.2013 10
The What? The legal requirements The scenarios for the training The Training Programs and their supervised effectiveness The form of examination The qualification of instructors and examiners APATS Bangkok 17.09.2013 11
The Why? Pilot Demand forecasted for the next 20 years The new concept of Competency- & Evidence Based Training Introduction of SMS in ATOs and other sources of operational data and the new opportunities from that Current flat trend lines of accident rates APATS Bangkok 17.09.2013 12
Pilot Demand Forecast APATS Bangkok 17.09.2013 13
The actual accident trend lines APATS Bangkok 17.09.2013 14
Two practical samples for human deficiencies They stem from a Research Project in which several German Universities from Bremen, Berlin and Munich cooperate with Lufthansa. The goal is, to develop valid Safety Performance Indicators for the SMS APATS Bangkok 17.09.2013 15
FINISHED FILES ARE THE RESULT OF YEARS OF SCIENTIFIC STUDY COMBINED WITH THE EXPERIENCE OF YEARS
FINISHED FILES ARE THE RESULT OF YEARS OF SCIENTIFIC STUDY COMBINED WITH THE EXPERIENCE OF YEARS
Task Related Probability Of Errors MTBFs Prof. Bubb TU-Munich Category Kategorie Error Fehler probability MTBF Simple and regularely performed tasks at a low stress level. Complex, regularely performed tasks in a well known working environment at a low stress level. Complex tasks in unusual situations at a high stress level and / or time pressure. 1. 10-3 1 10-2 1 10-1 ~30 min ~5 min ~30 sec
Today we fly APATS Bangkok 17.09.2013 20
and that s today s A380! APATS Bangkok 17.09.2013 21
and here the new B787 APATS Bangkok 17.09.2013 22
What shall we update? A simple update of the actual set of Training Programs is not enough! We must initiate a more sophisticated approach! How could we achieve that? We should start with the newly defined Pilot Core Competencies and Behavioral Indicators. APATS Bangkok 17.09.2013 23
The new set of Pilot Competencies as defined by IATA/ICAO/IFALPA February 2012 APATS Bangkok 17.09.2013 24
How can we adapt the training to that new approach? Integration of the SMS of operators and the ATOs Defined feedback loops based on data from the SMS, incident reports & analysis This feedback loop must cover the entire system from initial to recurrent training Research to support the SMS utilization APATS Bangkok 17.09.2013 25
An Example from our Research Project Simulator study measuring manual flying skills regarding Performance Shaping Factors APATS Bangkok 17.09.2013 26
Lehrstuhl für Ergonomie Technische Universität München Simulator study measuring manual flying skills regarding Performance Shaping Factors Dipl.-Ing. Andreas Haslbeck Dipl.-Ing. Ekkehart Schubert, Cpt. Manfred Müller Simulatorstudie zu manuallen Flugfertigkeiten
Lehrstuhl für Ergonomie Technische Universität München Simulatorstudie zu manuallen Flugfertigkeiten
Lehrstuhl für Ergonomie Technische Universität München 2. Demographic Data Captains A346 FOs A320 Age MW 50,0 SD 4,1 MW 30,4 SD 3,1 20.000 18.000 16.000 14.000 12.000 10.000 8.000 6.000 4.000 2.000 0 Flugerfahrung Flight in hours Flugstunden 14.851 2.562 3.374 2.093 20 15 10 5 0 Landings Landungen as PF als within PF in last 30 30 Tagen days 17 3 A346 on type Flugstunden Typ Total hours A320 Flugstunden gesamt A346 A320 Simulatorstudie Study: zu manuallen flying Flugfertigkeiten skills
Lufthansa seven two seven Continental oneoohsix Traffic Scenarios Quality nine three kilo Whitestar one eight Eurotrans zero seven Lufthansa seven three one Lufthansa one seven eight niner Singapore three two eight
Lehrstuhl für Ergonomie Technische Universität München Scenario 1 (summary) EDDM ILS APP 08L with >15Kts tailwind (WX: VIS >3.000m OVC 800ft) G/A Standard Missed APP Raw Data ILS APP 26R (WX >non-precision: VIS 2.000m OVC 350ft) Simulatorstudie study zu manuallen flying Flugfertigkeiten skills
Lehrstuhl für Ergonomie Technische Universität München Measuring basic (manual) flying performance The Simulator crew measures deviations: from Localizer altitude and sinkrate (Glideslope) approach speed touchdown point A Computer program integrates all measured deviations. In addition pilot performance assessment by an experienced training pilot. Simulatorstudie zu manuallen Flugfertigkeiten
A320-Pilots Grading Technische Universität Berlin VP Level Loc GS speed visual landing average grade 1 6 1 1 2 1 1 2 2 2 5 3 2 2 2 4 3 3 weiblich 3 2 1 1 1 2 1 1,33333 1 4 1 1 1 1 1 1 1 1 5 2 1 1 2 2 1 1,5 2 6 2 1 1 1 2 1 1,33333 1 7 3 1 1 1 2 2 1,66667 2 8 1 1 1 1 1 1 1 1 9 1 1 1 2 2 1 1,33333 1 10 5 1 2 2 1 1 2 2 11 1 1 1 1 2 2 1,33333 1 12 1 1 1 1 1 2 1,16667 1 13 2 1 1 1 3 3 1,83333 2 14 3 2 2 2 2 2 2,16667 2 15 3 1 1 2 2 1 1,66667 2 16 1 1 3 2 4 2 2,16667 2 17 1 2 2 2 3 3 2,16667 2 weiblich 18 2 1 1 1 2 2 1,5 2 19 4 1 1 2 2 1 1,83333 2 20 2 1 1 2 2 1 1,5 2 21 1 1 1 1 2 1 1,16667 1 22 2 1 1 2 2 1 1,5 2 23 3 1 2 2 2 2 2 2 24 3 1 1 2 1 1 1,5 2 25 2 1 1 1 2 2 1,5 2 26 1 1 1 1 1 1 1 1 27 3 1 1 1 2 1 1,5 2 28 6 6 6 4 6 3 5,16667 5 weiblich 29 3 1 1 1 1 1 1,33333 1 30 2 1 1 2 2 1 1,5 2
Datarecording (A340 & A320 FFS) Technische Universität Berlin 20 x 104 Topansicht XERUM 15 BURAM A340-600 A320 10 WLD Y [ft] MIQ 5 DM429 0 DM431 MAGAT EDDM 26R GUDEG -5-2.5-2 -1.5-1 -0.5 0 0.5 1 1.5 2 X [ft] x 10 5 34
eye tracking camera
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% FMA-Checks Anteil getätigter FMA-Checks 4 8 9 10 12 13 14 15 16 17 18 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 49 50 51 52 53 54 55 56 58 60 Ø CPT Ø FO Performance variability (all kinds of FMA-Checks) CPT FO
Speed-Checks Anteil der Piloten 100% 90% 80% 70% 60% Speed / Vertical Speed Checks below 100ft RA Speed/vert.Speed-Checks unter 100ft CPT FO F/O A320 F/O A320 50% 40% 30% 20% 10% 0% CPT A346 19,0% 64,0% Speed überprüft zwischen 100ft und TD Speed checked below 100ft and TD CPT A346 9,5% 60,0% vert. Speed überprüft zwischen 100ft und TD V/S checked below 100ft and TD
SaMSys Simulator Experiments Measuring Economical Flight Performance FMS Mode Awareness / Automation Handling unclear situations (Warsawa) Redundancy and team-interaction Go-Around-decision and -performance Basic Flying Skills (AI and Boeing) Predictability of pilot performance (DLR-test) 19.05.2014 Seite 38 Bitte diesen Text mit dem Präsentationstitel überschreiben Deutsche Lufthansa AG
Test Scenario 2 The tower reported wind is within limits. However the actual surface wind is beyond the limit (18Kts instead of max. 10Kts) A Go-Around should be performed! In 70ft a Go-Around is triggered by the PM Runway change after the Go-Around. Simulator Study: Manual Pilot Skills
Go-Around Anslysis (MSc of Alexander Schmidt) Technische Universität Berlin 39% of all pilots discussed the tailwind situation during the approach briefing. 89% of all pilots talked about the strong tailwind during approach above 1000ft AGL. 50% of all pilots checked the wind indication during final approach below 1000ft AGL. 64% of all pilots would have landed without the wind information given by the PM (TW of 18 Kts). 14% of all pilots decided to land, knowing the tail wind was 18Kts. 44% of all A340 pilots performed the Go-Around only after the PM gave the command in ~70ft AGL. 40
SaMSys Simulator Experiments Measuring Economical Flight Performance FMS Mode Awareness / Automation Handling unclear situations (Warsawa) Redundancy and team-interaction Handling the Go-Around Basic Flying Skills (AI and Boeing) Predictability of pilot performance (DLR-test) 19.05.2014 Seite 41 Bitte diesen Text mit dem Präsentationstitel überschreiben Deutsche Lufthansa AG
Go-Around Analysis (MSc of Alexander Schmidt) Technische Universität Berlin a) 1 A340-600 captain (3,7%) performs the Go-Around correctly. 10 A320-F/O (33%) perform the Go-Around correctly. b) In general more than one SOP-violation per Go-Around Average: A340-600 Cpt.: 2, A320 F/O: 1,3. c) Common SOP violations: Missed Appr. Proc. not correct ( NAV-Mode not activated). Flaps not operated according SOP. Thrust Reduction to late and/or MCT is selected instead of CLB thrust. Vertical speed is not reduced during the last 1000ft before reaching Go-Aroud Altitude. 42
Go-Around Analysis Technische Universität Berlin 80,0 70,0 67,7% relative Häufigkeit [%] 60,0 50,0 40,0 30,0 20,0 10,0 0,0 51,6% 35,5% 12,9% 16,1% 12,9% 3,2% 0,0% < 1000 1000-1500 1500-2000 >2000 A340 A320 Steiggeschwindigkeit v/s in ft/min [ft/min] Vertical Speed within the last 1.000ft before reaching the level off altitude Masterarbeit Alexander Schmidt
Fligth Recorder Data Analysis Only 5% of all Go-Arounds are flown correctly. In average 2,7 SOP-violations can be observed per Go-Around. Analysis of 150 Go-Arounds showed 415 SOP-violations. Maximum number of violations per Go-Around: 10. Go Around Campaign FRA CF, Annina Müller 23. Februar 2010 Page 44
But a word of BIG CAUTION Research is necessary but must be performed by competent researchers Human monitoring of data acquisition is a must Computerized data analysis of OPS data has risks of misleading results Identification of root causes is paramount but difficult
What are the next steps? A Process for data driven revision and update of training programs must be implemented the Inner Loop Control the effectiveness of the training in an Outer Loop by measuring the performance in the real OPS through valid Safety Performance Indicators (SPIs) Develop such valid SPIs for the Airline Operations and the related training APATS Bangkok 17.09.2013 46
Main topics for improvement Full application of all available training media backed up by a regular training transfer control process Global harmonization and joint efforts in the development of the new Evidence Based Training System Safety shall be the globally agreed driving factor for this new training system APATS Bangkok 17.09.2013 47
Conclusion Initiate a human centered approach to training, teaching and learning by using all available means of training know-how & technology and validate the efficiency Design the testing of knowledge, capabilities and the resulting performance as close as possible to the requirements of the aviation system Ensure regular updates driven by data APATS Bangkok 17.09.2013 48
Thank you for your attention! APATS Bangkok 17.09.2013 49