QUANTIFICATION OF FAULT TREES FOR A CAUSAL MODEL

Transcription

1 Executive summary QUANTIFICATION OF FAULT TREES FOR A CAUSAL MODEL OF AIR TRANSPORT SAFETY Problem area A causal model for air transport safety (CATS) is being developed. The purpose of the model is to describe the air traffic system and its safety functions in such a way that it is possible to analyse risk reduction alternatives and to serve as a means of communication between experts and managers within the industry. The model combines Event Sequence Diagrams (ESDs), Fault Trees and Bayesian Belief Nets into a single structure. The ESDs are used to represent accident scenarios like e.g. mid-air collision, structure overload, and fire/explosion. The scenarios terminate in one or more end states, e.g. runway overrun, collision with ground, and aircraft continues flight. Description of work An important part in the development of the causal model is the quantification of fault trees leading to initiating and pivotal events in the ESDs. In a first effort by DNV, estimates of the probabilities of occurrence of some events in the fault trees had to be based on pure judgement due to lack of data. The objective of this study was to use the NLR Air Safety Database for quantification of the probability of occurrence of those fault tree events that previously had been based on judgement. Results and conclusions This report describes the results of the quantification of fault trees based on accident and incident data, Air Safety Reports and Service Difficulty Reports. Approximately 50% of the events that were previously determined by pure judgement could be quantified. For the probability of occurrence of other events, there was insufficient data to quantify them. Report no. Author(s) B.A. van Doorn J.G. Verstraeten A. Kurlanc A.D. Balk J.A. Coelho H.T.H. van der Zee A.L.C. Roelen Report classification UNCLASSIFIED Date August 2008 Knowledge area(s) Safety & Security Descriptor(s) Safety Aviation Risk modelling UNCLASSIFIED

2 NLR Air Transport Safety Institute UNCLASSIFIED Anthony Fokkerweg 2, 1059 CM Amsterdam, P.O. Box 90502, 1006 BM Amsterdam, The Netherlands Telephone , Fax , Web site:

3 QUANTIFICATION OF FAULT TREES FOR A CAUSAL MODEL OF AIR TRANSPORT SAFETY B.A. van Doorn J.G. Verstraeten A. Kurlanc A.D. Balk J.A. Coelho H.T.H. van der Zee A.L.C. Roelen No part of this report may be reproduced and/or disclosed, in any form or by any means without the prior written permission of the owner. Customer Ministry of Transport, Public Works and Water Management Contract number Ra / DGL Owner Division Distribution Classification of title Ministry of Transport, Public Works and Water Management Air Transport Limited Unclassified August 2008 Approved by: Author Reviewer Managing department

4

5 CONTENTS 1 INTRODUCTION Background Objective Contents of this report 14 2 QUANTIFICATION APPROACH Introduction NLR Air Safety Database ASR ADREP and Airclaims SDR Quantification issues Top down versus bottom up Denominator General rules for apportioning in case of unknowns Confidence levels Data protocol 22 3 ESD1 AIRCRAFT SYSTEM FAILURE Fault tree events Quantification Overview of results 25 4 ESD2 ATC EVENT Fault tree events Quantification Overview of results 28 5 ESD3 AIRCRAFT HANDLING BY FLIGHT CREW INAPPROPRIATE Fault tree events Quantification Overview of results 31 6 ESD4 - AIRCRAFT DIRECTIONAL CONTROL RELATED SYSTEM FAILURE Fault tree events 32 August

6 6.2 Quantification Overview of results 34 7 ESD5 INCORRECT CONFIGURATION Fault tree events Quantification Overview of results 41 8 ESD6 - AIRCRAFT TAKES OFF WITH CONTAMINATED WING Fault tree events Quantification Overview of results 46 9 ESD7 AIRCRAFT WEIGHT AND BALANCE OUTSIDE LIMITS DURING TAKE-OFF Fault tree events Quantification Overview of results ESD8 AIRCRAFT ENCOUNTERS A PERFORMANCE DECREASING WIND SHEAR AFTER ROTATION Fault tree events Quantification Overview of results ESD9 - SINGLE ENGINE FAILURE DURING TAKE-OFF Fault tree events Quantification Overview of results ESD10 - PITCH CONTROL PROBLEMS Fault tree events Quantification Overview of results ESD11 - FIRE ONBOARD AIRCRAFT Fault tree events Quantification Overview of results 65 August

7 14 ESD12 - FLIGHT CREW SPATIALLY DISORIENTED Fault tree events Quantification Overview of results ESD13 - FLIGHT CONTROL SYSTEM FAILURE Fault tree events Quantification Overview of results ESD14 - FLIGHT CREW INCAPACITATION Fault tree events Quantification Overview of results ESD15 - ANTI-ICE/DE-ICE SYSTEM NOT OPERATING Fault tree events Quantification Overview of results ESD16 - FLIGHT INSTRUMENT FAILURE Fault tree events Quantification Overview of results ESD17 - AIRCRAFT ENCOUNTERS ADVERSE WEATHER Fault tree events Quantification Overview of results ESD18 - SINGLE ENGINE FAILURE IN FLIGHT Fault tree events Quantification Overview of results ESD19 - UNSTABLE APPROACH Fault tree events Quantification Overview of results 108 August

8 22 ESD21 AIRCRAFT WEIGHT AND BALANCE OUTSIDE LIMITS DURING APPROACH Fault tree events Quantification Overview of results ESD23 - AIRCRAFT ENCOUNTERS WIND SHEAR DURING APPROACH Fault tree events Quantification Overview of results ESD25 AIRCRAFT HANDLING BY FLIGHT CREW DURING FLARE INAPPROPRIATE Fault tree events Quantification Overview of results ESD26 - AIRCRAFT HANDLING BY FLIGHT CREW DURING LANDING ROLL INAPPROPRIATE Fault tree events Quantification Overview of results ESD27 - AIRCRAFT DIRECTIONAL CONTROL RELATED SYSTEM FAILURE DURING LANDING Fault tree events Quantification Overview of results ESD28 - SINGLE ENGINE FAILURE DURING LANDING Fault tree events Quantification Overview of results ESD29 - THRUST REVERSER FAILURE Fault tree events Quantification Overview of results 133 August

9 29 ESD30 - AIRCRAFT ENCOUNTERS UNEXPECTED WIND Fault tree events Quantification Overview of results ESD31 AIRCRAFT ARE POSITIONED ON COLLISION COURSE Fault tree events Quantification Overview of results ESD32 INCORRECT PRESENCE OF AIRCRAFT/VEHICLE ON RUNWAY IN USE Fault tree events Quantification Overview of results ESD33 CRACKS IN AIRCRAFT PRESSURE CABIN Fault tree events Quantification Overview of results ESD35 - FLIGHT CREW DECISION ERROR/OPERATION OF EQUIPMENT ERROR (CFIT) Fault tree events Quantification Overview of results 152 REFERENCES 153 August

10 ABBREVIATIONS A/C ACAS ADREP AOA APU ASI ASR ATA ATC ATCO ATFCM CAT CATS CFIT DMC DNV EASA ECAM ECCAIRS ESD FAA FAR FMC FMS FOD FWC GPWS ICAO LLWAS MET MSAW MTOW NLR Aircraft Airborne Collision Avoidance System Accident/incident Data REPorting Angle Of Attack Auxiliary Power Unit AirSpeed Indicator Air Safety Reports Air Transport Association Air Traffic Control Air Traffic Controller Air Traffic Flow and Capacity Management Clear Air Turbulence Causal model for Air Transport Safety Controlled Flight Into Terrain Display Monitoring Computer Det Norske Veritas European Aviation Safety Agency Electronic Centralised Aircraft Monitor European Co-ordination centre for Accident and Incident Reporting Systems Event Sequence Diagram Federal Aviation Administration Federal Aviation Regulations Flight Management Computer Flight Management System Foreign Object Damage Flight Warning Computer Ground Proximity Warning System International Civil Aviation Organisation Low Level Windshear Alert System Meteorology Minimum Safe Altitude Warning Maximum Take-Off Weight Nationaal Lucht- en Ruimtevaartlaboratorium August

11 PF PFD PNF PWS RTO SDR TO TOCW(S) Pilot Flying Primary Flight Display Pilot Not Flying Predictive Windshear System Rejected Take-Off Service Difficulty Reports Take-Off Take-Off Configuration Warning (System) August

12 This page is intentionally left blank. August

13 1 INTRODUCTION 1.1 BACKGROUND The Netherlands Ministry of Transport has initiated a research effort to develop a causal model for aviation safety [Ale et al, 2006]. The purpose of the model is to describe the air traffic system and its safety functions in such a way that it is possible to analyze risk reduction strategies and to support the communication between experts and managers within the industry. The model combines Event Sequence Diagrams, Fault Trees and Bayesian Belief Nets into a single structure. The causal model uses a backbone structure of generic accident scenarios. In a previous study [Roelen & Wever, 2005] those generic accident scenarios that form the upper layer of the integrated risk model have been developed. Main accident types have been defined based on the ICAO definition of an accident, in order to systematically develop accident scenarios. The accident scenarios are grouped by accident type and different flight phases. The Event Sequence Diagram (ESD) methodology is used for representing accident scenarios. In Roelen & Wever [2005] generic accident scenarios have been developed based on a combination of retrospective analyses and prospective analyses. These scenarios describe the sequence of events at a high level of abstraction. The high level of abstraction is required to make the scenarios easy to understand for users and to keep the model transparent and simple at the top layer of the integrated risk model. 1.2 OBJECTIVE The causal model combines Event Sequence Diagrams, Fault Trees and Bayesian Belief Nets into a single structure. An important part in the development of the model is the quantification of events. The quantification of the ESDs is described in Roelen et al [2006]. Development of the fault trees leading to initiating and pivotal events in the ESDs, including quantification, was conducted by DNV [DNV, 2008]. Due to lack of data, estimates of the probabilities of occurrence of some events in the fault trees had to be based on pure judgement. The objective of this study is to use the NLR Air Safety Database for quantification of those probabilities that previously had been based on judgement. The results of this quantification are described in the current report. August

14 1.3 CONTENTS OF THIS REPORT In Section 2 of this report, the approach to the quantification of fault tree events in the causal model is elaborated. The various steps are described, and some specific issues related to the quantification are discussed, e.g. pros and cons of the approach, dealing with uncertainty, a top down vs. bottom up quantification approach. After that, the results of the quantification are described in Section 3 Section 33. August

15 2 QUANTIFICATION APPROACH 2.1 INTRODUCTION The starting point for the quantification is version 5 of the DNV spread sheet containing all the fault trees developed by DNV [DNV, 2008]. Fault tree events for which the quantification is based on judgement are colour coded yellow in that spread sheet. The objective of this study is to quantify these yellow events by using the NLR Air Safety Database. The approach in this study was to consider the fault trees and fault tree event definitions as fixed. No attempts were made to modify the fault tree structure or to redefine the event definitions to better suit the data. Comments on how the definitions were interpreted are provided, if necessary, under a remarks heading in each section. 2.2 NLR AIR SAFETY DATABASE The NLR Air Safety Database was used as a primary source of data. Four main sources in that database were used for the quantification: ADREP, Airclaims, Service Difficulty Reports (SDR) and Air Safety Reports (ASR). ADREP and Airclaims are the primary accident data sources, whereas the SDR and ASR databases are the primary sources for incident data. For the quantification, the scope in these databases is limited to commercial air transport with western-built fixed wing aircraft heavier than 5,700 kg. Piston engine aircraft are not considered. There are no geographical restrictions. More specific scoping issues for the three databases are described in the following sections. Because of the size of the databases involved, much of the initial analysis is done by running queries, e.g. looking for particular key words. Each incident in the resulting dataset is then individually analyzed to verify whether it fits the particular fault tree under consideration ASR Air Safety Reports (ASRs) are reports by pilots of unsafe conditions and hazardous situations that occurred during operations. ASR systems are company specific: Criteria for reporting and data coding and storage may vary from airline to airline. ASRs are company confidential and are usually not shared with third parties. NLR has access to a number of ASR datasets from European and non- August

16 European airlines. The data protocol described in Section 2.4 is applied to protect the data. The data concern commercial operations with western aircraft of more than 5700 kg maximum take-off weight and cover 14 million flights between 1992 and The ASR database was mainly used in this study to quantify fault tree events leading to an initiating event in an event sequence diagram. The ASR data matches the scope for this study (see second paragraph of Section 2.2), except that it includes helicopters. Therefore practically the entire database could be used for the quantification ADREP AND AIRCLAIMS The Airclaims database provides brief details of all known major operational accidents to jet and turboprop aircraft worldwide. The subset of the Airclaims database purchased by the NLR contains data and descriptive information about all known airline accidents since The accident details have been drawn from many sources both official and unofficial (including press reports). Therefore, they may be incomplete or otherwise incorrect. The ICAO ADREP database is based on the accident/incident data report supplied to the ICAO organization. ADREP is an acronym for Accident Data REPorting system. The database includes worldwide accident/incident data of aircraft (fixed wing and helicopter) heavier than 5,700 kg since The ADREP and Airclaims sources were used in this study to quantify fault tree events leading to a pivotal event close to the end state of an event sequence diagram. The time period for ADREP and Airclaims data considered in this study is In addition to the generic scope mentioned in Section 2.2, this leads to a dataset covering 452 million flights. Note that the ADREP and Airclaims data have been included in the ECCAIRS database. This database has been used for the queries for the quantification of the fault trees SDR The objective of FAA's SDR program is to correct conditions adversely affecting aircraft safety. To do this, FAA collects mechanical reliability reports, analyzes the reports and disseminates trends, problems, and safety alert information to the aviation industry and FAA. FAR and FAR require that holders of certificates issued under part 121 (air carriers) or part 135 (air taxi), August

17 respectively, submit reports to the FAA on certain failures, malfunctions, or defects of specific systems and on all other failures, malfunctions, or defects that have endangered or may endanger the safe operation of an aircraft. In addition, FAR and contain provisions for certificated US and non-us repair stations, respectively, to report to the FAA serious defects in, or other recurring unairworthy conditions of an aircraft, powerplant, propeller, or component. Under FAR , an airline must report each aircraft malfunction incident within 72 hours to the FAA Flight Standards District Office responsible for that airline. After an initial review, the district office mails reports to FAA'S National Safety Data Branch in Oklahoma City, Oklahoma, which screens and enters them into a national computerized data base. The FAA SDR program has been criticised in the past [GAO, 1991]. Some of the criticism is irrelevant for the purpose of this study. However, one of the points of criticism considers underreporting, and this is an important issue. The number of SDRs submitted by airlines operating similar aircraft varies significantly among airlines. Airline officials attribute reporting differences to vague reporting requirements, leading to varying interpretations of what should be reported and to airlines concerns over the public s access to malfunction reports in accordance with the Freedom of Information Act. Concerned about public disclosure of SDR data, some airlines are reluctant to submit malfunction reports to FAA. Differences among airlines reporting practices would diminish the quality of the data because they would not reflect the actual occurrence of mechanical malfunctions. SDR data is limited to US airlines only. Because the level of safety of US airlines is similar to that of EASA operators [IVW, 2004] it is assumed that SDR data is also representative for European air carriers. In this study the SDR database was used to quantify system failures. In addition to the generic restrictions specified in Section 2.2, the SDR baseline dataset was limited to the timeframe. Only events which resulted in precautionary procedures were included in the dataset. Finally, events which were the result of a false warning were excluded. The total data sample included reports. The associated number of flights is 224 million. Because not every aircraft is equipped with the same systems (propellers is an obvious example), exposure data had to be matched to the SDR data. This was done by comparison of aircraft make/model. August

18 By limiting the dataset to occurrences that resulted in precautionary procedures, the analysis is restricted to significant occurrences. This has two advantages: a) the reliability of the data is better, and b) only events that really have a potentially adverse effect on safety are considered. 2.3 QUANTIFICATION ISSUES TOP DOWN VERSUS BOTTOM UP Quantification of a fault tree can be done top down or bottom up depending on the logic gates in the fault tree. The fault trees, as developed by DNV [DNV, 2008] contain 3 types of logic gates: AND, OR and MOR (OR gate where the base events are mutually exclusive). Both the top down and bottom up approaches are used for the quantification as described in this report. In principle, a bottom up approach is preferred because it is possible to use this approach for all types of logic gates in a fault tree. The top down approach is used in those case where the bottom up approach cannot be used. This is true when the base events are described in very specific detail, for which it is unlikely that a specific category is implemented in the various data souces (see Section 2.2) such that searching for a reliable dataset becomes very difficult. Also, events on a very low level in a fault tree could not be worthwile to be reported for inclusion in incident or accident databases al all. The advantage of the top down approach is that the data search is initiated from the more severe event, and these events are more likely to be reported and included in the databases than the lesser severe events. Note that using the top down approach also has its restrictions. This is explained in the following subsections. T Logic gate B1 B2 B3 Figure 1: Generic fault tree. August

19 Top down quantification Consider the simple fault tree of Figure 1 with top event T and base events B1, B2 and B3. Top down quantification starts with finding events that match top event T. Four situations are possible, as illustrated in Figure 2. If the logic gate is an OR, i.e. if T occurs if at least one of the events B1, B2 or B3 occurs, and if B1, B2 and B3 are mutually exclusive, top down quantification is possible, provided of course that the event descriptions contain sufficient detail. If we have a complete set of T, we can obtain a complete set of B1, B2 and B3 from T. Each occurrence of T can be uniquely classified as a B1, a B2 or a B3 and there are no occurrences of B1, B2 or B3 outside T. The number of events (or the probabilities) assigned to B1, B2 and B3 add up to the number of events (or the probability) of T. If the logic gate is an OR, and B1, B2 and B3 are not mutually exclusive, topdown quantification is also possible. If we have a complete set of T, we can obtain a complete set of B1, B2 and B3 from T, provided that the event descriptions contain sufficient detail. There are no occurrences of B1, B2 and B3 outside T, but in this case the number of events (or the probabilities) assigned to B1, B2 and B3 do not add up to the number of events (or the probability) of T, unless the rare event approximation is applied. If the logic gate is an exclusive OR, i.e. T occurs if exactly one of the events B1, B2 or B3 occurs, and if B1, B2 and B3 are not mutually exclusive, top-down quantification is not possible. If we have a complete set of T, we cannot obtain a complete set of B1, B2 and B3 from T, unless we apply the rare event approximation. If the logic gate is an AND, i.e. if T occurs if all of the input events B1, B2 and B3 occur, top-down quantification is not possible. If we have a complete set of T, we cannot obtain a complete set of B1, B2 and B3 from T. The advantage of the top down approach is that the data search is initiated from the more severe event, and these events are more likely to be reported and included in the databases than the lesser severe event. The disadvantage of the top down approach is that it cannot be used in case of an AND gate and in the case of an OR gate it can only be used if the base events are mutually exclusive or the rare event approximation is applied. August

20 T B3 T B3 B1 B2 B1 B2 OR gate. B1, B2 and B3 are mutually exclusive. AND gate. B1, B2 and B3 are not mutually exclusive. T B3 T B3 B1 B1 B2 B2 OR gate. B1, B2 and B3 are not mutually exclusive. Exclusive OR gate. B1, B2 and B3 are not mutually exclusive. Figure 2: Venn diagrams of possible combinations for the fault tree in Figure 1. Bottom up quantification Bottom up quantification starts with determining the number (or probability) of events B1, B2 and B3. The number (or probability) of T follows from B1, B2 and B3 according to the equations from probability theory. The disadvantage of the bottom up approach is that sometimes base events B1, B2 and B3 are not severe enough to be reported, yet the top event T is reported. If the occurrence database is then searched for B1, B2 or B3 we obtain no results (not because they do not occur but because the occurrences are not reported). Rare event approximation If B1, B2 and B2 are relatively small the rare event approximation can be applied to obtain: P(T) = P(B1) + P(B2) + P(B3) for all types of or gates and regardless of mutual exclusiveness. In the case that P(B1), P(B2) and P(B3) are <0.1 the rare event approximation is accurate to within 10% of the true probability. August

21 2.3.2 DENOMINATOR Quantification in this report always starts with retrieving matching events from the NLR Air Safety Database. Probabilities are then calculated by dividing the number of matching events (the numerator) by the proper denominator. The philosophy used throughout the fault trees is that of safety barriers which are being challenged. One possible denominator therefore is the number of challenges. The event probability then describes the failure probability per challenge. The disadvantage of this approach is that it requires the number of challenges and this may not be so easy to obtain. A different approach is to use the total number of flights as the denominator, regardless of the type of challenge. This approach is often used for the quantification in this report GENERAL RULES FOR APPORTIONING IN CASE OF UNKNOWNS Occurrences in the database containing insufficient information to decide for which base event in the fault tree it is applicable have to be somehow assigned to one (or more) base events. A general rule for this is illustrated by means of the following example. Consider the fault tree of Figure 1, and assumed the logic gate is an OR gate and the base events are mutually exclusive. In this case the top-down approach for quantification is allowed. Assume that as a result of the database search we have obtained 21 cases of event T. In the top-down approach these 21 cases have to be apportioned over B1, B2 and B3. However, sometimes the event description of the T occurrence provides insufficient details to determine whether it is a B1, a B2 or a B3. The result of the analysis may then be, for instance, 21 T, of which 4 B1, 8 B2, 2 B3 and 7 unknown. The approach throughout this document is to apportion the unknown cases proportionally over the known cases. In the case of the example, of the 7 unknowns we assume that 2 are B1, 4 are B2 and 1 is B3. In case the number of occurrences in which there is not enough information is much larger than the number of occurrences for which it is clear how they have to be assigned to a base event, the general rule is that the quantification of the base events results in insufficient data CONFIDENCE LEVELS This document only provides point estimates. Confidence levels of these point estimates are not calculated because we can only estimate uncertainty due to randomness in the data and not due to inaccuracies in the dataset or due to the data analysis process. August

22 In case no matches are found, we may be tempted to conclude that the associated probability is zero, but of course the probability might also be so small (yet non-zero) that the dataset from which we started the analysis is too small for the event to be expected. Probabilities of zero have to be avoided however, because then one might be tempted to discard the event from the fault tree altogether and this would result in loss of (qualitative) information (the event may describe a possible, although not very likely part of an accident pathway). Therefore, if no matching events are found, the conclusion throughout this document is that there is insufficient data to quantify the probability of occurrence of the event. 2.4 DATA PROTOCOL This report describes the data analysis process and the final results. For each result (probability estimate) the following information is registered: 1) The data source (uniquely identified by the database name and version number) that was used; 2) Data query; 3) The manual selection of applicable and not applicable data; 4) A copy of the resulting dataset. Because some of the databases contain confidential data, intermediate results such as data base search queries and resulting data samples are not described in this report. NLR is not at liberty to disclose this information to a non-authorised party, yet third parties must have a means to ensure that the result is indeed correctly derived. Therefore, upon request, the Ministry of Transport and Water Management can issue an official statement that the results are correctly derived from the available data. For this purpose a representative of the Ministry of Transport and Water Management will be allowed to inspect the data handling process and original data. The representative will sign a non-disclosure statement after each inspection. August

23 3 ESD1 AIRCRAFT SYSTEM FAILURE 3.1 FAULT TREE EVENTS The following fault tree events are quantified. Table 1 ESD1 event definitions Event id Event name Definition [DNV, 2008] Pivotal event: Failure to achieve maximum braking directional control TO01B32 Brakes not functioning correctly Excluding: engine failures and system failures that can result in Inability to reduce the speed of the aircraft after the decision to reject the take-off due to failure of the brake system lack of braking power or decelerating devices TO01B33 Brakes not applied correctly Inability to reduce the speed of the aircraft after the decision to reject the take-off due to a delay or failure to follow standard operating procedure (or brake system configuration error) August

24 Remark According to the definition of the pivotal event failure to achieve maximum braking as provided in [Roelen et al, 2006], events in which brake system configuration errors (e.g.: anti-skid system activation, selection of braking action, selection of auto or manual braking) or other decelerating devices errors (reversed trust or propeller reverse, lift dumpers) are in scope for the quantification of the fault tree leading to this pivotal event. 3.2 QUANTIFICATION Pivotal event Failure to achieve maximum braking The event TO01B32 (brakes not functioning correctly) is quantified using the SDR database. For this system failure ATA code 3240 is relevant. With 3425 occurrences in 216 million flights, this leads to a probability of brakes not functioning correctly of per flight. Since no relation exist between not correctly functioning brakes and a rejected take-off after a system failure, this probability can also be expressed as per take-off rejection with aircraft system failure. The quantification of event TO01B33 (brakes not applied correctly) was conducted by analysis of a selection of occurrences from the ECCAIRS database (period Jan 1990 through Dec 2005) which included incidents and accidents related to take-offs with a system failure (excluding direction control related failures and engine failures, which are covered by other ESDs). The ECCAIRS database has an exposure of 452 million flights in the considered period. The total amount of occurrences as derived from the query was 223 of which there were 15 records which did not include an event narrative or any information which could be used to match to the criteria of ESD 1. These are assumed to be not applicable to ESD 1 and are therefore subtracted from the data selection. With this, 208 occurrences are used for the quantification of base event TO01B33 (Brakes not applied correctly). From the 208 occurrences there where a total of 23 occurrences which regarded a rejected take-off after a system failure (excluding engine failures). From the 23 rejected take-offs, there where 15 occurrences where the rejected take-off was initiated before the decision speed V1 and 8 occurrences with an RTO speed above V1. From the 15 occurrences where the rejected take off was initiated before the decision speed, 2 resulted in a runway overrun of which 1 (one) occurrence could August

25 be related to event TO01B33 (Brakes not applied correctly). There was insufficient information to determine the cause of the second overrun. In the 12 remaining occurrences, the aircraft stopped on the runway after the take-off was rejected. 3.3 OVERVIEW OF RESULTS The results of Section 3.2 are summarised in Table 2. Table 2 Overview of ESD1 results Event id Event name Frequency Per TO01B32 Brakes not functioning correctly flight TO01B33 Brakes not applied correctly flight August

26 4 ESD2 ATC EVENT Air traffic related event includes for instance an air traffic controller s instruction to abort take-off because of other traffic. Note that runway incursions are excluded here. These are treated in another ESD. 4.1 FAULT TREE EVENTS The following fault tree events are quantified: Table 3 ESD2 event definitions Event id Event name Definition [DNV, 2008] Initial event: Air Traffic related event TO02B11214 Separation infringement with departing a/c caused by aircraft taking off Aircraft loses separation with an aircraft departing which is caused by the aircraft preparing to take-off Pivotal event: Flight Crew rejects take-off TO02B211 Pilot Misdiagnoses The pilot fails to understand the air traffic situation and as a result aborts the take-off above V1 August

27 TO02B212 Pilot Misjudgement The pilot diagnoses the air traffic situation but misjudges the response and incorrectly aborts the take-off above V1. Remarks For event TO02B11214 Separation infringement with departing a/c caused by aircraft taking off, we are looking for situations in which an aircraft is preparing for take-off and it causes a separation infringement with another departing aircraft. Of interest for this event are those situations in which the crew of the departure starts the take-off without a take-off clearance, causing a separation infringement with another departure on the same runway or on another departure runway. ATCo instruction errors are included in another part of the fault tree, and runway incursions are not in scope of this ESD. Note that the distinction between event TO02B11211 (Separation infringement with departing aircraft caused by other aircraft) and TO02B11214 (Separation infringement with departing aircraft caused by aircraft taking off) can be a superficial one. If there is a conflict between two departing aircraft and one is making a mistake, it depends on the viewpoint of an occurrence whether this aircraft is the other aircraft or the aircraft taking off. 4.2 QUANTIFICATION Initiating event Air Traffic related event An ASR dataset has been retrieved showing occurrences of aircraft in take off that are in a separation conflict. This dataset contains 495 occurrences of different types, divided as follows: 234 occurrences (other than runway incursions) are not applicable to this ESD; 56 runway incursions, which are treated in ESD 32; 117 occurrences in which departures are in conflict with go arounds or landing aircraft; 14 occurrences in which departures are in conflict with other departures; 21 occurrences in which departures are in conflict with other traffic (other than departures and landings); 11 bird strikes; and 42 occurrences in which a take-off clearance was cancelled by ATC for unknown reasons. Because the 42 occurrences where the reason was unknown seem to be relevant for this ESD, they are divided proportionally over the 56 runway incursions, 117 conflicts with arrivals, 14 conflicts with departures, 21 other types of conflict and 11 bird strikes. August

28 The 17 occurrences (14 plus 3 additional occurrences out of the 42 unknown occurrences) between two departures are relevant for the quantification of event TO02B11214 (Separation infringement with departing aircraft caused by aircraft taking off). The causes for the conflict are divided as follows: 0 take-off; 5 other departure; 1 ATC instruction error; 4 are related to wake turbulence (which are treated in ESD 37, so they are not applicable for ESD 2); 7 have unknown cause. Assuming that the 7 unknown causes can be caused by either the take-off, the other departure or ATC, in total 1 occurrence is applicable to event TO02B11214 (Separation infringement with departing aircraft caused by aircraft taking off). With an exposure of 14 million flights, this leads to a probability of this event of per take-off. Pivotal event Flight Crew rejects take-off To quantify the events TO02B211 (Pilot misdiagnoses) and TO02B212 (Pilot misjudgement), the ECCAIRS database has been consulted. This resulted in two data sets: one focussing on an ATC event and the other one on bird strikes during take-off. The first data set does not contain any occurrences that are applicable to this ESD. The second data set contains 28 occurrences, in 10 of which a take-off is aborted above (or close to) V1 after a bird strike. When analysing the narratives of these 10 occurrences it is not clear whether the crew misjudged or misdiagnosed the situation, or whether they did the right thing. Clearly, the crew diagnosed that there is a bird strike. Apparently they decided that it is better to abort the take-off than to continue the flight with a damaged aircraft. All in all, the retrieved information from the ECCAIRS database is insufficient to quantify the two events TO02B211 (Pilot misdiagnoses) and TO02B212 (Pilot misjudgement). 4.3 OVERVIEW OF RESULTS The results of Section 4.2 are summarised in Table 4. Table 4 Overview of ESD2 results Event id Event name Frequency Per TO02B11214 Separation infringement with departing take-off a/c caused by aircraft taking off TO02B211 Pilot Misdiagnoses insufficient data TO02B212 Pilot Misjudgement insufficient data August

29 5 ESD3 AIRCRAFT HANDLING BY FLIGHT CREW INAPPROPRIATE 5.1 FAULT TREE EVENTS The following fault tree events are quantified: Table 5 ESD3 event definitions Event id Event name Definition [DNV, 2008] Pivotal event: Flight crew rejects take-off TO03B211 Pilot Misdiagnosis The pilot either fails to realise the problem or diagnoses the problem as something else, perhaps more serious and as a result aborts the take-off. Pivotal event: Flight crew fails to maintain control TO03B51 Uncontrollable No input to controls will allow the pilot to maintain control of the aircraft when take off continued August

30 TO03B52 Lack of control The pilot makes no attempt to control the aircraft when takeoff continued TO03B53 Incorrect Control The pilot applies incorrect control to the aircraft when take-off continued. This can be due to improper training, stress and fatigue. TO03B54 Insufficient control The pilot applies correct measures but are not enough to prevent aircraft leaving off the side of the runway. Pivotal event: Flight crew fails to maintain control TO03B33 Incorrect Control The pilot applies incorrect control to the aircraft, which has speed less than V1. This can be due to improper training, stress and fatigue. TO03B34 Insufficient control The pilot applies correct measures but are not enough to prevent aircraft leaving off the side of the runway. 5.2 QUANTIFICATION A data sample retrieved from ECCAIRS database contains 42 occurrences, from which 35 are relevant for the overall number of inappropriate aircraft handlings by the crew during the take-off, resulting in a (serious) incident or an accident. The other 8 occurrences from the total 42 are not applicable due to the fact the events happened during the landing and not during the take-off. The 34 occurrences can be divided as follows: 5: RTO with V >V1 leading to a runway overrun (including 3 occurrences in which there was actually a veer off); 12: RTO with V<V1, o 10 leading to a runway veer-off; o 1 leading to an overrun; o 1 where control was maintained 15: no RTO, leading to a runway veer-off (at least, no mention was made about an RTO); 1: no RTO, control was maintained and flight was continued; and 1: insufficient information to decide on one of the others. August

31 Pivotal event Flight crew rejects the take-off Regarding the pivotal event Flight crew rejects the take-off TO03b1, 5 events are relevant, of which 4 occurrences explicitly are related to event TO03B211. Of 1 occurrence it can be stated that it is no pilot misdiagnosis. This leads to a probability of 4 events on an exposure of 452 million flights, which is per flight. Pivotal event Flight crew fails to maintain control (TO03c3) In the dataset, 15 occurrences were found that are relevant here. Of these 15, 2 are clearly applicable to event TO03B54 Insufficient control. The narratives of the other 13 events do not contain sufficient information to distinguish between the various base events. Because of this, none of the base events can be quantified. Pivotal event Flight crew fails to maintain control (TO03d2) For the quantification of the base events leading to this pivotal event, out of 12 occurrences related to RTO with V<V1, 9 are applicable for the events to be quantified: TO03B33 Incorrect Control : 1 occurrence; TO03B34 Insufficient Control 8 occurrences; The remaining 3 events are: 1 occurrence relevant to the pilot incorrect control, but leading to an overrun; 1 occurrence where control was maintained; 1 occurrence for which the pilot actions are not clear, so this could be any of the 4 base events. With this, the probability of event TO03B33 Incorrect Control is per flight, and the probability of event TO03B34 Insufficient Control is per flight. 5.3 OVERVIEW OF RESULTS The results of Section 5.2 are summarised in Table 6. Table 6 Overview of ESD3 results Event id Event name Frequency Per TO03B211 Pilot Misdiagnosis Flight TO03B33 Incorrect Control Flight TO03B34 Insufficient Control Flight TO03B51 Uncontrollable No sufficient data TO03B52 Lack of control No sufficient data TO03B53 Incorrect Control No sufficient data TO03B54 Insufficient control No sufficient data August

32 6 ESD4 - AIRCRAFT DIRECTIONAL CONTROL RELATED SYSTEM FAILURE Accident type: uncontrolled collision with ground. Flight phase: take-off. Initiating event: aircraft directional control related system failure Aircraft directional control related system failure Loss of traction and steering capability Flight crew rejects take-off V > V1 Runway overrun no yes Flight crew fails to maintain control Unrecovered loss of control Runway veer-off Failure to achieve maximum braking Runway overrun Aircraft stops on runway Flight crew fails to maintain control Unrecovered loss of control Runway veer-off Aircraft continues take-off 6.1 FAULT TREE EVENTS The following events are quantified: Table 7 ESD4 event definitions Event id Event name Definition [DNV, 2008] Initial event: Aircraft directional control systems failure TO04B121 Brake System Failure Failure in any part of the brake system that results in asymmetric braking force being applied to the wheels and hence causes directional instability. Pivotal event: Flight crew fails to maintain control August

33 TO04B32 Lack of control The pilot makes no attempt to control the aircraft with speed less than V1. TO04B33 Incorrect Control The pilot applies incorrect control to the aircraft, which has speed less than V1. this can be due to improper training, stress and fatigue. TO04B34 Insufficient Control The pilot applies correct measures but are not enough to prevent aircraft leaving off the side of the runway. Pivotal event: Fail to achieve maximum braking TO04B41 Insufficient Runway Length The runway is too short under wet or icy runway conditions for the plane to come to a halt even if the take-off is aborted before V1 is reached. TO04B43 Brakes not applied Failure of the flight crew to apply all the braking systems correctly immediately after take-off rejection. 6.2 QUANTIFICATION Initial event Aircraft directional control systems failure For the quantification of the event TO04B121 Brake system failure the SDR database was used. This event corresponds to ATA code 3240 Landing gear brake system. The number of occurrences is 3425 and the number of flights is 216 million. With this, the probability is per flight. Pivotal event Flight crew fails to maintain control For the quantification of events in the fault trees leading to this pivotal event in this ESD a dataset derived from ECCAIRS database was used. The data sample contains 131 occurrences, from which 121 are relevant for the directional control systems failure. The remaining 10 occurrences are not applicable, mainly due to the fact that the directional control systems failed during the landing and not take-off, or the problems during the take-off were caused by a pilot s mistake and not a system failure. The 121 relevant occurrences can be divided as follows: 115 occurrences which are relevant for the ESD but not relevant for the events that have to be quantified, e.g. because the take-off was not aborted and the flight was continued to the destination or the flight crew decided to divert to the airport of departure, or because take-off was aborted above V1, or there is an RTO but control is maintained; 2 veer-offs after an RTO; and 3 runway overruns after an RTO. August

34 For the quantification of events TO04B32 Lack of control, TO04B33 Incorrect Control and TO04B34 Insufficient control, the runway veer-offs are relevant. The 2 occurrences show that the crew applied corrective actions but without success and with this, these 2 occurrences are relevant for event TO04B34 Insufficient control. Based on the total exposure of flights between 1990 and 2005 in the ECCAIRS data base equal to 452 million, the probability of this event is per flight. There is insufficient data to quantify events TO04B32 Lack of control and TO04B33 Incorrect Control. Pivotal event Fail to achieve maximum braking For the events leading to this pivotal event, the 2 runway overruns after an RTO with speed less than V1 are relevant. No data was found to be applicable for the base event TO04B41 Insufficient Runway Length ; therefore the probability of occurrence cannot be quantified. For the event TO04B43 Brakes not applied correctly only 1 occurrence was found in the ECCAIRS dataset. As a consequence, this event has the probability equal to per flight. For 1 overrun, the pilot applied maximum braking but the system apparently were not functioning correctly (Event TO04B42, but this event does not have to be quantified) and in the 3 rd overrun it is not clear what happened. In this case this one occurrence will not be divided over the 3 possible causes because it is difficult to divide it proportionally over the 3 causes. 6.3 OVERVIEW OF RESULTS The results of Section 6.2 are summarised in Table 8. Table 8 Overview of ESD4 results Event id Event name Frequency Per TO04B121 Brake System Failure Flight TO04B32 Lack of control No sufficient data TO04B33 Incorrect Control No sufficient data TO04B34 Insufficient Control Flight TO04B41 Insufficient Runway Length No sufficient data TO04B43 Brakes not applied correctly Flight August

35 7 ESD5 INCORRECT CONFIGURATION Accident type: uncontrolled collision with ground. Flight phase: take-off. Initiating event: Incorrect configuration. Incorrect configuration Take-off configuration warning Flight crew rejects take-off V > V1 Runway overrun no yes Failure to achieve maximum braking Runway overrun Aircraft stops on runway Aircraft continues take-off Aircraft stalls after rotation Flight crew fails to regain control Unrecovered loss of control Collision with ground Aircraft continues flight Aircraft continues flight 7.1 FAULT TREE EVENTS The following fault tree events are quantified: Table 9 ESD5 event definitions Base Name Definition [DNV, 2008] event Initial event: Incorrect configuration T05B111 Unsuccessful TO configuration checklist Co-pilot fails to determine the position of the flap and slats T05B112 Unsuccessful checklist verification required for a successful take-off Captain fails to identify the incorrect position of the flap and slats determined by co-pilot August

36 T05B12 Flap & slat positions entered into FMC incorrectly Co-pilot fails to enter the correct flap and slat settings into the FMC that the aircraft is incorrectly configured prior to push-back from the stand T05B21 Verification not conducted Captain fails to perform the takeoff configuration check prior to the application of take-off power T05B22 Verification unsuccessful Captain performs the take-off configuration check but fails to notice that the aircraft is configured incorrectly. T05B3 (intermediate) T05B33 T05B311 T05B313 T05B321 T05B322 T05B412 Pivotal event: Take-off configuration warning TOCW horn fails to sound Aircraft takes-off with incorrect configuration Unsuccessful manufacture (TOCWS Failure) Unsuccessful operation (TOCWS Failure) Unsuccessful manufacture (TOCWS Power Supply Failure) Unsuccessful maintenance (TOCWS Power Supply Failure) Pivotal event: Take-off rejected Pilot misjudgement (TO incorrectly rejected above V1) TOCW fails to sound when an incorrect Take-off configuration is entered Aircraft is still able to take-off even with the incorrect configuration TOCW system fails due to unsuccessful manufacture and hence the take-off is not rejected TOCW system fails because the flight crew operate it incorrectly. This includes the failure of the flight crew to check that the TOCW is working prior to taxi or the failure of the crew to reset the TOCW circuit breaker following testing TOCW power supply fails due to unsuccessful manufacture and hence the take-off is not rejected TOCW power supply fails due to unsuccessful maintenance and hence the take-off is not rejected The pilot diagnoses the TOCW but misjudges the situation and allows the aircraft to reach V1 before incorrectly aborting the take-off Pivotal event: Aircraft stalls after rotation T05B61 Stall Unavoidable No input to controls will allow the flight crew to avoid the stall T05B622 Pilot ignores stick shaker Flight crew take no action to the activated stick-shaker August

37 T05B6211 Stick-Shaker failure Stick-shaker fails due to improper manufacture or maintenance T05B6212 Stall AOA too low Stall occurs at an AOA that is less than the AOA required to activate the stick-shaker Pivotal event: Flight crew fails to regain control T05B72 Lack of control The pilot makes no attempt to control the aircraft. T05B73 Incorrect control The pilot applies incorrect control to the aircraft. This can be due to improper training, stress and fatigue T05B74 Insufficient control The pilot applies correct measures but are not enough to prevent aircraft leaving off the side of the runway Remarks Event TO05B12 Flap & slat positions entered into FMC incorrectly only mentions incorrect flap & slat positions and as such, only flap/slat positions are considered for the quantification. However, for the initiating event of the ESD as defined in [Roelen et al, 2006], 9 types of incorrect configurations are considered: thrust not set to take-off thrust, thrust reverser not stowed, parking brake not released, flaps not in take-off position, spoilers/speed brakes not stowed, stabiliser trim not within green band, main landing gear not aligned, rudder trim not centred, flight control system not properly set (e.g. yaw damper not switched on). Events TO05B3 TOCW horn fails to sound and TO05B33 Aircraft takes off with incorrect configuration address take-off configuration warning failures. The fault tree mainly considers technical failures, such as a TOCW system failure and a TOCW system power supply failure. In the definition of the pivotal event [Roelen et al. 2006] two other reasons for the absence of a warning are given: inhibition of the warning and no TOCW system available. 7.2 QUANTIFICATION Initiating event incorrect configuration The Air Safety Reports (ASR) database is searched for events involving a take-off with incorrect configuration and/or take-off configuration warnings. The ASR database that is used for the quantification of ESD 5 covers 14 million flights between 1992 and There are more than 1600 occurrences involving a take-off configuration (warning). To limit the amount of effort for the analysis, only the occurrences in the years 2000 and 2001 are analysed, because these two years present the most complete occurrence reporting in the period of August