FAA-NEXTO NAS/ATM Performance Indexes Dr. Alexander (Sasha) Klein CENTE FO AI
Acknowledgments This research was funded by the FAA-NEXTO-GMU contract #DTFAWA-4-D-13 Many thanks to our FAA sponsors: Steve Bradford, ich Jehlen, Diana Liang FAA Also to: Stephane Mondoloni CSSI Glenn oberts - MITE CAASD George Donohue, Lance Sherry GMU Barry Davis, Carlton Wine FAA; Doug Williamson - Crown 2
Project Objectives Develop a framework for assessing NAS/ATM performance on a recurring / daily basis 3 Come up with a simple yet informative index of weather-related ATM performance on a given day ( one number ) Produce charts for each season Compare different seasons: Did we do better this year than last year? Account for major external factors: Weather Traffic Demand Enhance existing methods for NAS performance analysis efine computation of the effects of both en-route and terminal weather Consider additional metrics alongside Delay Traffic Demand (Schedules) Procedures Facilities Weather NAS ATM Delays Costs
Operational esponse Index (OI), 24 Components, Excluding Highest-Cost Days OI Component Analysis, 24 (excluding a few highest-cost snowstorm outliers) OI: 5-Period Moving Average of $$ Cost $14,, $12,, $1,, $8,, $6,, $4,, OI = Total daily OPSNet cost of Excess block time vs. Schedule, Excess distance vs. Flight-planned, Cancellations, and Diversions per flight Excess Block Time (pink) Cancellations (light blue) Diversions (yellow) Direct airline operating costs per flight for an averaged narrowbody fleet All OPSNet flights daily A cost-derived metric Computation details here $2,, Excess Distance (navy blue) $ 1 21 41 61 81 11 121 141 161 181 21 221 241 261 281 31 321 341 361 Day, from lowest to highest cost 4
OI for 366 Days 1/1 12/31/24, sorted by Date OI, 24, All days 1, 9 8 7 6 5 4 3 2 1 5 9/23/24 1/7/24 1/21/24 11/4/24 11/18/24 12/2/24 12/16/24 12/3/24 Snowstorms Volatile Wx in early spring Convective season Quiet period Hurricanes and convective Wx Snowstorms Somewhat quieter period 1/1/24 1/15/24 1/29/24 2/12/24 2/26/24 3/11/24 3/25/24 4/8/24 4/22/24 5/6/24 5/2/24 6/3/24 6/17/24 7/1/24 7/15/24 7/29/24 8/12/24 8/26/24 9/9/24
Weather-Impacted Traffic Index (WITI) Combined En-oute and Terminal Wx 6 En-oute WITI: Find intersections of each flow (GC track) with hex cells where convective Wx was reported Multiply by each hex cell s total NCWD count (reflects Wx duration) and by # of daily flights on this flow Add up all flows: En-oute WITI Terminal WITI: Hourly surface Wx observations at major airports Capacity degradation % for each Wx type * hourly movement rate Add up all airports: Terminal WITI Combined WITI (CWITI): Weighted sum of En-route and Terminal WITI eflects front-end impact of Wx on intended flights
Combined WITI and NAS Performance Metrics Example: OI & Delays, June-Oct 24 Including Outliers ASPM Arr Delays and Operational esponse Index (OI) vs. Combined WITI 9 8 7 6 5 Hurricane Frances 9/5 9/4 Hurricane Jeanne 9/26 9/3 ASPM Arrival Delay Hurricane Ivan 9/16 Even in ideal weather, there are significant residual delays and costs 4 3 2 OI 1 1 2 3 4 5 6 7 Combined WITI 7
OI and Delays vs. Combined WITI Example 1: Convective Weather 9,, ASPM Arr Delay and OI vs. Combined WITI Outliers (Exceedingly high costs on or around hurricane days) are removed Jun-Nov 24 July 14, 24 8,, 7,, 6,, 5,, 4,, ASPM Arrival Delay Very high OI ($485/flight) Very high delays Medium-high WITI Checking 3,, 2,, 1,, OI 1,, 2,, 3,, 4,, 5,, Combined WITI 8
Zooming In on July 14, 24 OI, En-oute and Terminal WITI 1,, En-route WITI is high (July ) 9,, Terminal WITI is low 8,, 7,, 6,, 5,, 4,, 3,, E-WITI OI Comb-WITI WITI A Terminal WITI OI Combined WITI Combined WITI is medium-high Very high % of cancellations: 3x the usual High operational response cost (OI) was caused by en-route thunderstorms leading to delays and cancellations 2,, 1,, T-WITI NAS performance was worse than usual on this day 784 794 714 7114 7124 7134 7144 7154 7164 7174 7184 9
OI and Delays vs. Combined WITI Example 2: Non-Convective Weather 9,, ASPM Arr Delay and OI vs. Combined WITI Outliers (Exceedingly high costs on or around hurricane days) are removed Jun-Nov 24 October 2, 24 8,, 7,, 6,, 5,, 4,, ASPM Arrival Delay OI is high but in line with average trend elatively low delays Very high WITI Checking 3,, 2,, 1,, OI 1,, 2,, 3,, 4,, 5,, Combined WITI 1
Zooming In on October 2, 24 OI, En-oute and Terminal WITI 7,, En-route WITI is low (late October) 6,, T-WITI But Terminal WITI is very high (rain, low ceilings etc) 5,, So the Combined WITI is high 4,, 3,, 2,, OI Comb WITI WITI A Terminal WITI OI Combined WITI That is, high OI ($/flight) and delays were caused mostly by terminal Wx NAS performance was actually good for this IMC day 1,, (Better in terms of delays than costs) E-WITI 11 1154 1164 1174 1184 1194 124 1214 1224 1234 1244 1254
OI and Delays vs. Combined WITI Example 3: Two Metrics Yield Different esults 9,, 8,, ASPM Arr Delay and OI vs. Combined WITI Outliers (Exceedingly high costs on or around hurricane days) are removed Jun-Nov 24 November 23, 24 High CWITI 7,, 6,, 5,, 4,, 3,, 2,, 1,, ASPM Arrival Delay OI 1,, 2,, 3,, 4,, 5,, Combined WITI High delays but just average OI NAS performance could be judged as poor if only delays were considered But considering costs (OI), it was about average given the weather and the demand 12
Comparing Delays for 24 and 25 May-September 25 delays were on average about the same as in 24 (1% diff.) ASPM Arrival Delays (avg delay for all flights, ASPM 55 airports, vs. schedule) 4 35 3 24 25 25 2 15 1 5 5/1 5/11 5/21 5/31 6/1 6/2 6/3 7/1 7/2 7/3 8/9 8/19 8/29 9/8 9/18 9/28 But, weather (CWITI) was on average better in May-Sep 25 (If 24 average = 1, then 25 average = 83) 13
Normalized Delay-to-Wx atio Comparison Delay-Based NAS/ATM Performance Index, 5 vs. 4 Normalized ASPM Arr Delay vs Weather, May-Sep 24 and 25 Against 24 Trendline (all days, including hurricane-impacted) 35 3 24 average Delay vs. 24 WITI = 1 25 average Delay vs. 25 WITI vs. 24 WITI = 12 25 2 24 25 15 1 5 14 25 average 24 benchmark 5/1/24 5/8/24 5/15/24 5/22/24 5/29/24 6/5/24 6/12/24 6/19/24 6/26/24 7/3/24 7/1/24 7/17/24 7/24/24 7/31/24 8/7/24 8/14/24 8/21/24 8/28/24 9/4/24 9/11/24 9/18/24 9/25/24
Delays and Traffic Demand Taking Exponential Delay-vs.-Demand Factor into Account Looking at 199-25 historical monthly averages 25: a 1% traffic increase 3 25 Monthly total delays vs operations, Jan 199 - Aug 25 1% increase in traffic (from 4.3M to 4.7M ops) can lead to a 45% increase in delays (from 1.25M to 1.8M minutes) This factor ought to be taken into account when we talk about NAS / ATM performance Monthly delay total, minutes 2 15 1 5 The trend doesn t depend on weather Adjusted chart is shown on next slide 1.2E+6 3.5E+6 3.7E+6 3.9E+6 4.E+6 4.1E+6 4.2E+6 4.3E+6 Monthly instrument ops, NAS OPSNet 4.4E+6 4.6E+6 15
ATM Performance Index, 25 vs. 24 Adjusted by Exponential Delay-vs-Demand Factor 24 benchmark = 1 25 average adjusted for Weather only = 12 Adjustment factor: divide by 145% (exponential delay increase rate), multiply by 11% (traffic increase rate; need to pro-rate 25 back to 24) 25 adjusted-for-weather-and-demand average = 12 / (1.45 / 1.1) = 91 Normalized ASPM Arr Delay vs Weather, May-Sep 24 and 25 Against 24 Trendline, Adjusted by Delay-vs-Demand Factor (all days, including hurricane-impacted) 35 3 25 adjusted average = 91 25 2 24 25 15 1 24 benchmark 25 adjusted average 5 16 5/1/24 5/8/24 5/15/24 5/22/24 5/29/24 6/5/24 6/12/24 6/19/24 6/26/24 7/3/24 7/1/24 7/17/24 7/24/24 7/31/24 8/7/24 8/14/24 8/21/24 8/28/24 9/4/24 9/11/24 9/18/24 9/25/24
Discussion Delays Did the NAS/ATM do 9% better in 25 than in 24? NAS delays were similar; delays vs. weather were worse in 25 But, relative to weather and traffic demand, the ATM component of the NAS did do better in 25 than in 24 D-VSM and other measures may have helped Even so, We are on the ascending slope of the exponential delay curve Peak delays in bad weather (July 25) were highest ever Delay variance is significant The exact proportion (45% delay increase due to 1% traffic demand growth) needs to be fine tuned Monthly delay total, minutes 3 25 2 15 1 5 We are here Monthly instrument ops, NAS OPSNet 17
3. 3 2.5 25 OI: Cost-to-Wx atio The uler for OI (1) is an Averaged Day Normalized Indices, Jun-Sep 24 "Cost", "Weather", and "NAS Cost-to-Wx Index", normalized (Current-day / Average) Outliers (hurricane-impacted days) removed Standard Deviation: Cost.22, Weather.4, NAS Cost-to-Wx Index.6 2. 2 1.5 15 Normalized Wx Normalized Cost (OI) 1. 1.5. 3 614 6114 6214 714 7114 7214 7314 814 8214 8314 9144 93 -.5-5 25-1. -1 2-1.5-15 Normalized-Cost-to-Normalized-Wx atio (is roughly inversely proportional to OI) -2. -2 1-2.5-25 -3. -3 18 Values below 1 are good High peaks are bad
Conclusions Delay, cost (OI) and weather (WITI) metrics computed for 24 and 25 Delay metric can be normalized vs. seasonal-average (e.g. 24 s) Normalized cost (OI) is a useful additional metric WITI calculation refined for both en-route and terminal parts Delay/Cost metrics should account for traffic demand, not just weather, if used as NAS/ATM performance indicators 1-15% traffic demand increase can cause 45-6% increase in delays Slightly better NAS/ATM performance in 25 if both weather and traffic demand are taken into account These metrics can advance our understanding of NAS response to external impacts 19
NAS esponse to External Impacts Traffic Demand NAS Weather Impact Unavoidable Avoidable Excess Demand vs. Capacity Unavoidable NAS esponse (Delays, Costs) Inefficiencies 2 1) What portion of delays/costs is due to system inefficiencies as opposed to unavoidable weather and traffic demand outside ATM s control? 2) Can we quantify positive impact of NAS/ATM efficiency improvements? A T M NAS/ATM Efficiency Improvements Excess D vs. C Unavoidable Inefficiencies
Back-up Slides 21
Operational esponse Index (OI) Components Using direct carrier costs only Passenger impact (value of pax time, ill will, re-issuing tickets etc) excluded Flights per day: OPSNet daily totals (varies between 37, and 5,) Simplifying assumption: all aircraft are narrowbodies Cost of 1 Minute of Delay Used $22/min (based on total non-fuel operating costs averaged for a narrowbody jet) Cost of 1 Extra Mile Flown (expressed in $/min) Equivalent to $18/min (based on 24 fuel cost average for a narrowbody in cruise at $1.25 / gallon) Cost of a [Narrowbody] Cancellation US carrier-reported average cost was $4,5 in 94 which equates to $6, per cancellation in 24 Cost of a [Narrowbody] Diversion Assuming 4 hrs extra block time and a $2,5 hourly operating cost for a narrowbody, we get $1, per diversion 22 Sources: OIG; BTS; MITE CAASD; FAA APO; FAA OPSNet database
Operational esponse Index (OI) Calculation OI = (Num_fl * Avg dist * Avg_fuelburn * Fuel_cost + Num_fl * Avg time * Avg_nonfuel_oper_cost Num_diversions * Avg_cost_of_diversion + Num_Cancellations * Avg_cost_of_cancellation) / Num_fl where: dist = average excess distance per flight (actual vs. flight-planned) time = average excess block time per flight (actual vs. scheduled) Avg_fuelburn = fuelburn for a generic narrowbody jet in cruise at FL33 Num_fl = daily number of OPSNet flights Sources of data: FAA APO Lab; FAA ASPM 23
Operational esponse Index eality Check Comparison with $$ Quoted in Literature For OPSNet flights: Total excess airline cost for 24 (all flights, all days) is $4.6B For a baseline ideal day (OI = $15/flight): if all days in 24 were like it, total cost would have been $2.7B Difference = $1.9B in direct operating costs eferences show comparable excess-cost estimates: Some figures have indicated that the total average direct annual costs of the irregular operations of ten U.S. major airlines for the period 1996-1999 have been about $1.9B (M.Janic TB eport, 23) the Air Transport Association estimated that delays cost the air carriers approximately $2.B in direct operating costs in 1999 : OIG eport, 2 The Air Transport Association's amount increases to nearly $5B when indirect costs and the value of passengers' lost time are included : OIG eport, 2 (extrapolation of our calculations to include indirect costs produces comparable numbers AK) Total operating costs of delays: $1.8-2.4B in 1987-94: FAA APO-13, Total Cost for Air Carrier Delay eport, 1996 1999 total cost of disruptions estimated at $1.8B (Z.Shavell Effects of Schedule Disruptions on the Economics of Airline Operations. In: Air Transportation Systems Engineering, 21, Chapter 8). 24
25 Delay/Wx: Linear vs. Exponential Trend A Sign of a Worsening Delay Situation? Exponential trendline: a better fit for 25 Delay-vs-Weather Plot? Linear trendline is a better fit for 24 data 25 ASPM Arr Delay vs CWITI, 24-25 May-Sep Excluding hurricane-impacted days 25 ASPM Arr Delay vs CWITI, 24-25 May-Sep Excluding hurricane-impacted days y = 47.823e.78x A rr D elay per Flight, N orm alized to 24 average 2 15 1 5 Linear trendline y =.832x + 3.734 Arr Delay per Flight, Normalized to 24 average 2 15 1 5 25 Linear (25) Exponential trendline 25 Expon. (25) 5 1 15 2 25 5 1 15 2 25 CWITI, Normalized to 24 average CWITI, Normalized to 24 average 25
Quiet Period Monthly ASPM Delays vs. Ops 1995-25 1-Year Trend: NAS Delays vs Ops, "Quiet months" (Mar,Apr,Oct,Nov) 32 3 28 26 24 22 2 18 16 14 12 1 8 6 4 2 26 Total monthly delay minutes vs. Schedule 388761 3919373 3949738 414168 443193 476275 497886 4117945 413877 4166976 4197472 423155 4254236 436868 4331362 4358776 439865 4451586 449993 4571369 4662789 Monthly ops
Terminal Capacity Degradation Weather factor Available airport capacity, % nominal THUNDESTOM 1 HEAVY_SNOW 3 HIGH_WIND (>3 kt)* 3 HEAVY_AIN 4 LOW_VISIBILITY 7 LOW_CEILING 7 SNOW 7 AIN 7 WIND (2-3 kt) 7 NO_WEATHE 1 *sustained wind above 3 kt, higher gusts 27
Flows and Actual Tracks Similarity 28