Uncertainty in Airport Planning Prof. Richard de Neufville Istanbul Technical University Air Transportation Management M.Sc. Program Airport Planning and Management / RdN Airport Planning and Management Module 06 January 2016
Objective of Presentation To demonstrate that: The Forecast is always Wrong in the sense that what actually occurs differs substantially from what had been forecast earlier This is due to all kinds of different special events and trend breakers that constantly arise
Outline of Presentation Empirical Evidence of the Forecast is always Wrong Estimation of Costs Forecasts of Prices FAA Analysis of their forecasts Worldwide examples Reflection on Reasons for Forecast is always Wrong
Estimation of Costs Three cases: 1.Simplest possible estimate of costs repaving of runway 2.Industry-wide variance between estimates in advance and actual costs 3.Variability of Jet Fuel Prices
Let s look at simplest case Simplest possible technology: asphalt paving of existing runways long flat surfaces with everything visible Simple issue: Cost estimation straightforward Sample of 59 cases of US airports, comparing estimated cost with actual cost, adjusted for inflation according to construction price index. RESULTS OFF IN RANGE OF A FACTOR OF 2!
Percent of Occurrences Ratio of Real Costs to Estimated Costs for Airport Projects 15 Costs expressed in constant dollars Median ~= 1.25 10 5 0 0 0.5 1 1.5 2 2.5 Real/Estimated Cost Ratio
Estimation of Costs Let s think a moment about why the estimates were so far from actual costs What could be the causes?
News report Cost of asphalt rising Local road repairs are likely to lag By Kay Lazar, Globe Correspondent June 8, 2006 http://www.boston.com/news/local/articles/2006/06/08/cost_of_asphalt_rising/ The soaring cost of petroleum -- a primary ingredient of asphalt -- has forced many communities to shoulder a 50 percent increase in costs as crews head out to repave roads this summer. Asphalt sticker shock is creating heartburn for legions of highway chiefs, prompting many to significantly curtail the number of roads they intend to repair this season. And the going may get even tougher. Two major asphalt suppliers for the region are warning of another 50 percent increase by Thanksgiving -- in addition to potential shortages
Ratio of Actual to Estimated Cost Cost Growth for Various Projects 3.0 2.5 2.0 1.5 1.0 0.5 0.0 DOD60 HWAY WATER BLDNG DOD50 ADHOC MAJOR ENRGY NASA NASA AVG St.Dev. CONST SAT m g
Ratio of Actual to Estimated Cost NASA Projects Cost Growth 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 HST GLL UARS GRO COBE MGL MOBS LSAT EUVE ERBE AVG St.Dev.
Forecasts of Prices Three cases: 1. Copper Prices by Chilean experts 2. Price of Crude Oil 3. Variability of Jet Fuel Prices You can replicate such charts yourself by searching web
cusd/(cu-pound) World Copper Prices 450 400 350 Range estimated by COCHILCO 300 250 COCHILCO Expected Value (first trimestre of the previous year) 200 Nominal Annual Copper Price reported by COCHILCO 150 2007 2008 2009 2010 2011 2012 (11/10/2012) Year Source: J. Esteban Montero, 2012 Note: Jan. 12 2016 = 225
Oil price forecasts Note: Jan 12 2016 = $31.40 EIA Annual Energy Outlook 2006 and BP Statistical Review of World Energy June 2012 Per Julia Kimmerly, 2012
Variability in Jet Fuel Prices Source: http://www.indexmundi.com/commodities/?commodity=jet-fuel&months=180
FAA Analyses of Errors Several cases: 1. Friedman 2004 2. FAA 2008 3. FAA Aerospace Forecasts 2006, 2013 The extensive, publically available data of the FAA, a most professional group, enable us to appreciate the difficulty of making forecasts
Results of a study of TAF (Terminal Area Forecasts for US) Errors in 5 year TAF Cumulative Distribution 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 50 Percent error (Absolute value) Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results Jerome Friedman, MITRE CAASD, Sept. 30, 2004
Results of a study of TAF Fregquency of Error (%) 20 15 10 5 0 0 3 7 10 Errors in 5 year TAF Note: Average error ~ 11% 13 17 20 23 27 30 33 36 40 Percent Error (Absolute value) Adapted from: Terminal Area Forecast (TAF) Accuracy Assessment Results Jerome Friedman, MITRE CAASD. Study dated Sept. 30, 2004, but data until 2000. Deliberate omission of 2001, 2002 when traffic dropped enormously
Error Data from U.S Source: MITRE CAASD and FAA City 5 YEAR FORECAST FROM / TO 92 93 94 95 97 98 99 2000 ATL 10.8-10 -15.5-13.3 BOS -0.8-2.1-0.2 8.9 BWI 27.4 6.3-5.8 7.5 CLE -13.4-14.6-17.5-13.3 CLT 7.8 15.8 10.9 18.1 CVG -0.9-4.3-14.5-9.5 DCA 7.5 13.8 2.7-5.1 DEN 18 19.6 19.9 5.5 DFW 2.6-3.2 7.8 14.2 DTW -13.2-6.5-8.6 3.1 EWR -12.2-5.9-0.1 5 FLL 17.2 3.4-3.8-11.6 HNL 28.2 40.9 23 14.9 IAD -6.4-9.5-41.4-41.2 IAH -8.1-12.3-8.9-25.4 JFK -0.9 6.2 11.7 3.8
Error Data from U.S Source: MITRE CAASD and FAA City 5 YEAR FORECAST FROM / TO 92 93 94 95 97 98 99 2000 LAS -5.1-2.2-10.4 8.5 LAX 2.2-1.6-1.4 0.6 LGA 7.7 3.9-3.2-4.8 MCO 15.9 15.1 18.6 14.9 MDW 33.9-21.3-32.9-5.2 MEM 9.1 16.3 10 4.7 MIA 3.9 3.8 13.1 21.5 MSP -5.4 0.4 5.6 0 ORD -6.8-5.1-3.4 2 PDX -3.9-12.2-9.5 3.5 PHL -3.2-7.2-4.2 0.1 PHX 6.1 6 4.1-2.8 PIT 9.4 9.9 8.2 6.6 SAN 21.6 20 15.6 20.6 SEA 6.3-0.8-15.3-9.3 SFO 25.7 22.5 17.4 10.7 SLC -6.7-0.2-0.3 9.8 STL -7.4-1.3-3.3 4.9 TPA 16.5 12.4 6.9 5.6
Locid City Airport Name Preliminary CY08 Total Enplanements Forecast 2004 TAF in thousands % Difference CVG Cincinnati Cincinnati/Northern Kentucky International 6,488,422 13193-103.3 IAD Washington Washington Dulles International 11,287,621 15861-40.5 MDW Chicago Chicago Midway International 8,019,338 10714-33.6 MSP Minneapolis St. Paul Minneapolis-St Paul International/Wold-Chamberlain 16,352,653 21361-30.6 BWI Baltimore Washington Baltimore/Washington International Thurgood Marshal 10,206,399 13135-28.7 HNL Honolulu Honolulu International 9,000,365 11483-27.6 DTW Detroit Detroit Metropolitan Wayne County 16,993,820 20183-18.8 LAX Los Angeles Los Angeles International 28,612,013 33628-17.5 ORD Chicago Chicago O'Hare International 33,668,545 38760-15.1 TPA Tampa Tampa International 8,869,806 10182-14.8 BOS Boston General Edward Lawrence Logan International 12,784,965 14658-14.7 LGA New York La Guardia 11,549,790 13225-14.5 PHL Philadelphia Philadelphia International 15,577,122 17817-14.4 PHX Phoenix Phoenix Sky Harbor International 19,433,827 22175-14.1 FLL Fort Lauderdale Fort Lauderdale/Hollywood International 11,018,382 12557-14.0 SLC Salt Lake City Salt Lake City International 9,889,030 11180-13.1 MCO Orlando Orlando International 17,271,885 19192-11.1 STL St. Louis Lambert-St Louis International 6,644,199 7359-10.8 ATL Atlanta Hartsfield - Jackson Atlanta International 43,737,608 47130-7.8 DFW Dallas/Ft.Worth Dallas/Fort Worth International 27,206,541 29202-7.3 LAS Las Vegas McCarran International 21,011,949 22424-6.7 SAN San Diego San Diego International 9,007,602 9437-4.8 SFO San Francisco San Francisco International 18,101,502 18496-2.2 IAH Houston George Bush Intercontinental/Houston 19,850,397 20070-1.1 EWR New York Newark Liberty International 17,578,856 17604-0.1 PDX Portland Portland International 7,073,767 6956 1.7 SEA Seattle Seattle-Tacoma International 15,815,133 15456 2.3 DCA Washington Ronald Reagan Washington National 8,692,131 8336 4.1 JFK New York John F Kennedy International 23,601,779 22306 5.5 DEN Denver Denver International 24,266,328 22817 6.0 MIA Miami Miami International 16,369,998 15369 6.1 CLT Charlotte Charlotte/Douglas International 17,271,119 14678 15.0 Actual 2008 traffic compared to that forecast in 2004 TAF Source: US FAA Sum -426.5 Average -13.3 Sum Absolute 507.7 Average Absolute 15.9
Actual vs. Forecast 10 years earlier Source: FAA Aerospace Forecast FY 2006-2017 Domestic Commercial Emplanements Year being forecast Actual vs. Forecast 1995 11.4 1996 12.2 1997 17.4 1998 14.9 1999 9.9 2000 5.5 2001 4.7 2002 14.5 2003 12.5 2004 20.0 2005 13.9 Median and Average 12.5 Note: These are aggregate data, in which greater local variations tend to cancel each other out
Older FAA Forecasts vs. Actual Data (% Difference) Forecast Year For Year Commercial Enplnmnts. Revenue Pax. Miles Hrs. General Aviation GA Ops at FAA Towers Tot. Ops at FAA Towers 1959 1964 (1.30) (6.50) (0.60) 4.60 9.70 1960 1965 (9.50) (9.70) (1.20) (27.80) (21.60) 1961 1966 (27.50) (26.00) (15.30) (37.70) (28.90) 1962 1967 (32.10) (31.40) (23.60) (34.70) (27.30) 1963 1968 (41.30) (41.30) * (38.40) (32.50) 1964 1969 (31.40) (33.60) (23.50) (27.30) (24.90) 1965 1970 (14.10) (19.80) (16.30) (2.60) (5.20) 1966 1971 9.40 0.50 (1.60) 53.70 42.20 1967 1972 23.60 13.00 9.10 72.50 54.90 1968 1973 23.90 15.90 7.40 78.30 58.40 1969 1974 21.10 21.20 4.60 53.60 42.40 1970 1975 26.30 33.00 (0.60) 80.90 25.90 1971 1976 19.00 28.60 (0.60) 42.90 22.90 1972 1977 22.30 33.70 (6.80) 36.90 4.50 1973 1978 14.00 18.30 (10.40) 14.80 8.80 1974 1979 (9.70) (7.40) (13.70) 11.80 9.40 1975 1980 (10.60) (17.30) (0.20) 34.60 25.70 1976 1981 4.30 (1.80) 15.70 41.30 32.10 Review of the FAA 1982 National Airspace System Plan
Available, Revenue Seat- Miles Note: Large AVERAGE errors in 5 years Some actual errors much greater! FAA Aerospace Forecast, Fiscal Years 2013-2033 https://www.faa.gov/data_research/aviation/ aerospace_forecasts/media/2013_forecast.pdf
Worldwide Airport Forecasts Four cases: 1. Japan International Traffic 2. Sydney Passenger Traffic 3. Miami Area Passenger Forecasts 4. US General Aviation Forecasts
Forecast vs. Actual International Pax in Japan Forecast Passengers (million) Percent Error For Done In Actual Forecast Difference/Actual 1980 1970 12.1 20.0 65 1985 1975 17.6 27.0 53 1990 1980 31.0 39.5 27 1995 1985 43.6 37.9 (13)
Forecast vs. Actual International Pax to Japan 450,000 400,000 350,000 10,000 ton 300,000 250,000 200,000 150,000 100,000 50,000 0 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Year Actual Forecast
Notice the Pattern! Forecasting is an exercise in projecting past into future like steering car by looking into rear view mirror! Past low growth => under estimation Past high growth => over estimation Almost never right!
Forecast vs Actual International Pax in Sydney Forecast For Year Source of Forecast, in Year Consultant Regional Study National Ministry 1974 1978 1983 1980 3.77 2.98-3.46 1985 7.4 3.87-4.34 2.674-3.047 1990 9.8 4.71-5.51 2.762-3.751 2000 projected 12.0 6.27-8.66 2.938-5.159 2000 actual 10
Forecast vs. Actual Miami/International What Happened? Completely Missed Lack of Growth in International for Miami Underestimated Doubling of Domestic Traffic, in 2013 30% above forecast for 2020
Growth of Domestic Traffic at Miami/Fort Lauderdale Traffic at Miami s other major airport (mostly domestic) grew by 150%: from 9 to 23.5 million passengers Largely due to Low Cost Carriers such as Southwest
ACTIVE GENERAL AVIATION AIRCRAFT 1980 FORECAST ACTUAL 80 FCST. 400 NUMBER OF AIRCRAFT 300 200 100 0 70 72 74 76 78 80 82 84 86 88 90
ACTIVE GENERAL AVIATION AIRCRAFT 1985 FORECAST 400 ACTUAL 80 FCST. 85 FCST. NUMBER OF AIRCRAFT 300 200 100 0 70 72 74 76 78 80 82 84 86 88 90 92 94
ACTIVE GENERAL AVIATION AIRCRAFT 1990 FORECAST 400 ACTUAL 80 FCST. 85 FCST. 90 FCST. NUMBER OF AIRCRAFT 300 200 100 0 70 74 78 82 86 90 94 98
ACTIVE GENERAL AVIATION AIRCRAFT 1998 FORECAST 400 ACTUAL 80 FCST. 85 FCST. 90 FCST. 98 FCST. NUMBER OF AIRCRAFT 300 200 100 0 70 74 78 82 86 90 94 98 02 06
Causes of Uncertainty Variability of phenomenon business cycles Difficulties in measurement transfer passengers Unpredictable circumstances or trend breakers airline mergers, hub closures, politics Limits to valid measurement Demand elasticity [which is what?]
Summary and Recommendations Summary Forecast Errors have been large Likely to continue Recommendations: Expensive Forecasting is cost-ineffective Use general trends...with large ranges Flexible Approach to Planning!!!