Operational Decision Tree Avalanche Forecasting

Transcription

1 Internatinal Snw Science Wrkshp (2002: Pentictn, B.C.) Operatinal Decisin Tree Avalanche Frecasting Walter Rsenthal Dnald Bren Schl fnvirnmental Science and Management University fcalifrnia, Santa Barbara Kelly lder Rcky Muntain Research Statin USDA Frest Service Frt Cllins, Clrad Rbert. Davis Cld Regins Research and ngineering Labratry US Army ngineer Research and Develpment Center Hanver, New Hampshire Abstract: Decisin tree mdels fmaximum avalanche size class run daily at Mammth Muntain, Califrnia. A classificatin tree grwn n an eight-year subset fall weather and avalanche recrds shws an abslute accuracy n avalanche cntrl days f frm abut 60-70% in a given year; accepting verestimates increases this t 70-80%. rrrs arise frm the rarity flarge events, exclusin fthe smallest mst frequent events, and tree sensitivity t small changes in key predictr variables. A cmplete 1-year data set yields a pair fdecisin trees frecasting bth maximum size class and maximum crwn size ver the entire muntain. Tested against a twentieth year, the size class tree may be mre accurate fr extreme events but perfrmed slightly wrse verall than the riginal tree. Cupling the size class and crwn trees identified bth class 5 avalanches during the test year. A third set ftrees, driven by hurly data frm a remte instrument netwrk, distributes maximum class and crwn sizes ver gegraphic sub-regins fthe muntain. These are striking fr bth small size and lw misclassificatin rate. Ifa majr surce ferrr is chatic avalanche behavir, decisin trees may prve mst valuable fr prviding prbability estimates frm given sets finitial cnditins. Keywrds: avalanche, decisin trees, avalanche frecasting 1. Intrductin Decisin tree mdels favalanche activity can be valuable frecasting and training tls. They rank imprtant relatinships amng avalanche variables and prvide clear graphic display f the relative imprtance f variables n a given day. They prvide prbabilities f different results and can alert wrkers t unusual cnditins. Binary decisin trees may be either classificatin r regressin trees (Breiman et al., 184). Data are arrayed in a learning sample fthe respnse variable (e.g., size class) and predictr variables (new snw depth, 24-hur wind speed, etc.). They are split n the single predictr value that prduces the tw mst *Cn'espnding authr: Walter Rsenthal, P.O. Bx 7457, Mammth Lakes, CA 3546; tel: ; walter@icess.ucsb.edu hmgenus respnse variable subsets. Fr example, mre than 0.5m fnew snw in 24 hurs may prduce mstly class 3-5 avalanches while thse with less new snw may prduce mstly class 1, 2 r n avalanches. ach fthe tw subsets is then reevaluated and recursively split until the final subsets shrink belw a specified size r meet a criterin fhmgeneity. Davis et al. (1) used decisin trees t evaluate and rank strm, snw and weather factrs influencing dry slab avalanches at Alta and Mammth Muntain. That paper details decisin tree methds, variables studied and rankings fthe variables as predictrs f maximum size class, sum favalanche sizes, and whether a given day is an avalanche day. We use their variables in all three parts fthis study. lder and Davis (2000) extended the previus wrk by develping classificatin trees fr avalanche educatin and training at Alta and Mammth Muntain. Thse trees, t large t reprduce, graphically demnstrate lgical and hierarchical relatinships amng the many variables affecting avalanche release. 152

2 - We fcus here n decisin trees fr avalanche frecasting, reprting n three studies at Mammth Muntain. The first tests the tree frm lder and Davis (2000) against avalanche data frm the last three seasns. The secnd develps classificatin trees fr maximum size class and crwn size frm all data cllected frm the winters f 183 thrugh 2001, testing them n data frm winter Finally, hurly data frm a netwrk fsnw and weather sensrs are used t frecast regin-specific avalanche activity arund the muntain. We describe methds and results in each sectin. The cmbined studies address five questins: 1. Beynd educatin, are such mdels useful frecasting tls? Can they alert avalanche wrkers t unusual cnditins that put them at risk? Avalanche Frecasting Table 1. Maximum class size tree tk:curtk:y fr the last 3 seasns at Mammth MunJain. N is the number f avalanche cntrl days; hits are days the tree crrectly frecast the maximum class size... Seasn N Hits i Hits + Overestimates 1/ (5%)' 32 (70%) 2000/0I?? J+ (?!5J45(* ) 2001/ (48%) : 25 (81%) 4 2. Des increasing learning sample size imprve the decisin tree frecasts, particularly fr larger, mre dangerus events? u l!l 3 OJ ::l ;:c 3. Size class is a subjective judgment. Can trees prvide accurate quantitative estimates fphysical features like maximum crwn size?? 0 f\ 4. Can spatially distributed hurly measurements drive regin-specific frecasts fclass and crwn size? 5. Are there fundamental limits t accuracy that can be anticipated fr these and ther statistical mdels? Hw are such mdels best used? Avalanche cntrl day 1/2000 Figure 1. Predicted and actual maximum avalanche size class, 1/2000. (e=predictedsize class; 0= actual size class). 2. A three-year retrspective study We tested the Mammth classificatin tree frm Davis and lder (2000) against all data frm the winters 2000 thrugh That tree, estimating maximum size class fr a given day, was grwn n an eight-year subset fall data acquired between 18 and 18. The learning sample cntained 80 cases, f which 160 were avalanche cntrl days. Mst fthse had a maximum avalanche size fclass 2 r 3, with nly a handful fclass 1 days, 24 class 4 days and seven class 5 days. Mst class 1 days were excluded because fincnsistencies and uncertainty abut recrd eliabi1ity. The largest size class estimated by the tree IS class 4... u l!l OJ ::l 2 )( Q ". :i:? A valanche cntrl day 2000/ The tree ran daily n avalanche cntrl mrnings dunng the winters f2001 and It has classified 132 avaanche cntrl days ver the last three years, as summarized in Table 1 and detailed in Figures 1-3. Figure 2. Predicted and actual maximum avalanche size class, 2000/2001. (e=predicted size class; 0= actual size class). 153

3 Internatinal Snw Science Wrkshp (2002: Pentictn, B.C.) ụ. 3 N Ui '" 2 'M '? - Data errrs, identified graphically, were crrected frm riginal recrds. Unlike Davis et ai., we included all class 1 events after crrecting r excluding bvius mistakes. We did nt shift avalanche data back int strm cycles t accunt fr slides released by cntrl wrk n clear days fllwing strms (abut 30% f avalanche cntrl days wuld qualify). We used their variables plus several thers indicative fclearing after a strm (e.g., mrning precipitatin, mrning clud cver, previus cntrl wrk dne n the upper muntain). This apprach fllwed frm the exigencies fsimple use during daily peratins Avalanche cntrl day 2001/2002 Fig. 3. Predicted and actual maximum avalanche size class, 2001/2002. (e=predicted size class; 0= actual size class). The maximwn class size tree has 65 terminal ndes and a misclassificatin errr rate f0.35. The tree fr maximum crwn size has 76 terminal ndes and a misclassificatin errr rate f0.43. Predicted and actual maximum class and crwn sizes fr winter are summarized in Table 3. We accept verestimates because the trees shqld alert wrkers t maximwn likely ccurrences and the factrs accunting fr them. Class bundaries are fuzzy and verestimating anticipated avalanche size carries n penalty. 3. Class and crwn size trees grwn frm a 1-year learning sample Table 3. Maximum size class and crwn size tree accuraciesfr winter 2001/2002 at Mammth Muntain. N =31. Respnse Hits size class.}}(?r) crwn size. 4 (13%) Hits + Overestimates 21 (68%) 20 (65%) Decisin trees are insensitive t rare ccurrences. These are smetimes islated in small terminal ndes but ften lwnped with ther events ccurring under similar circumstances. The primary benefit anticipated frm a larger learning sample is better classificatin f large avalanches. Mammth Muntain has recrded 742 avalanche cntrl days during mre than 3300 days fperatin since 182/83. We grew trees predicting maximum size class and maximwn crwn size frm 66 cntrl days during the first 1 years fthe data set was withheld as a test year. Table 2 gives the size distributin fr the largest avalanches n each cntrl day in the sample..... u 3 Qj '" 2 'M Avalanche cntrl day, Table 2. Distributin fmaximum avalanche size class fr 66 cntrl days at Mammth Muntain, 182/183 thrugh 2000/2001. Figure 4. Maximum size class during the 2001/2002 seasn, actual andpredicted by the tree grwn n a 1 year learning sample. (e = predicted size class; 0 = actual size class). Size: Occurrences: Nne The maximwn class size tree (Figure 4) crrectly predicts ne fthe tw class 5 avalanches. Hwever, it exhibits smewhat lwer verall accuracy than the tree 154

4 in Sectin 2. Davis et al. excluded mst class 1 tes lanche days while we excluded all nn-cntrl days, v: neither tree did well with class 1 r 0 (i.e., nne) lt year. ach crrectly identified nly 2 ut f8 cases. Further, Table 2 shws ur learning sample t be II balanced with respect t number fcases per wealanche class. The effect f shifting avalanche ntrl days back int the strm cycles remains t be investigated. Avalanche Frecasting thse tw days. rrrs are then apprximately nrmally distributed. Applying the t-test t thse data, the 5% cnfidence interval fr the mean is frm t 0.17m and the null hypthesis that the mean errr equals 0 cannt be rejected _ = 2.6 iii " Nde prbability f 4.6m crwn = ;c L...:::...-.:t Tl 2 31 Avalanche cntrl day, 2001/2002 Figure 5. Maximum crwn size during the seasn, actual C01d as predicted by the decisin tree grwn n a 1 year learning sample. (.=predicted size class; 0= actual size class). Figure 5 shws predicted and actual maximum crwn size fr the 31 cntrl days during 2001/2002. The fit between predicted and actual crwn sizes is much better than suggested by few hits shwn in Table 3. The relatively high misclassificatin rate is als misleading. The tree tracks crwn size well thrughut the seasn, with nly tw exceptins, which we cnsider first. Day 10 is a clear miss, 'with prbabilities f0.05 and 0.24 fr crwns fo.m and 0.6m respectively. Dy 7, als underestimated by the size class tree (FIgure 4), prduced a class 5 avalanche with a 3.7m crwn. Hwever, day 7 falls int ne fthe tw terminal ndes with the largest mean crwn sizes. The tree predicts a 105m crwn that day but estimates a 0.17 prbability fa larger crwn (4.6m, cmparable t the actual ne). Pl Figure 6 shws days 7 and 10 t be anmalies. tt are the errrs between estimated and actual m. axtmum crwn size. The pltted mean (0.06m) and standard deviatins (±0.2m) are fr the set withut 0.00 :s.2s c.g (J :g.0.75 G> N OJ _ Avalanche cntrl day, Figure 6. Crwn size errrs_ Mean (slid line) and standarddeviatin (dtted lines) are shwnfr the set withut days 7 and 10. A strng case exists fr jint use fcrwn size and size class trees. Class 5 avalanches ccurred n tw days. The size class tree predicted the first while the crwn size tree issued a significant prbability fthe secnd. Had bth trees been peratinal in late 2001 there wuld have been warning fclass 5 avalanches n bth days. 4. Reginal trees driven by hurly remte instrument data The trees described s far predict ccurrences ver the entire muntain, but different sets fstarting znes receive different strm lading and respnd differently t that lading. Since fall 1, a netwrk fremte instruments has lgged hurly data frm six sites arund the muntain. Fur sites (tw n the summit ridge, tw at mid-muntain) prvide wind speed and directin, maximum gust, temperature and relative humidity near the majr starting znes. A fifth site measures the same variables at the base fthe muntain while the sixth lgs snw depths, water equivalence, temperature and relative humidity. Lcalleaming samples fr maximum size class and crwn size n summit ridge and mid-muntain avalanche paths were prepared by averaging hurly 155

5 Internatinal Snw Science Wrkshp (2002: Pentictn, B.C.) < degdays< Figure 7. Classificatin frees fr maximum crwn size (tp) andsize class (bttm) fr 37paths ffthe summit ridge fmammth Muntain. Trees are driven by lcal instnlment data and snw plt data nly. Splitting criteria are givenfr all ndes. Lesser values g t the left descendent nde, greater values t the right. Step size is prprtinal t the reductin in nde deviance at each split. Key: snw48 = 48hr snolifall; mdepth = snw depth at master stake; wind24 = average 24hr windspeed; vgi = Vapr GradientIndex (Davis andlder [2000]); ppt72 == 74hr water eqldvalence; accll1nsnw =seasn ttal new snw accumulatin t date; degdays = degree days; ltemp/hitemp = 24hr lw and high temperatures; winddir = wind directin (clckwise frm n011h). 156

6 Avalanche Frecasting ---1ather measurements int 24-, 48- and 72-hur lea. fi we the last three years and cup l' mg t h ese WIt. h th e brs r ndino" precipitation.,. ttals. Till s pr d uce dl ca 1 crresp '" th. th 1. edi tr variables analgus t se m e earmng pr c les fr the trees discussed earlier. sall1p Trees were grwn fr bth maximum class and size fr the summit ridge and lwer muntain crwn h il 1 che paths. Trees fr bt regions were SlID ar' ava an.' d' dh thugh nly the summit set IS Iscusse ere. Figure 7 displays the full trees fr maximum. wn size and size class fr 37 paths n the sumnut e. Mst trikig is their small size cupled wit 1wmisclassl fication errr rates. The crwn and SlZe cass trees have 13 and 17 terminal ndes respectivey, with misclassificatin rates f0.24 and Allwmg as few as tw cases per terminal nde and splits prducing single-case ndes nly.increass the number fcrwn size terminal ndes by SiX and SIze class terminal ndes by ten. The hierarchical splits in Figure 7 illustrate bth predictive and explanatry aspcts fdecisin trees..in bth trees the first split prducmg the greatest reduchn in tree deviance is n 48-hur snwfall. In bth cases the critical value is abut 0.6m fnew snw, with higher values prducing larger avalanches. Fllwing the right main branch fthe crwn size tree, the next imprtant factr is 24-hur average wind speed. Nte that in bth trees all critical values fr 24-hur wind speed are abut 20m/s. That sustained winds ver 1700s shuld prduce n crwn is at first cunterintuitive, but such winds (depending n directin) either strip the muntain r prduce hard slabs resistant t artillery, hand charges and ski cutting. Belw the 17m/s threshld crwns are abut a meter thick, but the larger nes tend t ccur in thinner packs with less than 2.6m fsnw at the master stake. Thinner snw packs are subject t larger temperature gradients, greater depth har develpment, and therefre larger releases. Table 4 gives the distributin fmaximum size classes fr the summit ridge fr the last three years. Nte the relative under-representatin fclass 1 events when cmpared with Table 2. Table 4. Distributin fmaximtm1 avalanche size class fr 37 paths n the summit ridge fmammth Mzmtainfr 104 cntrl days, 1/2000 thrugh G Size: Nne Occurr-;;}cesu ] Scarcity fsmall events is a likely cause fsize class tree simplificatin. Similarly, crwn sizes n the upper muntain are ften much larger than thse n the lwer. The mean maximum crwn size fr paths ff the summit ridge is 2.5 times that fr mid-muntain. Average wind speeds, taken at the tp and at midmuntain, shw similar ratis fabut 2.5 acrss seasns, during strm perids and n cntrl days. T the extent that regins are mre hmgeneus in their lading and avalanche characteristics than the muntain as a whle, tree size shuld shrink and accuracy shuld imprve. 5. Discussin ach split in a decisin tree divides the data with a plane perpendicular t the axis fthe splitting variable. A parallelepiped classifier emerges in which predictr variables in a given sub-vlume fthe data space prduce (ideally) similar respnses. T the extent that small differences in the initial cnditins prduce large differences in the respnse, the apprach will fail. Sensitive dependence n initial cnditins is the classic definitin fchas. Chatic behavir wuld accunt fr the failure fthe cmplete, much larger learning sample t prduce decisin trees fgreater verall accuracy. It wuld als accunt fr bth the large tree size and relatively large errr rates: adjacent and even identical pints in the data space prduce vastly different avalanche utcmes. Fr example, the largest crwn recrded at Mammth Muntain is 6.6m. It ccurred n a day that prduced nly ten ther avalanches, all class 1 and 2, with crwns frm 0.05 O.3m. Inspecting the terminal ndes fthe decisin trees shws this is cmmn. Rsenthal and lder (2002, this vlume) present evidence that slab avalanching is a chatic prcess. This accrds with ther findings fsize distributins cnsistent with self-rganized criticality (Birkeland and Landry, 2002). Ifa majr surce fdecisin tree errr is chatic avalanche behavir, prbability estimates fmultiple avalanche measures may prve mre imprtant than the abslute accuracy fanyne tree. Terminal ndes are ften hetergeneus and their values are simply the means r mdes (depending n tree type) fthe respnses within them. This suggests perturbing initial cnditins t prduce ensemble frecasts fmeasures like crwn size, size class and path length. New prblems will arise such as hw t emply such frecasts and hw t reslve inevitable cnflicts amng them. 157

7 Internatinal Snw Science Wrkshp (2002: Pentictn, B.C.) '--' ' Cnclusins References We return t the five questins psed in the intrductin, drawing n the results fthe three studies. 1. Decisin tree mdels can be useful frecasting tls and are generally cnservative in that they err n the side fverestimating the size favalanches t which wrkers may be expsed. 2. Increasing the size fthe learning sample des nt imprve verall tree accuracy but may imprve estimates fextreme events. The use fmultiple trees estimating different avalanche features like size class and class size imprves estimates f extreme events. 3. Physical features such as maximum crwn size can be well mdeled and frecast with decisin tree methds. 4. Hurly data frm a remte instrument netwrk can drive regin-specific frecasts fclass and crwn size. The trees are small, with lw misclassificatin rates cmpared with thse fr mdels applied t the entire muntain. This is partly because fthe size f the learning sample and partly because fmre hmgeneus reginal cnditins. Birkeland, KW. and C.C. Landry. Pwer-laws and snw avalanches. Gephysical Research Letters 2(11), /2001GLOI4623, p. 4-1, Breiman, L., IH. Friedman, RA. Olshen, and C.I Stne, Classificatin and Regressin Trees, Wadswrth, Belmnt, CA, 184. Davis, R., Klder, D. Hwlett and. Buzaglu, Relating strm and weatherfactrs t dry slab activity at Alta, Utah and Mammth Muntain, Califrnia, using classificatin and regressin trees. Cld Regins Science and Technlgy 30, p. 7, 1. Davis, R. and Klder, A between-strm indicatr f avalanche activity. ISSW 2000 Prceedings, p lder, K and R. Davis, Decisin trees predicting avalanche respnse: Tlsfr training?, ISSW 2000 Prceedings, p Rsenthal, Wand K. lder, vidence fchas in slab avalanching. ISSW 2002 Prceedings (this vlume). 5. Chas may impse fundamental limits n the accuracy that can be anticipated frm any statistical mdel. The mst useful prducts fsuch mdels may be prbability distributins fwrkers being expsed t large avalanches. 5. Acknwledgements We thank the Mammth Muntain Ski Patrl fr weather and avalanche data and fr their cntinuing supprt fthis prject. We als thank the Sierra Nevada Aquatic Research Labratry fthe University fcalifrnia fr their supprt. This wrk was partially funded by U.S. Army Prject 4A762784AT