Ordnance Component Dynamic Test Requirements: Observations, Challenges, Recommended Investigation John Niehues United Launch Alliance Copyright 2015 United Launch Alliance, LLC. Unpublished Work. All Rights Reserved.
Introduction Ordnance devices provide a unique challenge in regards to environmental testing: the flight units cannot be functioned prior to use in flight Lot Acceptance Testing (LAT) is an example of where Test Like You Fly cannot be fully realized This situation has led to conservative test requirements that have the potential to negatively impact cost, schedule, and risk on a program AIAA-S-113 specifies qualification levels for all recurring LATs (MPE + 6.0 db) This presentation will highlight the significance of ordnance test requirements, observations of the existing state, the resulting quantified reliability, and recommendations for future areas of study June 2015 1
Ordnance Test Requirements Impacts All of Us Ordnance is used for many mission critical functions Separation (Spacecraft, Fairing, Staging, SRM) Pyrovalves Flight termination Ordnance test requirements impact both SC & LV communities Failure to properly function in flight may impact mission success Unnecessary failures increase cost, cause schedule delays, and question the functional reliability No historical evidence could be found by ULA justifying the selection of existing requirements Refining these test requirements will lead to: More perceptive testing Lower testing costs Fewer false positives in testing (avoiding unnecessary failures) Reduced schedule risk June 2015 2
Non-Environmental Screening Methods Ordnance components cannot be functionally tested prior to flight, but all units can be screened using non-destructive tests Visual/Dimensional Inspection Electrical Testing (if applicable) Leak Testing Radiographic Inspection X-Ray N-Ray Many ordnance failure modes are not related to dynamics Additional test amplitude will not drive out non-dynamics related failures Does additional test amplitude really provide more confidence? This may cause more test failures, but if we accept these failures and fly the remaining units as is, no additional benefit is gained June 2015 3 This presentation will focus on environmental testing, but keep in mind it is not the only screening tool we have available to us
Evident Shortcomings AIAA-S-113 specifies that qualification and acceptance be performed with the same margin (6.0 db) This approach is inherently flawed: random failures are expected! Caused by minor build to build variability What acceptance failure rate would we expect to see if we treated avionics boxes the same way? This approach ignores the benefits of the initial successful qualification If we do not trust the initial qualification covers subsequent builds/lots, why call it acceptance and not re-qual? What benefits does qualification buy us then? Component capability If the same test is repeated, the next unit selected may be a slightly lower performer Units sampled for qualification; All pass test June 2015 4 Test Level
Causes of LAT Failures at MPE + 6 db No shift in performance curve (random failure) Units would survive flight and lot is flown as is Minor shift in performance curve Units would survive flight and lot is flown as is Major shift in performance curve Units would fail flight and lot is scrapped Nominal design capability Major shift Minor shift Flight Environment µ Flight MPE Qual (6 db) June 2015 5 Can we design a test that will screen out major shifts in performance without flagging false positives?
Current Avionics/Propulsion Practice Qual Unit All units that pass IAT (capability above MPE) are used for flight Qual Component Capability MPE Failure in IAT demonstrates Flight unit capability below MPE *Qualitative assessment for example purposes June 2015 6 Time / Build
Qual Units Current Ordnance Practice High Performer Tested Low Performer Qual/LAT Component Capability = LAT Failure MPE Many lots that fail LAT are still used for flight Failures concluded to be non-flight like and unrealistic with all units in lot having capability above MPE Appropriate LAT failure *Qualitative assessment for example purposes June 2015 7 Time / Lot
Qual Units Desired Ordnance Future State High Performer Tested Low Performer Qual Component Capability LAT MPE No failure in LAT demonstrates flight unit / low performer capability above MPE Failure in LAT demonstrates flight unit / low performer capability below MPE *Qualitative assessment for example purposes June 2015 8 Time / Lot
Quantified Reliability Using a reliability model based upon classical statistics, we can evaluate the probability of passing a test and subsequently failing in flight Accounts for dynamics testing and dynamics failure modes only Standard Requirement NASA-HDBK-7005 Qual 1 unit @ 3.0 db, IAT flight unit 0.4% MIL-STD-1540 (Propulsion) Qual 1 unit @ 6.0 db, No IAT on flight unit 0.1% Maximum Flight Risk MIL-STD-1540 (Avionics) Qual 1 Unit @ 6.0 db, IAT flight unit 0.08% EWR 127-1 Qual 3 units @ 6.0 db, IAT flight unit 0.01% AIAA-S-113 (Non-EEDS) LAT 9 units @ 6.0 db, no IAT on flight unit 0.003% AIAA-S-113 (EEDS) LAT 30 units @ 6.0 db, no IAT on flight unit 0.001% These numbers do not account for the initial qualification! June 2015 9 The flight risk associated with ordnance test requirements is inconsistent with other industry test requirements
Summary & Recommendation for Future Study Ordnance test requirements are overly conservative This is increasing the cost of the system without improving the overall system reliability Auto manufacturers optimize cost by designing components to all have similar life capabilities Do not design a transmission lasts 500K if the engine only lasts 100K If the weakest component only lasts X miles, why design rest to last longer? We all need to evaluate our own history of ordnance testing When acceptance failures occur, do we scrap the lot or do we disposition the remaining units as fly as it due to excessive test severity? What are the past financial and schedule impacts of these false positives in test? Is a higher amplitude test truly more perceptive? June 2015 10