Project Overview

A heat shield is the unique sub-system of a space capsule that is designed to protect the rest of the space capsule, and its human or scientific payload, from the extreme and uncertain temperature and pressure loads encountered during the capsule’s entry into an atmosphere. For the National Aeronautics and Space Administration (NASA) Orion space capsule (Figure 1), the heat shield is designed to withstand temperatures of thousands of degrees, and pressures several times that of the normal atmosphere, during the entry of the capsule into the Earth’s atmosphere.  Heat shields typically employ a variety of thermal mechanisms (e.g., conduction, convection radiation, ablation and melting) to accomplish this extremely important objective.  The heat shield is usually the only sub-system of the space capsule for which no form of redundancy exists. Hence, even if everything else in the mission goes perfectly, the success of the entire mission ultimately depends entirely upon the success of the heat shield. This piece of equipment is so important, that it is routinely over-designed; in fact, the mission goals and objectives may be limited in order to accommodate this over design.

The NASA Space Transportation System (STS, i.e., “space shuttle”) Columbia (Figure 2) is the most obvious example of a heat shield failure.  The vehicle was completely destroyed (February 1, 2003, Figure 3) during re-entry into Earth’s atmosphere because of unforeseen and undetected damage to the tile-based heat shield. The damage was caused by pieces of insulating foam shedding from the fuel tanks during the ascent phase of the mission from Earth two weeks earlier. The Columbia failure grounded all subsequent STS flights for about two and a half years while an in-depth investigation was conducted. The findings, observations and recommendations of the investigation resulted in significant cultural changes throughout the agency and expense increases and operational changes to provide a variety of heat shield inspections for the remaining 21 STS flights through 2011. 

The overall goals of the current project were to: 1) quantify the reliability of the Orion thermal protection system  (TPS, i.e., “heat shield”) to withstand the a wide variety of possible re-entry conditions and thermal loads, and 2) identify the key sensitivities of the predicted reliability to uncertainties and TPS design variables. Within these goals, one specific objective was to develop an optimal design of experiment (DOE) proposal for a planned arc jet testing campaign. Arc jet testing (Figure 4) is an expensive and limited-availability testing procedure that is used for the high-temperature materials used in space capsule re-entry heat shields. NASA only has two operational facilities for arc jet testing, at the Johnson Space Center (JSC) and the Ames Research Center (ARC) arc jet facilities. The two facilities have capabilities that are somewhat complimentary and consistency between the two facilities may be an issue. Arc jet tests must be well planned in advance so as to acquire as much information as possible with the minimum cost and time invested, and to avoid conflicts for these limited-availability facilities with other NASA, university and international space programs. This effort defined a set of test conditions to be used during the arc jet testing campaign for the TPS Project and Orion Program. 

Problem Definition

Defining the optimal DOE test plan employed a combination of probabilistic technology (PT), a selected DOE approach, Analysis of Variance (ANOVA), classical statistical methods, response surface modeling (RSM) techniques, and subject matter expert judgment. There are several choices within each of the PT, DOE and RSM realms that needed to be evaluated for their suitability to the project. There are also several characteristics of the resulting DOE that must be evaluated such as randomization, statistical replication, blocking and orthogonality.

A combination of effects were analyzed including: the existing data conditions, the originally proposed new test matrix by the Orion Program, relevant flight trajectories, variable and uncertainty bounds, arc jet test reproducibility uncertainties (for recession, rate, and the detailed thermocouple responses over time), the material / environmental / carrier structure design sensitivities, critical time-averaged mission conditions, analysis code RSM modeling uncertainties; and test differences with computations (same conditions). During the development of the optimal DOE proposal, it was discovered that as many as five different thermo-physics regimes may exist within a series of arc jet testing cases (Figures 5, 6 and 7). These additional factors needed to be considered within the development of the test proposal, as well.

PredictionProbe’s Solution

In this example, PT was used to construct probability levels associated with various degrees of uncertainty in the model. A fully-bounded D-Optimal approach to defining the DOE was adopted (so as not to waste data points in flight regimes out of the domain of interest) and various orders of RSM were evaluated, while considering various boundary condition options that are consistent with the chosen RSM polynomial orders. Full randomization was used to establish the test order, facility, and test hardware cut pattern identification parameters. Given the Orion Program priority to acquire new data, only a small number of actual replicate points were defined; it was realized that given the ensemble of data (both existing and new) that would exist, estimates of the data repeatability could be inferred from the use of RSM constructed with various subsets of all the available points.

Results & Conclusions

The proposed DOE-based test matrix is shown in Figure 8 (red circles), along with the previously existing data (black squares) and six more additional new data points proposed later (purple diamonds) using similar techniques to augment the existing data set as the original DOE proposal. The proposed DOE-based test matrix was fully adopted and implemented by the NASA Orion Program during the summer 2011 arc jet testing campaign. The recommended test matrix consisted of 30 test cases with four replicated test conditions.  Each replicated test condition required two samples (i.e., a replicate pair) tested at the same conditions. Two of the four pairs were cross-facility replicates with one sample tested at the JSC facility and the other sample tested at the ARC facility. The remaining two replicate pairs were tested within the same facility, with one replicate pair tested at JSC and one replicate pair at ARC. Results from these tests were also leveraged in separate study related to using vintage Apollo TPS Data to support the Orion TPS reliability assessment (NESC RP-12-00790).

The current proposed DOE-based test matrix: 1) provided coverage of more relevant flight conditions than the original test matrix proposed by the Orion Program, with adequate reproducibility estimates; 2) significantly reduced critical uncertainties; and 3) produced a set of new data that was used to estimate the vehicle TPS reliability and to show that the target reliability requirement is met under most conditions.