Overview of the Technology Readiness Level (TRL) system
Blog
In case you haven’t noticed, designing and building aerospace stuff is complicated. You may have heard your seventh-grade gym teacher say something like, “This is just rope-climbing… it ain’t rocket science!!” Well, Mrs. Pommelhorst, this actually IS rocket science, and as aerospace and space systems became more complex, in the 1970s NASA saw the need for a system for describing, assessing, and segmenting the process of new technological development to determine how ready any critical technology is for meeting a particular mission objective. This system has since been adopted across multiple other industries as well.
It may seem pretty difficult to understand what happens at each stage of the TRL system. Let’s see if we can break it down.
What does TRL mean?
TRL is an acronym that stands for Technology Readiness Level, and is a method developed by NASA for determining the maturity of a technology. In simple terms, the TRL system describes how close a particular technology, item, component, system, or idea is to being fully functional, operational, integrated, and supported.
The (current) 9-step TRL scale helps project stakeholders and collaborators determine how mature, or how ready, a particularly technology is, in order to manage risk, reduce confusion, and, as NASA says, streamline “the evolution of an idea from a thought, perhaps written on a cocktail napkin or the back of an envelope, to the full deployment of a product in the marketplace.”
What were NASA’s original 7 TRL levels?
When the TRL concept was first conceived in 1974 by NASA researcher Stan Sadin, it had seven levels, though they were not formally defined until 1989. Here are the original TRL levels.
- Basic principles observed and reported
- Potential application validated
- Proof-of-concept demonstrated, analytically and/or experimentally
- Component and/or breadboard laboratory validated
- Component and/or breadboard validated in simulation or real space environment
- System adequacy validated in a simulated environment
- System adequacy validated in space
Though the original 7-level TRL scale was useful, it had its shortcomings. Sadin reports some confusion between management and researchers around levels 6 and 7. For this reason, in 1995 NASA refined the TRL scale from 7 steps to 9, and added some clarifying descriptions of each level.
A description of NASA’s 9 Technology Readiness Levels (TRL)
Let’s go through NASA’s brief descriptions of each of the 9 levels, and attempt to provide some additional clarification on what actually happens at each TRL stage (with thanks to the Australian government’s Defence Science and Technology Group).
TRL 1: Basic principles observed and reported
“Transition from scientific research to applied research. Essential characteristics and behaviours of systems and architectures. Descriptive tools are mathematical formulations or algorithms.”
This level is where initial scientific research begins, and principles are qualitatively postulated, observed, and reported. Research studies on basic material properties may be performed. The focus here is on a fundamental understanding of a material or process, or what may be termed “basic research.”
TRL 2: Technology concept and/or application formulated
“Applied research. Theory and scientific principles are focused on specific application area to define the concept. Characteristics of the application are described. Analytical tools are developed for simulation or analysis of the application.”
Once the basic principles are observed in TRL 1, teams can then identify potential practical applications for the technology. These applications are purely speculative at this point, and there may be no proof or detailed analysis to support initial assumptions. Research is still limited to analytic or “paper studies.” Researchers, engineers, or developers may use supporting information including publications or other references that outline the application being considered and that provide analysis to support the initial concept. In short, the step from TRL 1 to TRL 2 transitions ideas from basic to applied research.
TRL 3: Analytical and experimental critical function and/or characteristic proof-of concept
“Proof of concept validation. Active Research and Development (R&D) is initiated with analytical and laboratory studies. Demonstration of technical feasibility using breadboard or brassboard implementations that are exercised with representative data.”
In this phase, analytical and initial laboratory studies are performed to see if the technology is viable, and are designed to physically validate the predictions of separate elements of the proposed technology. The results of laboratory tests may be compared to analytical predictions in order to measure key parameters for critical components. Basically, at TRL 3, experimental work is performed with the intent to verify that the concept works as expected. Components of the technology are validated, but there is no strong attempt at this level to integrate the components into a complete system. Modeling and simulation may be used to support and expand physical experiments.
TRL 4: Component/subsystem validation in laboratory environment
“Standalone prototyping implementation and test. Integration of technology elements. Experiments with full-scale problems or data sets.”
Here’s where individual components or elements are tested for compatibility and viability within a larger system. The basic technological components are integrated and tested to establish whether the pieces will work together. The results of the integrated experiments are analyzed to see how components and experimental test results differ from the predicted system performance goals. TRL steps 4 through 6 represent the bridge from scientific research to engineering, and from development to demonstration. Per the DSTG, “The goal of TRL 4 should be the narrowing of possible options in the complete system.”
TRL 5: System/subsystem/component validation in relevant environment
“Thorough testing of prototyping in representative environment. Basic technology elements integrated with reasonably realistic supporting elements. Prototyping implementations conform to target environment and interfaces.”
At TRL 5, the basic technological components of the technology are integrated so that the system configuration matches the final application in nearly all aspects. This is essentially the beta prototype component phase. The major difference between TRL 4 and TRL 5 is the increase in the alignment of the experimental system and its environment with the actual application. The system tested is almost prototypical. Scientific risk should be retired at the end of TRL 5, and the results presented should be statistically relevant.
TRL 6: System/subsystem model or prototyping demonstration in a relevant end-to-end environment (ground or space)
“Prototyping implementations on full-scale realistic problems. Partially integrated with existing systems. Limited documentation available. Engineering feasibility fully demonstrated in actual system application.”
While TRL 5 was the beta prototype component phase, TRL 6 is the beta prototype system phase. Here, engineering-scale models or prototypes are tested in a relevant environment, and TRL 6 is a major step up in a proposed technology’s demonstrated readiness level. Devices or components may be fabricated on an engineering pilot line. TRL 6 is where engineers begin true development of the proposed technology as an integrated, operational system. The major difference between TRL 5 and TRL 6 is the transition from laboratory scale to engineering scale. Any engineering pilot scale demonstration should be capable of performing all the functions that will be required of a full manufacturing system. The operating environment for testing the system should closely represent the actual operating environment (for space systems, this can be extremely difficult, as you can imagine). New learnings from the engineering pilot line will lead to refinement of the predicted project cost model on this level. The primary goal while at TRL 6 is to reduce engineering risk.
TRL 7: System prototyping demonstration in an operational environment (ground or space)
“System prototyping demonstration in operational environment. System is at or near scale of the operational system, with most functions available for demonstration and test. Well integrated with collateral and ancillary systems. Limited documentation available.”
At TRL 7, the final design is essentially complete, and the actual system prototype is demonstrated in a relevant and realistic environment. The goal here is to eliminate engineering and manufacturing risk, which is a huge milestone and it can be incredibly difficult. When scaling up manufacturing and engineering between TRL 6 and TRL 7, many major issues can surface that need to be addressed.
TRL 8: Actual system completed and “mission qualified” through test and demonstration in an operational environment (ground or space):
“End of system development. Fully integrated with operational hardware and software systems. Most user documentation, training documentation, and maintenance documentation completed. All functionality tested in simulated and operational scenarios. Verification and Validation (V&V) completed.”
In a commercial environment, TRL 8 is the pre-commercial demonstration phase. Here, the technology will be proven to work in its final form and under expected conditions. In nearly all cases, TRL 8 represents the end of true system development. Full-scale volume manufacturing of a commercial end product may occur at this phase. True manufacturing costs will be determined, and any discrepancy compared to predicted models will need to be highlighted, and plans developed to address it. Similarly, if a technology’s performance differs from expected, teams will develop plans to address and remedy any issues.
TRL 9: Actual system “mission proven” through successful mission operations (ground or space)
“Fully integrated with operational hardware/software systems. Actual system has been thoroughly demonstrated and tested in its operational environment. All documentation completed. Successful operational experience. Sustaining engineering support in place.”
At TRL 9, the new technology and related system is proven and ready for full deployment, whether in a space or ground, military, .gov, or commercial setting. The actual system is proven through successful operation in the intended environment. The new technology is in its final form and has been shown to meet expectations under the full range of operating conditions. Support systems are in place. From here on, it’s all gravy.
How does NASA’s TRL system differ from military aerospace or commercial/industrial variations?
Since NASA’s 9-step TRL system is easily applicable to many industrial, commercial, and military project development processes, it has been adopted as-is or modified by R&D, engineering, and product development teams across multiple arenas. For example, the United States Department of Defense (DoD), Department of Energy, and Navy have adapted the TRLs for assessments of technology, with some initial modifications of terms like “flight” or “space.” The US Department of Energy’s version modifies levels 4 to 9. The newly added language specifies the type of prototype, whether it is a component or a system, and whether it is for pre-commercial demonstration.
However, the basic steps and concepts are broadly applicable to any organization developing new technology for commercial sale or military use. It’s not a flawless system, however, as pointed out by authors George Salazar & M. Natalia Russi-Vigoya in their TRL abstract summary. While the adoption of the TRL has many advantages, such as providing a common understanding of the technology status, facilitating communication, enhancing risk management, and improving decision making regarding funding and transition of technology, “Readiness does not necessarily fit with appropriateness or technology maturity. For instance, a mature product may pose a greater or lesser degree of readiness for use in a system context than one of lower maturity. Levels are limited to an operational environment and product-system architecture.”
Regarding the TRL scheme’s lack of representation of the integration readiness of the technology into an operational system, the authors point out that the TRL “declares that the system is ready, but only in terms of hardware and software without integrating the human element. [Researchers] studied seven different organizations (NASA, Raytheon, BP, Bombardier, John Deere, Alstom, and google)[,] examined documentation collected from industry standards and organizational guidelines related to technology development and demonstration [and] found 15 TRL’s challenges that fit into 3 main categories. These included system complexities (e.g., integration and connectivity), planning and review (e.g., backup plans and product road mapping), and validity of assessment (Subjectivity of the assessment and impression scale).”
Dan Newman, a senior technical fellow at Boeing, describes the gulf between proven technological readiness and actual successful implementation as a “Valley of Death,” especially where there is inherent resistance to all-out changes and backwards compatibility is a key factor. “Both old and new systems must be in place and functioning until the entire change is complete,” Newman says. “Often it is necessary that a new component must be compatible with some or all interfacing parts or systems, as well as the existing infrastructure. For example, a new US Navy aircraft must be compatible with the current air-capable ships, and the new ships must be compatible with all the existing aircraft. This ‘backwards compatibility’ requirement limits the freedom to improve.”
Crucially, it’s often not necessarily the technology, but government regulators who are not ready. Newman continues, “Sometimes the technology is nearly there, and the customers are ready, but the regulators are not. Unmanned aircraft are proven extensively in the military, but to date, drone pizza delivery is still limited to New Zealand. Similarly, the advent of urban air taxis is a ways off, according to civil regulators. As the aerospace industry relies so heavily on the support and approval of these necessary partners—both military qualification and civil certification—I offer that we need to develop and establish a Regulatory Readiness Level (RRL) as another risk measure.”
Well, when you throw in the subjective human element and government regulators, nothing ever goes strictly according to plan. But the Technology Readiness Level system can still provide broadly applicable benefits to multiple industries with allowances or adjustments for the human element where appropriate.
–By Jeff Davis, Intergalactic Scribe
Sources:
https://esto.nasa.gov/files/trl_definitions.pdf
https://www.nasa.gov/topics/aeronautics/features/trl_demystified.html
https://www.dst.defence.gov.au/sites/default/files/basic_pages/documents/TRL%20Explanations_1.pdf
https://www.boeing.com/features/innovation-quarterly/may2017/feature-thought-leadership-newman.page