Why Use Simulation? - Return on Investment
The available findings show that simulators are cost-effective for initial flight and maintenance training in institutions: they train as well as does actual equipment and cost less to procure and use. This finding applies also to computer-based instruction as compared to conventional classroom training. Simulators are a good investment. The cost of their procurement can be amortized in periods of one to four years.
A figure of merit for simulator training effectiveness is Transfer Effectiveness Ratio (TER). It is a ratio of actual equipment training time saved as a function of time spent training on a simulation. Large values of this ratio indicate that simulations train relatively well in comparison to operating actual equipment, and small values indicate that simulations train poorly relative to actual equipment.
TER comparisons for military flight simulators from the literature suggest that the majority (59%) of tasks trained have TERs greater than 0.33. This means that for every three hours spent in the simulator, one hour of actual flight time could be eliminated for 54% of the tasks.
Apache Longbow Force Development Test and Experimentation (FDT&E)
Cost to operate an actual tank is estimated at $75 per mile.
Cost to operate a Tank Driver Trainer simulator is $2.50 per mile.
Also, included was the army saved $2.5M training 2,200 soldiers. That is a savings of $1,136 per soldier which equates to about 15 hours of training per soldier.
In tank gunnery, the introduction of the Conduct of Fire Trainer reduced the annual expenditure of ammunition from 134 to 100 rounds per tank and improved marksmanship. This resulted in an annual cost avoidance of approximately $29M.
The 1990 REFORGER (Return of Forces to Germany) exercise made extensive use of constructive simulation to train leaders at brigade, division, and corps level. Benefits of such training were emphasis on battle planning, staff procedures, and command and control; more efficient use of training time; focus on higher echelons that would otherwise be cost prohibitive; reduced adverse environmental and political impacts. The 1990 exercise saved more than $4M in transportation and cargo handling costs as compared to costs historically [GAO 1991]. In 1992, constructive simulation was used to avoid $34M in costs as compared with the equivalent exercise done without simulation in 1988.
The Army estimates that it substitutes simulation for $68M of flight operations training in the active force and $55M in the Reserves each year; the Navy considers simulation to be effective in initial training in unfamiliar aircraft, as is reflected in the ratio of simulator to actual aircraft training flights (40 to 77) in the fleet replacement training program for F/A-18 aircraft; the Air Force Air Mobility Command plans to replace up to 50 percent of flight training hours with flight simulators and other training devices for training air transport crews.
The operating cost of flight simulators is estimated to be between 5-20% of the cost of aircraft. Many studies have shown that skills learned in flight simulators can be performed successfully in aircraft; the use of simulators for training can reduce flight time. In a more recent study, the median cost ratio of simulators to aircraft was estimated to be 8%.
The Army aviation training program budget is a billion dollar program. To identify the cost effectiveness of flight simulators in relation to flight hours, this study compared the reported actual flight hour costs for the CH-47D, UH-60, and AH-64 in the FY95 budget and the cost to operate the same weapon systems simulator. A further comparison applied the FY94 actual flying hours and simulator utilization hours to the aircraft and simulator operating costs. The chart below indicates the difference in cost per hour for the aircraft and simulator.
Hourly Cost Differential
The simulator cost includes parts and labor provided by the CLS contractor. The aircraft hourly cost includes fuel, other consumables, and repairables. The cost differences between hourly simulation and actual flight cost is substantial.
The following chart identifies the actual FY94 flight hours, the cost per hour, and the total cost for the three weapon systems. The AH-64, with sophisticated weapons systems, is the most expensive program followed by the UH-60 and the CH-47 programs.
FY94 Flying Hour / Program Cost
The simulator hourly cost were applied to the FY94 simulator utilization hours, then compared to the cost of the equivalent flight hours from the chart. The results are not unexpected and indicate the cost savings of simulators. The equivalent hours represent a FHP cost avoidance. The equivalent cost avoidance of 47,840 simulator hours in the AH-64 was $148.0M dollars, for the CH-47 $14.6M, and the UH-60 $44.1M. The cost savings of simulation in a total training program is enormous.
FY94 Simulator Utilization Cost Differential
The impact of the cost savings shown in this analysis reinforces the Armys strategy to pursue advanced technology applications to improve training efficiency, while augmenting the expense to train in the traditional field or flight environment. Army aviation will be fiscally challenged to sustain the current FHP and OPTEMPOs as flying hour costs escalate and training budgets are scrutinized for efficient training methodologies.
In FY90 alone, pilots logged nearly 29,000 hours in the F/A-18 operational flight trainers and weapons tactics trainers. Assuming, on average, that each hour in the simulator equates to each hour in aircraft for training purposes, the value of this training can be stated in terms of a cost avoidance of $43 million.
Relative Cost of Simulated Versus Actual Flight Hour
Army Missile Systems
The Army Missile Command (USAMICOM) Research, Development and Engineering Center (RDEC) uses modeling and simulation extensively in the development of Army Missile Systems [Jolly and Ward 1995]. During the course of numerous simulation projects, the benefits of hardware-in-the-loop (HWIL) simulations have translated into cost savings and avoidance for my weapon system development programs. Examples of cost saving and avoidance exceeding $320M.
Army Missile Systems Cost Effects
Several studies, relating to the use of simulation in lieu of live fire, indicate that performance with simulation is at least equal to live fire training, but that cost is lower. Soldiers with MACS (Multipurpose Arcade Combat Simulator) training expended less rounds during live-fire qualifications and fewer soldiers failed to qualify as compared to those trained usingtraditional methods. Several studies with the Squad Engagement Training System (SETS) have shown positive transfer from SETS to live fire. Training with the Indoor Simulated Marksmanship Trainer (ISMT) has been demonstrated to benefit live-fire performance. The Precision Gunnery Training System (PGTS), an inexpensive trainer for TOW and Dragon missiles, whose rounds are prohibitively expensive ($11,500 and $19,145, respectively, per round), has been demonstrated to be cost-effective, and also permits training that would otherwise cost several hundred million dollars per year if actual missiles were used.
A review of maintenance simulators found that they are as effective for training as actual equipment trainers when measured by student achievement in school. In the majority of cases examined, the cost to develop and fabricate one unit was less than 60% of actual equipment and the cost of fabricating a second unit was less than 20%. Acquisition and use of a maintenance simulator over a 15-year period costs 38% as much as actual equipment. In studies where time to train was reported, simulators took 25-50% less time than actual equipment.
TECOM Virtual Proving Grounds (VPG)
For modeling and simulation to be useful and valid, the applicable tools must be based on real data derived from testing--ground truth [TECOM 1995]. The Armys Test and Evaluation Command (TECOM) supports this concept with an approach known as Simulation and Modeling Anchored by Real Testing (SMART). TECOM is developing the VPG as a means for researchers and developers to assure that their models and simulations are based on real test data.
VPG is a network of models and simulations enabling interactive testing within a synthetic environment. A number of projects undertaken by TECOM use these models and simulations to determine the various effects on systems and to replicate actions without undertaking the time and expense of actual testing.
The chart represents a summary of selected TECOM systems that used VPG in conducting tests and evaluations with cost avoidance as a measure of effectiveness. Actual cost includes investment in simulation when appropriate and available.
TECOM VPG Cost Avoidance
Target Interaction, Lethality, and Vulnerability (TILV)
TILV refers to the science of understanding the mechanisms by which a warhead or other damage mechanism can defeat a target [TILV 1995]. The TILV area addresses the tools, methods, databases, and supporting techniques needed to assess the lethality and vulnerability of all weapon systems, including aspects of design, effectiveness, and survivability. TILV tools provide essential inputs to milestone decisions and the COEA process. TILV analyses play a crucial role throughout the development cycle of all major military systems. The chart depicts the Return on Investment (ROI) of 20 systems. The typical ROI was between $20 and $30 returned for each $ invested.
TILV Return on Investment
Agile Provider (AP) was a Joint exercise sponsored by USACOM replaced by Unified Endeavor (UE) in 1995. AP was a field exercise last held in 1994. Unified Endeavor was supported by a Joint Training Confederation (JTC) of models interacting through the aggregate level simulation protocol (ALSP). The models replace ships at sea and flying hours, and focus on the primary training audience, the JTF commander and staff. Total cost for AP-94 were $40M with $8M in strategic lift costs. UE-95s costs totaled $2.9M with approximately $.5M in strategic lift. Approximately 85% of the UE-95 participants rated their training good and 82% rated it better than a similar field exercise like AP-94. The conclusion was better training at 7.5% of the cost.
USMC Indoor Simulated Marksmanship Trainer (ISMT)
Ability to fire at moving targets, both aircraft and vehicles
Actually see if rounds are having an effect on target
The ISMT reduced the cost of operation requirement for the School of Infantry by more than $16M, without reducing the quality of training.
The following table illustrates the cost difference between using simulation and live-fire.
Cost Comparison of Training with PGTS Versus with the Operational Equipment
TOW gunners trained initially with PGTS-TOW in Southwest Asia during Desert Shield said it was "just like the real thing." In Desert Storm, TOW crews achieved high rate of hits and/or kills per missile fired; one crew destroyed 10 tanks or armored vehicles with 10 missiles. Similar success with PGTS-Dragon has also been evident in the large improvement in target tracking proficiency and first-round hits in live-fire exercises. The cost savings incidental to PGTS are dramatic: one TOW practice round cost $8.1 thousand while a PGTS for TOW costs $30.3 thousand; one Dragon practice round costs $3.8 thousand while a PGTS for Dragon costs $22.9 thousand. To sustain gunnery skills, 24 practice firings are required per month. The cost of practice rounds would be $194.4 thousand for each TOW gunner and $91.2 thousand for each Dragon gunner, compared to $30 thousand and $23 thousand, respectively, for the entire training systems.
Marine Corps Flight Training Programs for Basic Pilot to Become Fully Combat Qualified (1994)
Simulator flight must be performed satisfactorily before follow-on aircraft flights can be scheduled. Because of their realism and their ability to host emergencies too risky to simulate in aircraft, simulators may be used for:
(1) up to 50 percent of minimum flying hour requirements;