Why Use Simulation? - Return on Investment

- back to previous page -

The available findings show that simulators are cost-effective for initial flight and maintenance training in institutions: they train as well as does actual equipment and cost less to procure and use. This finding applies also to computer-based instruction as compared to conventional classroom training. Simulators are a good investment. The cost of their procurement can be amortized in periods of one to four years.

A figure of merit for simulator training effectiveness is Transfer Effectiveness Ratio (TER). It is a ratio of actual equipment training time saved as a function of time spent training on a simulation. Large values of this ratio indicate that simulations train relatively well in comparison to operating actual equipment, and small values indicate that simulations train poorly relative to actual equipment.

TER comparisons for military flight simulators from the literature suggest that the majority (59%) of tasks trained have TERs greater than 0.33. This means that for every three hours spent in the simulator, one hour of actual flight time could be eliminated for 54% of the tasks.

Apache Longbow Force Development Test and Experimentation (FDT&E)

Resources	Phase I Manned Simulation	Phase II Field Test
Cost (O&M Army)	$.712M	$4.049M
Equipment	1 Simulator	4 AH-64D 2 UH-60 14 M1 Tanks 10 M3 Fighting Vehicles 2 2S6 20 + Air Defense Units 47 + Vehicles
Personnel (Government)	27	663
Mission Turn-Around Time	2 Hours	6 Hours
Data Reduction Time	4 Hours	80 Hours
Number of Trials	32	16
Test Period	4 Weeks	6 Weeks
Safety	No Risk	Moderate Risk

Tank Training

Cost to operate an actual tank is estimated at $75 per mile.

Cost to operate a Tank Driver Trainer simulator is $2.50 per mile.

Also, included was the army saved $2.5M training 2,200 soldiers. That is a savings of $1,136 per soldier which equates to about 15 hours of training per soldier.

In tank gunnery, the introduction of the Conduct of Fire Trainer reduced the annual expenditure of ammunition from 134 to 100 rounds per tank and improved marksmanship. This resulted in an annual cost avoidance of approximately $29M.

Single-Service Training

The 1990 REFORGER (Return of Forces to Germany) exercise made extensive use of constructive simulation to train leaders at brigade, division, and corps level. Benefits of such training were emphasis on battle planning, staff procedures, and command and control; more efficient use of training time; focus on higher echelons that would otherwise be cost prohibitive; reduced adverse environmental and political impacts. The 1990 exercise saved more than $4M in transportation and cargo handling costs as compared to costs historically [GAO 1991]. In 1992, constructive simulation was used to avoid $34M in costs as compared with the equivalent exercise done without simulation in 1988.

Aviation

The Army estimates that it substitutes simulation for $68M of flight operations training in the active force and $55M in the Reserves each year; the Navy considers simulation to be effective in initial training in unfamiliar aircraft, as is reflected in the ratio of simulator to actual aircraft training flights (40 to 77) in the fleet replacement training program for F/A-18 aircraft; the Air Force Air Mobility Command plans to replace up to 50 percent of flight training hours with flight simulators and other training devices for training air transport crews.

The operating cost of flight simulators is estimated to be between 5-20% of the cost of aircraft. Many studies have shown that skills learned in flight simulators can be performed successfully in aircraft; the use of simulators for training can reduce flight time. In a more recent study, the median cost ratio of simulators to aircraft was estimated to be 8%.

The Army aviation training program budget is a billion dollar program. To identify the cost effectiveness of flight simulators in relation to flight hours, this study compared the reported actual flight hour costs for the CH-47D, UH-60, and AH-64 in the FY95 budget and the cost to operate the same weapon systems simulator. A further comparison applied the FY94 actual flying hours and simulator utilization hours to the aircraft and simulator operating costs. The chart below indicates the difference in cost per hour for the aircraft and simulator.

Hourly Cost Differential

Weapon System	SimulatorCost / Hour	AircraftCost / Hour
CH-47	$256	$1,771
UH-60	$59	$1,448
AH-64	$70	$3,101

The simulator cost includes parts and labor provided by the CLS contractor. The aircraft hourly cost includes fuel, other consumables, and repairables. The cost differences between hourly simulation and actual flight cost is substantial.

The following chart identifies the actual FY94 flight hours, the cost per hour, and the total cost for the three weapon systems. The AH-64, with sophisticated weapons systems, is the most expensive program followed by the UH-60 and the CH-47 programs.

FY94 Flying Hour / Program Cost

System	Flying Hours	Cost / Hour	Flight Cost ($M)
AH-64	108,981	$3,101	$337.9
CH-47D	59,792	$1,771	$105.9
UH-60	182,225	$1,448	$263.9

The simulator hourly cost were applied to the FY94 simulator utilization hours, then compared to the cost of the equivalent flight hours from the chart. The results are not unexpected and indicate the cost savings of simulators. The equivalent hours represent a FHP cost avoidance. The equivalent cost avoidance of 47,840 simulator hours in the AH-64 was $148.0M dollars, for the CH-47 $14.6M, and the UH-60 $44.1M. The cost savings of simulation in a total training program is enormous.

FY94 Simulator Utilization Cost Differential

System	Sim Hours	Cost / Hour	Sim Cost ($M)	EquivalentFlight ($M)
AH-64	47,840	$70	$3.3	148.0
CH-47D	9,663	$256	$2.5	14.6
UH-60	31,775	$56	$1.9	44.1

The impact of the cost savings shown in this analysis reinforces the Armys strategy to pursue advanced technology applications to improve training efficiency, while augmenting the expense to train in the traditional field or flight environment. Army aviation will be fiscally challenged to sustain the current FHP and OPTEMPOs as flying hour costs escalate and training budgets are scrutinized for efficient training methodologies.

In FY90 alone, pilots logged nearly 29,000 hours in the F/A-18 operational flight trainers and weapons tactics trainers. Assuming, on average, that each hour in the simulator equates to each hour in aircraft for training purposes, the value of this training can be stated in terms of a cost avoidance of $43 million.

Relative Cost of Simulated Versus Actual Flight Hour

Airframe	Cost/Actual Flight Hour	Cost/Simulated Flight Hour	Ratio
F-16	$5000	$500	10/1
FA-18A	$3955	$217	18/1
P-3C	$2903	$119	24/1
S-3A	$4360	$143	30/1
SH-60B	$1724	$118	15/1
CH-47	$3000	$435	7/1
		Average Ratio:	17/1

Army Missile Systems

The Army Missile Command (USAMICOM) Research, Development and Engineering Center (RDEC) uses modeling and simulation extensively in the development of Army Missile Systems [Jolly and Ward 1995]. During the course of numerous simulation projects, the benefits of hardware-in-the-loop (HWIL) simulations have translated into cost savings and avoidance for my weapon system development programs. Examples of cost saving and avoidance exceeding $320M.

Army Missile Systems Cost Effects

Project	Application of HWIL and DIS Simulation	Save / Avoid ($M)
MLRS-TGSM	45% reduction in flight / drop test program	6
FOG-M/NLOS	HWIL simulation identified all hardware and software faults prior to flight tests resulting in reduction in flight test costs	15
LONGBOW	Successful Proof-of-Principle and EMD flight test programs with prevention of at least 2 test failures and reduction of risk in several other cases	6.5
Classified Program	Viability of this development program possible only through HWIL simulation; estimated flight test cost savings	60
HAWK	Flight test cost savings on counter ECM and other system PIPs	80
STINGER	Flight test cost savings for benign, countermeasured, and untestable scenarios	90
ATACMS	Analysis of flight test anomaly possible only with HWIL simulation; rapid identification of anomaly source saved extensive investigation	0.5
JAVELIN	Performance assessment data for milestone 3 decision produced by simulations, avoiding several flight tests	5
Foreign Material Exploitation	ECM hardware / software / techniques evaluation and optimization against foreign threat missiles (Desert Storm payoff in identified saving of at least one aircraft and pilot)	25
FAADS-BSFV (DIS)	Evaluate options using real soldiers, without requiring costly development of prototype systems, and save substantially on field testing	32.1
	Total Cost Savings or Avoidance	320.1

Small Arms

Several studies, relating to the use of simulation in lieu of live fire, indicate that performance with simulation is at least equal to live fire training, but that cost is lower. Soldiers with MACS (Multipurpose Arcade Combat Simulator) training expended less rounds during live-fire qualifications and fewer soldiers failed to qualify as compared to those trained usingtraditional methods. Several studies with the Squad Engagement Training System (SETS) have shown positive transfer from SETS to live fire. Training with the Indoor Simulated Marksmanship Trainer (ISMT) has been demonstrated to benefit live-fire performance. The Precision Gunnery Training System (PGTS), an inexpensive trainer for TOW and Dragon missiles, whose rounds are prohibitively expensive ($11,500 and $19,145, respectively, per round), has been demonstrated to be cost-effective, and also permits training that would otherwise cost several hundred million dollars per year if actual missiles were used.

Maintenance

A review of maintenance simulators found that they are as effective for training as actual equipment trainers when measured by student achievement in school. In the majority of cases examined, the cost to develop and fabricate one unit was less than 60% of actual equipment and the cost of fabricating a second unit was less than 20%. Acquisition and use of a maintenance simulator over a 15-year period costs 38% as much as actual equipment. In studies where time to train was reported, simulators took 25-50% less time than actual equipment.

TECOM Virtual Proving Grounds (VPG)

For modeling and simulation to be useful and valid, the applicable tools must be based on real data derived from testing--ground truth [TECOM 1995]. The Armys Test and Evaluation Command (TECOM) supports this concept with an approach known as Simulation and Modeling Anchored by Real Testing (SMART). TECOM is developing the VPG as a means for researchers and developers to assure that their models and simulations are based on real test data.

VPG is a network of models and simulations enabling interactive testing within a synthetic environment. A number of projects undertaken by TECOM use these models and simulations to determine the various effects on systems and to replicate actions without undertaking the time and expense of actual testing.

The chart represents a summary of selected TECOM systems that used VPG in conducting tests and evaluations with cost avoidance as a measure of effectiveness. Actual cost includes investment in simulation when appropriate and available.

TECOM VPG Cost Avoidance

Project	Use	Simulation	Actual Cost ($M)	Cost Avoidance ($M)
Firing Impulse Simulator	Recoil loads and ballistic shock effects	Replicate actual firing without the use of ammunition for tanks and howitzers	6.9	23
M830E1 Fuse Testing	Evaluates tank vs. helicopter engagements	Virtual test range simulation using simulated helicopter engagements with actual manned tank	26	1.5
Moving Target Simulator	Immersion of entire weapon system (air or ground) into moving visual target environment	Asses the ability of an M1A2 tank crew to track and simulate firing on images of simulated maneuvering targets		1.5per year
Simulation / Test Acceptance Facility (STAF)	Test millimeter wave radar-guided missiles	Hardware-in-the-loop simulator providing test of a fully assembled "live" missile with multiple computer-based test scenarios		10.6per year
Aerial Cable Range (ACR)	Test missile tracking of heat sources	Uses a 3-mile long suspended Kevlar cable that serves as path for captive vehicles	.7	13.8
Test Item Simulators (TIS)	Non-radiating simulated digital message traffic to C3 systems	Test of Enhanced Position Location Reporting System (EPLRS)	4.7	2
Trajectory Sense and Destroy Armor Simulation (SADARM)	Model ballistic simulation for the SADARM projectile	Enables downrange auto-trackers to acquire and track incoming projectiles and transition quickly to acquire end-game data		12
Physical Simulation of Bridge Crossing	Bridge durability tests	Mix of physical and simulated bridge crossings	.325	.11

Target Interaction, Lethality, and Vulnerability (TILV)

TILV refers to the science of understanding the mechanisms by which a warhead or other damage mechanism can defeat a target [TILV 1995]. The TILV area addresses the tools, methods, databases, and supporting techniques needed to assess the lethality and vulnerability of all weapon systems, including aspects of design, effectiveness, and survivability. TILV tools provide essential inputs to milestone decisions and the COEA process. TILV analyses play a crucial role throughout the development cycle of all major military systems. The chart depicts the Return on Investment (ROI) of 20 systems. The typical ROI was between $20 and $30 returned for each $ invested.

TILV Return on Investment

Program	Tool	Type	Total Invest($M)	Direct Savings($M)	ROI	Program Result
AMRAAM	SHAZAM	End Game	6.5	250	38	Continued
MK Series Bomb Fragment Data		Arena Tests	.0825	.9	11	Continued
BLU-109		Lethality Testing	.0825	3	36	Continued
Air-to-Missile Analysis	ACEVAL/ AIMVAL	Lethality plus Engagement	20.0	75	4	Continued
Wide Area Anti-Armor Munition		Lethality Analysis	.75	30	40	Canceled
Hypervelocity Missile		Lethality Analysis	.5	10	25	Canceled
ISAS		Lethality Analysis	.75	40	53	Canceled
Kinetic Energy Penetrator (KEP)		Lethality Analysis	1.1	50	45	Canceled
JP 233 Runway Attack Munition		Lethality and Vulnerability Analysis	1.1	54	49	Canceled
Boosted KineticEnergy Penetrator		Runway Vulnerability Models	2.75	130	47	Canceled
JAVELIN ATGM		Analytic Simulation	.62	14	23	Accepted
M2 Bradley FVS		Engineering Design	.88	30	34	Accepted
Abrams M1A2Vulnerability Test		Damage Prediction	1.83	30	16	Less Test & Damage
Block 3 M1A2		Design Vulnerability	1.76	100	57	Terminated
Standard MissileSM-2 BLK IIIA	COVART WHDEVAL	Cost Reduction	2.25	47	21	Accepted
PHALANX CIWS		Performance Evaluation	8.12	125	15	Continued
PHALANX CIWS Upgrade		Product Upgrade	6.63	200	30	Accepted
AIM-7P Sea Sparrow	SCAN	Lethality Analysis, End Game	.7	16	23	Accepted
Phoenix Missile	SCAN	Lethality Analysis, End Game	2.23	70	31	Accepted
ECM vs. AMRAAM	SCAN	Lethality Analysis, End Game	.58	10.5	18	Evaluation Continue

Agile Provider (AP) was a Joint exercise sponsored by USACOM replaced by Unified Endeavor (UE) in 1995. AP was a field exercise last held in 1994. Unified Endeavor was supported by a Joint Training Confederation (JTC) of models interacting through the aggregate level simulation protocol (ALSP). The models replace ships at sea and flying hours, and focus on the primary training audience, the JTF commander and staff. Total cost for AP-94 were $40M with $8M in strategic lift costs. UE-95s costs totaled $2.9M with approximately $.5M in strategic lift. Approximately 85% of the UE-95 participants rated their training good and 82% rated it better than a similar field exercise like AP-94. The conclusion was better training at 7.5% of the cost.

USMC Indoor Simulated Marksmanship Trainer (ISMT)

Ability to fire at moving targets, both aircraft and vehicles

Actually see if rounds are having an effect on target

The ISMT reduced the cost of operation requirement for the School of Infantry by more than $16M, without reducing the quality of training.

The following table illustrates the cost difference between using simulation and live-fire.

Cost Comparison of Training with PGTS Versus with the Operational Equipment

PGTS		Operational Equipment
TOW Trainer	$30,325	Operational System	$119,242
Dragon Trainer	$22,880	TOW Missile	$ 19,145
Cost of Missiles	$ 0	Dragon Missile	$ 11,500

TOW gunners trained initially with PGTS-TOW in Southwest Asia during Desert Shield said it was "just like the real thing." In Desert Storm, TOW crews achieved high rate of hits and/or kills per missile fired; one crew destroyed 10 tanks or armored vehicles with 10 missiles. Similar success with PGTS-Dragon has also been evident in the large improvement in target tracking proficiency and first-round hits in live-fire exercises. The cost savings incidental to PGTS are dramatic: one TOW practice round cost $8.1 thousand while a PGTS for TOW costs $30.3 thousand; one Dragon practice round costs $3.8 thousand while a PGTS for Dragon costs $22.9 thousand. To sustain gunnery skills, 24 practice firings are required per month. The cost of practice rounds would be $194.4 thousand for each TOW gunner and $91.2 thousand for each Dragon gunner, compared to $30 thousand and $23 thousand, respectively, for the entire training systems.

Marine Corps Flight Training Programs for Basic Pilot to Become Fully Combat Qualified (1994)

Aircraft	Aircraft Hours	Simulator Hours	Sim Hr./AC Hr.
F/A-18	243	95	.39
AV-8	235	140	.60
CH-46	131	69	.58
CH-53E	128	20	.16

Simulator flight must be performed satisfactorily before follow-on aircraft flights can be scheduled. Because of their realism and their ability to host emergencies too risky to simulate in aircraft, simulators may be used for:

(1) up to 50 percent of minimum flying hour requirements;
(2) flights to evaluate operational procedures;
(3) instrument and NATOPS check flights; and
(4) 50 percent of minimum flying requirements for instrument rating.

- back to previous page -

2111 Wilson Boulevard, Suite 400 • Arlington, VA 22201
Phone: 703.247.9471 • Fax: 703.243.1659
For more information, e-mail Patrick Rowe
Directions to NTSA