About NTSA Membership Events Publications Contact Us Home
 

Why Use Simulation? - Return on Investment

- back to previous page -

Return-On-Investment - EfficiencyThe available findings show that simulators are cost-effective for initial flight and maintenance training in institutions: they train as well as does actual equipment and cost less to procure and use. This finding applies also to computer-based instruction as compared to conventional classroom training. Simulators are a good investment. The cost of their procurement can be amortized in periods of one to four years.

A figure of merit for simulator training effectiveness is Transfer Effectiveness Ratio (TER). It is a ratio of actual equipment training time saved as a function of time spent training on a simulation. Large values of this ratio indicate that simulations train relatively well in comparison to operating actual equipment, and small values indicate that simulations train poorly relative to actual equipment.

TER comparisons for military flight simulators from the literature suggest that the majority (59%) of tasks trained have TERs greater than 0.33. This means that for every three hours spent in the simulator, one hour of actual flight time could be eliminated for 54% of the tasks.

Apache Longbow Force Development Test and Experimentation (FDT&E)

Resources

Phase I Manned Simulation

Phase II Field Test

Cost (O&M Army)

$.712M

$4.049M

Equipment

1 Simulator

4 AH-64D
2 UH-60
14 M1 Tanks
10 M3 Fighting Vehicles
2 2S6
20 + Air Defense Units
47 + Vehicles

Personnel (Government)

27

663

Mission Turn-Around Time

2 Hours

6 Hours

Data Reduction Time

4 Hours

80 Hours

Number of Trials

32

16

Test Period

4 Weeks

6 Weeks

Safety

No Risk

Moderate Risk

Tank Training

Cost to operate an actual tank is estimated at $75 per mile.

Cost to operate a Tank Driver Trainer simulator is $2.50 per mile.

Also, included was the army saved $2.5M training 2,200 soldiers. That is a savings of $1,136 per soldier which equates to about 15 hours of training per soldier.

In tank gunnery, the introduction of the Conduct of Fire Trainer reduced the annual expenditure of ammunition from 134 to 100 rounds per tank and improved marksmanship. This resulted in an annual cost avoidance of approximately $29M.

Single-Service Training

The 1990 REFORGER (Return of Forces to Germany) exercise made extensive use of constructive simulation to train leaders at brigade, division, and corps level. Benefits of such training were emphasis on battle planning, staff procedures, and command and control; more efficient use of training time; focus on higher echelons that would otherwise be cost prohibitive; reduced adverse environmental and political impacts. The 1990 exercise saved more than $4M in transportation and cargo handling costs as compared to costs historically [GAO 1991]. In 1992, constructive simulation was used to avoid $34M in costs as compared with the equivalent exercise done without simulation in 1988.

Aviation

The Army estimates that it substitutes simulation for $68M of flight operations training in the active force and $55M in the Reserves each year; the Navy considers simulation to be effective in initial training in unfamiliar aircraft, as is reflected in the ratio of simulator to actual aircraft training flights (40 to 77) in the fleet replacement training program for F/A-18 aircraft; the Air Force Air Mobility Command plans to replace up to 50 percent of flight training hours with flight simulators and other training devices for training air transport crews.

The operating cost of flight simulators is estimated to be between 5-20% of the cost of aircraft. Many studies have shown that skills learned in flight simulators can be performed successfully in aircraft; the use of simulators for training can reduce flight time. In a more recent study, the median cost ratio of simulators to aircraft was estimated to be 8%.

The Army aviation training program budget is a billion dollar program. To identify the cost effectiveness of flight simulators in relation to flight hours, this study compared the reported actual flight hour costs for the CH-47D, UH-60, and AH-64 in the FY95 budget and the cost to operate the same weapon systems simulator. A further comparison applied the FY94 actual flying hours and simulator utilization hours to the aircraft and simulator operating costs. The chart below indicates the difference in cost per hour for the aircraft and simulator.

Hourly Cost Differential

Weapon System

SimulatorCost / Hour

AircraftCost / Hour

CH-47

$256

$1,771

UH-60

$59

$1,448

AH-64

$70

$3,101

The simulator cost includes parts and labor provided by the CLS contractor. The aircraft hourly cost includes fuel, other consumables, and repairables. The cost differences between hourly simulation and actual flight cost is substantial.

The following chart identifies the actual FY94 flight hours, the cost per hour, and the total cost for the three weapon systems. The AH-64, with sophisticated weapons systems, is the most expensive program followed by the UH-60 and the CH-47 programs.

FY94 Flying Hour / Program Cost

System

Flying Hours

Cost / Hour

Flight Cost ($M)

AH-64

108,981

$3,101

$337.9

CH-47D

59,792

$1,771

$105.9

UH-60

182,225

$1,448

$263.9

The simulator hourly cost were applied to the FY94 simulator utilization hours, then compared to the cost of the equivalent flight hours from the chart. The results are not unexpected and indicate the cost savings of simulators. The equivalent hours represent a FHP cost avoidance. The equivalent cost avoidance of 47,840 simulator hours in the AH-64 was $148.0M dollars, for the CH-47 $14.6M, and the UH-60 $44.1M. The cost savings of simulation in a total training program is enormous.

FY94 Simulator Utilization Cost Differential

System

Sim Hours

Cost / Hour

Sim Cost ($M)

EquivalentFlight ($M)

AH-64

47,840

$70

$3.3

148.0

CH-47D

9,663

$256

$2.5

14.6

UH-60

31,775

$56

$1.9

44.1

The impact of the cost savings shown in this analysis reinforces the Armys strategy to pursue advanced technology applications to improve training efficiency, while augmenting the expense to train in the traditional field or flight environment. Army aviation will be fiscally challenged to sustain the current FHP and OPTEMPOs as flying hour costs escalate and training budgets are scrutinized for efficient training methodologies.

In FY90 alone, pilots logged nearly 29,000 hours in the F/A-18 operational flight trainers and weapons tactics trainers. Assuming, on average, that each hour in the simulator equates to each hour in aircraft for training purposes, the value of this training can be stated in terms of a cost avoidance of $43 million.

 

Relative Cost of Simulated Versus Actual Flight Hour

Airframe

Cost/Actual Flight Hour

Cost/Simulated Flight Hour

Ratio

F-16

$5000

$500

10/1

FA-18A

$3955

$217

18/1

P-3C

$2903

$119

24/1

S-3A

$4360

$143

30/1

SH-60B

$1724

$118

15/1

CH-47

$3000

$435

7/1

 

 

Average Ratio:

17/1

Army Missile Systems

The Army Missile Command (USAMICOM) Research, Development and Engineering Center (RDEC) uses modeling and simulation extensively in the development of Army Missile Systems [Jolly and Ward 1995]. During the course of numerous simulation projects, the benefits of hardware-in-the-loop (HWIL) simulations have translated into cost savings and avoidance for my weapon system development programs. Examples of cost saving and avoidance exceeding $320M.

Army Missile Systems Cost Effects

Project

Application of HWIL and DIS Simulation

Save / Avoid ($M)

MLRS-TGSM

45% reduction in flight / drop test program

6

FOG-M/NLOS

HWIL simulation identified all hardware and software faults prior to flight tests resulting in reduction in flight test costs

15

LONGBOW

Successful Proof-of-Principle and EMD flight test programs with prevention of at least 2 test failures and reduction of risk in several other cases

6.5

Classified Program

Viability of this development program possible only through HWIL simulation; estimated flight test cost savings

60

HAWK

Flight test cost savings on counter ECM and other system PIPs

80

STINGER

Flight test cost savings for benign, countermeasured, and untestable scenarios

90

ATACMS

Analysis of flight test anomaly possible only with HWIL simulation; rapid identification of anomaly source saved extensive investigation

0.5

JAVELIN

Performance assessment data for milestone 3 decision produced by simulations, avoiding several flight tests

5

Foreign Material Exploitation

ECM hardware / software / techniques evaluation and optimization against foreign threat missiles (Desert Storm payoff in identified saving of at least one aircraft and pilot)

25

FAADS-BSFV (DIS)

Evaluate options using real soldiers, without requiring costly development of prototype systems, and save substantially on field testing

32.1

 

Total Cost Savings or Avoidance

320.1

Small Arms

Several studies, relating to the use of simulation in lieu of live fire, indicate that performance with simulation is at least equal to live fire training, but that cost is lower. Soldiers with MACS (Multipurpose Arcade Combat Simulator) training expended less rounds during live-fire qualifications and fewer soldiers failed to qualify as compared to those trained usingtraditional methods. Several studies with the Squad Engagement Training System (SETS) have shown positive transfer from SETS to live fire. Training with the Indoor Simulated Marksmanship Trainer (ISMT) has been demonstrated to benefit live-fire performance. The Precision Gunnery Training System (PGTS), an inexpensive trainer for TOW and Dragon missiles, whose rounds are prohibitively expensive ($11,500 and $19,145, respectively, per round), has been demonstrated to be cost-effective, and also permits training that would otherwise cost several hundred million dollars per year if actual missiles were used.

Maintenance

A review of maintenance simulators found that they are as effective for training as actual equipment trainers when measured by student achievement in school. In the majority of cases examined, the cost to develop and fabricate one unit was less than 60% of actual equipment and the cost of fabricating a second unit was less than 20%. Acquisition and use of a maintenance simulator over a 15-year period costs 38% as much as actual equipment. In studies where time to train was reported, simulators took 25-50% less time than actual equipment.

TECOM Virtual Proving Grounds (VPG)

For modeling and simulation to be useful and valid, the applicable tools must be based on real data derived from testing--ground truth [TECOM 1995]. The Armys Test and Evaluation Command (TECOM) supports this concept with an approach known as Simulation and Modeling Anchored by Real Testing (SMART). TECOM is developing the VPG as a means for researchers and developers to assure that their models and simulations are based on real test data.

VPG is a network of models and simulations enabling interactive testing within a synthetic environment. A number of projects undertaken by TECOM use these models and simulations to determine the various effects on systems and to replicate actions without undertaking the time and expense of actual testing.

The chart represents a summary of selected TECOM systems that used VPG in conducting tests and evaluations with cost avoidance as a measure of effectiveness. Actual cost includes investment in simulation when appropriate and available.

TECOM VPG Cost Avoidance

Project

Use

Simulation

Actual Cost ($M)

Cost Avoidance ($M)

Firing Impulse Simulator

Recoil loads and ballistic shock effects

Replicate actual firing without the use of ammunition for tanks and howitzers

6.9

23

M830E1 Fuse Testing

Evaluates tank vs. helicopter engagements

Virtual test range simulation using simulated helicopter engagements with actual manned tank

26

1.5

Moving Target Simulator

Immersion of entire weapon system (air or ground) into moving visual target environment

Asses the ability of an M1A2 tank crew to track and simulate firing on images of simulated maneuvering targets

 

1.5per year

Simulation / Test Acceptance Facility (STAF)

Test millimeter wave radar-guided missiles

Hardware-in-the-loop simulator providing test of a fully assembled "live" missile with multiple computer-based test scenarios

 

10.6per year

Aerial Cable Range (ACR)

Test missile tracking of heat sources

Uses a 3-mile long suspended Kevlar cable that serves as path for captive vehicles

.7

13.8

Test Item Simulators (TIS)

Non-radiating simulated digital message traffic to C3 systems

Test of Enhanced Position Location Reporting System (EPLRS)

4.7

2

Trajectory Sense and Destroy Armor Simulation (SADARM)

Model ballistic simulation for the SADARM projectile

Enables downrange auto-trackers to acquire and track incoming projectiles and transition quickly to acquire end-game data

 

12

Physical Simulation of Bridge Crossing

Bridge durability tests

Mix of physical and simulated bridge crossings

.325

.11

Target Interaction, Lethality, and Vulnerability (TILV)

TILV refers to the science of understanding the mechanisms by which a warhead or other damage mechanism can defeat a target [TILV 1995]. The TILV area addresses the tools, methods, databases, and supporting techniques needed to assess the lethality and vulnerability of all weapon systems, including aspects of design, effectiveness, and survivability. TILV tools provide essential inputs to milestone decisions and the COEA process. TILV analyses play a crucial role throughout the development cycle of all major military systems. The chart depicts the Return on Investment (ROI) of 20 systems. The typical ROI was between $20 and $30 returned for each $ invested.

TILV Return on Investment

Program

Tool

Type

Total Invest($M)

Direct Savings($M)

ROI

Program Result

AMRAAM

SHAZAM

End Game

6.5

250

38

Continued

MK Series Bomb Fragment Data

 

Arena Tests

.0825

.9

11

Continued

BLU-109

 

Lethality Testing

.0825

3

36

Continued

Air-to-Missile Analysis

ACEVAL/ AIMVAL

Lethality plus Engagement

20.0

75

4

Continued

Wide Area Anti-Armor Munition

 

Lethality Analysis

.75

30

40

Canceled

Hypervelocity Missile

 

Lethality Analysis

.5

10

25

Canceled

ISAS

 

Lethality Analysis

.75

40

53

Canceled

Kinetic Energy Penetrator (KEP)

 

Lethality Analysis

1.1

50

45

Canceled

JP 233 Runway Attack Munition

 

Lethality and Vulnerability Analysis

1.1

54

49

Canceled

Boosted KineticEnergy Penetrator

 

Runway Vulnerability Models

2.75

130

47

Canceled

JAVELIN ATGM

 

Analytic Simulation

.62

14

23

Accepted

M2 Bradley FVS

 

Engineering Design

.88

30

34

Accepted

Abrams M1A2Vulnerability Test

 

Damage Prediction

1.83

30

16

Less Test & Damage

Block 3 M1A2

 

Design Vulnerability

1.76

100

57

Terminated

Standard MissileSM-2 BLK IIIA

COVART WHDEVAL

Cost Reduction

2.25

47

21

Accepted

PHALANX CIWS

 

Performance Evaluation

8.12

125

15

Continued

PHALANX CIWS Upgrade

 

Product Upgrade

6.63

200

30

Accepted

AIM-7P Sea Sparrow

SCAN

Lethality Analysis, End Game

.7

16

23

Accepted

Phoenix Missile

SCAN

Lethality Analysis, End Game

2.23

70

31

Accepted

ECM vs. AMRAAM

SCAN

Lethality Analysis, End Game

.58

10.5

18

Evaluation Continue

Agile Provider (AP) was a Joint exercise sponsored by USACOM replaced by Unified Endeavor (UE) in 1995. AP was a field exercise last held in 1994. Unified Endeavor was supported by a Joint Training Confederation (JTC) of models interacting through the aggregate level simulation protocol (ALSP). The models replace ships at sea and flying hours, and focus on the primary training audience, the JTF commander and staff. Total cost for AP-94 were $40M with $8M in strategic lift costs. UE-95s costs totaled $2.9M with approximately $.5M in strategic lift. Approximately 85% of the UE-95 participants rated their training good and 82% rated it better than a similar field exercise like AP-94. The conclusion was better training at 7.5% of the cost.

USMC Indoor Simulated Marksmanship Trainer (ISMT)

Ability to fire at moving targets, both aircraft and vehicles

Actually see if rounds are having an effect on target

The ISMT reduced the cost of operation requirement for the School of Infantry by more than $16M, without reducing the quality of training.

The following table illustrates the cost difference between using simulation and live-fire.

Cost Comparison of Training with PGTS Versus with the Operational Equipment

PGTS

 

Operational Equipment

 

TOW Trainer

$30,325

Operational System

$119,242

Dragon Trainer

$22,880

TOW Missile

$ 19,145

Cost of Missiles

$ 0

Dragon Missile

$ 11,500

TOW gunners trained initially with PGTS-TOW in Southwest Asia during Desert Shield said it was "just like the real thing." In Desert Storm, TOW crews achieved high rate of hits and/or kills per missile fired; one crew destroyed 10 tanks or armored vehicles with 10 missiles. Similar success with PGTS-Dragon has also been evident in the large improvement in target tracking proficiency and first-round hits in live-fire exercises. The cost savings incidental to PGTS are dramatic: one TOW practice round cost $8.1 thousand while a PGTS for TOW costs $30.3 thousand; one Dragon practice round costs $3.8 thousand while a PGTS for Dragon costs $22.9 thousand. To sustain gunnery skills, 24 practice firings are required per month. The cost of practice rounds would be $194.4 thousand for each TOW gunner and $91.2 thousand for each Dragon gunner, compared to $30 thousand and $23 thousand, respectively, for the entire training systems.

Marine Corps Flight Training Programs for Basic Pilot to Become Fully Combat Qualified (1994)

Aircraft

Aircraft Hours

Simulator Hours

Sim Hr./AC Hr.

F/A-18

243

95

.39

AV-8

235

140

.60

CH-46

131

69

.58

CH-53E

128

20

.16

Simulator flight must be performed satisfactorily before follow-on aircraft flights can be scheduled. Because of their realism and their ability to host emergencies too risky to simulate in aircraft, simulators may be used for:

(1) up to 50 percent of minimum flying hour requirements;
(2) flights to evaluate operational procedures;
(3) instrument and NATOPS check flights; and
(4) 50 percent of minimum flying requirements for instrument rating.

- back to previous page -

President's Corner
IITSEC
MSPCC
M & S Awards
M & S FAQs
Upcoming Events

April 15-17, 2014
MODSIM World 2014

May 13, 2014
Simulation & Training Community Forum

May 20-22, 2014
ITEC 2014 - ITEC is an NTSA Supported Event

NTSA Spotlight
IITSEC
   

 

2111 Wilson Boulevard, Suite 400 • Arlington, VA 22201
Phone: 703.247.9471 • Fax: 703.243.1659
For more information, e-mail Patrick Rowe
Directions to NTSA