Why Use Simulation? - Return on Investment

- back to previous page -

Multi-Service and Joint Training

The Defense Science Board concluded that computer-based, simulated scenarios offer the only practical and affordable means to improve the training of Service operational commanders, their staffs, and the commanders and staffs who report to them. Battle simulation offers the only opportunity to practice the use of certain weapon systems, sensors, tactics, and techniques against a skilled adversary [Defense Science Board 1988].

Apache Longbow

Phase I of Force Development Test and Experimentation (FDT&E) was based on manned simulation. Phase II employed approximately the same test scenario and activities but used live equipment. Twice as many trials were conducted in Phase I than in Phase II, at less cost, with fewer personnel, in less time. FDT&E allows the helicopter crews to train on the new equipment without the risk associated with flying real equipment. It allows development and practice of new tactics, techniques, and procedures. Those responsible for developing scenarios for Initial Operational Test and Evaluation (IOT&E) have the opportunity to structure the very expensive operational test to gain the most critical information.

Simulator Effectiveness

The operating cost of flight simulators is estimated to be between 5-20% of the cost of aircraft. Many studies have shown that skills learned in flight simulators can be performed successfully in aircraft; the use of simulators for training can reduce flight time. In a more recent study, the median cost ratio of simulators to aircraft was estimated to be 8%.

The available findings show that simulators are cost-effective for initial flight and maintenance training in institutions: they train as well as does actual equipment and cost less to procure and use. This finding applies also to computer-based instruction as compared to conventional classroom training. Simulators are a good investment. The cost of their procurement can be amortized in periods of one to four years.

Examples of Successful Training with Simulation

Landing on the Moon	Actual Training not possible on Earth
Top Gun Fighter Weapon School	Improved Navy exchange ratio in air combat over North Vietnam from 2.4 to 12.5
Commercial Airlines	FAA: Simulator training alone qualifies a pilot to fly a new airplane for the first time on a revenue flight
Nuclear power plants	No significant accidents (after Three Mile Island) in military and civilian operations
Canadian Armor Trophy	Extensive training with SIMNET and UCOFT helped Army win CAT for the first time in 1987
73 Easting	Participants report that sucess in Gulf War battle was based on tactical experience gained in SIMNET, National Training Center and Grafenwoehr

Studies indicate the relative cost of military flight simulators is 10% of actual equipment wben both actual equipment and simulators are already in the inventory. If the simulators must be both procured and maintained the figure is 33%.

[Simpson et al. 1995] drew these general conclusions about the effectiveness of individual skills training simulators:

..in aggregate, simulators provide significant beneficial transfer from simulator to aircraft at a median operating cost of about one-tenth of an aircraft..Because of their scope, the body of studies probably provides the strongest case for the value of any type of simulation.

USMC Indoor Simulated Marksmanship Trainer (ISMT)

• Ability to fire at moving targets, both aircraft and vehicles
• Actually see if rounds are having an effect on target

The ISMT reduced the cost of operation requirement for the School of Infantry by more than $16M, without reducing the quality of training.

Modeling and Training Simulation at DoD Labs

• Leg Clearance Ejection Simulations for JPATS.

The simulations quantified the possible foot and / or shin strikes with the instrument panel. The JPATS SPO was able to take this information to their contractors and improve the cockpit design. The simulators identified any problems before designs were complete. Avoiding costly design changes and improving new member accommodation.

• "BURNSIM" Model

The Lab BURNSIM Model has eliminated the need to use animals in tests of burn hazard associated with aircraft and life support systems. Thus far, it has been applied to problems as diverse as effectiveness of pilots' flight suits, hazards from freon replacements and burn potential of aerothermal heating at very high speed escape.

Live-Fire Versus Simulation: A Review of the Literature (May 1994)

Many of the empirical studies have demonstrated that performance using simulation is "at least equal to" using live-fire. If the resulting performance using simulation is "at least equal to" performance using live-fire, then cost becomes the major driver in the selection of a training methodology. A significant difference in performance is not needed to demonstrate the value of simulation.

The following table illustrates the differences in training capabilities between simulation and live-fire training.

Comparison of the Training Capabilities: PGTS Versus Live-Fire Training

PGTS	Live-Fire Training
Target Identification	Target engagement only (and only when missiles are available)
Target Tracking	Feedback limited to HIT or MISS
Target Engagement	No replay capabilities
Missile Flight Control	Limited to available targets and terrains
Visual Replay of Mission	Performance measures: HIT or MISS
Recording/Playback of missile flight and target movement
Unlimited choices of terrains
Unlimited choices of targets, including helicopters; static or moving
Target size can be changed
Obstruction may be varied from 0 to 4 seconds
Performance measures: HIT, HIT-KILL, MISS, missile tracking score, pull down force and target selection
Visual feedback to the student

The true test of the training effectiveness of the PGTS-TOW occurred during the Persian Gulf War. The Naval Air Warfare Center Training Systems Division took a number of PGTS Field Tactical Trainers for the TOW to Saudi Arabia and conducted intensive training in close proximity to hostile forces. The Marine Corps TOW anti-armor gunners described the training as being "just like the real thing." Marines experiencing their first combat indicated that they felt extremely confident to function in the situation. this was attributed, in a large part, to the confidence they had gained by repeated practice with the PGTS TOW trainers shortly before the ground war started. One USMC TOW crew destroyed ten tanks/armored vehicles out of ten missiles fired. Overall, the percentage of first hits by personnel trained on the PGTS exceeded those not trained on the PGTS.

Using Aircraft Simulators to Train Fleet Aviators (May 1995)

Training in a simulator has advantages [over training in the actual aircraft], such as more cockpit time for the money, greater flexibility in designing the scenario, and more effective interaction with the instructor. More specifically, simulation provides the following:

• The simulator is not constrained by the logistics and cost of scheduling opposition aircraft, range time, warning areas, etc.
• A simulator scenario is not constrained by safety, environmental, diplomatic, security, and other real-world peacetime limitations.
• A simulator scenario does not have to contend with unrealistic physical requirements. For example, when simulated aircraft are killed, they can be immediately removed from the scenario.
• The design of a simulator scenario can be more flexible, which means it can more easily be tailored to situations that are particularly important or to situations that the student is having difficulty with.
• The simulator can provide the student with more trials in a given block of time by eliminating tasks that are not central to the training objective (e.g., launching, refueling, repositioning).
• Simulator scenarios are reproducible, so they can be used to teach lessons that require repetition. And ... repetition is a highly effective training technique.
• The instructor's critique of the student's actions can be more precise because the instructor has an accurate reconstruction of what happened.
• The instructor can provide the student with cause-and-effect feedback almost immediately, when it is most effective.

Shipboard Firefighting In Immersive Virtual Environments (VE) October 1995

http://www.ait.nrl.navy.mil/ DamageControl/VR.html

Results show that there was a measurable improvement in the performance of firefighters that used VE training over firefighters without VE training in both phases of the tests. In the Phase 1 (navigation) test, the VE Training group was an average of 30 seconds faster over a two minute run. In addition, all of the Traditional Training group made at least one wrong turn, while only one VE Training group member made any wrong turns. In Phase 2 (the firefighting test), the majority of the participants in the Traditional training group made wrong turns but no one in the VE Training group did.

Timing data was recorded in Phase 2 but individual differences in team leadership style and differences in the actions of the firefighting team made timing measurements inconclusive.

In addition to the quantifiable results obtained during the tests, participants expressed their increased confidence in performing their tasks because of the familiarization of the spaces and situation awareness that they received through VE. Most members of the VE Training group used VE to actively investigate the scene and plan their strategies, so that during the fire they were able to concentrate on fighting the fire instead of finding their way through unfamiliar spaces.

On-line Report

A copy of the report titled "Virtual Environment Firefighting / Ship Familiarization Feasibility Tests Aboard the Ex-USS Shadwell" is available on-line at http://www.ait.nrl.navy.mil/DamageControl/VETest.html.The on-line report is a reprint of: D. L. Tate, L. Sibert, F. W. Williams, T. King, and D. H. Hewitt, "Virtual Environment Firefighting / Ship Familiarization Feasibility Tests Aboard the Ex-USS Shadwell", NRL Ltr Rpt 6180/0672A.1, Oct. 17, 1995

Fleet Supports Virtual Environment Training for Firefighting

At the Damage Control/Firefighting Working Group Safety and Survivability Conference on 7-9 Nov 1995, NRL submitted an action item for Virtual Environment (VE) Training Technology to the Training Sub-group. Based on the successful results of the VE Feasibility Tests conducted on the ex-USS Shadwell, the recommendation was to support the development of techniques for enhancing VE as a training system.

Motorola University Virtual Trainer

http://www.genreality.com/app.html

Motorola University has demonstrated that using virtual reality to simulate training on a robotic manufacturing line has the potential to save tens of thousands of dollars over traditional hands-on experience in an instructor-led environment.

Using a standard 66MHz Pentium computer, VR software from Superscape and a CyberEye head-mounted display from General Reality Company, Adams Consulting set out to model a virtual duplicate of Motorola's Pager Robotic Assembly Line in Boynton Beach, Florida.

Three training groups of seven students each were trained to perform various tasks. Each group was taught by the same Master Instructor. Everyone had the same classroom instruction. The two VR groups each had 20 minutes of navigation instruction and familiarization, so they could move easily in the virtual lab.

The control group spent one hour in the real lab. With the help of job aid checklists, they worked through the procedures on the real equipment. The instructor was nearby to answer questions. The Desktop VR group used the virtual lab, which they saw on monitors, and used a 2D mouse for navigation. The HMD VR group used the same virtual lab, but saw it in their HMD's. They also navigated using 2D mice. The VR groups also had one hour for learning in the virtual lab, with an instructor and checklists. Following the familiarization time, each person individually went into the real lab, and without the checklists performed the procedures under the eye of the instructor. Each student was graded on the number of errors and missed steps.

Results of the testing showed that the VR Groups spent more time on any given training task. They were also more focused, using as a measure the nearly complete lack of conversation among the VR Groups' members. There were three perfect scores: two in the HMD VR Group, one in the Desktop VR Group, and none in the class that was trained using the actual manufacturing line itself. The HMD VR Group had essentially no errors, and scored in the 80th percentile of all students. The total number of errors possible is in the hundreds.

Integrated Damage Control Training Technology

http://www.tekamah.com/public/idctt/

The need to train the Damage Control Assistant (DCA) under realistic, stressful circumstances has long been recognized by the Navy. However, the cost, time and resource availability associated with Damage Control (DC) exercises have posed limitations to this much needed training. With the belief that DC training could be implemented more cost-effectively, a multi-disciplined group consisting of education professionals, experienced multimedia developers, subject matter experts, and other technical personnel convened to address this issue. Their collective work resulted in the Integrated Damage Control Training Technology (IDCTT), a prototype which utilized multimedia technology to train prospective DCAs through an interactive battle damage scenario. This prototype provided trainees a mechanism to test their knowledge and ability in situations made stressful through interactive video technology without the associated cost of bringing an entire ship to general quarters.

In November of 1993, the IDCTT initial R&D prototype was demonstrated at the CNO Firefighting Damage Control Conference in Norfolk, Va. It was so well received that in early 1994 it was placed in operation at both the SWOS DCA School Newport, RI and the Afloat Training Group Pacific (ATGPAC) San Diego, CA. Subsequent field testing at these training commands received great acceptance. Training content and effectiveness were evaluated, and IDCTT's educational value was documented both by instructor feedback and through a Navy Post-Graduate School (NPGS) Masters Thesis. The Masters Thesis from NPGS addressed the educational value and compared IDCTT with traditional training methods. Feedback from both students and instructors at the installed sites showed and enthusiasm for the concept of the trainer; however, IDCTT was developed as a prototype and was limited in its capabilities. A full-lifecycle, multimedia trainer system was therefore recommended to compensate for the shortcomings in the IDCTT prototype.

In response to this feedback, NAVSEA is enhancing the IDCTT trainer to extend its capabilities. Building on the technology developed under IDCTT, the new trainer will provide standardized and realistic training to naval personnel acting in the roles of both DCA and Repair Party Leader (RPL). The current DCA scenario will be ported to a modern, scalable architecture, and two RPL scenarios will be provided.

Although initially targeted at shore-based training sites, this trainer has potential as an embedded training module in an operational shipboard system, much like the Damage Control System (DCS). The RPL shall be packaged as a COTS product designed to run on any MPC II compliant system running a 32 bit operating system , allowing dual use of hardware in stand-alone mode, but shall also include a network interface to the DCS system to integrate with the operational system as embedded trainer.

Red Flag Remote Briefing Tool

http://www.afit.af.mil/ENGgraphics/veprojects/ve.html

The Red Flag Remote Debriefing Tool (RDT) research provides a capability to monitor and review Red Flag missions at remote locations. This is the first ARPA project to successfully integrate real-world and virtual environment actors into one large-scale distributed virtual environment. Before RDT, a Red Flag mission could only be monitored and critiqued on-site. RDT strips out aircraft position, motion, telemetry, and weapon control status data from the Red Flag Monitoring and Debriefing System in real-time and uses the DIS protocol to send this information to any location with a DSI connection.

This information could also be sent over any other network as long as the DIS protocols are used. The receiving site can present the transmitted data as either a 2D or 3D view of the range. RDT provides much of the functionality of the debriefing system located at Nellis, but on a 21" monitor. As a result, the system is inexpensive and portable.

Virtual Emergency Room

http:// www.afit.af.mil/ENGgraphics/veprojects/ve.html

We are developing a system usable by Emergency Department (ED) personnel for the diagnosis and treatment of patients who have trauma, myocardial infarction, aneurysms, tension pneumothorax, poisonings, acute hemorrhage, or other time-critical medical emergencies. This system may serve to reduce hospital costs and the length of stay for trauma patients admitted through the ED. In addition, the system is suitable for use in mobile military field hospitals.

The technological opportunity now exists to perform the research required to place patient diagnostic and radiological information at the physician's fingertips within an Emergency Department setting using virtual environment technology. The vehicle for this step is the recent advances made within virtual environment technologies. Virtual environment technology can be defined as the human experience of perceiving and interacting through displays, sensors, and effectors with a virtual environment and its contents as if it were real. To implement a virtual environment requires the use of several different technologies. Users of the environment must be given visual and audio cues that are sufficiently accurate to entice the user to suspend disbelief in the virtual environment presentation. In addition, sensors to determine the users position and orientation and a mapping for them from the real to virtual world are needed. Finally, devices that allow the user to control appropriate portions of the environment and the display of the environment are needed. To meet these requirements researchers have investigated rendering techniques and requirements, image display devices,input devices, output devices, sensors, environment descriptions, user interface paradigms, and the virtual environments that could be built with these advances in equipment.

The principal objective of our project is to develop a state-of-the-art virtual reality environment for use within level I and II Emergency Rooms. The facility will allow Emergency Department doctors and other emergency department personnel to access patient textual data, review radiological medical imaging records, examine current radiological medical imaging results, monitor patient vital signs, and monitor output at the patient's side. ED personnel can force important data to the doctor's HMD and vice-versa. The medical imaging data is moved into the ED from the Radiology Department using picture archiving and communication systems (PACS) and is segmented according to the doctor's commands and registered against other modalities and/or previous medical imaging data for presentation. The system is now under construction.

The P-3C SASP Acoustic Trainer

Intermetrics has developed a number of ground-based trainers as well. The P-3C SASP Acoustic Trainer, for example, provides real-time simulation of target acoustic signals and aircraft navigation data. Employing extensive databases on targets, ocean environments, and sonobuoy patters, this system enables fleet personnel to create multiple scenarios during training sessions.

In the area of laboratory simulation, Intermetrics developed the External Interface Simulator which simulates inputs from the Naval Tactical Data System and the Link-11 net so that CV-ASWM analytical functions can be tested. We provided a full gamut of engineering services for this effort including design, code, test, debug, and integration of all computer program modules.

Intermetrics has also developed and integrated several ASW support laboratories. One such facility, the LAMPS Avionics Integration Laboratory (AIL), helps resolve problems within the LAMPS operational system and supports the implementation of new functions. Another lab, the ASW Engineering Laboratory will be a platform independent facility used to assess available and advancing technologies. As a generic evaluation lab, it is expected to increase the overall effectiveness of fleet operations.

For more information, please contact Bruce Waldron at (215) 674-2913 or waldron@warm.inmet.com

Manufacturing Case Studies Caterpillar VR

http://nemo.ncsl.nist.gov/~sressler/projects/mfg/mfg_cs_cat.html

Researchers at Caterpillar Inc. have used VR to improve the design process for heavy equipment. Dave Stevenson and John Bettner engineers with Caterpillar in collaboration with the staff of NCSA (National Center for Supercomputing Applications) have put together a system which allows them to quickly prototype wheel loader and backhoe loader designs.

In particular the team is able to perform visibility assessment of the new design. Engineers put on a helmet mounted display and have a full 360 degrees of vision to see how the environment looks and to evaluate obstructions. A Silicon Graphics is used to generate the real time graphics display and to simulate the operation of the equipment. The engineers can "operate" the equipment and evaluate visual obstructions in a natural manner without having to build a physical prototype.

This image from the Virtual Backhoe project illustrates an quot;operator" driving the virtual equipment at the NCSA VR lab. Select it to view a short MPEG movie of the facility in action.

The Caterpillar team was awarded the 1993 NCSA Industrial Challenge Award for VR Use. In the press release announcing the award:

Simulation-Based Design (SBD)

http://dmsttiac.sc.ist.ucf.edu/services/ir/benefits/

We have had a major impact on ship construction and we have only scratched the surface with a Phase I feasibility demo for my program Simulation-Based Design (SBD). General Dynamics Electric Boat has indicated to us that based on feasibility results, the project a 30% savings on their next submarine construction project. This showed up in briefing material provided to senior DoD people at the end of June 1994.

Other companies have developed business strategies around similar results from Phase I SBD. Lockheed may be able to provide some specific targets that they are trying to achieve utilizing SBD.

SIF Conversion Utility SW - IST/STRICOM

http://dmsttiac.sc.ist.ucf.edu/services/ir/benefits/

In response to the DMSO request below, one outstanding example of modeling and simulation work that is continuing to save the government and industry time and money is the work that IST accomplished under contract from STRICOM wherein the SIF Conversion Utility Software was developed. This software provides a ready interface of DoD standard software in the Simulator Interchange Format (SIF) to operate with various Image Generators used in the simulation field. This software has been developed and placed i n the DMSTTIAC Orlando Service Center to be made available to government agencies and contractors. To date there have been seven separate requests satisfied with copies of this very useful software. The development of this software is estimated to save a user approximately 6 man months of effort.

In addition to the completed software identified above, we at STRICOM have received two separate requests for copies of our developmental software used in our Dynamic Terrain project even though the software is in a transitional state. One request was for the ARMORED VEHICLE LAUNCHED BRIDGE (AVLB) software and the other request was for our D-7 BULDOZER FLT MODEL. Both of these projects together are estimated to represent a developmental effort of approximately two man years.

Gulf War Analyses

http://dmsttiac.sc.ist.ucf.edu/services/ir/benefits/

Modeling and Simulation has contributed to innumerable decisions involving system evaluation and force sizing. However, it has also contributed significantly to combat operations. In 1990 and 1991, the Air Force Studies and Analyses Agency (AFSAA) perf ormed a series of Gulf War analyses that Lt General Glosson (then chief of CENTAF Special Projects) asserts "...saved literally hundreds of lives."

A team of AFSAA analysts quickly deployed to the Air Force Operations Center in Riyadh, Saudi Arabia, where they analyzed the air campaign both before and after it began. For their analyses, they used primarily the Army's Space and Strategic Defense Comm and's EADSIM (also called the C3ISIM) model.

EADSIM was a brand new model to AFSAA at the time and was selected because it did an excellent job of analyzing command,control, and communications. It is a hybrid model with Monte Carlo and deterministic features. The model afforded the enormous advan tage to the combat operations planner of playing out a God's eye view of the air attack as it unfolded. Thus the planner was able to watch a preview of the attack as it unfolded in a way to reveal graphically the plan's strengths and weaknesses. Since the modeled air defenses, unlike the actual air defenses, acted in a rational manner, the simulation results showed a worst case scenario for the actual air assault.

One main contribution was to choreograph the masses of aircraft into and out of the Kuwaiti Theater of Operations to avoid mid-air collisions and to schedule the rendezvousing of tankers with attack aircraft. They also analyzed the best use of defense suppression assets, and alerted planners of missions that were too hazardous for some aircraft.

For instance, their analyses indicated that it would be too dangerous to carry out plans to send A-6 and Tornado aircraft directly over Baghdad. As a result only F-117 stealth fighters, none of which were lost, were assigned targets in that highly defended area. These changes undoubtedly saved lives and the needless loss of aircraft. When they determined that SCUD sites in Western Iraq were too well defended and (as existing prior to the attack) too hazardous for F-15E attacks, defense suppression missions were reconfigured to correct the problem. When aircraft losses occurred, computer simulations were used to help determine the most likely cause so that later missions could be made less dangerous. To ensure that aerial tankers would make their rendezvous with fighters in need of refueling, missions were played out in advance. To avoid mid-air collisions, attacks were carefully choreographedespecially the first day's intense activity.

A complete description of these operations is contained in the AFSAA report, Analysis of Air Operations During Desert Shield/Desert Storm, by Major F.T. Case, November 1991.

The report is available for review in the AFSAA Library, (703) 697- 5213 (file No. 28467), and is available in hard copy from the Defense Technical Information Center, (703) 274-6434 (DTIC report number B161849). A similar description may also be found in, "The Wizard Warriors of Desert Storm," The Journal of Electronic Defense, Mar ch 1992, p.56.

Navy China Lake Weapon System Support Facilities (WSSFs)

http://dmsttiac.sc.ist.ucf.edu/services/ir/benefits/

The Naval Air Warfare Centers, Weapons Division has made extensive use of modeling and simulation (M& S) since the 1970s to help develop and test aircraft and weapons upgrades. The laboratories used for aircraft navigation and weapon integration are called the Weapon System Support Facilities (WSSFs). The WSSFs use a combination of actual aircraft hardware and software, together with extensive modeling and simulation to provide a cost effective test platform. In addition to saving money, WSSFs enable testing to be conducted that would otherwise be impossible. The following paragraphs look at the impact of M&S on aircraft upgrade development, Safety-of-Flight, and Verification and Validation.

When adding or modifying new aircraft subsystems or weapon systems it is necessary to know the impact the changes have on the entire aircraft/weapons system. Each subsystems is modeled, and all subsystems are combined to form a system level simulation. This simulation is used to prototype initial developments--allowing pilots and engineers to refine their requirements prior to actual hardware and software development. During development, each upgrade is tested in the WSSF to determine if it meets requirements and to ensure that the modification does not adversely affect other aspects of system performance.

Prior to flight testing any changes, a standard set of tests are performed in the WSSF to ensure there are no problems that will be hazardous the pilot or his aircraft.

Once the aircraft and weapon system upgrades are completed, the system is thoroughly tested in the WSSF. The majority of system errors are found in these simulation laboratories. The WSSFs are a very cost effective method for testing. Extensive regression testing is performed to ensure that no performance degradation is inadvertently introduced during development. In addition, these laboratories have capabilities to perform tests that could not otherwise be performed. For instance, M&S provides software control of the system that enables parametric testing to be performed. These tests look at massive amounts of data collected for a wide range of flight conditions. In addition, the aircraft can be "flown" in potentially hazardous conditions that would otherwise be outside the test profile. Using "flight playback", all the bus traffic from a real or laboratory flight is recorded so that a flight can be "reflown" repeatedly in the laboratory. This enables engineers to examine all aspects of system performance. In addition, the same flight can be "reflown" after modifications to the system have been performed, thereby providing an inexpensive means to evaluate the impact of the changes.

While WSSFs are capable of validating the majority of the aircraft system, they are specialized laboratories, and aren't capable of testing the entire system. The ability to perform system level testing is being increased by connecting the WSSFs to other laboratories and ranges. For instance, the Electronic Combat Range (ECR) is an extensive EW flight range. By strategically placing EW hardware on a mountain top at the range, and linking this equipment with the WSSF, we are able to essentially bring much of the ECR environment into the laboratory. Similarly, the missile hardware-in-the-loop laboratories provide excellent missile modeling and simulation of the missile systems. By linking the WSSF with these facilities we are able to test the aircraft/weapon system from captive carry, through launch and into post launch data linking. While the WSSFs' simulations are capable of representing a dynamic environment for many of the tests, some hardware components, such as the Global Positioning System require more sophicicated simulations to accurately represent the satellite constellations, signal drop out, etc. The test problems associated with this limitation are overcome by linking the WSSFs with a GPS simulator. The interoperability of the WSSF with other facilities is in the initial development stages. We are currently enhancing the capability to develop this network into a viable tool that will reduce the time to integrate complex EW and missile systems onto the aircraft while significantly reducing the number of flight tests.

Additional F-14 WSSA (Snapshot)

One M&S application, of particular note, developed for the F-14D WSSA that had a large impact on tactical software verification and testing is a tool called Snapshot and its successor, Weapon Emulation and Evaluation System (WEES). These tools employ selection and engagement models of various weapons hosted by the F-14D avionics. The models are written in "C" or FORTRAN and run in real time on an external computer. The arguments for the logic and algebraic expressions are abstracted from the F-14D data buses and the resultant computed to derived output values are compared to the time and event correlated values, computed by the F-14D avionics system, that are used for weapon selection and engagement. This process allows the detection of two types of events (1) The detection of direct programming errors in the tactical software and (2) the detection of timed-out processes, i.e. lack of CPU speed or over-written memory resulting from over tasking and memory infringement by other computational processes.

Snapshot is an event driven, i.e. pickle push, tool that collects data over a six second interval and then the data is used to exercise the off line programs. WEES is a real-time version of Snapshot that runs continuously (i.e. the computed reference values are compared in real time to the equivalent system values computed by the F-14D avionics. These tools have substantially improved the code verification process as well or greatly reduced the software integration testing necessary to merge independently developed software modules and/or software subsystems. Efficiency increases of 100% are cited by software developers performing software verification.

Additional EA-6B input

The navigation and environment simulations allow the EA-6B V& V team to "fly" the system, while testing the tactical software at different Directions of Arrival (DOS's) of threat data. These simulations reduce the amount of flight testing required.

Two WRA's were combined into one, and included a single card Mission Computer. Our weapon system model, along with the environment simulator provided a very dense threat environment which allowed for most of the integration testing to be completed before the new WRA was installed in the aircraft. Integration in the aircraft would have been a much more expensive option.

IRIAM

The DMSO-funded IRIAM project has achieved acceptance and savings of note. IRIAM is being developed to serve as the DoD test, evaluation and integration testbed for comparison of synthetic environments, targets, and instruments with their real counterparts. For the next several years, IRIAM will serve as the prototype Joint Service sensor simulation testbed for the Distributed Interactive Simulation. IRIAM will serve as a prototype testbed for the training of engineers, scientists, and decision-makers in the effective use of models and distributed interactive simulations for developmental and operational test and evaluation of weapons systems.

IRIAM will help define standards for the presentation of test and simulation results, help validate models and present displays representative of the wargame performance. IRIAM will provide for the integration of models into advanced distributed simulations and serve as a testbed for proof-of-concept demonstrations for cross-model access capability and the assessment of database technology.

An activity that achieved savings by using IRIAM is the 445th Test Squadron Radar Flight Group at Edwards Air Force Base. The following is a quotation: "Our organization has benefited considerably from the in-house interactive data analysis which resulted from our participation in the IRIAM project. We are now able to provide a 24-hour quick-look performance analysis to support subsequent flight test decisions. In addition, we are now capable of reducing data from a flight for technical performance re porting in a few days. The IRIAM Team has provided us with hardware and software configured to bring us into commonality with the AMRAAM missile community, in support of our missile integration work. We have applied and extended this capability to other areas of our radar flight test analysis. The use of the IRIAM system has resulted in a cost savings to the Squadron of $2.1M in the last two years. "An activity that expects to achieve savings using the IRIAM-developed technology is the Brilliant Eyes System Program Office (BE SPO) of the Air Force Space and Missile Systems Center (AFMC). Following is a quote from the Chief, Test and Mission Operations Branch, AFMC "value beyond the numbers just quoted."

2111 Wilson Boulevard, Suite 400 • Arlington, VA 22201
Phone: 703.247.9471 • Fax: 703.243.1659
For more information, e-mail Patrick Rowe
Directions to NTSA