Simulation Evaluation

What types of simulations are available?
What is advanced/distributed distance learning (ADL)?
How do you determine the effectiveness of representations?
How do I measure simulator fidelity?
What is the relationship between fidelity and training effectiveness?

What types of simulation are available?

There are many types of simulators and simulations on the market for training purposes. Their level of sophistication depends on the specific nature of the system. Medical simulations have different grades depending on the amount of interaction between the user, patient, and other doctors. For example, a simulation representative of the human body is far less advanced then one which simulates the interaction of many different entities conducting a complicated heart transplant (Meller, 1997). Similar levels of complexity exist with flight simulation. At the lowest level, "fishtank VR" systems allow users to view a virtual environment through a desktop window of a standard computer monitor (Knott, 2000). At the next level of complexity, low-end flight simulators provide a basic flight simulation experience. Low-end flight simulators generally operate on the three degrees of freedom (3DOF) flight model. This means that the equations used in the flight model determine only three levels of motion, in comparison to the high-end simulators that incorporate all six degrees of freedom (6DOF). High-end simulators are designed to provide existing or future pilots with realistic practice through high fidelity simulations and graphics. Logically, high-end simulators tend to be much more expensive then their low-end counterparts (Wittick, 1996). (Image courtesy of Advanced Medical Simulations, Ltd.)

See also 6DOF

What is advanced distributed/distance learning (ADL)?

The Advanced Distributed Learning (ADL) initiative is an effort between government, industry, and academia to collaborate in a learning environment that brings together state-of-the-art technology and advanced networking capabilities. The DoD established the program in 1997 to modernize training and education and to facilitate cooperation between government, academia, and industry leaders. A major component of ADL is the establishment of interoperable learning tools and content that can be accessed and reused on a global scale. Constructive simulations will be used extensively because they represent an innovative way of conducting training without exhausting financial resources. In addition, simulations have been shown to effectively improve military readiness (Howell, 2000). The ADL initiative has attempted to create new markets for training materials, reduce the cost of development, and increase the return rate on investments. The vision of ADL is to “provide access to the highest quality education and training, tailored to individual needs, and delivered cost-effectively anywhere, anytime (ADLNet, 2002).” Training and learning content should be interoperable, reusable, accessible, durable, and affordable in order to succeed in achieving this vision (ADLNet, 2002). Additional information on ADL news, events, resources, and current projects can be found at www.adlnet.org.

How do I measure simulator fidelity?

Simulator fidelity can be measured both subjectively and objectively. Fidelity in both virtual environments and simulators are often measured using subjective scales of "presence", or degree of realism felt by the user. Subjective evaluation scales, such as the Cooper-Harper rating (Cooper and Harper, 1969), have been used to evaluate the fidelity of aircraft simulation. Although these scales are valuable, they can provide inconsistent results because the individual opinions and biases of the raters make it difficult to generalize across scores. Self-reports can also be used to measure presence, although again, there is likelihood of bias and individual differences. An alternative to subjective evaluation scales is the measure of psychological and physiological responses and reflexes in the virtual environment. By measuring the optokinetic (vision), vestibular (balance), and postural (body position) reflexes in response to a virtual environment, researchers can gain some understanding of how realistic the synthetic environment appears to the user (Sadowski & Stanney, 2000). For example, if the postural response is considered delayed, it can be concluded that the user does not view his or her body as fully immersed in the simulation (Ellis, 1995). Rapid reflexes in response to the virtual environment or simulator would indicate a high degree of fidelity. (Image courtesy of FlightSimulator.ch)

What is the relationship between fidelity and training effectiveness?

The degree of fidelity in a simulator is closely linked to training effectiveness. In general, it is assumed that the transfer of training from a simulator to a real world task will be greater when the conditions in the simulation better match those in the actual task. Theoretically, the more similar the stimulus and response elements in "A" are to "B", the more likely positive transfer of training will occur between the two (Martin, 1981). Since high fidelity simulators are associated with more realistic representations of the actual task, this assumption has been the driving force behind obtaining high fidelity simulators for training purposes. The only exception to this rule is the use of platform motion in a simulator. It would seem logical that a simulation with added motion would increase transfer of training from one piece of equipment to another. However, numerous studies have found that platform motion provides no added training benefit to simulator users (Martin, 1981). In fact, the use of platform motion is more likely to cause nausea, dizziness, and other symptoms associated with simulator sickness.