Anderson, H. C., Euhus, L. E., & Parker, D. D. (n.d.). Utilizing simulators in operator training. Retrieved June 5, 2002 from the World Wide Web: http://www.pacsim.com/Articles/OpTrain/art1tr.htm

The Weyerhaeuser Corporation initiated the modernization of a pulp and paper complex by implementing high tech equipment necessary to meet customer and community demands and upcoming environmental regulations. The new equipment and modernized controls required a sophisticated training program. The cornerstone of the training program was the use of a high fidelity simulation. Advances in desktop simulation software have improved the quality and benefits of simulator based training. A real time dynamic simulation was interfaced to a distributed control system and programmable logic controller for use in operator training and control system checkout.

Anonymous. (n.d.). Human-machine system design: When does simulation pay? Retrieved June 5, 2002 from the World Wide Web: http://olias.arc.nasa.gov/cognition/papers/freed/interact99.html

Badler, N. L., Bindiganavale, J. B., Allbeck, J., Shi, J., & Palmer, M. (1997). Real time virtual humans. Retrieved June 17, 2002 from the World Wide Web: http://www.cis.upenn.edu/home.html

The last few years have seen great maturation in the computation speed and control methods needed to portray 3D virtual humans suitable for real interactive applications. Various dimensions of real-time virtual humans are considered, such as appearance and movement, autonomous action, and skills such as gesture, attention, and locomotion. A virtual human architecture includes low level motion skills, mid-level PaT-Net parallel finite-state machine controller, and a high level conceptual action representation that can be used to drive virtual humans through complex tasks. This structure offers a deep connection between natural language instructions and animation control.

Baliozian-Chalabian, J., & Dunnington, G. (2000). Cognitive task analysis to identify educational objectives for procedural skill training. Retrieved June 5, 2002 from the World Wide Web: http://www.surgery.usc.edu/research/education.htm

Researchers from the University of Southern California performed a pilot study with use of Cognitive Task analysis (CTA) to identify educational objectives based upon expert-novice performance differences in Minimally Invasive Surgical (MIS) technical skills. Rapid advance in medical technology requires development of new procedural skills and expertise. Procedural skill acquisition requires both, development of technical skills and cognitive or decision making components (ie. when, where, and how) of implementation. The military developed Cognitive Task Analysis (CTA) to facilitate rapid and effective acquisition of expertise by enlisted personnel in complex cognitive-technical skills. Performance differences existed on each test item between the expert, intermediate and novice pilot groups. Quantitative and qualitative data suggest study group differences in content and structure related declarative knowledge, knowledge schemes, pattern recognition etc., corresponding to differences predicted from study of cognitive psychology and expertise.

Bangay, S., & Preston, L. (1999). An investigation into factors influencing immersion in interactive virtual reality environments. Retrieved January 28, 2002 from the World Wide Web: http://www.cs.ru.ac.za

Researchers from Rhodes University examined the multitude of factors influencing immersion in interactive virtual reality (VR) environments. Bangay and Preston conducted two experiments, with the goal of the first being to identify the factors affecting immersion and the second being to determine the effect of immersion on stress and simulator sickness. To identify immersion factors, subjects were placed in a “swimming with the dolphins” simulation, intended to calm stress levels and reduce their heart rates. Based on participant questionnaires, the following factors were found to influence the effectiveness of immersion in a VR environment: excitement of the experience, comfort of environment, quality of sound and images, and age. To determine the effect of immersion on stress and simulator sickness, subjects were placed in a “virtual roller coaster” experience. The responses given by participants indicated that the following factors were dependent on the degree of immersion in VR environments: simulator sickness, control, excitement of the experience, and desire to repeat the experience.

Boyd, C. (1995). Human and machine dimensions of 3D interfaces for virtual environments. Retrieved February 14, 2002 from the World Wide Web: http://www.acm.org/sigchi/chi95/proceedings/doctoral/cb1bdy.htm

Persistent problems arise from characteristics of interfaces of virtual reality devices related not to the limitations of the hardware, but to the perceptual, cognitive, and sensory-motor abilities of humans. The interface characteristics explored in this paper include three-dimensional interface metaphors, computer graphics, models of navigation, and the three-dimensional joystick. In addition to interface characteristics, this paper also examines human characteristics and how they affect virtual environments. There are cognitive and physical differences between individuals using virtual environment systems. The researchers propose the use of studies that test the ability to understand abstractions, visualize spatial relationships, and determine field dependence/independence in a visual context. Examining these abilities will help designers understand how people differ in their speed and type of learning in a virtual environment.

Boyd, C. (1997). Does immersion make a virtual environment more usable? Retrieved February 14, 2002 from the World Wide Web: http://www.acm.org/sigchi/chi97/proceedings/short-talk/cb.htm

       Researchers at the University of Colorado at Boulder performed usability tests on three different types of virtual environments. Two environments were categorized as non-immersive, while one environment was considered to be immersive. The non-immersive environments consisted of hand-trackers that allowed the user to explore the virtual environment through a desktop monitor. The immersive environment consisted of a stereoscopic head-mounted display, which simulated a walk through a natural environment. The head-mounted display allowed turning of the head and other search and navigation features provide a feeling of “being there” for the user. The sixty participants in this study were each given simple task with each interface, and asked to perform the task ten times consecutively. Based on the findings from the experiment, participants in the immersive virtual environment completed their tasks significantly faster then participants in the non-immersive environments. The researchers concluded that an immersive environment is therefore more usable, or user-friendly, than non-immersive environments.

Brenner, T., Sheehan, K., Arthur, W., & Bennett, W. (1998). Behavioral and cognitive task analysis integration for assessing individual and team work activities. Retrieved June 5, 2002 from the World Wide Web: http://www.ijoa.org/currentwork/98symposium/brenneretal.html

The U.S. Air Force Research Laboratory has been developing cost effective methods for gathering occupational and training requirements information. This information has most often been collected at an individual level of analysis focusing on the more behavioral aspects of work. Recent interest in both team and cognitive requirements for work has prompted renewed interest in team task analysis and accurately representing knowledge/cognitive components of work. The U.S. and Allied Military Services have pioneered the development of exemplar methods that serve as the foundation for recent advances. This paper highlights recent explorations and advances in team task analysis and cognitive task analysis methods. Implications for increasing the accuracy and efficiency of the requirements analysis process will also be discussed.

Caretta, T. R. & Dunlap, R. D. (1998). Transfer of training effectiveness in flight simulation: 1986-1997 (Report No. AFRL-HE-AZ-TR01998-0078). Mesa, AZ: Air Force Research Laboratory. (DTIC No. ADA362818)  

The purpose of this report was to review recent studies regarding the effectiveness of flight simulators as augmentation for "hands-on" flying training. Simulation-based training has been proposed to reduce costs, extend aircraft life, maintain flying proficiency, and provide more effective training, especially in areas difficult to train in operational aircraft. Studies of simulator effectiveness for training landing skills constituted a majority of the transfer studies, although a few examined other flying skills such as radial bombing accuracy and instrument and flight control. Results indicate that simulators are useful for training landing skills, bombing accuracy, and instrument and flight control. Generally, as the number of simulated sorties increases, performance improves, but this gain levels off after approximately 25 missions. Further, several studies indicate that successful transfer may not require high-fidelity simulators or whole-task training, thus reducing simulator development costs. Evaluation of this literature is difficult for many reasons. Typically, researchers fail to report sufficient detail regarding research methods, training characteristics, and simulator fidelity. In addition to these methodological concerns, there is a lack of true simulator-to-aircraft transfer studies involving complex pilot skills. This may be due to problems such as inadequate simulator design, cost, and availability, and access to simulators in operational flying units. Future directions in simulator transfer of training are discussed.

Davies, T. (2001). RNZAF C-130 simulator training: The future of a costly necessity. Retrieved June 5, 2002 from the World Wide Web: http://papers.maxwell.af.mil/research/ay2000/acsc/00-051.htm

This paper aims to assist the RNZAF in realizing that some long-term strategic decisions are required about the future of its C-130 training. At present, there exists a disturbing trend showing pilot experience, aircraft continuation training, and simulator refresher training to be decreasing insidiously. Most of this is due to budgetary constraints, which have reduced resources to a bare minimum. The current simulator-training scheme is too expensive to yield the required minimum frequency for refresher training and new ways must be identified to enable more training for less cost. This paper reviews the rational for using flight simulators and then compares the RNZAF C-130 with other air forces to assess whether its c-130 pilots are fulfilling the minimum level of simulator training recommended by those countries.

Dorr, K., Schiefele, J., & Kubbat, W. (2000). Virtual cockpit simulation for pilot training. Darmstadt, Germany: Institute for Flight Mechanics and Control. (DTIC No. ADP010789)

For some of today's simulations very expensive heavy and large equipment is needed. Examples are driving shipping, and flight simulators with huge and expensive visual and motion systems. In order to reduce cost immersive Virtual Simulation becomes very attractive. Head Mounted Displays (HMD), CAVEs (Computer Animated Virtual Environments), and datagloves are used to generate a stereoscopic virtual environment (VE) for the trainee. IVS enhances training quality and quantity for classroom-teaching and Computer Based Training (CBT). It allows trainees to visualize and animate teaching material in a more natural stereoscopic environment. Data of before unseen complexity can be revealed and complex models easily visualized. For the first time the trainee can interact with a Data-Glove in the environment and collect cockpit experience long before his maiden flight. CAVEs and Immersive Projection Screens enable "group training" to collect personal and shared experience while further enhancing training quality.

Draper, M.H. (1996). Can your eyes make you sick? Investigating the relationship between the vestibulo-ocular reflex and virtual reality. Retrieved January 22, 2002 from the World Wide Web: http://www.hitl.washington.edu/publications/r-96-3/

This paper addresses the vestibulo-ocular reflex and speculations about its relationship to simulator sickness. The vestibular apparatus is a small structure in the inner ear that is used to sense and signal movements of the head. Its function contributes to coordination of motor responses, eye movements, and posture and has been linked to motion sickness. One theory that is highly regarded is the sensory conflict theory, or the idea that there is some conflict between the visual and vestibular systems. For example, if a sailor believes he is looking at a stationary point (such as the horizon), but his vestibular system is sensing head movement (movement of the ship), a mismatch of cues may occur and physical symptoms such as dizziness, nausea, and headaches may occur. A similar mismatch of cues may occur in simulator sickness, in that a head-mounted display may provide inappropriate cues in comparison to the exact motion that an individual is experiencing. Contributors to simulator sickness include time delays, tracking inaccuracies, and poor image quality and resolution, and badly designed displays.

Driscoll, M. P. (1994). Psychology of learning for instruction. Needham Heights, MA: Allyn & Bacon.

This book focuses on learning and instruction. Specific implications and applications of learning theories are discussed, and examples are drawn from educational situations and educational problems. Theoretical concepts are illustrated in concrete terms for the reader. Topics covered include learning theory, behaviorism, cognitive information processing, meaningful reception learning, schema theory, mental models, knowledge development, motivation, instructional theories, and constructivism.

Dzindolet, M. T., Gleason, D. R., Hernandez, C., Hill, K., Jaquint, M., Jette, F., Larson, J., Oxley, N. L., & Pierce, L. G. (1998). Defining training objectives and performance measures in a Janus battle simulation (ARL Report No. CR-433). Aberdeen Proving Ground, MD: Army Research Laboratory. (DTIC No. ADA357112)

The United States Army Field Artillery School (USAFAS) at Fort Sill, Oklahoma, uses the Janus battle simulation for training officers in fire support operations and battle command decision making. In conjunction with the training exercises, a series of experiments was run to determine the effectiveness of the Janus simulation for training students in the Officer Basic Course (OBC). Course instructors were interviewed to determine the training objectives expected to be met using the Janus simulation. These objectives were measured using both objective and subjective measurement approaches. Objective measures were performance indicators that could be derived from the simulation. Characteristics of the simulation, instructor involvement, and ability of the interactors limited the validity of the objective measures. Subjective measures, obtained using self-report measures from instructors and students, indicated that the Janus simulation exercises, as used in the OBC, were useful in increasing the confidence and proficiency of the students. Results are presented by training objective and measurement approach.

Ellis, S. R. (1995). Virtual environments and virtual instruments. In K. Carr & R. England (Eds), Simulated and virtual realities: Elements of perception (pp. 11-39). Bristol, PA: Taylor & Francis.

This section of the book covers virtual environments and virtual environment instruments. The author emphasizes the media’s influence on the design of virtual environments, optimal design, and extension of the desk-top metaphor. In addition, the author also goes through the necessary steps in identifying and defining a virtual environment, including its content, geometry, and dynamics. Virtualization is defined and distinguished by three levels: virtual space, virtual image, and virtual environments. Lastly, this chapter discusses the history and early origins of virtual environments and what role virtual environments play in the current technological society.

Gillian, D. J., & Cooke, N. J. (1995). Methods of cognitive analysis for HCI. Retrieved June 5, 2002 from the World Wide Web: http://www.acm.org/sigchi/chi95/proceedings/tutors/dg2_bdy.htm

This paper teaches participants about methods used to measure cognitive content, structure, and processes in an active hands-on manner, and how to apply those methods to HCI. The structure of the paper centers on the phases of a design process. The areas of cognition addressed are perception, memory, language, and thinking. For the initial analytical phase of design, the paper describes methods for measuring visual search, the structure of semantic memory, and process tracing. Methods for measuring readability and comprehension, as well as memory recall and recognition are applied to data from the second phase, which is design and diagnostic testing. For the third phase, system testing, the discussion focuses on scaling methods and statistical techniques.

Healy, A. F., & Sinclair, G. P. (1994). Long term retention of trained skills (Technical Report No 94-001). Orlando, FL: Naval Air Warfare Center Training Systems Division. (DTIC No. ADB187295)

Knowledge and skills, acquired during training, often are not activated for considerable time periods. Disuse can lead to skill deterioration that, in the extreme, might mean that a skill is no longer functional when needed. Training techniques must be developed to maintain skills at functional levels over periods of disuse. Although there has been considerable research designed to develop training techniques for efficient acquisition of skills, relatively much less is known about training or refresher techniques that lead to maintenance over time. In this document, researchers review empirical results and theories from basic experimental research concerning the long-term retention of trained skills. Throughout this review, they discuss the implications of the basic research findings for real-world applications. In particular, the researchers develop a set of guidelines concerning the means to improve the conditions of training and instruction so as to facilitate long-term retention of knowledge and skills. These guidelines apply to complex tasks such as those used in the military, not just to the simple tasks used in the specific experiments.

Heintzman, R. J. (1996). Determination of force cueing requirements for tactical combat flight training devices (ASC-TR-97-5001). Wright Patterson AFB, OH: Aeronautical Systems Center, Training Systems Product Group.

 This report documents the study that covered many considerations relating to force cueing in a simulator, including its effect on pilot behavior and performance and how force cues may relate to different flight tasks and the performance of force cuing devices. The study also included a review of the evolution of platform motion systems, other force cuing devices and past force cuing research. Recommendations are provided for designing a simulator with force cueing capabilities, in addition to less expensive force cuing alternatives for use in training simulators and simulations. Finally, an attempt is made to define a procedure to objectively determine the need for force cueing and its value as part of a ground based combat flight training system.

Howell, D. Y. (2000). The feasibility of conducting constructive combat training simulations via distributed training technology methods (Report No. 20000801043). Carlisle Barracks, PA: U.S. Army War College. (DTIC No. ADA380131)

The dispersion of National Guard units and limited training time has always made training readiness a challenge. In addition, sustaining reserve forces at the same level as their active component counterparts is crucial with the major reduction in active component forces. Current constructive combat training simulations available in the Army may be a viable training option to training National Guard units for combat. It is possible that training with simulations may reduce the number of days required to train a unit for mobilization. GUARDNET XXI, the National Guard Distributive Learning Network offers a mode to make current constructive combat training simulations distributive. This research determines the feasibility of conducting constructive simulations via GUARDNET XXI. Conducting simulations will enhance and supplement collective training during the limited time and other resource constraints in the National Guard. The methodology is to determine the value of training Army National Guard units with simulations; determine requirements to operate the simulation system and capabilities of the Distributive Learning Network; and consider the National Guard's posture as it relates to simulation management. At the conclusion, Army National Guard leadership will have an alternative to train the Reserve forces at an equivalent level of the Active component forces and complete the training requirements, all at a reduced cost.

Jones, R. M. (1993). Realistic intelligent agents for training simulators. Paper presented at the International Joint Conference on Artificial Intelligence Workshop on Models of Teaching and Models of Learning. Chambery, France: International Joint Conference on Artificial Intelligence, Inc. 

This research paper focuses on skills that require interaction with other humans within a simulated environment. In order for automated simulated agents to provide an effective training situation, they must behave as realistically as possible. Simulation is advantageous because it can provide effective training in a more controlled environment then the real world. Example domains to which this paper is applicable include air-traffic control, public transportation, cooperative activities, economics, and military tactics. Appropriate behavior in these domains relies heavily on the interaction between multiple intelligent agents, each with their own goals, skills, and equipment. In a training simulation, some of these agents may be other humans taking part in the simulation, whether they are trainees or teachers.

Jones, R. M., Laird, J. E., & Nielsen, P. E. (1998). Automated intelligent pilots for combat flight simulation. Proceedings of the Tenth Annual Conference on Innovative Applications of Artificial Intelligence (pp. 1047 –1054). Menlo Park, CA: AAAI Press.

This paper describes the technological capabilities of application called TacAir-Soar. TacAir-Soar is an intelligent, rule-based system that generates believable “human-like”behavior for military simulations. The innovation of the application is primarily a matter of scale and integration. The system is capable of executing most the airborne missions that the United States military flies in fixed-wing aircraft. It accomplishes this by integrating a wide variety of intelligent capabilities, including reasoning about interacting goals, reacting to rapid changes in real time or faster, communicating and coordinating with other agents including humans, maintaining situational awareness, and accepting new orders while in flight. The system is currently deployed at the Oceana Naval Air Station WISSARD training facility, and has participated in a variety of tests, technology demonstrations, and operational training exercises.

Jones, R. M., Tambe, M, Laird, J. E., & Rosenbloom, P. S. (1993). Intelligent automated agents for flight training simulators. In Proceedings of the Third Conference on Computer Generated Forces and Behavioral Representation (pp. 33-42). Orlando, FL: Simulated Interoperability Standards Organization.

 Training in flight simulators will be more effective if the agents involved in the simulation behave realistically. Accomplishing this requires that the automated agents be under autonomous, intelligent control. Researchers in this paper use the SOAR cognitive architecture to implement intelligent agents that behave as much like real humans as possible. In order to approximate human behavior, the agents must integrate planning and reaction in real time, adapt to new and unexpected situations, learn with experience, and exhibit the cognitive limitations and strengths of humans. This paper describes two simple tactical flight scenarios and the knowledge required for an agent to complete them. In addition, the paper describes an implemented agent model that performs in limited tactical scenarios on three different flight simulators.

Kolasinski, E. M., Goldberg, S. L., & Hiller, J. H. (1995). Prediction of simulator sickness in virtual environments. Retrieved May 29, 2002 from the World Wide Web: http://kb.hitl.washington.edu/virtual-worlds/kolasinski/rol.html 

The Army has made a commitment to improving the effectiveness of training and preparing soldiers for the electronic battlefield. This report describes the literature review by the U.S. Army Research Institute for the Behavioral and Social Sciences of virtual environment (VE) technology with regard to the phenomenon of simulator sickness. Simulator sickness results from exposure to training simulators and simulations, and has a pattern of symptoms similar to motion sickness, headaches, eyestrain, and disorientation. Simulator sickness has implications for both training effectiveness and safety. This report consists of the identification of the types of equipment and features of a simulator which most contribute to the development of the sickness in addition to providing recommendations for countermeasures and prevention.

Lombard, M., & Ditton, T. At the heart of it all: The concept of presence. Retrieved February 19, 2002 from the World Wide Web: http://www.ascusc.org/jcmc/vol3/issue2/lombard.html

This article examines the concept of presence in virtual reality and in virtual environments. Six conceptualizations of presence found in a diverse set of literature are identified. The six conceptualizations are as follows: presence as social richness, presence as realism, presence as transportation, presence as immersion, presence as a social actor, and presence as medium. These concepts are all diverse, yet the authors tie them together to form a complete conceptualization of the term presence. The researchers also examine the characteristics that contribute to presence in a virtual environment, including visual display characteristics such as image quality, image size, color, motion, and dimensionality. Finally, researchers examine some of the effects associated with presence on the user of the virtual environment. Presence has been associated with arousal, vection, simulator sickness, and enjoyment depending on the degree of realism achieved.

Macpherson, D., Patterson, C., & Mirabella, A. (1989). Application of ARI Skill Retention Model to wheel vehicle maintenance tasks (ARI Report No. 1538). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. (DTIC No. AD A219 684)

The Skill Retention Model is a model that describes how people forget technical tasks in terms of such task characteristics as the number of steps in the task and the quantity of the job aids. Researchers administered the Skill Retention Model to seven expert wheeled vehicle mechanical/instructors (SMEs) at the Ordnance Center School, Aberdeen, MD. A total of nine tasks were covered, five tasks in five hours on each of two days, with one task being evaluated on both days and one evaluator serving on both days. They identified three out of nine tasks that are rapidly forgotten. Out of ten task characteristics, five differentiated the three tasks easily forgotten from those that are not easily forgotten. The researchers recommended ways to improve training and calculated how much retention would increase with improved training.

Martin, E. L. (1981). Training effectiveness of platform motion: Review of motion research involving the advanced simulator for pilot training and the simulator from air-to-air combat (Report No. AFHRL-TR-79-51). Williams AFB, AZ: Air Force Human Resources Laboratory. (DTIC No. ADA095930)

This report presents a summary review of the transfer-of-training studies conducted by the Operations Training Division of the Air Force Human Resources Laboratory investigating the training effectiveness of six-degrees-of-freedom platform motion cueing. A total of six studies are reviewed. Of the six studies, five were conducted on the Advanced Simulator for Pilot Training (ASPT) located at Williams AFB and one on the Simulator for Air-to-Air Combat (SAAC) located at Luke AFB. Tasks investigated included basic and advanced contact, instruments, basic fighter maneuvers, and conventional weapons delivery. The review of each study contains a statement of objectives, a summary of the method and results, a data excerpt representative of the findings, and a critique. The report also contains a description of the research strategy from which the studies were derived, a discussion of transfer-of-training methodology, and a discussion of the relationship between the results of these six studies and research findings from other agencies or facilities. Implications for future research are discussed.

Meister, J. A. (2000). Individual perceptions of team learning experiences using video-based or virtual reality environments. Doctoral dissertation, University of Houston, Texas.

Collaborative learning, in small groups coupled with the user of computer-mediate technologies, is becoming a very popular instructional strategy. Case study methodology was used to analyze individuals’ perceptions of their experience and performance in a team training environment which employed different training technologies (videotape and virtual reality). The aim of the study was to investigate team process and evolution, as well as the impact of virtual environment training, through the perceptions of training participants. Findings supported previous work on the roles of positive interdependence, individual accountability, and group goals in team training and performance. Also, level of cohesion among teammates was found to influence group processes, particularly with regard to assessing team member’s skills, abilities, and willingness to receive help. Finally, participants’ perceptions of training in a virtual environment supported the importance of fidelity of representations.

Morrison, J. E., & Hammon, C. (2000). On measuring the effectiveness of large-scale training simulations (IDA paper P-3570). Alexandria, VA: Institute for Defense Analysis. (DTIC No. ADA394491)

This paper identifies and evaluates methods for measuring the benefit or utility of large-scale training simulations (LSTSs). The intended audience is those people who are directly responsible for assessing the training effectiveness of LSTSs. The researchers examine literature related to the measurement of LSTS training effectiveness including guidance provided by the individual military services and the Office of the Secretary of Defense (OSD), the body of research on performance measurement, and research practices related to experimental design and analysis. They then synthesize the findings from these literature sources to provide seven recommendations for performing training effectiveness analyses of LSTSs: (1) identify specific measurement issues, (2) devise a measurement plan that specifies performance measurement and research design, (3) use valid and reliable performance measures, (4) obtain the most valid measure of proficiency possible within the constraints of the evaluation, (5) to the extent possible and reasonable, impose experimental control on the research situation, (6) use analytic models in early stages and continue using these models throughout system development, (7) and include user reactions as an adjunct to performance data and analysis.

Olson, M. J., & Popplewell, H. E. (n.d.). Integrating multi-media CBI approaches into simulator training programs. Retrieved June 5, 2002 from the World Wide Web: http://www.agentwaresystems.com/Publications/media.htm

This paper describes a unique, high technology training system currently under development for the U.S. Marine Corps and the U.S. Navy’s Space Warfare Systems Command by the General Electric Company. The system includes a broad range of instructional media, including conventional instruction, Computer Based Instruction (CBI), two dimensional simulation, interactive graphics and video-disc, a high fidelity training device, dummy mission equipment, actual Prime Mission Equipment (PME), and managed on the job training. The integration of these media and the myriad types of personnel required to develop such a system is a complex challenge from both the technical and management perspectives. This paper attempts to show how educational technology may be intelligently applied to meet complex training requirements.

Orlansky, J., Taylor, H. L, Levine, D.B., & Honig, J. G. (1997). The cost and effectiveness of the multi-service distributed training testbed (MDT2) for training close air support (IDA Paper P-3284). Alexandria, VA: Institute for Defense Analysis.

The purpose of this study was to determine the effectiveness and cost of the Multi-Service Training Testbed (MDT2). MDT2 is the first system to link simulators of different Services in an interactive network. This network provides multi-Service training for a common mission. In one 5-day exercise held May 1994 and a second held February 1995, the MDT2 was used to train personnel in Close Air Support (CAS). Eight different types of simulators representing friendly and enemy forces were linked to conduct exercises modeled after those that take place at the National Training Center, Fort Irwin, CA. Participating in the exercise were Army, Marine Corps, and Air Force personnel and simulators at three different locations. Researchers found strong evidence on the utility of distributed interactive simulation as a significant way to provide complex multi-Service training.

Pausch, R., Profitt, D., & Williams, G. (1997). Quantifying immersion in virtual reality. Retrieved January 28, 2002 from the World Wide Web: http://www-2.cs.cmu.edu/~stage3/publications/97/conferences/siggraph/immersion

Researchers from the University of Virginia examined the types of applications most suitable to virtual reality (VR). From prior studies, the experimenters hypothesized that users of VR systems developed a better frame of reference for the space or environment in which they are working. Users of VR systems also have higher immersion levels, or a feeling of actually “being there” in the simulation. To compare the performance of users of VR systems and users of a hand-based input device with a stationary monitor, the experimenter placed both types of users in the center of a virtual room and asked them to look for targets that were camouflaged. In this task, users of VR systems did not do significantly better then desktop users. However, when both users were asked to search the room and conclude if a target existed, VR users were substantially better at determining whether they had searched the entire room. These findings lend salience to the idea that VR users develop a better frame of reference for their environment. Lastly, the experimenters also studied the transfer of training effects related to learning in a virtual environment. Interestingly, they discovered that there is a positive transfer of training from VR systems to desktop displays and a negative transfer of training from stationary displays to VR systems.

Peterson, B. (1998). The influence of whole-body interaction on wayfinding in virtual reality. Retrieved January 28, 2002 from the World Wide Web: http://www.hitl.washington.edu/publications/r-98-3/ch1.html

This study explored the properties of a virtual reality interface that affect user navigation and maneuvering through a virtual environment. The experience of wayfinding, or navigating and maneuvering, through a virtual world is much different from the experience of wayfinding through the actual world. In the real world, maneuvering engages motor skills, while navigating involves cognitive skills. When in a simulated environment, wayfinding becomes more complicated because of two interface-related factors. First, visual stimuli may present problems if they are of low-resolution, narrow field of view, or coupled with time delays. Second, humans must communicate their intentions through an interface, where maneuvering and navigating may be less natural and require more conscious control then normal wayfinding. Peterson examined the input and output properties of the interface as to influence one’s wayfinding performance, and hypothesized that if the interface accepts inputs more congruent to human vestibular and kinesthetic properties, spatial awareness will improve and wayfinding performance will be enhanced. The results of the study indicated that the user of sufficient motion interface devices may improve one’s sense of direction in virtual environments.

Peterson, B., Wells, M., Furness, T., and Hunt, E. (1998). The effects of the interface on navigation in virtual environments. In Proceedings of Human Factors and Ergonomics Society 1998 Annual Meeting (pp. 1496-1505). Santa Monica, CA: Human Factors and Ergonomics Society.

Researchers at the University of Washington conducted an experiment to determine how interfaces affect users’ search and navigation abilities. Thirty participants explored two virtual mazes in this experiment, and half used a Virtual Motion Controller (VMC) or body-controller interface while the other half used a hand-controller or joystick to navigate through the environment. The results indicated that maneuvering performance was slightly better with the joystick. Survey knowledge, however, or the ability to form a mental map of the space and use it to find alternative routes, was significantly better with the VMC. In addition, the enhancement of performance using the VMC on the maze task was correlated with the difficulty of the maze. In other words, the more difficult the maze was, the greater the benefit of using the VMC. This experiment provides support for the idea that using the body in virtual environments may enhance certain components of navigation for users.

Rheingold, H. (1991). Virtual reality. New York, NY: Touchstone.

This book discusses the revolutionary technology of computer-generated artificial worlds and how it promises to transform society. The author speaks of his personal definition of reality, and what role illusion plays in the modern world. Binocular illusion is described in detail with respect to the natural capabilities of the human eye and how science and technology are attempting to recreate this natural phenomenon for the purposes of realistic simulations and illusions. The history of virtual reality is discussed, including the influence of NASA and the birth of VR business. Next, the idea of intelligent virtual beings is examined, which leads into a discussion of the future of virtual reality and the effect of cyberspace on mankind.

Rickel, J., & Johnson, W. L. (1999). Animated agents for procedural training in virtual reality: perception, cognition, and motor control. Applied Artificial Intelligence, 13, 343-382.

This paper describes Steve, an animated agent that helps students learn to perform physical, procedural tasks. The student and Steve cohabit a three-dimensional, simulated mock-up of the student’s work environment. Steve can demonstrate how to perform tasks and can also monitor students while they practice tasks, providing assistance when needed. This paper describes Steve’s architecture in detail, including perception, cognition, and motor control. The perception module monitors the state of the virtual world, maintains a coherent representation of it, and provides this information to the cognition and motor control modules. The cognition module monitors the state of the virtual world, maintains a coherent representation of it, and provides this information to the cognition and motor control modules. The cognition module interprets its perceptual input, chooses appropriate goals, constructs and executes plans to achieve those goals, and sends out motor commands. The motor control module implements these motor commands, controlling Steve’s voice, locomotion, gaze, and gestures, and allowing Steve to manipulate objects in the virtual world.

Romano, D. M., Brna, P. (2002). ACTIVE World: Manipulating time and point of view to promote a sense of presence in a collaborative virtual environment for training in emergency situations. Retrieved July 8, 2002 from the World Wide Web: http://www.cbl.leeds.ac.uk/~daniela/1998_ACTIVE_world.html

The use of virtual environments (VEs) to train members of a team to make good quality decisions is advocated. Researchers have designed a system, ACTIVE world, that incorporates some of the strengths of VEs, but also supports various ways of reflecting on aspects of the training process. These additional support mechanisms have been categorized in terms of additional powers (superpowers) given to the trainees that they can exercise within the training environment. Researchers describe an ACTIVE world that supports collaboration, provides a sense of presence and some additional powers. How these superpowers interact with the sense of presence and degree of effective collaboration will be the object of research in the near future.

Rose, A. M., Radtke, P. H., Shettel, H. H., & Hagman, J. D. (1985). User’s manual for predicting military task retention (ARI Report No. 85-26). Alexandria, VA: U. S. Army Research Institute for the Behavioral and Social Sciences. (DTIC No. ADA163710)

A methodology was developed whereby unit commanders and trainers can estimate unit proficiency on individual tasks over periods of no practice. The method will enable persons making training decisions to organize training schedules to maximize unit readiness. The method consists of 10 rating questions concerning the retention characteristics of individual tasks: the presence and quality of job aids, the number of steps to complete the task, the requirement to complete all or part of the task steps in a certain sequence or within a given time limit, the presence of feedback for correct or incorrect performance, the character and level of mental demand, and the level of motor control required to do the task.

Rosett, A. (1987). Training needs assessment. Englewood Cliffs, NJ: Educational Technology Publications, Inc.

This book is focused on human performance. The intended audience includes trainers, performance technologists, instructional designers, educational specialists, and course developers. It focuses solely on training needs assessment. Training needs assessment is the systematic study of a problem or innovation, incorporating data and opinions from varied sources, in order to make effective decisions or recommendations. Topics addressed include tools and techniques, performance problems, subject matter analysis, interviewing, and working with groups.

Schaab, B. B., & Moses, F.L. (2001). Six myths about digital training (Report No. 1774). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

The Army Research Institute has conducted a research study about current training practices given to soldiers who work with digital systems and information intensive technologies. The purpose of the study was to evaluate current training methods and examine how future training practices should be modified to prepare soldiers for the use of digital systems and the effective management of large quantities of information. Key findings included the notion that once they are learned, digital skills remain present in soldiers despite the passage of time. In addition, soldiers quickly adapted to software upgrades and system changes, showing that training practices should focus more on mission performance and less on the specific aspects of the operating system. Also, although skills are not perishable, soldiers need routine exposure to digital systems if they are to maintain solid digital skills and effectively manage information. This training should take place in context and should vary depending on the different levels of knowledge acquisition. Development training materials are needed that support learning and expand emerging skills. Finally, soldiers in the study believed that current training is not enough to support digital performance and future training should be modified to better support the systems used in the 21^st Century.

Schraagen, J. M., Chipman, S. F., & Shalin, V. L. (2000). Cognitive task analysis. Mahweh, NJ: Lawrence Erlbaum Associates, Inc.

This book discusses cognitive task analysis. Cognitive task analysis is the extension of traditional task analysis techniques to yield information about the knowledge, thought processes, and goal structures and underlie observable task performance. Analysis of jobs and their component tasks may be undertaken for a wide variety of purposes, including the design of computer systems to support human work, the development of training, or the development of tests to certify job competence. Topics covered in this book include cognitive task analysis for individual training, performance assessment, selection, human system interaction, and teamwork situations.

Seaman, K. A. (1999). A comprehensive analysis of air combat simulation training studies provides insight on how to improve F-15C air combat training using DMT. Retrieved June 5, 2002 from the World Wide Web: http://www.au.af.mil/au/database/research/ay1999/acsc/99-183.htm

Air Combat Command is investing in Distributed Mission Training (DMT) to provide realistic mission training to the Combat Air Forces (CAF) using advances in simulation technology. DMT will network advanced simulators and some real-world systems to provide combat aircrews with team training in a synthetic wartime environment. F-15C units will be first in the CAF to incorporate DMT. They are confronted with developing training programs utilizing this new tool without previous experience of how to exploit the benefits of simulation for air combat training. This paper seeks to assist syllabus developers by providing a summary of lessons learned from years of air combat simulation study, and applying those lessons to DMT.

Shneiderman, B. (1998). Designing the user interface: Strategies for effective human-computer interaction. Reading, MA: Addison Wesley Longman, Inc.

This book examines strategies for effective human-computer interaction, specifically in designing the user interface. It begins by introducing the role of human factors in interactive software, and the various physical, cognitive, psychological, and cultural factors that go into designing an effective user interface. The author also describes the difficulties and challenges of designing user interfaces, specifically response time and display rate. Presentation styles and graphics are also discussed. Next, the author examines the theories, principles, and guidelines that govern the design of the user interface. There is also a fairly large section on expert review, usability testing, surveys, and continuing assessments. Overall, this book reviews new technologies and methods of designing virtual environments that impact modern day society.

Simpson, H., West, W., & Gleisner, D. (1995). The use of simulation in military training: value, investment, and potential (DMDC Report No. TR-95-007). Seaside, CA: Defense Manpower Data Center.

A study was conducted to assess capabilities and limitations of simulation for military training; DoD investments, plans, and programs for simulation; and cost-saving potential. Findings were (1) simulation technology is advancing, but faces a number of technical challengies; (2) the Services accept and use live, stand-along single-system, and constructive simulations, and are making increasing use of virtual simulation; (3) cost data on simulation are not reported regularly or consistently but approximate relative levels of investment can be estimated; and (4) the Services have a multitude of simulation programs, with much of their planned technology development work in the areas of virtual simulation and range instrumentation.

Smith, J. F. (1981). Experience with flight simulators – training effectiveness and future developments (AFRL Report No. ARHRL-TP-81-41). Williams AFB, AZ: Air Force Operations Training Division. (DTIC No. ADA108087)

The paper provides a discussion of experience in the use of simulators for pilot training in the United States of America. The use of ground based flight simulators in pilot training programs as alternatives to more expensive training media has been practiced at some level for over 50 years. This paper opens with a brief discussion of the very early experiences of Mr. Ed Link and his 'PILOT MAKER.' Progress is then summarized in three time frames; 1934-1949, 1950-1970 and 1971-1980. Simulator training objectives retain their focus on instruments and procedures to which are added weapon system operations. Trainers are procured for nearly all jet aircraft weapon systems. The final section of the paper provides the author's opinion as to future simulation applications. Three areas are discussed; the first includes comments on how we may improve utilization of existing equipment, the second provides some ideas concerning future simulator training requirements, and the third provides a broad summary of planned and needed research.

Stansfield, S. & Sobel, A. (1998). Creating virtual humans for simulation-based training and planning (Report No. SAND98-0081C). Albuquerque, NM: Sandia National Laboratories. (DTIC No. ADA343236)

Sandia National Laboratories has developed a distributed, high fidelity simulation system for training and planning small team Operations. The system provides an immersive environment populated by virtual objects and humans capable of displaying complex behaviors. The work has focused on developing the behaviors required to carry out complex tasks and decision making under stress. Central to this work are techniques for creating behaviors for virtual humans and for dynamically assigning behaviors to CGF to allow scenarios without fixed outcomes. Two prototype systems have been developed that illustrate these capabilities: MediSim, a trainer for battlefield medics and VRaptor, a system for planning, rehearsing and training assault operations.

This report describes the Lockheed-Martin VET team efforts and accomplishments during the seventh quarter of the contract. Activity is reported for each of the software components of the Training Studio: VRIDES, STEVE, and VISTA, as well as domain development and initial software evaluation plans. This report contains material submitted by Dr. Allen Munro at BTL and Dr. Lewis Johnson at ISI, and Dr. Craig Hall, USAF AL/TRAIN.

Tambe, M., Johnson, W. L, Jones, R. M., Koss, F., Laird, J.E., Rosenbloom, P.S., & Schwamb, K. B. (1995). Intelligent agents for interactive simulation environments. AI Magazine, 16(1), 15-39.

This paper describes a project aimed at developing humanlike, intelligent agents that can interact with each other, as well as with humans, in virtual environments. The current target of the project is intelligent, automated pilots for battle-field simulation environments. These dynamic, interactive, multi-agent environments pose interesting challenges for research on specialized agent capabilities as well as the integration of these capabilities in the development of the complete pilot agents. This article provides an overview of this domain and project by analyzing the challenges that automated pilots face in battlefield simulations.

Tao, Y. H. ,& Guo, S. H. (2001). The design of a web-based training system for simulation analysis. In Proceedings of the 2001 Winter Simulation Conference (pp.645-652). San Diego, CA: Society of Computer Simulation.

In order to help teach students the statistical analysis associated with simulation, a computer-based training (CBT) module was developed to better prepare students for simulation problem solving. CBT's are widely regarded as a unique and effective way of enabling distance learning. The researchers chose college students who are in the process of learning about simulation as the targeted users. The objective of the research was to design a CBT system that provides effective features for beginners to learn simulation analysis. The experimenters found that in order to retain the learners' interests after a long period of time, the training must be designed with brief paragraphs of content, wide spaces between different ideas, the hiding of unnecessary information with hyperlinks, and the use of color to separate different types of information. The CBT should also provide instant access to online help, and allow user feedback throughout the training.

Taylor, W. (1997). Student responses to their immersion in a virtual environment. Retrieved on February 14, 2002 from the World Wide Web: http://www.hitl.washington.edu/publications/r-97-11

Students in grades 4-12 were immersed in virtual environments to determine their degree of presence, enjoyment, and ease of navigation. The students were first given a brief introduction to virtual reality and three-dimensional computer graphics. Each student was then placed in a head-mounted display (HMD) where they visited a virtual world that was considered to be fully immersive. After visiting the virtual environment, students were given questionnaires inquiring into their physical and mental state during and after the simulation. Results indicated that all students experienced enjoyment in the virtual environment, and almost all students reported a feeling of presence, or that they were really interacting in the environment. In terms of navigation abilities, some students found it easier then others to navigate through the virtual world. In addition, some students discovered new objects with ease, while others had more trouble locating objects in the virtual world. Finally, no students reported serious simulator sickness symptoms, indicating that age may be a significant factor in the development of such symptoms.

Towne, D. A. (1995). Learning and instruction in simulation environments. Englewood Cliffs, NJ: Educational Technology Publications.

The objective of this book is to provide a clear view of the potential benefits that simulation-oriented training offers and the development process required to implement the approach. It is written to combine a number of topics including the construction of interactive device models, the support of learning by discover, the acquisition and storage of domain expertise, and the automated presentation of instruction in the context of a working device model. It is hoped that this book allows specialists to formulate a complete instructional program in which simulation techniques address some very difficult training areas.

Weiss, R. J., & Craiger, J. P. (1997). Traveling in cyberspace: computer-based training. Retrieved February 11, 2002 from the World Wide Web: http://www. siop.org/tip/backissues/tipapr97/weiss.htm

Researchers at the University of Nebraska in Omaha examined the advantages and disadvantages of computer-based training (CBT). They also introduce the various forms of computer-based training and the differences that exist between them. In general, CBT is a method by which computers are used to deliver training in a particular knowledge or skill. CBT offers a more personalized environment for the learner, and allows students to learn at their own pace. CBTs vary in sophistication of presentation, amount of interaction, and fidelity of the instruction. They provide numerous advantages to learners and to organizations. Organizational benefits include increased training efficiency, reduced expenses, and improved tracking of learner progress. For the individual, CBTs provide individualized and systematic instruction, active and interactive learning, and immediate feedback. However, there are also disadvantages associated with CBTs. There is difficulty in developing a CBT, for it requires a large investment in time and money. Also, it is hard to maintain a CBT, because it must be constantly updated to ensure it is providing the most current information. Finally, a CBT is considered inflexible because it has to be preprogrammed with anticipated learner responses, and it can only receive input from a few methods of communication, namely the keyboard, mouse, and touchpad.