U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000
Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations
REPORT |
This report is an archived publication and may contain dated technical, contact, and link information |
Publication Number: FHWA-HRT-12-073 Date: December 2012 |
Publication Number: FHWA-HRT-12-073 Date: December 2012 |
PDF files can be viewed with the Acrobat® Reader®
Since 2005, approximately 4,500 fatalities each year have resulted from single-vehicle run-off-road (ROR) crashes along two-lane rural roads in the United States. Crash investigations suggest that many of these crashes involve drivers who are familiar with the roadway but are inattentive and in a hurry. It has been suggested that methods to mitigate ROR crashes involving these factors might be studied in driving simulators, where environmental conditions can be controlled and drivers are not at risk of injury or fatality. The present study addresses how being inattentive or in a hurry affects the performance of drivers who are familiar with a roadway. The efficacy of eliciting these conditions in a driving simulation environment and whether these elicitations affect driving behavior in measureable ways is investigated. The potential for using these driving simulator elicitation methods to evaluate engineering countermeasures to ROR crashes on two-lane rural-roadway horizontal curves is also discussed. The results should be useful to researchers studying factors associated with increased risk for ROR crashes and engineering countermeasures to prevent such crashes.
Monique R. Evans
Director, Office of Safety
Research and Development
Notice
This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The U.S. Government assumes no liability for the use of the information contained in this document. This report does not constitute a standard, specification, or regulation.
The U.S. Government does not endorse products or manufacturers. Trademarks or manufacturers’ names appear in this report only because they are considered essential to the objective of the document.
Quality Assurance Statement
The Federal Highway Administration (FHWA) provides high-quality information to serve Government, industry, and the public in a manner that promotes public understanding. Standards and policies are used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality improvement.
Technical Report Documentation Page
1. Report No. FHWA-HRT-12-073 |
2. Government Accession No. |
3. Recipient's Catalog No. |
||||
4. Title and Subtitle Evaluating Driver Performance on Rural Two-Lane Horizontal Curved Roadways Using a Driving Simulator |
5. Report Date December 2012 |
|||||
6. Performing Organization Code |
||||||
7. Author(s) Mary Anne Bertola, Stacy A. Balk, Jim Shurbutt |
8. Performing Organization Report No. |
|||||
9. Performing Organization Name and Address SAIC 6300 Georgetown Pike McLean, VA 22101 |
10. Work Unit No. (TRAIS) |
|||||
11. Contract or Grant No. DTFH61-08-C-00006 |
||||||
12. Sponsoring Agency Name and Address Office of Safety Research and Development Federal Highway Administration 6300 Georgetown Pike McLean, VA 22101 |
13. Type of Report and Period Covered |
|||||
14. Sponsoring Agency Code |
||||||
15. Supplementary Notes The Contracting Officer's Representative (COR) was C.Y. David Yang, HRDS-30. |
||||||
16. Abstract Between 2005 and 2009, single-vehicle run-off-road (ROR) crashes on two-lane rural-roadway horizontal curves have resulted in an average of 4,748 fatalities per year. Driving while familiar with the roadway, inattentive, and in a hurry have been identified as factors that increase the likelihood of these crashes. The goal of the present study was to evaluate methods that elicit familiarity with the roadway, inattention, and driving in a hurry. Fourteen participants drove a simulated two-lane rural roadway. Familiarity was elicited by having participants drive the test route seven times. Two methods to elicit inattention were used: mental mathematics problems and noun classification. Two methods to elicit being in a hurry were used: with visual travel time feedback and without. The method to elicit familiarity with the roadway resulted in significant changes in behavior when compared to driving while unfamiliar. Results suggested that solving mental mathematics problems was more effective in producing driving behaviors associated with inattention than noun classification. Both methods to elicit being in a hurry were effective in producing noticeable changes in behavior when compared to driving while not under time pressure. The promise of using these elicitation methods in driving simulators to evaluate engineering countermeasures to ROR crashes on two-lane rural-roadway horizontal curves is discussed. |
||||||
17. Key Words Driving simulation, Driver inattention, Time pressure, Driver performance, Curve negotiation, Driver familiarity, Cognitive distraction, Cognitive workload, Run-off-road crash, Rural roads |
18. Distribution Statement No restrictions. This document is available to the public through NTIS: National Technical Information Service Springfield, VA 22161 |
|||||
19. Security Classif. (of this report) Unclassified |
20. Security Classif. (of this page) Unclassified |
21. No. of Pages 37 |
22. Price |
Form DOT F 1700.7 (8-72) Reproduction of completed page authorized
SI (Modern Metric) Conversion Factors
FARS | Fatality Analysis Reporting System |
FHWA | Federal Highway Administration |
GEE | Generalized estimating equations |
ROR | Run-off-road |
NMVCCS | National Motor Vehicle Crash Causation Survey |
DCZ | Data collection zone |
NASA | TLX National Aeronautics and Space Administration Task Load Index |
MAS | Mean average speed |
MSDLP | Mean standard deviation of lane position |
MW | Mean workload |
OR | On-road |
According to the Fatality Analysis Reporting System (FARS), a total of 23,740 fatalities resulted from run-off-road (ROR) crashes on the horizontal curve sections of two-lane rural roadways from 2005 to 2009, an average of 4,748 fatalities per year.(1) After peaking at 5,071 fatalities in 2007, the number decreased to a period low of 4,182 fatalities in 2009. However, there exists the need to continually decrease the number of roadway fatalities.
Liu and Ye utilized the National Motor Vehicle Crash Causation Survey (NMVCCS) to examine fatal and nonfatal collisions involving passenger vehicles to better understand the critical events leading up to ROR crashes.(2) Information about the driver, vehicle, weather, roadway, and environment were collected onsite immediately following crashes. Odds ratios were calculated to further quantify the effect of each precrash event on the outcome of a ROR crash. When determining the odds ratios, ROR crashes were considered one element of the binary outcome and on-road (OR) single-vehicle crashes were considered the other element. In other words, noted odds ratios compared ROR versus OR events and not ROR versus non-crash events. Three of the critical precrash events identified by the NMVCCS as contributing to ROR crashes were inattention, in a hurry, and familiarity with the roadway. Details of the three critical precrash events and their odds ratios are as follows:
The NMVCCS data collection occurred prior to the harmonization efforts under the National Automotive Sampling System. Because of this, the current Federal Highway Administration (FHWA) definition of roadway departure cannot be applied to the NMVCCS data. Specifically, the FHWA definition requires variables published in FARS that are not available in the data collected through NMVCCS.
The goal of the current research was to apply human factors techniques and methodologies to investigate motorists' behavior and performance on two-lane rural-roadway horizontal curves. Variables leading up to and setting the occasion for single-vehicle ROR crashes were researched, as were the critical precrash events identified by Liu and Ye.(2)
Study objectives were as follows:
Multiple studies have investigated the factors associated with ROR crashes. A ROR crash is an incident in which a moving vehicle exits the roadway and collides with natural or artificial objects or overturns on non-traversable terrain. ROR crashes typically involve a single vehicle and can result in serious injuries and fatalities. Around 70 percent of single-vehicle crash fatalities are considered ROR crashes.(3) Based on data from FARS, Liu and Subramanian noted that although the total number of fatal single-vehicle crashes fluctuated from 1991 to 2007, the percentage of single-vehicle crashes classified as ROR has shown an upward trend, from 68 percent in 1991 to 73 percent in 2007 (see figure 1).(3)
Figure 1. Graph. Total passenger vehicle fatal single-vehicle crashes and percent of ROR crashes, 1991-2007.(3)
A variety of factors have been associated with ROR crashes. Driver-related factors such as fatigue, alcohol or drug use, health conditions, gender, age, and non-driving stress; environment-related factors such as roadway surface conditions, roadway alignment, roadway function class, number of lanes, posted speed limit, weather, and lighting conditions; and vehicle-related factors such as vehicle occupancy and vehicle type have all been found to be associated with ROR crashes.(2,3) The currentresearch focuses on evaluation methods of manipulating the three critical precrash events (inattention, in a hurry, and familiarity with the roadway) identified by Liu and Ye, whose findings have been supported by numerous other studies. (See references 2-6.) Other variables showing evidence of being linked to ROR crashes are outside the scope of the study and will not be addressed.
Research specific to inattention, in a hurry, and familiarity with the roadway is discussed in the following sections.
Attention is defined as "the mental energy or resource necessary for completing mental processes, believed to be limited in quantity and under the control of some executive control mechanism."(7) Most activities are governed by the limits of participant attention. For example, learning in the classroom is dependent upon, among other factors, the attention of the students present. Attention allocation can then be defined as how a person prioritizes or distributes attentional resources to a given task. This concept is important to understand because humans have the ability to successfully engage in two non-competing tasks simultaneously (e.g., talking while walking).
One way to understand the relationship between available resources and task demand is by assessing mental workload. Mental workload, as a concept, answers the following questions: "How busy is theoperator? How complex are the tasks? Can any additional tasks be handled above and beyond those that are already performed? Will the operator be able to respond to unexpected events? How does the operator feel about the difficulty of the tasks being performed?"(8) Physiological and subjective methods, among others, exist for assessing mental workload. Two multidimensional subjective methods are the National Aeronautics and Space Administration Task Load Index (NASA TLX) and the Subjective Workload Assessment Technique.(9,10) NASA TLX was used in the current research because it is more sensitive to different workload levels.(11)
As a general definition, inattention can be instances when a person diverts his/her attention away from the task at hand. Driver inattention can further be defined as "diminished attention to activities critical for safe driving in the absence of a competing activity."(12) For Liu and Ye, engagement in cognitive thought was the source of driver inattention.(2) This is what Regan, Hallett, and Gordon regarded as non-driving-related driver diverted attention; the authors assert that this form of inattention "includes the diversion of attention to internalized mental activities (i.e., internalized thoughts and daydreams)."(13) Inattention was operationally defined in this form for the present study. To see how this definition relates to forms of inattention used in previous research, such as internal and external driver distractions, see Regan et al.(13)
Wickens and Hollands proposed the multiple resources theory as a framework for understanding attention allocation when engaging in two tasks simultaneously.(8) This approach models resourceprioritization using three dichotomous dimensions: stages (perception, working memory, and cognition versus responding), perceptual modalities (auditory versus visual), and processing codes (verbal versus spatial). Under this theory, attentional resources are predicted to be more effectively shared between two tasks when the tasks are not in direct competition with one another. For example, the act of a driver listening to a passenger in the vehicle (an auditory and verbal task) is usually not in direct competition with the primary act of driving (a visual and spatial task). Hence, the multiple resources model suggests that the driver would be able to successfully attend to both conversation and driving tasks concurrently.
One way to elicit cognitive thought is to have drivers perform mental mathematics problems. For example, drivers could be tasked with adding two numbers and the difficulty of the task varied by adding either single- or double-digit integers (e.g., 6 + 9 versus 45 + 72).(14) In other settings, researchers could first ask the driver to add two single-digit numbers. Next, a third single-digit number is provided, and the driver is asked to add this value to the original sum. Another single-digit number is then given, and the driver is told to add this value to the second sum (e.g., 2 + 4 = 6, followed by 6 + 5 = 11, followed by 11 + 3 = 14). The process is repeated for the duration of the drive.(15) Mental mathematics problems are useful because it is relatively easy to control the task demand and no specific training is required. Another way to elicit cognitive thought is to have drivers repeat a spoken message in a certain amount of time. This task is also known as "shadowing."(7) For example, the driver could be tasked to repeat a sequence of single-digit numbers read aloud by the researcher.(15) Such tasks place demands on short-term memory, the part of the human memory system that holds information for up to 20 s.(7) Because short-term memory is also loosely tied to attention, it is reasonable to assert that solving mental mathematics problems and shadowing would elicit inattention. Under the multiple resources theory, neither task would be in direct competition with the primary task of driving. Forms of both tasks were used in the current research.
Primary task performance can be analyzed to determine effects of a secondary task. For example, if a participant is solving mathematics problems while driving, then his/her driving data can be reviewed for behavioral changes. A secondary task does not necessarily degrade performance, though. Consider the case when an individual has low arousal while driving, perhaps due to drowsiness. The Yerkes-Dodson Law suggests that execution of a secondary task could increase driver performance if the task increases the driver's level of arousal.(16) Conversely, if the driver was already in a state of high arousal when the secondary task was executed, possibly due to traveling on a curved roadway, then the secondary task could push arousal past the optimum level and performance would decline. Therefore, inclusion of a secondary task aimed at eliciting inattention may yield improved or deteriorated performance dependent on the driver's level of arousal when the secondary task is implemented.
Popular synonyms for being in a hurry are "driving under haste" and "time pressure."(17,18) Such terms describe drivers who may be trying to avoid being late (e.g., for work, school, or a business appointment) or who are pursuing or fleeing. Alternatively, some researchers include being in a hurry as typical aggressive driving behavior.(2) In the current research, participants were assumed to be driving in a hurry if time constraints were supplied by the researcher. It was hoped that time constraints would induce a sense of hurry for the driver.
Wickens and Hollands suggest a closed-loop model for how humans deal with task stress.(8) In this model are two key components: appraisal and strategic choice. First, an individual under stress will evaluate his/her perceived levels of stress (i.e., time pressure for a driver). Next, one of four choices is made: (1) recruit more resources, where the person attempts to accomplish more in less time; (2) remove the stressor, where the person simply tries to eliminate the source of stress; (3) change the goals of the task, where the person can abandon compensatory strategies for simpler, non-compensatory ones; or (4) do nothing. Due to the nature of the present study, choosing the second or third option was not feasible, and an incentive was used in the study to curtail the driver from choosing the fourth option. The first choice (also known as "acceleration") can lead to speed-accuracy trade-offs. Therefore, drivers who were in a hurry were expected to increase speed and show signs of more error-prone driving behaviors, such as increased variability in lane position.
Situational contexts with and without time limits can be provided to elicit being in a hurry. For example, drivers could be told, "Today you woke up late, and you have little time to arrive at work. You should arrive in 3 min and 30 s."(18) To make the time constraint salient, current travel times can be displayed at predetermined intervals (e.g., every 30 s) to remind the driver of his/her time on-task.(19) As an alternative, specific time constraints can be removed by simply telling the driver that he/she is late for a meeting.(20) With these techniques, researchers induce time pressure and restrict the amount of time the driver has to complete a task. In previous studies, there were no rewards if the driver succeeded or penalties if the driver failed to decrease the amount of time to complete the given task.(18-20)
Jenkins, Gupta, Mitra, and Shaw examined the relationship between financial incentives and performance.(21) A meta-analytic review of empirical research showed that financial incentives were related to performance quantity but not performance quality. Performance quantity refers to the amount of product generated by the worker, and performance quality is the caliber of the product. In the current research, performance quantity is represented by the driver's completion time. Correspondingly, performance quality is how realistically the participant drove the route while in a hurry. Thus, using the results from Jenkins et al., incentivizing the participant to drive in a hurry should cause the participant to shorten his/her completion time but not at the expense of driving in an abnormal fashion.(21) It should be noted that the meta-analytic review did not discuss the effect of a disincentive on quality sacrifice due to extreme increases in quantity production. For example, there was no inclusion of empirical research where products were mass produced and quality controlled. The present study, however, does not involve such a situation, so disincentives were not used.
Frisch and Dickinson investigated whether the size of the incentive affected quantity performance.(22)Incentive size ranged from 10 to 100 percent of base pay. Results indicated that the size of the incentive did not affect productivity levels; only the presence of the incentive itself affected productivity. In other words, all incentive levels successfully affected productivity compared to those situations without an incentive. However, the degree to which productivity was affected did not differ among the various incentive sizes. Hence, in the present study, an incentive (regardless of size) should cause the participant to shorten his/her completion time. Moreover, there should be no direct relationship between the size of the incentive and the completion time (i.e., a 100 percent incentive would not cause completion time to decrease more than a 10 percent incentive).
Lotan suggested that driver familiarity is determined by the interaction of personal characteristics with route and trip attributes.(23) For example, a driver may be familiar with a roadway during specific times of day (e.g., morning and evening rush-hour commute) while simultaneously being unfamiliar with the same segment during other times of day (e.g., night hours). Level of familiarity can also vary with other route circumstances, such as roadway conditions.
Multiple definitions exist for characterizing a driver as familiar with a roadway. In some cases, a driver was categorized as familiar with the roadway if he/she traveled the segment at least once a week or at least once a month.(2,24) Familiarity was also defined by distance away from the driver's residence. In particular, town limits of the driver's residence could dictate which areas are consideredfamiliar and unfamiliar.(25) Such a definition has practical value because mileage limits can be used to better quantify levels of familiarity. For example, one author characterized crashes where the driver was classified as speeding, driving too fast for roadway conditions, or failing to maintain proper speed.(5) The author's data was collected from nine counties in Virginia, Pennsylvania, Michigan, and Florida. In all States, more than half of the crashes occurred within 20 mi of the driver's home, with a maximum of 91 percent of the 52 crashes studied in Michigan occurring within that distance. However, the results did not appear to control for level of exposure. To investigate familiarity in a simulated environment, Lotan examined familiarity in an experimental simulation based on real-life road networks.(23) Familiar drivers were those who lived in close proximity to the real-life road networks and unfamiliar drivers were those participants who had never traveled to the real-life area. In the current research, which utilized a driving simulation, all participants were first characterized as unfamiliar because the simulation did not mimic particular roadways.
Lotan attempted to identify the point when unfamiliar drivers became familiar with the road networks used in the experimental simulation.(23) Route choice behavior was studied to determinewhen patterns of the two populations (i.e., unfamiliar and familiar drivers) became similar. This concept could also be applied to other variables to characterize driver performance of the two groups and determine when familiarity had been established. However, Lotan acknowledged that although patterns became similar, there were still distinct differences in behavior among unfamiliar and familiar drivers. Thus, in the current research, a threshold was predetermined to identify when unfamiliar drivers became familiar with the route.
If familiar roadways are defined by the distance from a driver's residence, then it is assumed the driver travels these roadways most often. Given this logic, familiarity can be elicited by repeatedly traveling the same road segments. Repeated travel could occur through multiple runs in one day or multiple runs over multiple days. Martens and Fox elicited familiarity by asking participants to drive the same route 24 times over 5 consecutive days (5 times on the first through fourth days and4 times on the fifth day).(26) Their results indicated that driver behavior changed between days and between runs on the same day. However, the largest behavior changes occurred between runs on the first day. This suggests that although familiarity may be elicited through multiple runs over multiple days, simply having multiple runs on one day may be enough exposure to minimize variance in driver performance (presumably because of increasing familiarity). Hence, familiarity was established in the present study by having participants drive the test route multiple times in one day.
The Yerkes-Dodson Law proposes that a driver's level of arousal will affect driving performance.(16) As the level of arousal increases, the level of performance should improve. Familiarity with the roadway, however, could cause a driver's arousal level to decrease because it has been suggested that familiarity can induce a daydream-like effect.(27) Under this assumption, familiar drivers in the current research may show signs of worsened driving, perhaps due to low levels of arousal. Liu and Ye suggested that changes in driving performance due to familiarity could take the form of increased speed.(2)
The literature review findings suggested that increased speed is one of the most prominent indicators of driving in a hurry. Drivers tend to accelerate more when feeling rushed than when that feeling is absent.(17) Vehicle speed can also be monitored to determine if speed significantly changes on more familiar segments compared to travel on less familiar roadways.(25,26) Similarly, vehicle mean speed can be recorded and analyzed to determine if different cognitive tasks produce different effects on a driver's choice of speed.(15) In addition, driver performance can be captured observationally by monitoring traffic violations and dangerous behaviors.(25) In such cases, results can be compared to determine if violation severity and the number of occasions of dangerous behaviors significantly differ while traveling on familiar roadways, engrossed in cognitive thought, or under time pressure. Drivers under high levels of anger and irritating conditions (possible results of being in a hurry) exhibit larger steering wheel movements. Thus, steering wheel locations or other indicators of steering wheel movements (e.g., standard deviation of lane position) are possible methods of capturing driver performance while in a hurry.(17)
Highway engineers commonly use the equation in figure 2 to describe the relationship between curve radius (Rv) in ft, vehicle velocity (V) in ft/s, gravitational force (g) in ft/s2, side friction (fs), and super elevation (e). The equation in figure 3 describes the relationship between the radius of a curve (R) in ft and the corresponding degree of curvature (D).(28)
Figure 2. Equation. Curve radius.
Figure 3. Equation. Degree of curvature.
As the radius increases so does the ability to maneuver a curve with greater velocity. Conversely, radius has an inverse relationship with degree of curvature so that as the radius increases, the degree of curvature decreases. Thus, one would expect lower velocities with larger degrees of curvature. In the present study, mean vehicle speed across various curve degrees is used to validate data from the driving simulator.
A MiniSim™ driving simulator was used in this study (see figure 4).(29) The driving simulator consisted of three 42-inch plasma displays, a driving buck (car seat, steering wheel, gear shift, and instrument panel), and computers for generating driving scenes and controlling vehicle dynamics. Two side mirrors and a rearview mirror are simulated on the displays. The MiniSim™ runs on the National Advanced Driving Simulator software platform. This apparatus is useful for evaluating driver performance in simple environments. Typical data collected include speed and standard deviation of lane position for up to 20 data collection zones (DCZs). In the current study, data collection began when the simulated vehicle crossed a trigger pad at the start of a DCZ and ended when the vehicle crossed a trigger pad at the conclusion of a DCZ.
Figure 4. Photo. Driving simulator with three 42-inch plasma displays and driving buck.
The test route was a simulated two-lane roadway that was rural in character and had many curved segments. Figure 5 depicts an overhead view of the test route. For the current research, 18 DCZs included curved segments and 2 DCZs included tangent segments. Nine unique curves were repeated in both directions (left and right approach) to complete the set of 18 curved DCZs. A summary of the unique curves is shown in table 1. In figure 5, the curved segments are indicated by lettered boxes that correspond to the curve IDs.
Figure 5. Illustration. Overhead view of route driven by participants.
Table 1. Summary of nine unique curves included in route.
Curve ID | Number of Times in Route | Radius (ft) | Degree of Curvature | Design Speed (mi/h) |
---|---|---|---|---|
A |
2 |
965.64 |
5.93 |
45 |
B |
2 |
396.09 |
14.47 |
30 |
C |
2 |
478.50 |
11.97 |
30 |
D |
2 |
478.50 |
11.97 |
30 |
E |
2 |
787.50 |
7.28 |
40 |
F |
2 |
213.27 |
26.87 |
20 |
G |
2 |
512.52 |
11.18 |
30 |
H |
2 |
1,725.00 |
3.32 |
55 |
I |
2 |
829.71 |
6.91 |
40 |
Throughout the route, the posted speed limit was 55 mi/h. Curves were marked with appropriate advisory speeds. If the route was driven at the posted speed limit, the drive would be completed in approximately 13 min. Three different oncoming car scenarios were used and rotated during data collection. Oncoming vehicles started in pseudorandom locations across the three scenarios and only passed the participant on tangent roadway segments. Figure 6 and figure 7 show one of the curved roadway segments. The discontinuities in the images are due to the simulation being displayed across three screens. The image appeared normal to the driver on the driving simulator.
Figure 6. Screenshot. Example curve before the curve warning sign and advisory speed panel.
Figure 7. Screenshot. Example curve after the curve warning sign and advisory speed panel.
Data collection trigger pads for curved DCZs were placed at the start and end of the respective curve. The focus of the current research was limited to ROR crashes on two-lane curved roadways. Hence, tangent DCZs were strictly used as baseline performance data for simulator validation because speed and lane-keeping performance in simulators is typically different from driving performance in the real world.
Pilot data were collected using employees who were not blind to the study objectives and who participated as part of their normal work schedule. Each employee drove the route 10 times to elicit familiarity with the roadway. Results indicated that travel time stabilized after seven runs (i.e., no significant differences in completion time during runs 7-10). Thus, run 7 was assumed to be the threshold where the variance in driver performance was minimized and drivers could be considered familiar with the route. Methods to elicit inattention and being in a hurry were also tested and modified, as necessary.
Fifteen participants were recruited from the Washington, DC metropolitan area via a secure participant database to complete the study. One participant departed the roadway several times, missing the starting trigger pads for some DCZs. This caused the data in those DCZs to be lost. Of the 14 participants (M = 39.5 years, range = 26-53 years) who provided complete data, 9 were male. All participants held a valid drivers' license and had a visual acuity of at least 20/40, corrected if necessary, as tested by the Freiburg Visual Acuity Landolt C measure.(30) All participants had also successfully completed at least one driving simulator study in the past, although none of the previous studies involved the simulator used in the present study.
Each participant drove two initial practice drives to become acquainted with the simulator. Only city roadways were included in the practice simulation to avoid exposing the participants to the rural roadways used in the test route. In addition to the driver performance data collected from the simulator, a stopwatch was used to determine the completion time for each experimental drive. Participants were directed to "drive as they normally would." No explicit directions to maintain thelane or drive the speed limit were provided because these were variables of interest. The simulator system crashes the vehicle if the user exceeds 100 mi/h. Thus, if the participant neared the 100 mi/h threshold, the researcher issued a warning not to exceed 100 mi/h.
Testing for each participant took place over two days. Seven drives to induce familiarity with the roadway were made on the first day, as were an additional seven drives to test the effects of being familiar. On the second day, each participant first drove a practice drive to become reacquainted with the simulator. Because participants were required to complete the second day of driving within five days, it was decided that they did not need to be refamiliarized with the test route. Either the inattention or in a hurry drives were made on the second day. Eight participants performed the inattention tasks, and six participants received the in a hurry elicitations on the second day. That is, participants were either exposed to the inattention tasks or the in a hurry tasks, not both.
Participants were paid a rate of $40 per hour at the end of each day. On average, each day took approximately 3.5 h. An additional $5 was given as a completion bonus at the end of the second day.
After two practice drives to become acquainted with the simulator, each participant drove the test route seven times on the first day. These seven drives will hereafter be referred to as the unfamiliar drives. Following the unfamiliar drives, participants drove the test route seven more times to complete the familiar drives. These 14 drives were completed on the first day.
On the second day, eight of the participants received two methods to elicit inattention. Each participant drove the test route seven times while each inattention elicitation method was active. Four completed drives using method 1 first, and four completed drives using method 2 first.
Under method 1, the participant was tasked to add 2 two-digit numbers every 10 s. There was a different sequence of number pairs for each drive. Four sequences had numbers between 10 and 50, inclusive, and three sequences had numbers between 10 and 99, inclusive. Only one response was allowed per pair of numbers, and the participant was marked as incorrect if he/she did not provide the correct sum or if he/she did not respond in the allotted time of 10 s. The randomly generated number pairs were prerecorded and played through the sound system of the simulator. Each correct response yielded a $0.05 financial incentive, and participants were made aware of the reward prior to the series of drives. There was no financial penalty for incorrect responses. This elicitation method is hereafter referred to as inattention math.
Under method 2, the participant was provided with a series of nouns and tasked to categorize each noun as either "alive," if the noun was a living thing such as a plant or animal, or "not alive." Nouns were provided at 5-s intervals. When a tone was played, the participant was asked to respond with the number of nouns in the chosen response category in the previously read list. Participants were notified of the chosen response category before the respective drive was initiated. Only one response was allowed per list, and the participant was marked as incorrect if he/she did not provide the correct number of nouns or if he/she did not respond in the allotted time of 5 s following the tone. The number of words preceding the tone varied from 1 to 15. There was a different sequence of nouns for each drive, and the response category was randomized across drives once for all subjects. Each correct response yielded a $0.05 financial incentive, and participants were made aware of the reward prior to the series of drives. No financial penalty was present for incorrect responses. This elicitation method is hereafter referred to as inattention noun.
On the second day, six of the participants received two methods to elicit being in a hurry. Each participant drove the test route seven times with each in a hurry elicitation method. Three completed drives using method 1 first, and three completed drives using method 2 first.
Under method 1, the participant was directed to imagine that he/she had a doctor's appointment and was running late. No other time cues were provided to the participant. This elicitation method is hereafter referred to as in a hurry no time.
Under method 2, the participant was also directed to imagine that he/she had a doctor's appointment and was running late. However, if the participant completed the drive in a specific amount of time, he/she would arrive on time. A timer was placed in front of the driver to show the elapsed time. To vary the degree of time pressure (and presumably the level of stress felt by the driver), the goal completion time varied between a 5, 10, 15, and 20 percent reduction from the average completion time of the familiar drives. For example, if the participant averaged a completion time of 10 min across the seven familiar drives, then a 10 percent reduction would correspond to a goal completion time of 9 min for the in a hurry drive. Goal completion times were randomized across drives once for all subjects. Each completion time shorter than the goal completion time yielded a $4 financial incentive. Conversely, if the drive completion time exceeded the goal completion time, then the financial incentive decreased to values ranging from $2 to nothing, depending on how much slower the participant drove the test route. This elicitation method is hereafter referred to as in a hurry with time.
Three dependent variables were identified for the analysis: (1) mean average speed (MAS), defined as the MAS within a DCZ and measured in mi/h; (2) mean standard deviation of lane position (MSDLP), defined as the MSDLP within a DCZ and measured in ft; and (3) mean workload (MW), defined as the mean perceived workload score calculated by the NASA TLX for a particular drive type and ranging from 0 to 100. MAS, MSDLP, and MW were analyzed using generalized estimating equations (GEE) with a gamma response distribution and log link function.(31) This statistical technique was used because responses were repeated measurements from a non-Gaussian population. Models for MAS and MSDLP included the main effect predictors of curve degree (measured in degrees), curve direction (left or right approach), and drive type (unfamiliar, familiar,inattention math, inattention noun, in a hurry no time, or in a hurry with time). For example, drive type had two levels (unfamiliar and familiar) when analyzing familiarity with the roadway. The model for MW included only the drive type main effect because the NASA TLX was administered once per seven-run period (i.e., once per elicitation method). The assumed significance level was = 0.05. Tukey-Kramer adjustments were used for post-hoc multiple comparisons of significant effects. Because GEE uses quasi-likelihood instead of maximum likelihood, Wald statistics were the chosen alternative to likelihood ratio statistics.(32)
To determine if familiarity with the roadway altered driver performance, data from the unfamiliar and familiar drives were compared.
Results of the Wald Type 3 tests indicated that both drive type (2 (1) = 56.69, p < 0.001) and curve degree (2 (1) = 149.81, p < 0.001) were significant predictors for MAS. Comparing familiar to unfamiliar indicated that familiar drives produced a higher average speed (M = 59.25 mi/h) than unfamiliar drives (M = 52.56 mi/h; see figure 8). For a fixed direction and drive type, the percent decrease in MAS for a 2-degree increase in curve degree was estimated to be 2.74 percent (with 95 percent confidence limits of 2.31 and 3.17 percent). The observed values of MAS ranged from 41.79 mi/h for a curve of 26.87 degrees to 60.45 mi/h for a curve of 5.93 degrees. Figure 9 shows the observed values of MAS for each curve degree in the study.
Figure 8. Graph. MAS by drive type for combined curve degree, familiarity.
Figure 9. Graph. MAS by curve degree for combined drive type, familiarity.
Drive type (2 (1) = 17.20, p < 0.001) and curve degree (2 (1) = 14.68, p < 0.001) were also significant predictors for MSDLP. Comparing familiar to unfamiliar indicated that familiar drives (M = 1.30 ft) resulted in more variability in lane position than unfamiliar drives (M = 1.17 ft; see figure 10). For a fixed direction and drive type, the percent increase in MSDLP for a 2-degree increase in curve degree was estimated to be 2.85 percent (with 95 percent confidence limits of 1.38 and 4.33 percent). Observed values of MSDLP ranged from 0.97 ft for a curve of 7.28 degrees to 1.54 ft for a curve of 26.87 degrees. Figure 11 shows the observed values of MSDLP for each curve degree in the study.
Figure 10. Graph. MSDLP by drive type for combined curve degree, familiarity.
Figure 11. Graph. MSDLP by curve degree for combined drive type, familiarity.
Drive type was not a significant predictor for MW. For unfamiliar and familiar drives, MW was 47.29 and 49.52, respectively.
To determine if inattention altered driver performance, data from the inattention noun, inattention math, and familiar drives were compared.
Results of the Wald Type 3 tests indicated that only curve degree (2 (1) = 113.31, p < 0.001) wasa significant predictor for MAS. For a fixed direction and drive type, the percent decrease in MAS for a 2-degree increase in curve degree was estimated to be 2.74 percent (with 95 percent confidence limits of 2.24 and 3.24 percent). Observed values of MAS ranged from 45.70 mi/h for a curve of 26.87 degrees to 65.56 mi/h for a curve of 5.93 degrees. Figure 12 shows the observed values of MAS for each curve degree in the study.
Figure 12. Graph. MAS by curve degree for combined drive type, inattention.
Drive type (2 (2) = 9.83, p = 0.007) and curve degree (2 (1) = 30.68, p < 0.001) were significant predictors for MSDLP. The contrast comparing the inattention math and inattention noun drives was significant (p = 0.006, adjusted), indicating that the inattention math (M = 1.31 ft) drives yielded a decrease in lane position variability when compared to the inattention noun drives (M = 1.42 ft). There was no significant difference in MSDLP between the inattention math and familiar drives (M = 1.40 ft) or between the inattention noun and familiar drives (see figure 13). Because drive type did not have a significant effect on MAS (see previous paragraph), the effect of drive type on MSDLP is most likely not due to differences in vehicle speed. For a fixed direction and drive type, the increase in MSDLP for a 2-degree increase in curve degree was estimated to be 3.91 percent (with 95 percent confidence limits of 2.51 and 5.33 percent). Observed values of MSDLP ranged from 1.12 ft for a curve of 7.28 degrees to 1.89 ft for a curve of 26.87 degrees. Figure 14 shows the observed values of MSDLP for each curve degree in the study.
Figure 13. Graph. MSDLP by drive type for combined curve degree, inattention.
Figure 14. Graph. MSDLP by curve degree for combined drive type, inattention.
Drive type was a significant predictor for MW (2 (2) = 7.32, p = 0.026). The contrast comparing the familiar and inattention math drives was significant (p = 0.020, adjusted), indicating that the inattention math drives (M = 66.69) yielded an increase in perceived workload when compared to the familiar drives (M = 49.01). There was no significant difference in MW between the inattention noun (M = 52.15) and familiar drives or between the inattention noun and inattention math drives (see figure 15). Although there was a significant difference between the inattention math and inattention noun drives prior to the application of the Tukey-Kramer adjustment, the difference was no longer statistically significant after the adjustment was made.
Figure 15. Graph. MW by drive type for combined curve degree, inattention.
To determine if time pressure altered driver performance, data from the in a hurry no time, in a hurry with time, and familiar drives were compared.
Results of the Wald Type 3 tests indicated that both drive type (2 (2) = 18.77, p < 0.001) and curve degree (2 (1) = 100.59, p < 0.001) were significant predictors for MAS. The contrast comparing familiar to in a hurry no time was significant (p < 0.001, adjusted), showing that in a hurry no time drives (M = 77.23 mi/h) produced a higher average speed than familiar drives (M = 60.29 mi/h). Similarly, in a hurry with time drives (M = 78.40 mi/h) yielded significantly higher average speeds compared to familiar drives (p < 0.001, adjusted). However, there was no significant difference in MAS when comparing in a hurry no time and in a hurry with time drives (see figure 16). For a fixed direction and drive type, the decrease in MAS for a 2-degree increase in curve degree was estimated to be 3.45 percent (with 95 percent confidence limits of 2.79 and 4.11 percent). Observed values of MAS ranged from 50.16 mi/h for a curve of 26.87 degrees to 78.81 mi/h for a curve of 5.93 degrees. Figure 17 shows the observed values of MAS for each curve degree in the study.
Figure 16. Graph. MAS by drive type for combined curve degree, in a hurry.
Figure 17. Graph. MAS by curve degree for combined drive type, in a hurry.
Drive type (2 (2) = 26.44, p < 0.001) and curve degree (2 (1) = 15.80, p < 0.001) were also significant predictors for MSDLP. The contrast comparing familiar to in a hurry no time was significant (p < 0.001, adjusted), showing that in a hurry no time drives (M = 1.64 ft) produced more variability in lane position than familiar drives (M = 1.18 ft). Similarly, in a hurry with time drives (M = 1.66 ft) yielded significantly more lane position variability when compared to familiar drives (p < 0.001, adjusted). However, there was no significant difference in MSDLP between in a hurry no time and in a hurry with time drives (see figure 18). For a fixed direction and drive type, the increase in MSDLP for a 2-degree increase in curve degree was estimated to be 4.26 percent (with 95 percent confidence limits of 2.14 and 6.42 percent). Observed values of MSDLP ranged from 1.09 ft for a curve of 7.28 degrees to 2.30 ft for a curve of 14.47 degrees. Figure 19 shows the observed values of MAS for each curve degree in the study.
Figure 18. Graph. MSDLP by drive type for combined curve degree, in a hurry.
Figure 19. Graph. MSDLP by curve degree for combined drive type, in a hurry.
Drive type was not a significant predictor for MW. For familiar, in a hurry no time, and in a hurry with time drives, MW was 50.17, 47.39, and 51.56, respectively.
Methods to elicit inattention, in a hurry, and familiarity with the roadway were used in an effort to better understand driver behavior and performance while navigating simulated two-lane rural-roadway horizontal curves. The following sections summarize the conclusions for the individual elicitation methods. Limitations of the current research and recommendations for future research are also discussed.
As the degree of curvature increases, one would expect vehicle velocity to decrease. Results from the current research indicated that curve degree had the expected effect on MAS as well as other driver performance variables. These data suggest that a similar pattern of results would be found in the real world.
The following three sections describe the conclusions related to the familiarity, inattention, and in a hurry elicitation methods.
Data from the current study showed that as participants became familiar with the test route, MAS and MSDLP increased. Upon identifying familiarity with the roadway as a critical event preceding ROR crashes, Liu and Ye provided the possible explanation that individuals who are familiar with a particular roadway may be less cautious while driving compared to individuals who are not as familiar with the same roadway.(2) Logic suggests that as drivers become familiar with different roadway segments they become more confident in their ability to successfully navigate the roadway. This increase in confidence could lead to more risky driving behaviors, such as those identified in the present research. Categorization of actions as risky, however, is dependent on the context of the situation. For instance, the researchers observed that as some drivers became more familiar with the test route they tended to use both lanes while traveling through the curved roadways. Although participants were directed to "drive as they normally would," there were no realistic consequences for those instances when drivers drove more riskily. Because no opposing traffic crossed the participant on a curve, elections to take advantage of the entire roadway in the current setting were not particularly dangerous. Additionally, there were no sightline restrictions that might have concealed oncoming vehicles had they been present during a curve.
Another possible explanation for the observed results is based on the Yerkes-Dodson law described at the beginning of this report. If it is true that familiarity decreases levels of arousal in the driver, then the increases in MAS and MSDLP could be signs of either worsening driving performance or attempts to increase arousal. For example, Yanko found familiar drivers had significantly slower reaction times to activate brakes compared to unfamiliar drivers when sudden events (e.g., a car pulling out) occurred in the roadway.(27) Increased speeds shown in the present study suggest that familiar drivers would need to allow more time to brake if a sudden event took place. Had sudden events been included in the current research, it would have been likely that drivers would have performed considerably worse during the familiar drives than the unfamiliar drives because they would have had slower reaction times and been traveling at higher speeds during the familiar drives.
Because drivers' MAS and MSDLP increased after becoming familiar with the test route, there was support that the familiar elicitation method was successful in generating significant changes in driver performance.
Two methods were used to elicit inattention. Although performance on the secondary task was not of primary interest, results were used to calculate payment of financial incentives. Participants answered correctly an average of approximately 78 percent of the time during the inattention math drives, with a minimum of about 63 percent and a maximum of about 91 percent. Similarly, participants answered correctly an average of approximately 79 percent of the time during the inattention noun drives, with a minimum of about 43 percent and a maximum of about 94 percent. Findings such as these suggest that the methods were matched in difficulty.
The NASA TLX was administered to verify whether levels of perceived cognitive workload had indeed been altered by the different drive types. There was no difference in mental workload between the two inattention methods, a result that is supported by the secondary task performance previously described. Inattention math did yield a significant difference in mental workload when compared to familiar; however, the same was not true when inattention noun was compared to familiar. Some questions on the NASA TLX refer to the mental demand of the task as well as the effort and frustration associated with performing the task. In the present study, researchers observed participants voluntarily voicing opinions of the inattention math task being more difficult than the inattention noun task. Given that there was no perceived workload difference between the two methods and no corresponding differences in secondary task performance, one possible explanation for these findings could be that participants simply assumed solving mathematics problems to be more difficult than counting a sequence of nouns (i.e., inattention math task was more mentally demanding or frustrating than the inattention noun task). These views could have been further developed because individuals tend to interact with nouns (e.g., places of work, referring to people with whom they correspond, etc.) on a daily basis more than they interact with mathematics. As such, these assumptions could have influenced their process for completing the NASA TLX. In other words, prior experiences may have had a greater influence on perceived workload than on actual task difficulty. Thus, participants may have perceived the mathematics task to yield a higher level of workload despite the two inattention tasks having similar levels of difficulty.
Drive type also had a significant effect on MSDLP but not on MAS. Inattention math differed significantly from inattention noun on MSDLP, but neither method differed significantly from familiar. Contrary to what some may hypothesize, however, inattention math yielded the smallest variability in lane position and inattention noun yielded the largest; familiar yielded results in the middle of the two inattention methods. Previous research has found that moderate cognitive distractions may actually minimize variability in lateral position.(12) It is feasible that inattention math was a greater cognitive distraction, although still of moderate intensity, compared to inattention noun.
It appeared that inattention math was more successful than inattention noun at eliciting inattention.
Two methods were used to create the in a hurry situation. Completion times were recorded to calculate financial incentive payments at the time of data collection. Goal times were only used for in a hurry with time drive because completion time feedback was provided under this elicitation method. For the in a hurry with time drives, goal times were met 86, 100, 75, and 38 percent of the time when reductions in completion time were 5, 10, 15, and 20 percent, respectively. Varying the time reduction goal enabled approximation of a participant's ceiling for speed. This ceiling may have been due to personal limits of the driver (i.e., confidence in ability to navigate the test route successfully).
Data from the current study showed that as participants were given completion time constraints, MAS and MSDLP increased. However, the inclusion of completion time feedback did not affect either response measure. This suggests that although in some cases a driver may have a specific goal time in mind (i.e., make it to the destination in 10 min), the simple act of the individual being in a hurry is what affects behavior.
According to the model of how humans deal with stress suggested by Wickens and Hollands, individuals under time pressure have four response options.(8) Drivers in the present research tended to select the first option of recruiting more resources through acceleration. This is evident because there were significant increases in MAS for the in a hurry drives. These increases lead one to consider speed-accuracy trade-offs, an outcome asserted by Wickens and Hollands. The observed increases in MSDLP indicated that in a hurry drivers also exhibited more corrections in lane position (either deliberately or not) due to increased speeds.
As noted by Rendon-Velez et al., drivers who are under time pressure tend to accelerate more and exhibit bigger steering wheel movements when compared to their non-pressured counterparts.(17) Because data from the present research reflect these behaviors, both in a hurry no time and in a hurry with time appeared successful in inducing a state of being in a hurry.
Results indicated no effect of curve direction (i.e., left or right approach) on the response variables as a result of familiarity. Removal of duplicated curves would decrease the amount of time required to complete the test route, though perhaps at the expense of how many drives the participant would need to perform to be considered familiar with the route. Deletion of some curves may decrease the amount of time needed for familiarization to occur. As an alternative, performance criteria (e.g., two trials with at least a 10 percent increase in lane position variability) could be used in real time to identify when an individual was considered familiar with the test route. At the time of data collection, though, this alternative was not possible since continuous data were not accessible.
One important aspect of assessing cognitive inattention is to identify potential tradeoffs with safety. This is particularly important because inattention was classified as a critical event preceding ROR crashes.(2) Detection of tradeoffs was not reasonable in the current research due to a number of factors. For instance, because only summary data were collected, it was impossible to precisely track where a participant was in the test route when he/she was responding. Although researchers wrote down observations, notes about driver behavior were recorded on a relatively macro level (i.e., per DCZ and not at a particular point in a DCZ). This is worth noting because participants heard multiple mathematics problems or noun sequences during each DCZ. If the participants exhibited an increase in lane position variability or exited the lane, it was not possible to correctly align a particular problem with the resultant driver behavior. For instance, if a participant crossed the edge line during an inattention noun drive there was no way of knowing (1) if he/she was passively or actively interacting (i.e., listening or responding) with the sequence and (2) the length of the present noun sequence. In the future, it would be helpful to link participant responses with driver performance on a micro level so as to more accurately identify potential safety tradeoffs. This knowledge could be used to create better distraction tasks. If, for example, it is found that more safety tradeoffs occur when the driver is listening than when the driver is responding, then the rate in which nouns are read could be modified and driving performance compared at the different levels to determine if doing so varies the intensity of cognitive distraction.
Under the multiple resources model, neither inattention task (auditory and verbal) was in direct competition with the primary task of driving (visual and spatial).(8) Thus, one would expect participants to successfully execute both tasks (i.e., one inattention task and the driving task) concurrently. Results from the current research supported this.
Liu and Ye provided multiple definitions of drivers who are considered being in a hurry.(2) In the current research, being in a hurry was elicited by instructing participants that they were late for a doctor's appointment. In the real world, though, different motivations for being in a hurry could yield different results. For instance, a driver who is fleeing from law enforcement would likely have more motivation to be in a hurry than a driver who is simply running late for a doctor's appointment. Incentive size could be varied or disincentives could be included in future studies in attempts to examine the various motivations for driving in a hurry.
Given the nature of the summary data, it was not possible to discern the direction of a lane departure (either over the center or edge line). However, researchers attempted to capture this information observationally. Driving misbehaviors were characterized as center line crossing, center line crossing followed by left edge line crossing, center line crossing followed by left edge line crossing followed by spin out, right edge line crossing, and right edge line crossing followed by spin out. Table 2 shows the frequency of each behavior and the respective distribution by drive type (row percentages). There were a total of 7,840 opportunities for driver misbehavior to occur (20 DCZs × 7 runs × 4 drive types × 14 participants). Given the low counts, the occurrence of these behaviors was a rare event. In general, there were more center line crossings than right edge line crossings when combining for all forms of each crossing. Only 14 spin outs were observed in the study.
Table 2. Count and percentage distribution of lane departures across drive type.
Behavior | Count | Distribution by Drive Type | |||||
---|---|---|---|---|---|---|---|
Unfamiliar | Familiar | Inattention Math | Inattention Noun | In a Hurry No Time | In a Hurry With Time | ||
Center line |
106 |
57.55 |
0.94 |
2.83 |
0.94 |
37.74 |
|
Center line + left edge line |
4 |
50.00 |
50.00 |
||||
Center line + left edge line + spin out |
6 |
33.33 |
50.00 |
16.67 |
|||
Right edge line |
71 |
5.63 |
2.82 |
22.54 |
8.45 |
29.58 |
30.99 |
Right edge line + spin out |
8 |
50.00 |
25.00 |
25.00 |
Blank cells indicate zero percent.
Because researchers were monitoring multiple objects in the data collection facility (participant, stop watch, etc.) it is unknown whether all lane crossings and spin outs were observed and recorded correctly. Use of continuous data would allow for correct evaluation of direction of lane departure. If only summary data are available, though, then a small camera or screen capture feature should be used to record the drives and ensure all lane departures and spin outs are observed and documented appropriately.
Using a simulator presents other general issues. For instance, there are no realistic consequences of risky driving behaviors (e.g., no threat of a speeding ticket or wrecking a personal vehicle). Additionally, one generally assumes (and the present data suggest) that driver performance in a simulator follows a similar pattern to that in the real world. However, the exact relationship between simulator and real-world performance is unknown. For example, it is possible that the number of center line crossings in a simulator could simply be due to differences in the handling of the virtual vehicle from the participant's personal vehicle. Also, there is a tendency to speed in driving simulators, and this tendency influences the ability to accurately interpret speeding data.(33) Speeding results in the current study were analyzed in a relative fashion as opposed to an absolute fashion (i.e., recorded speed in relation to posted advisory speed). Furthermore, the simulator used in this experiment has a limited field of view and provided limited proprioceptive feedback. Both are important cues in safe real-world driving behaviors.
The various elicitation methods used in the present study are not simulator-specific. For example, Martens and Fox elicited familiarity in the same manner as the current research.(26) Additionally, previous investigations have used similar methods to elicit both inattention and being in a hurry (19,21). (See references 14, 15, 18, and 20.) Thus, elicitation methods included in the present study could be modified, as necessary, and used as a foundation point for future research in this area. Each method was able to have a significant effect on driver performance (i.e., average speed and standard deviation of lane position). Therefore, it could be possible to use these elicitation methods to test various engineering countermeasures for two-lane rural-roadway horizontal curves when drivers are familiar with the roadway, inattentive, or in a hurry.
1. National Highway Traffic Safety Administration (2011). Fatality Reporting Analysis System, Washington, DC. Accessed online: October 15, 2012. (http://www-fars.nhtsa.dot.gov/ Main/index.aspx)
7. Ashcroft, M.H. (2006). Cognition, Fourth Edition, Pearson Education, Upper Saddle River, NJ.
29. University of Iowa (2010). NADS MiniSim, Iowa City, IA. Accessed online: November 2011. (http://www.nads-sc.uiowa.edu/minisim)