U.S. Department of Transportation
Federal Highway Administration
1200 New Jersey Avenue, SE
Washington, DC 20590
202-366-4000


Skip to content
Facebook iconYouTube iconTwitter iconFlickr iconLinkedInInstagram

Federal Highway Administration Research and Technology
Coordinating, Developing, and Delivering Highway Transportation Innovations

Report
This report is an archived publication and may contain dated technical, contact, and link information
Publication Number: FHWA-RD-03-092
Date: May 2006

Verification of LTPP Virtual Weather Stations Phase I Report: Accuracy and Reliability of Virtual Weather Stations

Task 3: Determine Accuracy of Vws Estimates and Verify The Vws Algorithm (Cont.)

Data Anomalies

Four different data problems were found during the course of this study and were reported to the LTPP program. These problems are related to the raw OWS daily data file, daily VWS calculations in the IMS, and SMP data problems. Following is a description of each data problem.

Duplicate Records in OWS Daily Data

The review of the OWS daily data in the owsdaily.txt file revealed that sometimes the temperature and precipitation data for 2 or more consecutive days were identical (duplicates). Duplicate records were identified when identical data for parameters were repeated for 1 or more days. It was found that fewer than 1 percent of the data were duplicates.

Duplicate records were probably used by NCDC to remedy the missing climatic data for some days. Although duplicate records may not affect the calculation of monthly data estimates, it may affect the daily climatic estimates.

Problems in VWS Data Calculations

The daily VWS data in the IMS were compared to the VWS data that were calculated from the original daily OWS data, and some differences were observed. While the VWS data for most SMP and AWS sections in the study (tables 28 through 31) matched the 1/R2-weight estimated values, the results for some of the sections were different (e.g., AWS site 200100 in table 28). A more detailed study of the problem indicated that the difference was caused by a number of weather stations used in the calculation of the VWS data in CLM_VWS_OWS_LINK. Some weather stations were eliminated from the calculations, but still were counted in the number of contributing weather stations shown in the table.

Table 36 shows the VWS daily maximum temperatures, the corresponding AWS daily data, and the data for the five closest OWS to AWS site ID 200100 (from the owsdaily.txt file). The first row of data (August 7, 1996) shows that the estimated VWS Max_Day_Temp was 33.2 °C and the measured AWS temperature was 26.5 °C. The weighted average of the five closest OWS was 26.72 °C. It was found by trial and error that the data for the closest weather station (KS02164, distance R = 264 m) were eliminated from the VWS calculations, but were still counted as a contributing weather station. Therefore, the Contribute_WS_No is actually only four (not five as reported in the CLM_VWS_DATA_DAILY table).

The maximum temperature for the eliminated weather station actually agreed closely with the measured AWS data (26.5 versus 26.7 °C). The VWS estimate is almost identical to the data for station no. 2 (KS03239, R = 4783 m) because this station is much closer to the AWS site compared to station nos. 3 through 5. Eliminating the closest weather station for this site has resulted in a much higher VWS estimate.

Table 36. VWS, AWS, and OWS maximum temperatures for AWS site 200100.
Virtual Weather Station AWS Operating Weather Stations
VWS_
Date
MAX_
DAY_TEMP
CONTRI-
BUTE_WS_NO
MAX_
DAY_TEMP
(1)KS02164 R=264 m (2)KS03239 R=4783 m (3)KS01704
R=37771 m
(4)KS04333
R=37588 m
(5)KS06549
R=40734 m
Weight
1/R2
07-Aug-96 33.2 5 26.5 26.7 33.3 30.0 33.9 32.2 26.72
08-Aug-96 25.7 5 28.6 27.8 25.6 27.8 27.8 28.3 27.79
09-Aug-96 27.2 5 24.6 23.9 27.2 25.0 30.0 27.2 23.91
10-Aug-96 26.1 5 26.8 27.2 26.1 28.3 25.0 26.7 27.20
11-Aug-96 25.7 5 26.0 26.7 25.6 27.8 28.3 27.2 26.70
12-Aug-96 25.7 5 24.5 24.4 25.6 27.8 27.8 25.6 24.40
13-Aug-96 24.6 5 28.7 28.3 24.4 28.3 27.2 28.3 28.29
14-Aug-96 28.4 5 30.6 30.0 28.3 30.6 28.9 30.6 29.99
15-Aug-96 30.1 5 30.6 28.9 30.0 30.6 31.1 32.2 28.90
16-Aug-96 30.0 5 29.3 28.3 30.0 29.4 32.2 29.4 28.31
17-Aug-96 30.5 5 28.4 27.2 30.6 28.3 29.4 28.9 27.21
18-Aug-96 31.1 5 32.4 32.2 31.1 32.2 28.9 32.2 32.20
19-Aug-96 31.8 5 34.5 33.3 31.7 33.9 32.8 33.9 33.30
20-Aug-96 33.8 5 27.3 26.1 33.9 29.4 35.0 32.2 26.12
21-Aug-96 27.9 5 31.4 30.6 27.8 31.1 28.9 32.2 30.59
22-Aug-96 31.1 5 32.1 30.6 31.1 31.7 31.7 32.8 30.60
23-Aug-96 31.1 5 29.4 27.2 31.1 30.6 33.3 30.0 27.21
24-Aug-96 28.9 5 29.7 28.3 28.9 28.9 30.0 28.9 28.30
25-Aug-96 29.4 5 27.6 27.8 29.4 26.7 30.6 28.3 27.80
26-Aug-96 27.8 5 27.3 25.6 27.8 26.1 28.9 27.8 25.61
27-Aug-96 27.2 5 27.6 27.2 27.2 28.9 26.7 27.2 27.20
28-Aug-96 27.2 5 27.8 27.8 27.2 25.6 28.9 28.3 27.80
29-Aug-96 26.8 5 29.4 28.9 26.7 28.9 28.3 29.4 28.89
30-Aug-96 28.8 5 26.9 23.3 28.9 27.2 28.9 28.9 23.32
31-Aug-96 26.8 5 25.6 25.0 26.7 26.1 31.1 31.7 25.01
01-Sep-96 25.6 5 26.0 24.4 25.6 25.6 25.6 28.9 24.40
02-Sep-96 25.7 5 28.6 28.3 25.6 28.3 26.7 29.4 28.29

The daily VWS temperature data that were calculated from the owsdaily.txt raw data for the AWS and SMP sections did not agree with the VWS data estimates in the IMS table CLM_VWS_DATA_DAILY for about 10 percent of the AWS and SMP sections. The criteria used to eliminate the weather stations in the CLM_VWS_OWS_LINK table are not known. While this problem has affected the daily VWS data, the effect on the monthly averages is not as significant. The stated number of contributing weather stations, however, is sometimes higher than the actual number.

Anomalies in Daily SMP Data

Different problems in daily SMP air temperature and precipitation data (table SMP_ATEMP_ RAIN_DAY) for some SMP sections were found. These problems are listed below:

  • Entire Data Suspect: All data for sections 370201, 370205, 370208, and 370212 are either missing or zero. The minimum and maximum temperature times are set to 0001.
  • Zero Minimum and Maximum Temperatures: The minimum and maximum temperatures for section 161010 (from 9/29/96 to 1/17/97) are zero. The minimum and maximum temperature times are set to 0001. For the following sections, the minimum and maximum temperature times are both set to midnight, as shown in table 37.

 

Table 37. Sections with anomalies in daily SMP time data.
ID SMP_DATE MAX_ATEMP_TIME MIN_ATEMP_TIME
271018 11/27/96 2400 2400
831801 12/10/96 2400 2400
493011 11/02/94 2400 2400
833802 11/10/93 2400 2400
276251 11/04/93 2400 2400
276251 11/03/95 2400 2400
469187 11/10/96 2400 2400
  • Unreasonable Temperature Change: Temperature differences of more than 20 °C in less than an hour are listed in table 38. Either the air temperature or the time is suspect.

 

Table 38. Sections with anomalies in daily SMP temperature change data.
ID SMP_DATE MAX_AIR
_TEMPERATURE
MIN_AIR
_TEMPERATURE
Temperature
Difference
MAX_ATEMP
_TIME
MIN_ATEMP
_TIME
510113 11/19/97 14.1 -6.3 20.4 0803 0826
561007 02/06/94 -0.5 -23.0 22.5 0009 0000
271028 04/06/97 5.5 -14.6 20.1 0101 0100
533813 03/28/98 12.1 -44.5 56.6 1442 1452
533813 04/25/98 15.4 -36.9 52.3 1608 1555
274040 09/29/96 28.0 4.1 23.9 0055 0000
484143 09/08/94 25.9 -15.5 41.4 0908 0941
276251 02/19/94 6.8 -15.3 22.1 0005 0000
276251 01/14/96 -5.5 -27.3 21.8 0001 0000
  • Unreasonable Daily Temperatures: Daily temperature differences of more than 50 °C are listed below in table 39. It appears that the minimum air temperature is unreasonable.

 

Table 39. Sections with anomalies in daily SMP temperature data.
ID SMP_DATE MAX_AIR
_TEMPERATURE
MIN_AIR
_TEMPERATURE
Daily
Temperature
Difference,
°C
533813 05/01/98 30.4 -41.4 71.8
131031 05/17/98 30.1 -36.2 66.3
131031 04/20/98 21.4 -44.5 65.9
533813 03/13/98 19.0 -46.0 65.0
533813 05/04/98 22.6 -39.4 62.0
533813 04/19/98 16.6 -45.2 61.8
484143 08/15/94 34.3 -24.3 58.6
533813 03/28/98 12.1 -44.5 56.6
533813 01/01/98 9.4 -45.1 54.5
533813 11/17/97 12.6 -41.1 53.7
533813 04/25/98 15.4 -36.9 52.3
484143 09/04/94 33.3 -17.3 50.6
484143 09/07/94 31.3 -19.1 50.4
533813 12/29/97 8.1 -42.3 50.4

Data Inconsistency from Two SMP Installations

The daily climatic data for LTPP sites with more than one simultaneous SMP installation were compared to verify data consistency. A study of the collected data using different instruments in the same location can reveal the data consistency. During this study, collected daily maximum and minimum air temperatures in the SMP_ATEMP_RAIN_DAY table for sections within an SMP site were compared. While most data were within a close range, for some sections, major differences in the temperatures were observed.

Figure 38 shows the maximum air temperatures for SMP site 010101 versus similar temperatures collected at SMP site 010102. Both SPS sections are located within the same site. As can be seen from figure 38, data for some days do not correspond very well.

Figure 39 includes the graph of minimum air temperatures for the same two sections. This figure also shows that SMP collected minimum temperatures for some days do not agree well. The temperature data for temperature differences of 8.5 °C and greater, which correspond to the part of the data with high scatter, are shown in table 40. The first row in table 40 shows a minimum temperature difference of 15.2 °C for data collected on 10/03/98. The difference in the maximum temperature was also significant (11.3 °C).

The same process was repeated for all SMP sites with more than one simultaneous installation, and similar, but less significant, inconsistency was observed for SMP sections 040113 and 040114.

Click to view alternative text

Figure 38. Graph. Maximum air temperature for SMP sections 010101 versus 010102.

Click to view alternative text

Figure 39. Graph. Minimum air temperature for SMP sections 010101 versus 010102.

Table 40. Temperature data for two SMP installations within the same site.
SMP Section 010101 SMP Section 010102 Difference
SMP_DATE MAX_AIR
_TEMPER
-ATURE
MIN_AIR
_TEMPER
-ATURE
TOTAL_RAIN SMP_DATE MAX_AIR_
TEMPER
-ATURE
MIN_AIR_
TEMPER
-ATURE
TOTAL_RAIN Maximum
Temper
-ature, °C
Minimum
Temper
-ature, °C
10/3/1998 6.2 -3 0 10/3/1998 17.5 12.2 0 11.3 15.2
15/04/98 25.8 16.6 0.2 15/04/98 23.4 2.4 0.1 2.4 14.2
6/1/1998 19.3 17.1 21.7 6/1/1998 21.7 2.9 20.1 2.4 14.2
4/1/1998 22.6 7.2 0 4/1/1998 16.1 -6.6 0 6.5 13.8
5/1/1998 18.7 12.7 7.8 5/1/1998 20.7 -0.3 7.6 2 13
6/3/1998 16.5 9.2 10.9 6/3/1998 17 -3.1 12.5 0.5 12.3
8/4/1998 22 14.9 40.2 8/4/1998 24.2 2.8 39.3 2.2 12.1
13/01/98 14.7 10.6 2.1 13/01/98 19.2 -1.4 2 4.5 12
9/1/1998 9.1 5.5 0.4 9/1/1998 21.3 17 0.3 12.2 11.5
21/11/97 20.6 11.8 18.9 21/11/97 17.5 0.3 20.3 3.1 11.5
1/11/1997 21 13.5 34.3 1/11/1997 21.8 2.3 28 0.8 11.2
22/11/97 18.4 9.2 0 22/11/97 19 -2 0 0.6 11.2
5/4/1998 21.4 4.4 0 5/4/1998 24.8 15.2 0 3.4 10.8
25/01/98 12.5 -2.7 0 25/01/98 14.2 7.9 0 1.7 10.6
3/1/1998 20.7 3.1 0 3/1/1998 9.6 -7.5 0 11.1 10.6
14/03/98 18.8 5 0 14/03/98 6.7 -5.5 0 12.1 10.5
14/04/98 18.4 11.9 26.4 14/04/98 23.7 1.4 27.8 5.3 10.5
15/03/98 21.8 4.1 0 15/03/98 13.4 -6.2 0 8.4 10.3
9/12/1997 10.4 4.3 1.2 9/12/1997 7.9 -5.9 1 2.5 10.2
22/01/98 12.9 7.5 19.7 22/01/98 12.1 -2.6 20.5 0.8 10.1
27/11/97 20.7 7.3 0 27/11/97 15.2 -2.6 0 5.5 9.9
16/11/97 8.2 -2.9 0 16/11/97 12 7 0 3.8 9.9
22/08/96 33.3 5.9 0 22/08/96 32.5 15.8 0 0.8 9.9
16/02/98 20.5 8.4 50.6 16/02/98 15.5 -1.4 46.8 5 9.8
11/2/1998 18.6 7.4 7.9 11/2/1998 15.6 -2.4 8.4 3 9.8
20/12/97 19.3 8.3 0 20/12/97 17.7 -1.4 0 1.6 9.7
8/1/1998 18.9 8 1.1 8/1/1998 19.9 17.7 1.3 1 9.7
23/01/98 13.7 7.5 0.1 23/01/98 14.5 -2.2 0.1 0.8 9.7
5/3/1998 13.5 7.4 6.5 5/3/1998 10.6 -2.1 7 2.9 9.5
28/04/98 17.5 14.7 0 27/04/98 23.6 5.4 0 6.1 9.3
2/2/1998 9.7 6.5 39.8 2/2/1998 14.8 -2.7 35.9 5.1 9.2
22/12/97 15.5 9.8 8.1 22/12/97 19.5 0.8 7.2 4 9
9/3/1998 12.4 2.8 0.1 9/3/1998 15.8 11.8 0.1 3.4 9
28/11/97 21.9 7.3 0 28/11/97 19.7 -1.7 0 2.2 9
10/12/1997 20.3 7.3 0.5 10/12/1997 7.3 -1.6 0.6 13 8.9
10/1/1998 12.3 -0.2 0 10/1/1998 19.5 8.4 0 7.2 8.6
25/03/98 24.3 8 0 25/03/98 20.1 -0.5 0 4.2 8.5
27/01/98 8.6 5 2.4 27/01/98 12.2 -3.5 2.6 3.6 8.5
27/02/98 21.3 11.3 0.4 27/02/98 24 2.8 0.2 2.7 8.5
Previous | Table of Contents | Next

 

Federal Highway Administration | 1200 New Jersey Avenue, SE | Washington, DC 20590 | 202-366-4000
Turner-Fairbank Highway Research Center | 6300 Georgetown Pike | McLean, VA | 22101