Verification of LTPP Virtual Weather Stations Phase I Report: Accuracy and Reliability of Virtual Weather Stations
TASK 3: DETERMINE ACCURACY OF VWS ESTIMATES AND VERIFY THE VWS ALGORITHM
Data Anomalies
Four different data problems were found during the course of this study and were reported to the LTPP program. These problems are related to the raw OWS daily data file, daily VWS calculations in the IMS, and SMP data problems. Following is a description of each data problem.
Duplicate Records in OWS Daily Data
The review of the OWS daily data in the owsdaily.txt file revealed that sometimes the temperature and precipitation data for 2 or more consecutive days were identical (duplicates). Duplicate records were identified when identical data for parameters were repeated for 1 or more days. It was found that fewer than 1 percent of the data were duplicates.
Duplicate records were probably used by NCDC to remedy the missing climatic data for some days. Although duplicate records may not affect the calculation of monthly data estimates, it may affect the daily climatic estimates.
Problems in VWS Data Calculations
The daily VWS data in the IMS were compared to the VWS data that were calculated from the original daily OWS data, and some differences were observed. While the VWS data for most SMP and AWS sections in the study (tables 28 through 31) matched the 1/R2-weight estimated values, the results for some of the sections were different (e.g., AWS site 200100 in table 28). A more detailed study of the problem indicated that the difference was caused by a number of weather stations used in the calculation of the VWS data in CLM_VWS_OWS_LINK. Some weather stations were eliminated from the calculations, but still were counted in the number of contributing weather stations shown in the table.
Table 36 shows the VWS daily maximum temperatures, the corresponding AWS daily data, and the data for the five closest OWS to AWS site ID 200100 (from the owsdaily.txt file). The first row of data (August 7, 1996) shows that the estimated VWS Max_Day_Temp was 33.2 °C and the measured AWS temperature was 26.5 °C. The weighted average of the five closest OWS was 26.72 °C. It was found by trial and error that the data for the closest weather station (KS02164, distance R = 264 m) were eliminated from the VWS calculations, but were still counted as a contributing weather station. Therefore, the Contribute_WS_No is actually only four (not five as reported in the CLM_VWS_DATA_DAILY table).
The maximum temperature for the eliminated weather station actually agreed closely with the measured AWS data (26.5 versus 26.7 °C). The VWS estimate is almost identical to the data for station no. 2 (KS03239, R = 4783 m) because this station is much closer to the AWS site compared to station nos. 3 through 5. Eliminating the closest weather station for this site has resulted in a much higher VWS estimate.
Table 36. VWS, AWS, and OWS maximum temperatures for AWS site 200100.
| Virtual Weather Station |
AWS |
Operating Weather Stations |
VWS_ Date |
MAX_ DAY_TEMP |
CONTRI- BUTE_WS_NO |
MAX_ DAY_TEMP |
(1)KS02164 R=264 m |
(2)KS03239 R=4783 m |
(3)KS01704 R=37771 m |
(4)KS04333 R=37588 m |
(5)KS06549 R=40734 m |
Weight 1/R2 |
| 07-Aug-96 |
33.2 |
5 |
26.5 |
26.7 |
33.3 |
30.0 |
33.9 |
32.2 |
26.72 |
| 08-Aug-96 |
25.7 |
5 |
28.6 |
27.8 |
25.6 |
27.8 |
27.8 |
28.3 |
27.79 |
| 09-Aug-96 |
27.2 |
5 |
24.6 |
23.9 |
27.2 |
25.0 |
30.0 |
27.2 |
23.91 |
| 10-Aug-96 |
26.1 |
5 |
26.8 |
27.2 |
26.1 |
28.3 |
25.0 |
26.7 |
27.20 |
| 11-Aug-96 |
25.7 |
5 |
26.0 |
26.7 |
25.6 |
27.8 |
28.3 |
27.2 |
26.70 |
| 12-Aug-96 |
25.7 |
5 |
24.5 |
24.4 |
25.6 |
27.8 |
27.8 |
25.6 |
24.40 |
| 13-Aug-96 |
24.6 |
5 |
28.7 |
28.3 |
24.4 |
28.3 |
27.2 |
28.3 |
28.29 |
| 14-Aug-96 |
28.4 |
5 |
30.6 |
30.0 |
28.3 |
30.6 |
28.9 |
30.6 |
29.99 |
| 15-Aug-96 |
30.1 |
5 |
30.6 |
28.9 |
30.0 |
30.6 |
31.1 |
32.2 |
28.90 |
| 16-Aug-96 |
30.0 |
5 |
29.3 |
28.3 |
30.0 |
29.4 |
32.2 |
29.4 |
28.31 |
| 17-Aug-96 |
30.5 |
5 |
28.4 |
27.2 |
30.6 |
28.3 |
29.4 |
28.9 |
27.21 |
| 18-Aug-96 |
31.1 |
5 |
32.4 |
32.2 |
31.1 |
32.2 |
28.9 |
32.2 |
32.20 |
| 19-Aug-96 |
31.8 |
5 |
34.5 |
33.3 |
31.7 |
33.9 |
32.8 |
33.9 |
33.30 |
| 20-Aug-96 |
33.8 |
5 |
27.3 |
26.1 |
33.9 |
29.4 |
35.0 |
32.2 |
26.12 |
| 21-Aug-96 |
27.9 |
5 |
31.4 |
30.6 |
27.8 |
31.1 |
28.9 |
32.2 |
30.59 |
| 22-Aug-96 |
31.1 |
5 |
32.1 |
30.6 |
31.1 |
31.7 |
31.7 |
32.8 |
30.60 |
| 23-Aug-96 |
31.1 |
5 |
29.4 |
27.2 |
31.1 |
30.6 |
33.3 |
30.0 |
27.21 |
| 24-Aug-96 |
28.9 |
5 |
29.7 |
28.3 |
28.9 |
28.9 |
30.0 |
28.9 |
28.30 |
| 25-Aug-96 |
29.4 |
5 |
27.6 |
27.8 |
29.4 |
26.7 |
30.6 |
28.3 |
27.80 |
| 26-Aug-96 |
27.8 |
5 |
27.3 |
25.6 |
27.8 |
26.1 |
28.9 |
27.8 |
25.61 |
| 27-Aug-96 |
27.2 |
5 |
27.6 |
27.2 |
27.2 |
28.9 |
26.7 |
27.2 |
27.20 |
| 28-Aug-96 |
27.2 |
5 |
27.8 |
27.8 |
27.2 |
25.6 |
28.9 |
28.3 |
27.80 |
| 29-Aug-96 |
26.8 |
5 |
29.4 |
28.9 |
26.7 |
28.9 |
28.3 |
29.4 |
28.89 |
| 30-Aug-96 |
28.8 |
5 |
26.9 |
23.3 |
28.9 |
27.2 |
28.9 |
28.9 |
23.32 |
| 31-Aug-96 |
26.8 |
5 |
25.6 |
25.0 |
26.7 |
26.1 |
31.1 |
31.7 |
25.01 |
| 01-Sep-96 |
25.6 |
5 |
26.0 |
24.4 |
25.6 |
25.6 |
25.6 |
28.9 |
24.40 |
| 02-Sep-96 |
25.7 |
5 |
28.6 |
28.3 |
25.6 |
28.3 |
26.7 |
29.4 |
28.29 |
The daily VWS temperature data that were calculated from the owsdaily.txt raw data for the AWS and SMP sections did not agree with the VWS data estimates in the IMS table CLM_VWS_DATA_DAILY for about 10 percent of the AWS and SMP sections. The criteria used to eliminate the weather stations in the CLM_VWS_OWS_LINK table are not known. While this problem has affected the daily VWS data, the effect on the monthly averages is not as significant. The stated number of contributing weather stations, however, is sometimes higher than the actual number.
Anomalies in Daily SMP Data
Different problems in daily SMP air temperature and precipitation data (table SMP_ATEMP_ RAIN_DAY) for some SMP sections were found. These problems are listed below:
- Entire Data Suspect: All data for sections 370201, 370205, 370208, and 370212 are either missing or zero. The minimum and maximum temperature times are set to 0001.
- Zero Minimum and Maximum Temperatures: The minimum and maximum temperatures for section 161010 (from 9/29/96 to 1/17/97) are zero. The minimum and maximum temperature times are set to 0001. For the following sections, the minimum and maximum temperature times are both set to midnight, as shown in table 37.
Table 37. Sections with anomalies in daily SMP time data.
| ID |
SMP_DATE |
MAX_ATEMP_TIME |
MIN_ATEMP_TIME |
| 271018 |
11/27/96 |
2400 |
2400 |
| 831801 |
12/10/96 |
2400 |
2400 |
| 493011 |
11/02/94 |
2400 |
2400 |
| 833802 |
11/10/93 |
2400 |
2400 |
| 276251 |
11/04/93 |
2400 |
2400 |
| 276251 |
11/03/95 |
2400 |
2400 |
| 469187 |
11/10/96 |
2400 |
2400 |
- Unreasonable Temperature Change: Temperature differences of more than 20 °C in less than an hour are listed in table 38. Either the air temperature or the time is suspect.
Table 38. Sections with anomalies in daily SMP temperature change data.
| ID |
SMP_DATE |
MAX_AIR _TEMPERATURE |
MIN_AIR _TEMPERATURE |
Temperature Difference |
MAX_ATEMP _TIME |
MIN_ATEMP _TIME |
| 510113 |
11/19/97 |
14.1 |
-6.3 |
20.4 |
0803 |
0826 |
| 561007 |
02/06/94 |
-0.5 |
-23.0 |
22.5 |
0009 |
0000 |
| 271028 |
04/06/97 |
5.5 |
-14.6 |
20.1 |
0101 |
0100 |
| 533813 |
03/28/98 |
12.1 |
-44.5 |
56.6 |
1442 |
1452 |
| 533813 |
04/25/98 |
15.4 |
-36.9 |
52.3 |
1608 |
1555 |
| 274040 |
09/29/96 |
28.0 |
4.1 |
23.9 |
0055 |
0000 |
| 484143 |
09/08/94 |
25.9 |
-15.5 |
41.4 |
0908 |
0941 |
| 276251 |
02/19/94 |
6.8 |
-15.3 |
22.1 |
0005 |
0000 |
| 276251 |
01/14/96 |
-5.5 |
-27.3 |
21.8 |
0001 |
0000 |
- Unreasonable Daily Temperatures: Daily temperature differences of more than 50 °C are listed below in table 39. It appears that the minimum air temperature is unreasonable.
Table 39. Sections with anomalies in daily SMP temperature data.
| ID |
SMP_DATE |
MAX_AIR _TEMPERATURE |
MIN_AIR _TEMPERATURE |
Daily Temperature Difference, °C |
| 533813 |
05/01/98 |
30.4 |
-41.4 |
71.8 |
| 131031 |
05/17/98 |
30.1 |
-36.2 |
66.3 |
| 131031 |
04/20/98 |
21.4 |
-44.5 |
65.9 |
| 533813 |
03/13/98 |
19.0 |
-46.0 |
65.0 |
| 533813 |
05/04/98 |
22.6 |
-39.4 |
62.0 |
| 533813 |
04/19/98 |
16.6 |
-45.2 |
61.8 |
| 484143 |
08/15/94 |
34.3 |
-24.3 |
58.6 |
| 533813 |
03/28/98 |
12.1 |
-44.5 |
56.6 |
| 533813 |
01/01/98 |
9.4 |
-45.1 |
54.5 |
| 533813 |
11/17/97 |
12.6 |
-41.1 |
53.7 |
| 533813 |
04/25/98 |
15.4 |
-36.9 |
52.3 |
| 484143 |
09/04/94 |
33.3 |
-17.3 |
50.6 |
| 484143 |
09/07/94 |
31.3 |
-19.1 |
50.4 |
| 533813 |
12/29/97 |
8.1 |
-42.3 |
50.4 |
Data Inconsistency from Two SMP Installations
The daily climatic data for LTPP sites with more than one simultaneous SMP installation were compared to verify data consistency. A study of the collected data using different instruments in the same location can reveal the data consistency. During this study, collected daily maximum and minimum air temperatures in the SMP_ATEMP_RAIN_DAY table for sections within an SMP site were compared. While most data were within a close range, for some sections, major differences in the temperatures were observed.
Figure 38 shows the maximum air temperatures for SMP site 010101 versus similar temperatures collected at SMP site 010102. Both SPS sections are located within the same site. As can be seen from figure 38, data for some days do not correspond very well.
Figure 39 includes the graph of minimum air temperatures for the same two sections. This figure also shows that SMP collected minimum temperatures for some days do not agree well. The temperature data for temperature differences of 8.5 °C and greater, which correspond to the part of the data with high scatter, are shown in table 40. The first row in table 40 shows a minimum temperature difference of 15.2 °C for data collected on 10/03/98. The difference in the maximum temperature was also significant (11.3 °C).
The same process was repeated for all SMP sites with more than one simultaneous installation, and similar, but less significant, inconsistency was observed for SMP sections 040113 and 040114.

Figure 38. Graph. Maximum air temperature for SMP sections 010101 versus 010102.

Figure 39. Graph. Minimum air temperature for SMP sections 010101 versus 010102.
Table 40. Temperature data for two SMP installations within the same site.
| SMP Section 010101 |
SMP Section 010102 |
Difference |
| SMP_DATE |
MAX_AIR _TEMPER -ATURE |
MIN_AIR _TEMPER -ATURE |
TOTAL_RAIN |
SMP_DATE |
MAX_AIR_ TEMPER -ATURE |
MIN_AIR_ TEMPER -ATURE |
TOTAL_RAIN |
Maximum Temper -ature, °C |
Minimum Temper -ature, °C |
| 10/3/1998 |
6.2 |
-3 |
0 |
10/3/1998 |
17.5 |
12.2 |
0 |
11.3 |
15.2 |
| 15/04/98 |
25.8 |
16.6 |
0.2 |
15/04/98 |
23.4 |
2.4 |
0.1 |
2.4 |
14.2 |
| 6/1/1998 |
19.3 |
17.1 |
21.7 |
6/1/1998 |
21.7 |
2.9 |
20.1 |
2.4 |
14.2 |
| 4/1/1998 |
22.6 |
7.2 |
0 |
4/1/1998 |
16.1 |
-6.6 |
0 |
6.5 |
13.8 |
| 5/1/1998 |
18.7 |
12.7 |
7.8 |
5/1/1998 |
20.7 |
-0.3 |
7.6 |
2 |
13 |
| 6/3/1998 |
16.5 |
9.2 |
10.9 |
6/3/1998 |
17 |
-3.1 |
12.5 |
0.5 |
12.3 |
| 8/4/1998 |
22 |
14.9 |
40.2 |
8/4/1998 |
24.2 |
2.8 |
39.3 |
2.2 |
12.1 |
| 13/01/98 |
14.7 |
10.6 |
2.1 |
13/01/98 |
19.2 |
-1.4 |
2 |
4.5 |
12 |
| 9/1/1998 |
9.1 |
5.5 |
0.4 |
9/1/1998 |
21.3 |
17 |
0.3 |
12.2 |
11.5 |
| 21/11/97 |
20.6 |
11.8 |
18.9 |
21/11/97 |
17.5 |
0.3 |
20.3 |
3.1 |
11.5 |
| 1/11/1997 |
21 |
13.5 |
34.3 |
1/11/1997 |
21.8 |
2.3 |
28 |
0.8 |
11.2 |
| 22/11/97 |
18.4 |
9.2 |
0 |
22/11/97 |
19 |
-2 |
0 |
0.6 |
11.2 |
| 5/4/1998 |
21.4 |
4.4 |
0 |
5/4/1998 |
24.8 |
15.2 |
0 |
3.4 |
10.8 |
| 25/01/98 |
12.5 |
-2.7 |
0 |
25/01/98 |
14.2 |
7.9 |
0 |
1.7 |
10.6 |
| 3/1/1998 |
20.7 |
3.1 |
0 |
3/1/1998 |
9.6 |
-7.5 |
0 |
11.1 |
10.6 |
| 14/03/98 |
18.8 |
5 |
0 |
14/03/98 |
6.7 |
-5.5 |
0 |
12.1 |
10.5 |
| 14/04/98 |
18.4 |
11.9 |
26.4 |
14/04/98 |
23.7 |
1.4 |
27.8 |
5.3 |
10.5 |
| 15/03/98 |
21.8 |
4.1 |
0 |
15/03/98 |
13.4 |
-6.2 |
0 |
8.4 |
10.3 |
| 9/12/1997 |
10.4 |
4.3 |
1.2 |
9/12/1997 |
7.9 |
-5.9 |
1 |
2.5 |
10.2 |
| 22/01/98 |
12.9 |
7.5 |
19.7 |
22/01/98 |
12.1 |
-2.6 |
20.5 |
0.8 |
10.1 |
| 27/11/97 |
20.7 |
7.3 |
0 |
27/11/97 |
15.2 |
-2.6 |
0 |
5.5 |
9.9 |
| 16/11/97 |
8.2 |
-2.9 |
0 |
16/11/97 |
12 |
7 |
0 |
3.8 |
9.9 |
| 22/08/96 |
33.3 |
5.9 |
0 |
22/08/96 |
32.5 |
15.8 |
0 |
0.8 |
9.9 |
| 16/02/98 |
20.5 |
8.4 |
50.6 |
16/02/98 |
15.5 |
-1.4 |
46.8 |
5 |
9.8 |
| 11/2/1998 |
18.6 |
7.4 |
7.9 |
11/2/1998 |
15.6 |
-2.4 |
8.4 |
3 |
9.8 |
| 20/12/97 |
19.3 |
8.3 |
0 |
20/12/97 |
17.7 |
-1.4 |
0 |
1.6 |
9.7 |
| 8/1/1998 |
18.9 |
8 |
1.1 |
8/1/1998 |
19.9 |
17.7 |
1.3 |
1 |
9.7 |
| 23/01/98 |
13.7 |
7.5 |
0.1 |
23/01/98 |
14.5 |
-2.2 |
0.1 |
0.8 |
9.7 |
| 5/3/1998 |
13.5 |
7.4 |
6.5 |
5/3/1998 |
10.6 |
-2.1 |
7 |
2.9 |
9.5 |
| 28/04/98 |
17.5 |
14.7 |
0 |
27/04/98 |
23.6 |
5.4 |
0 |
6.1 |
9.3 |
| 2/2/1998 |
9.7 |
6.5 |
39.8 |
2/2/1998 |
14.8 |
-2.7 |
35.9 |
5.1 |
9.2 |
| 22/12/97 |
15.5 |
9.8 |
8.1 |
22/12/97 |
19.5 |
0.8 |
7.2 |
4 |
9 |
| 9/3/1998 |
12.4 |
2.8 |
0.1 |
9/3/1998 |
15.8 |
11.8 |
0.1 |
3.4 |
9 |
| 28/11/97 |
21.9 |
7.3 |
0 |
28/11/97 |
19.7 |
-1.7 |
0 |
2.2 |
9 |
| 10/12/1997 |
20.3 |
7.3 |
0.5 |
10/12/1997 |
7.3 |
-1.6 |
0.6 |
13 |
8.9 |
| 10/1/1998 |
12.3 |
-0.2 |
0 |
10/1/1998 |
19.5 |
8.4 |
0 |
7.2 |
8.6 |
| 25/03/98 |
24.3 |
8 |
0 |
25/03/98 |
20.1 |
-0.5 |
0 |
4.2 |
8.5 |
| 27/01/98 |
8.6 |
5 |
2.4 |
27/01/98 |
12.2 |
-3.5 |
2.6 |
3.6 |
8.5 |
| 27/02/98 |
21.3 |
11.3 |
0.4 |
27/02/98 |
24 |
2.8 |
0.2 |
2.7 |
8.5 |