Verification of LTPP Virtual Weather Stations Phase I Report: Accuracy and Reliability of Virtual Weather Stations
Task 3: Determine Accuracy of Vws
Estimates and Verify The Vws Algorithm (Cont.)
Data Anomalies
Four different data problems were found during the course of
this study and were reported to the LTPP program. These problems
are related to the raw OWS daily data file, daily VWS calculations
in the IMS, and SMP data problems. Following is a description of
each data problem.
Duplicate Records in OWS Daily
Data
The review of the OWS daily data in the owsdaily.txt file
revealed that sometimes the temperature and precipitation data for
2 or more consecutive days were identical (duplicates). Duplicate
records were identified when identical data for parameters were
repeated for 1 or more days. It was found that fewer than 1 percent
of the data were duplicates.
Duplicate records were probably used by NCDC to remedy the
missing climatic data for some days. Although duplicate records may
not affect the calculation of monthly data estimates, it may affect
the daily climatic estimates.
Problems in VWS Data
Calculations
The daily VWS data in the IMS were compared to the VWS data that
were calculated from the original daily OWS data, and some
differences were observed. While the VWS data for most SMP and AWS
sections in the study (tables 28 through 31) matched the
1/R2-weight estimated values, the results for
some of the sections were different (e.g., AWS site 200100 in table
28). A more detailed study of the problem indicated that the
difference was caused by a number of weather stations used in the
calculation of the VWS data in CLM_VWS_OWS_LINK. Some weather
stations were eliminated from the calculations, but still were
counted in the number of contributing weather stations shown in the
table.
Table 36 shows the VWS daily maximum temperatures, the
corresponding AWS daily data, and the data for the five closest OWS
to AWS site ID 200100 (from the owsdaily.txt file). The first row
of data (August 7, 1996) shows that the estimated VWS Max_Day_Temp
was 33.2 °C and the measured AWS temperature was 26.5 °C. The
weighted average of the five closest OWS was 26.72 °C. It was found
by trial and error that the data for the closest weather station
(KS02164, distance R = 264 m) were eliminated
from the VWS calculations, but were still counted as a contributing
weather station. Therefore, the Contribute_WS_No is actually only
four (not five as reported in the CLM_VWS_DATA_DAILY table).
The maximum temperature for the eliminated weather station
actually agreed closely with the measured AWS data (26.5 versus
26.7 °C). The VWS estimate is almost identical to the data for
station no. 2 (KS03239, R = 4783 m) because this station
is much closer to the AWS site compared to station nos. 3 through
5. Eliminating the closest weather station for this site has
resulted in a much higher VWS estimate.
Table 36. VWS, AWS, and OWS maximum temperatures for AWS
site 200100.
Virtual Weather Station |
AWS |
Operating Weather Stations |
VWS_
Date |
MAX_
DAY_TEMP |
CONTRI-
BUTE_WS_NO |
MAX_
DAY_TEMP |
(1)KS02164 R=264 m |
(2)KS03239 R=4783 m |
(3)KS01704
R=37771 m |
(4)KS04333
R=37588 m |
(5)KS06549
R=40734 m |
Weight
1/R2 |
07-Aug-96 |
33.2 |
5 |
26.5 |
26.7 |
33.3 |
30.0 |
33.9 |
32.2 |
26.72 |
08-Aug-96 |
25.7 |
5 |
28.6 |
27.8 |
25.6 |
27.8 |
27.8 |
28.3 |
27.79 |
09-Aug-96 |
27.2 |
5 |
24.6 |
23.9 |
27.2 |
25.0 |
30.0 |
27.2 |
23.91 |
10-Aug-96 |
26.1 |
5 |
26.8 |
27.2 |
26.1 |
28.3 |
25.0 |
26.7 |
27.20 |
11-Aug-96 |
25.7 |
5 |
26.0 |
26.7 |
25.6 |
27.8 |
28.3 |
27.2 |
26.70 |
12-Aug-96 |
25.7 |
5 |
24.5 |
24.4 |
25.6 |
27.8 |
27.8 |
25.6 |
24.40 |
13-Aug-96 |
24.6 |
5 |
28.7 |
28.3 |
24.4 |
28.3 |
27.2 |
28.3 |
28.29 |
14-Aug-96 |
28.4 |
5 |
30.6 |
30.0 |
28.3 |
30.6 |
28.9 |
30.6 |
29.99 |
15-Aug-96 |
30.1 |
5 |
30.6 |
28.9 |
30.0 |
30.6 |
31.1 |
32.2 |
28.90 |
16-Aug-96 |
30.0 |
5 |
29.3 |
28.3 |
30.0 |
29.4 |
32.2 |
29.4 |
28.31 |
17-Aug-96 |
30.5 |
5 |
28.4 |
27.2 |
30.6 |
28.3 |
29.4 |
28.9 |
27.21 |
18-Aug-96 |
31.1 |
5 |
32.4 |
32.2 |
31.1 |
32.2 |
28.9 |
32.2 |
32.20 |
19-Aug-96 |
31.8 |
5 |
34.5 |
33.3 |
31.7 |
33.9 |
32.8 |
33.9 |
33.30 |
20-Aug-96 |
33.8 |
5 |
27.3 |
26.1 |
33.9 |
29.4 |
35.0 |
32.2 |
26.12 |
21-Aug-96 |
27.9 |
5 |
31.4 |
30.6 |
27.8 |
31.1 |
28.9 |
32.2 |
30.59 |
22-Aug-96 |
31.1 |
5 |
32.1 |
30.6 |
31.1 |
31.7 |
31.7 |
32.8 |
30.60 |
23-Aug-96 |
31.1 |
5 |
29.4 |
27.2 |
31.1 |
30.6 |
33.3 |
30.0 |
27.21 |
24-Aug-96 |
28.9 |
5 |
29.7 |
28.3 |
28.9 |
28.9 |
30.0 |
28.9 |
28.30 |
25-Aug-96 |
29.4 |
5 |
27.6 |
27.8 |
29.4 |
26.7 |
30.6 |
28.3 |
27.80 |
26-Aug-96 |
27.8 |
5 |
27.3 |
25.6 |
27.8 |
26.1 |
28.9 |
27.8 |
25.61 |
27-Aug-96 |
27.2 |
5 |
27.6 |
27.2 |
27.2 |
28.9 |
26.7 |
27.2 |
27.20 |
28-Aug-96 |
27.2 |
5 |
27.8 |
27.8 |
27.2 |
25.6 |
28.9 |
28.3 |
27.80 |
29-Aug-96 |
26.8 |
5 |
29.4 |
28.9 |
26.7 |
28.9 |
28.3 |
29.4 |
28.89 |
30-Aug-96 |
28.8 |
5 |
26.9 |
23.3 |
28.9 |
27.2 |
28.9 |
28.9 |
23.32 |
31-Aug-96 |
26.8 |
5 |
25.6 |
25.0 |
26.7 |
26.1 |
31.1 |
31.7 |
25.01 |
01-Sep-96 |
25.6 |
5 |
26.0 |
24.4 |
25.6 |
25.6 |
25.6 |
28.9 |
24.40 |
02-Sep-96 |
25.7 |
5 |
28.6 |
28.3 |
25.6 |
28.3 |
26.7 |
29.4 |
28.29 |
The daily VWS temperature data that were calculated from the
owsdaily.txt raw data for the AWS and SMP sections did not agree
with the VWS data estimates in the IMS table CLM_VWS_DATA_DAILY for
about 10 percent of the AWS and SMP sections. The criteria used to
eliminate the weather stations in the CLM_VWS_OWS_LINK table are
not known. While this problem has affected the daily VWS data, the
effect on the monthly averages is not as significant. The stated
number of contributing weather stations, however, is sometimes
higher than the actual number.
Anomalies in Daily SMP Data
Different problems in daily SMP air temperature and
precipitation data (table SMP_ATEMP_ RAIN_DAY) for some SMP
sections were found. These problems are listed below:
- Entire Data Suspect: All data for sections
370201, 370205, 370208, and 370212 are either missing or zero. The
minimum and maximum temperature times are set to 0001.
- Zero Minimum and Maximum Temperatures: The
minimum and maximum temperatures for section 161010 (from 9/29/96
to 1/17/97) are zero. The minimum and maximum temperature times are
set to 0001. For the following sections, the minimum and maximum
temperature times are both set to midnight, as shown in table
37.
Table 37. Sections with anomalies in daily SMP time
data.
ID |
SMP_DATE |
MAX_ATEMP_TIME |
MIN_ATEMP_TIME |
271018 |
11/27/96 |
2400 |
2400 |
831801 |
12/10/96 |
2400 |
2400 |
493011 |
11/02/94 |
2400 |
2400 |
833802 |
11/10/93 |
2400 |
2400 |
276251 |
11/04/93 |
2400 |
2400 |
276251 |
11/03/95 |
2400 |
2400 |
469187 |
11/10/96 |
2400 |
2400 |
- Unreasonable Temperature Change: Temperature
differences of more than 20 °C in less than an hour are listed in
table 38. Either the air temperature or the time is suspect.
Table 38. Sections with anomalies in daily SMP temperature
change data.
ID |
SMP_DATE |
MAX_AIR
_TEMPERATURE |
MIN_AIR
_TEMPERATURE |
Temperature
Difference |
MAX_ATEMP
_TIME |
MIN_ATEMP
_TIME |
510113 |
11/19/97 |
14.1 |
-6.3 |
20.4 |
0803 |
0826 |
561007 |
02/06/94 |
-0.5 |
-23.0 |
22.5 |
0009 |
0000 |
271028 |
04/06/97 |
5.5 |
-14.6 |
20.1 |
0101 |
0100 |
533813 |
03/28/98 |
12.1 |
-44.5 |
56.6 |
1442 |
1452 |
533813 |
04/25/98 |
15.4 |
-36.9 |
52.3 |
1608 |
1555 |
274040 |
09/29/96 |
28.0 |
4.1 |
23.9 |
0055 |
0000 |
484143 |
09/08/94 |
25.9 |
-15.5 |
41.4 |
0908 |
0941 |
276251 |
02/19/94 |
6.8 |
-15.3 |
22.1 |
0005 |
0000 |
276251 |
01/14/96 |
-5.5 |
-27.3 |
21.8 |
0001 |
0000 |
- Unreasonable Daily Temperatures: Daily
temperature differences of more than 50 °C are listed below in
table 39. It appears that the minimum air temperature is
unreasonable.
Table 39. Sections with anomalies in daily SMP temperature
data.
ID |
SMP_DATE |
MAX_AIR
_TEMPERATURE |
MIN_AIR
_TEMPERATURE |
Daily
Temperature
Difference,
°C |
533813 |
05/01/98 |
30.4 |
-41.4 |
71.8 |
131031 |
05/17/98 |
30.1 |
-36.2 |
66.3 |
131031 |
04/20/98 |
21.4 |
-44.5 |
65.9 |
533813 |
03/13/98 |
19.0 |
-46.0 |
65.0 |
533813 |
05/04/98 |
22.6 |
-39.4 |
62.0 |
533813 |
04/19/98 |
16.6 |
-45.2 |
61.8 |
484143 |
08/15/94 |
34.3 |
-24.3 |
58.6 |
533813 |
03/28/98 |
12.1 |
-44.5 |
56.6 |
533813 |
01/01/98 |
9.4 |
-45.1 |
54.5 |
533813 |
11/17/97 |
12.6 |
-41.1 |
53.7 |
533813 |
04/25/98 |
15.4 |
-36.9 |
52.3 |
484143 |
09/04/94 |
33.3 |
-17.3 |
50.6 |
484143 |
09/07/94 |
31.3 |
-19.1 |
50.4 |
533813 |
12/29/97 |
8.1 |
-42.3 |
50.4 |
Data Inconsistency from Two SMP
Installations
The daily climatic data for LTPP sites with more than one
simultaneous SMP installation were compared to verify data
consistency. A study of the collected data using different
instruments in the same location can reveal the data consistency.
During this study, collected daily maximum and minimum air
temperatures in the SMP_ATEMP_RAIN_DAY table for sections within an
SMP site were compared. While most data were within a close range,
for some sections, major differences in the temperatures were
observed.
Figure 38 shows the maximum air temperatures for SMP site 010101
versus similar temperatures collected at SMP site 010102. Both SPS
sections are located within the same site. As can be seen from
figure 38, data for some days do not correspond very well.
Figure 39 includes the graph of minimum air temperatures for the
same two sections. This figure also shows that SMP collected
minimum temperatures for some days do not agree well. The
temperature data for temperature differences of 8.5 °C and greater,
which correspond to the part of the data with high scatter, are
shown in table 40. The first row in table 40 shows a minimum
temperature difference of 15.2 °C for data collected on 10/03/98.
The difference in the maximum temperature was also significant
(11.3 °C).
The same process was repeated for all SMP sites with more than
one simultaneous installation, and similar, but less significant,
inconsistency was observed for SMP sections 040113 and 040114.
Figure 38. Graph. Maximum air temperature
for SMP sections 010101 versus 010102.
Figure 39. Graph. Minimum air temperature
for SMP sections 010101 versus 010102.
Table 40. Temperature data for two SMP installations
within the same site.
SMP Section 010101 |
SMP Section 010102 |
Difference |
SMP_DATE |
MAX_AIR
_TEMPER
-ATURE |
MIN_AIR
_TEMPER
-ATURE |
TOTAL_RAIN |
SMP_DATE |
MAX_AIR_
TEMPER
-ATURE |
MIN_AIR_
TEMPER
-ATURE |
TOTAL_RAIN |
Maximum
Temper
-ature, °C |
Minimum
Temper
-ature, °C |
10/3/1998 |
6.2 |
-3 |
0 |
10/3/1998 |
17.5 |
12.2 |
0 |
11.3 |
15.2 |
15/04/98 |
25.8 |
16.6 |
0.2 |
15/04/98 |
23.4 |
2.4 |
0.1 |
2.4 |
14.2 |
6/1/1998 |
19.3 |
17.1 |
21.7 |
6/1/1998 |
21.7 |
2.9 |
20.1 |
2.4 |
14.2 |
4/1/1998 |
22.6 |
7.2 |
0 |
4/1/1998 |
16.1 |
-6.6 |
0 |
6.5 |
13.8 |
5/1/1998 |
18.7 |
12.7 |
7.8 |
5/1/1998 |
20.7 |
-0.3 |
7.6 |
2 |
13 |
6/3/1998 |
16.5 |
9.2 |
10.9 |
6/3/1998 |
17 |
-3.1 |
12.5 |
0.5 |
12.3 |
8/4/1998 |
22 |
14.9 |
40.2 |
8/4/1998 |
24.2 |
2.8 |
39.3 |
2.2 |
12.1 |
13/01/98 |
14.7 |
10.6 |
2.1 |
13/01/98 |
19.2 |
-1.4 |
2 |
4.5 |
12 |
9/1/1998 |
9.1 |
5.5 |
0.4 |
9/1/1998 |
21.3 |
17 |
0.3 |
12.2 |
11.5 |
21/11/97 |
20.6 |
11.8 |
18.9 |
21/11/97 |
17.5 |
0.3 |
20.3 |
3.1 |
11.5 |
1/11/1997 |
21 |
13.5 |
34.3 |
1/11/1997 |
21.8 |
2.3 |
28 |
0.8 |
11.2 |
22/11/97 |
18.4 |
9.2 |
0 |
22/11/97 |
19 |
-2 |
0 |
0.6 |
11.2 |
5/4/1998 |
21.4 |
4.4 |
0 |
5/4/1998 |
24.8 |
15.2 |
0 |
3.4 |
10.8 |
25/01/98 |
12.5 |
-2.7 |
0 |
25/01/98 |
14.2 |
7.9 |
0 |
1.7 |
10.6 |
3/1/1998 |
20.7 |
3.1 |
0 |
3/1/1998 |
9.6 |
-7.5 |
0 |
11.1 |
10.6 |
14/03/98 |
18.8 |
5 |
0 |
14/03/98 |
6.7 |
-5.5 |
0 |
12.1 |
10.5 |
14/04/98 |
18.4 |
11.9 |
26.4 |
14/04/98 |
23.7 |
1.4 |
27.8 |
5.3 |
10.5 |
15/03/98 |
21.8 |
4.1 |
0 |
15/03/98 |
13.4 |
-6.2 |
0 |
8.4 |
10.3 |
9/12/1997 |
10.4 |
4.3 |
1.2 |
9/12/1997 |
7.9 |
-5.9 |
1 |
2.5 |
10.2 |
22/01/98 |
12.9 |
7.5 |
19.7 |
22/01/98 |
12.1 |
-2.6 |
20.5 |
0.8 |
10.1 |
27/11/97 |
20.7 |
7.3 |
0 |
27/11/97 |
15.2 |
-2.6 |
0 |
5.5 |
9.9 |
16/11/97 |
8.2 |
-2.9 |
0 |
16/11/97 |
12 |
7 |
0 |
3.8 |
9.9 |
22/08/96 |
33.3 |
5.9 |
0 |
22/08/96 |
32.5 |
15.8 |
0 |
0.8 |
9.9 |
16/02/98 |
20.5 |
8.4 |
50.6 |
16/02/98 |
15.5 |
-1.4 |
46.8 |
5 |
9.8 |
11/2/1998 |
18.6 |
7.4 |
7.9 |
11/2/1998 |
15.6 |
-2.4 |
8.4 |
3 |
9.8 |
20/12/97 |
19.3 |
8.3 |
0 |
20/12/97 |
17.7 |
-1.4 |
0 |
1.6 |
9.7 |
8/1/1998 |
18.9 |
8 |
1.1 |
8/1/1998 |
19.9 |
17.7 |
1.3 |
1 |
9.7 |
23/01/98 |
13.7 |
7.5 |
0.1 |
23/01/98 |
14.5 |
-2.2 |
0.1 |
0.8 |
9.7 |
5/3/1998 |
13.5 |
7.4 |
6.5 |
5/3/1998 |
10.6 |
-2.1 |
7 |
2.9 |
9.5 |
28/04/98 |
17.5 |
14.7 |
0 |
27/04/98 |
23.6 |
5.4 |
0 |
6.1 |
9.3 |
2/2/1998 |
9.7 |
6.5 |
39.8 |
2/2/1998 |
14.8 |
-2.7 |
35.9 |
5.1 |
9.2 |
22/12/97 |
15.5 |
9.8 |
8.1 |
22/12/97 |
19.5 |
0.8 |
7.2 |
4 |
9 |
9/3/1998 |
12.4 |
2.8 |
0.1 |
9/3/1998 |
15.8 |
11.8 |
0.1 |
3.4 |
9 |
28/11/97 |
21.9 |
7.3 |
0 |
28/11/97 |
19.7 |
-1.7 |
0 |
2.2 |
9 |
10/12/1997 |
20.3 |
7.3 |
0.5 |
10/12/1997 |
7.3 |
-1.6 |
0.6 |
13 |
8.9 |
10/1/1998 |
12.3 |
-0.2 |
0 |
10/1/1998 |
19.5 |
8.4 |
0 |
7.2 |
8.6 |
25/03/98 |
24.3 |
8 |
0 |
25/03/98 |
20.1 |
-0.5 |
0 |
4.2 |
8.5 |
27/01/98 |
8.6 |
5 |
2.4 |
27/01/98 |
12.2 |
-3.5 |
2.6 |
3.6 |
8.5 |
27/02/98 |
21.3 |
11.3 |
0.4 |
27/02/98 |
24 |
2.8 |
0.2 |
2.7 |
8.5 |