Verification of LTPP Virtual Weather Stations Phase I Report: Accuracy and Reliability of Virtual Weather Stations

Task 2: Compare Estimated and Onsite Climatic Data

During this task, climatic estimates in the LTPP database were compared to the measured data. The daily, monthly, and yearly VWS data estimates were compared to the onsite SMP and AWS measured data. The results of the comparisons were presented in various charts and graphs for numerical and visual verification of the accuracy of the VWS estimates. An explanation of the comparisons for each climatic parameter follows.

Minimum and Maximum Temperatures

Daily and monthly temperature estimates (minimum and maximum) were compared to AWS and SMP measured data for the same periods. The results are presented below.

Precision and Bias of Estimate

The VWS estimates in the database were compared to the daily measured data from the AWS and SMP sites. For every SMP and AWS site, the collected daily low (minimum) and high (maximum) air temperatures were compared to the current VWS estimates in table CLM_VWS_DATA_DAILY, which is based on up to five nearby weather stations. Figures through 5 show the estimated versus measured maximum daily temperature for three sample sites. Figure 3 shows a close relationship between the estimated and collected data, while the data shown in figure 4 show more variation between the two. Figure 5 is an example of a relationship that is close, but has a shift in the data.

The mean and standard deviation of the difference between the estimated and measured data (error of estimate) were used for the purpose of numerical comparison of the data. The mean of the error was used as an estimate bias and shows a shift in the data (e.g., figure 5). The standard deviation of the error was used as estimate variation, which relates to the precision of the estimated data (low precision for data in figure 3 and higher precision in figure 4).

Comparing Daily Minimum and Maximum Temperatures

Table 14 includes the mean and standard deviation of error (bias and precision of estimate) for minimum and maximum temperatures for AWS and SMP sites. There are data for 82 sites (24 AWS and 58 SMP sites) in this table. The mean error for the maximum temperature ranged from –1.2 to 6.3 degrees Celsius (°C), with an average of 0.57 °C. The overall mean error was 0.51 °C (for all days of data). The standard deviation of error for the maximum temperature ranged from 0.73 to 5.8 °C, with an average of 2.75 °C per section. The standard deviation for all days of data (overall) was 3.14 °C. The overall value is usually higher than the average per section since it includes the effect of bias, while this effect is removed by averaging standard deviations for the sections. The average mean error per section and the overall mean error, however, are usually comparable.

The mean error for the minimum temperature was between –2.6 and 2.8 °C, with an average of –0.13 °C per section (–0.14 °C overall), and the standard deviation of error was between 1.0 and 7.0 °C, with an average of 2.26 °C per section (2.55 °C overall).

More than one set of climatic instrumentation was installed at some LTPP project sites. For example, the first three sections in table 14 were in the same vicinity since they were all installed at the same SPS–1 site (typically within a 1.6-kilometer (km) stretch). Despite their proximity, some differences in the precision and bias of the estimated value were evident. For example, the mean error for the minimum temperature estimate was between 0 and 1.33 °C, and the standard deviation ranged from 2.06 to 2.41 °C.

Similarly, SMP section numbers 4–8, 15–16, 37–38, 41–42, 47–48, 56–57, and 71–73 are built within the same project site. Data for these sites may be used to determine the variation between onsite measurements and thus the precision and bias of the measured data within a site. This provides a useful reference for comparing onsite and estimated data, and for determining the required accuracy of the estimated data.

Figure 3. Graph. Estimated versus measured maximum temperature for section 331001.

Figure 4. Graph. Estimated versus measured maximum temperature for site 010100.

Figure 5. Graph. Estimated versus measured maximum temperature for site 040100.

The distribution of errors for maximum and minimum temperatures is shown in figures 6 and 7 for AWS sites. Figures 8 and 9 show the same distribution for SMP sites. It shows that errors in these figures are approximately normally distributed.

Figure 6. Bar chart. Error distribution of maximum temperature estimates for AWS sites.

Figure 7. Bar chart. Error distribution of minimum temperature estimates for AWS sites.

Figure 8. Bar chart. Error distribution of maximum temperature estimates for SMP sites.

Figure 9. Bar Chart. Error distribution of minimum temperature estimates for SMP sites.

Comparing Monthly Minimum and Maximum Temperatures

The monthly average maximum and minimum daily temperatures were calculated for all SMP and AWS sites and were compared with the monthly estimates in LTPP database table CLM_VWS_TEMP_MONTH. Table 15 shows the mean and standard deviation of error (estimated minus measured) for all AWS and SMP sites and contains 1120 months of measured and estimated data. Some sections had as few as 1 month of data and some had as many as 32 months.

The mean error for the maximum temperature ranged from –1.2 to 6.26 °C, with an average of 0.58 °C. The overall mean error was 0.51 °C. The highest mean error (bias) belongs to site 040100 (section numbers 4–6). The bias is mainly caused by the high elevation difference between the site and the five closest weather stations and can be somewhat improved.

The standard deviation of error for the maximum temperature for the sections ranged from 0.19 to 1.11 °C, with an average of 0.47 °C. The overall standard deviation for all 1120 months of data was 1.2 °C. The standard deviation for the majority of the sites was less than 0.5 °C.

The mean error (bias) for the minimum temperature ranged from –2.5 to 2.55 °C, with an average mean error of –0.14 °C. The standard deviation of error for the sections ranged from 0.11 to 2.22 °C, with an average of 0.53 °C. The overall standard deviation for all months of data was almost twice as much (1.04 °C).

The average estimated bias (mean error) for the daily and monthly maximum temperatures is very close (0.57 and 0.58 °C, respectively). This was also the case for the minimum temperature (–0.13 and –0.14 °C, respectively). The average standard deviation (used as an estimate of precision) for the monthly maximum temperature estimate was substantially less than the daily estimates (0.47 versus 2.75 °C, respectively). The same is true for the minimum temperature (0.53 versus 2.26 °C, respectively).

Precipitation and Freezing Index

The monthly and yearly precipitation estimates in the LTPP database were compared with the monthly SMP and AWS measured values, and the results are presented below.

Comparing Monthly Precipitation and Freezing Index

Table 16 includes the mean and standard deviation of error for the monthly precipitation and the freezing index (FI) estimates for 82 SMP and AWS sections. There were between 1 and 28 months of data for each section and a total of 1120 months of data. The overall values (mean and standard deviation for all months of data) and the average values per section are highlighted at the end of table 16.

The mean error of the precipitation estimate was between –5.8 and 38 millimeters (mm), with an average of 9.1 mm per section. The overall mean error for all months of data was 8.58 mm, which is not significantly different. The standard deviation of error was between 1.3 and 84.3 mm, and the average standard deviation per section was 18.6 mm. The overall standard deviation (22.3 mm) was higher.

The FI was calculated as the sum of all the mean daily temperatures less than 0 °C and is reported as °C-days. The mean error of the FI ranged from –8.6 to 55.8 °C-days, and the average mean error per section was 1.7 °C-days (overall average was 0.97 °C-days). The standard deviation of error was between zero and 26.8 °C-days, and the average standard deviation per section was 4.7 °C-days (overall standard deviation was 8.4 °C-days).

Comparing Yearly Precipitation and Freezing Index

Table 17 includes the mean and standard deviation of error for yearly total precipitation and the FI estimates for SMP and AWS sites. Only 37 sites had 1 or 2 years of measured and estimated (VWS) data, and 7 sites had 2 years of data.

As shown in table 17, the average of the mean error of the yearly precipitation estimates per section was 99.8 mm (overall mean was 97.5 mm), and the average standard deviation per section was 31.5 mm (overall standard deviation was 78.7 mm).

The average mean error of the FI estimate was about 9.4 °C-days (8.1 °C-days overall), and the average standard deviation per section was 9.5 °C-days. This value was substantially less than the overall standard deviation (around 50 °C-days).

Since precipitation and FI are accumulated values (rather than monthly mean values in the case of temperature), their variation is higher for yearly estimates (since yearly values accumulate the error). For this reason, both the average per section and the overall yearly statistics are significantly higher than the monthly estimates.

Humidity

The daily estimates of maximum and minimum humidity in the LTPP database were compared with the AWS measured humidity to assess the accuracy of the estimated humidity values. Only 10 AWS sections had both measured and estimated humidity data. These data were combined into a database that had data for about 4100 days.

Comparing Daily Maximum and Minimum Humidity

Table 18 includes the mean error and standard deviation for daily maximum and minimum humidity estimates. The average mean error of the maximum humidity per section was –4.8 percent (–4.2 percent overall), and the standard deviation (Std. Dev. column) was 6.1 percent (7.3 percent overall).

The average mean error of the minimum humidity was 2.1 percent (2.5 percent overall), and the standard error was 7.8 percent (8.6 percent overall). The results indicated that the maximum humidity estimates were made slightly more accurately than the minimum humidity estimates. Generally, daily humidity estimates were within 10 percent of the measured data.

Comparing Monthly Maximum and Minimum Humidity

The mean error and standard deviation of the monthly humidity estimates (minimum and maximum) are shown in table 19. This table includes 142 months of data (from 5 to 29 months per site).

The average mean error per section was similar to the daily estimates (–4 percent and 2.6 percent for maximum and minimum humidity, respectively); however, the average standard deviation was less than half of the daily estimates (2.1 percent and 3.4 percent for maximum and minimum humidity, respectively).

Windspeed

The daily and monthly windspeed estimates were compared to the measured AWS data. As with humidity, only 10 AWS sites had both average windspeed estimates (VWS) and measured data. However, only four sites had maximum windspeed estimates and measured data. This is mainly a result of the lack of data for maximum windspeed in the VWS monthly table.

Comparing Daily Average and Maximum Windspeeds

Table 20 includes the mean error and standard deviation for the daily average and maximum windspeeds. There were 4183 days with estimated and measured average windspeed data, but only 1365 days for maximum windspeed (because of lack of estimates).

The mean error of the average daily windspeed estimate was 1.4 kilometers per hour (km/h), and the standard deviation was 1.1 km/h. The mean error of the maximum daily windspeed was 1.5 km/h, and the standard deviation was 2.2 km/h. The average per section values was similar to the overall values.

These data indicate that the daily windspeed estimates were estimated within 1 to 2 km/h of the measured values.

Comparing Monthly Average and Maximum Windspeeds

Table 21 includes the mean error and standard deviation for the monthly average and maximum windspeeds. There are 142 months of estimated and average windspeed data and only 47 months of data for the maximum windspeed.

The mean error of the average and maximum monthly windspeeds was similar to the daily data (1.3 and 1.5 km/h). However, the standard deviations were lower for the monthly data (0.8 and 1.3 km/h). The average per section mean error was similar to the overall error; however, the average standard deviation was less than half of the overall value.

In general, the monthly windspeed estimates were estimated within 1 km/h of the measured values.

Summary of Comparisons

A summary of all of the comparisons performed between the estimated (VWS) and measured (AWS and SMP) LTPP climatic data is presented in table 22. This table includes the per section average for the mean and standard deviation of the error. Depending on the parameter, the results are shown either for daily and monthly or monthly and yearly estimates. The following are some general observations drawn from the comparisons:

• Daily and monthly temperatures were estimated within 2.5 and 0.5 °C, respectively.
• Monthly and yearly precipitation were estimated within 19 and 32 mm, respectively.
• Monthly and yearly FI were estimated within 5 and 10 °C-days.
• Daily and monthly humidity were estimated within 7 and 3 percent.
• Daily and monthly windspeeds were estimated within 2 and 0.5 km/h.