Appendix F

Conceptually, for the t-test in which the sample variances are equal, the equation used to calculate the t-value divides the difference between two means by the pooled standard deviation. The pooled standard deviation is the square root of the pooled variance that is the weighted average of the two variances, using the degrees of freedom for each sample as the weighting factor. (Again, conceptually, this is similar to the Z-equation in which the difference between the mean and a point of interest is expressed in standard deviation units. But because small sample sizes are used, the t-distribution is used.)

To determine the critical t value, t_crit, against which the computed t-value is compared, it is necessary to select the level of significance, a. Again, a value of a = 0.01 is recommended. Next, the critical t-value, t_crit, is obtained from the t-table (see table 36 at the end of this appendix) for the pooled degrees of freedom. The pooled degrees of freedom for the case where the sample variances are assumed equal are (n_c + n_a - 2). If t > t_crit, then decide that the two sets of tests have significantly different means. If t < t_crit, then decide that there is no reason to believe the means are significantly different.

t-test Example Problem 1: Sample Variances Assumed to Be Equal.

Use F-test example problem 1 above in which a contractor has run 12 asphalt content tests and the agency has run 6 tests over the same period of time using the same sampling and testing procedures. Based on their means, is it likely that the tests came from the same population?

Use the t-test for the case of equal variances (determined above in F-test example problem 1) to determine whether or not to assume the mean of the contractor's tests differs from the mean of the agency's tests.

In F-test example problem 1, it was determined that = 0.061 and .

Step 1. Compute the sample mean, , for each set of tests.

(50, 51)

Step 2.Compute the pooled variance, , using the sample variances from above.

Step 3. Compute the t-statistic, t, using the equation for equal variances.

Step 4. Determine the critical t value, t_crit, for the pooled degrees of freedom.

Degrees of freedom = (n_c + n_a - 2) = (12 + 6 - 2) = 16.

From table 36, for a = 0.01 and 16 degrees of freedom, t_crit = 2.921.

Conclusion: Since 2.981 > 2.921, we reject the null hypothesis, and assume that the sample means are not equal. We therefore assume that they came from different populations. We therefore conclude that it is unlikely (but not impossible) that the contractor and agency test results represent the same process. In other words, the agency tests do not verify the contractor tests.

t-test Example Problem 2: Sample Variances Assumed to be Different

The F-test example problem 2 above in which a contractor has run 10 air void tests from cores and the agency has run 5 tests over the same period of time using the same sampling and testing procedure is used. Based on their means, is it likely that the tests came from the same population?

In F-test example problem 2, it was determined that =1.036 and .

Step 1. Compute the mean, , for each set of tests.

(54, 55)

Step 2. Compute the t-statistic, t, using the equation for unequal variances.

Step 3. Determine the critical t value, t_crit, for the effective degrees of freedom, f'.

The calculated value for effective degrees of freedom is rounded to the closest integer in this example. The critical value could also be obtained by interpolation or by truncating to the lowest integer. This equation is an approximation and there is not a universally accepted method for arriving at the effective degrees of freedom. In general, rounding to a smaller value for degrees of freedom gives a larger critical value, thereby making it less likely to reject the null hypothesis of equal means.

Note that the value for effective degrees of freedom is less than would have been used if the variances had been assumed to be equal.

From the t-table, table 36, for a = 0.01 and 5 degrees of freedom, t_crit = 4.032.

Conclusion: Since 0.734 < 4.032, there is no reason to reject the assumption that the means are equal. Therefore, we assume that it is possible (but not certain) that they came from the same population.

Note: The difference in sample means is much greater in this example (7.32 - 6.24 = 1.08) than in the previous example (6.10 - 5.70 = 0.40). However, in the previous example it was concluded that the means were different, while in this example it was not concluded that the means were different. The larger ratio of variance values in this example is the reason that it was not possible to conclude that the means were different.

Computer Programs for the F- test and T- test Calculations

As can be seen from the example problems, the required computations can be quite complex and time consuming. This introduces the possibility of human error.

Using Microsoft Excel.

As noted above, spreadsheet programs such as Microsoft Excel often have built-in functions for conducting both F-tests and t-tests. These tests can be performed by anyone with a basic knowledge regarding how to use spreadsheet functions. Excel has a function for conducting F-tests. Excel can also conduct paired t-tests, as well as two-sample t-tests for the cases of both equal and unequal variances.

To illustrate the use of spreadsheets for conducting F-tests and t-tests, Excel was used to compare the data sets used in Example Problem 1 above. The following paragraphs show the steps necessary in using Excel for these calculations.

The first step is to input the contractor and agency data into two different columns in Excel. The data for this example are shown in figure 48.

The F-test is then conducted before the t-test. This is done by using the Excel function

FTEST(array1,array2)

For the example in figure 48, the contractor data are in array1, and it is input as A2:A13, while the agency data are in array2 and it is input as B2:B7. The function that is entered into cell B15 is therefore =FTEST(A2:A13,B2:B7).

Figure 48. Excel Results for Data from Example Problem 1

The test that is conducted by Excel is a one-sided F-test. The value that is displayed in cell B15 is the probability of getting an F-value as large as the one for these data sets if the two data sets have the same variance. In other words, the lower the probability value returned by this function, the less likely it is that the two sets of data have the same variance. For example, if the level of significance for the test were selected as 0.05, for a one-tailed test you would reject the assumption of equal variances whenever the probability value that is returned by the function is less than 0.05.

To compare the results of function FTEST with the critical values in table 35, which is based on a two-sided F-test and a = 0.01 therefore, you would reject the assumption of equal variances whenever the Excel FTEST function returned a probability value less than 0.005. Figure 48 shows that for the example data a probability value of 0.484 is returned by the FTEST function. Therefore, the conclusion would be to assume that the variances are equal.

Once the results of the F-test are known, the t-test can then be conducted using the Excel function

TTEST(array1,array2,tails,type)

For the example in figure 48, the contractor data are in array1, and it is input as A2:A13, while the agency data are in array2 and it is input as B2:B7. Since a two-tailed is desired, tails is input as 2, and, since from the F-test the variances were assumed to be equal, type is input as 2. The function that is entered into cell B17 is therefore =TTEST(A2:A13,B2:B7,2,2). Figure 48 shows that for the example data a probability value of 0.00986 is returned by the TTEST function. Therefore, at the a = 0.01 level of significance, the conclusion would be to assume that the means are not equal since the probability value is less than 0.01.

Similarly, Excel can be used to perform the F-test and t-test on the data sets from Example Problem 2 above. This is illustrated in figure 49.

Figure 49. Excel Results for Data from Example Problem 2

The results in figure 49 (see cell B13) indicate that the variances are assumed to be not equal. This means that the type input for the TTEST function will be 3, for an unequal variance t-test. The tails input will still be 2 for a two-tailed test. The results in figure 49 (see cell B15) indicate that the means are assumed to be equal since the probability in cell B15 is much greater than the level of significance of a = 0.01.

Using Program DATATEST

Another software program that can be used for performing F-test and t-test comparisons is the FHWA Demonstration Project No. 89 program DATATEST. ⁽¹⁸⁾ This program demonstrates how simply the F-tests and t-tests can be performed with a personal computer. To illustrate this, the DATATEST program was used to compare the data sets used in the example problems above. To illustrate the use of the program, the input and output screens for these examples are presented in the figures beginning on the next page.

DATATEST Screens for the Data from Example Problem 1

The program first asks for the number of values and then allows the user to input the values for the first set of data.

The program then asks for the number of values and then allows the user to input the values for the second set of data.

The program then asks the user to select a level of significance, a.

Finally, the program conducts the F-test and then, based on the F-test results, the appropriate form of the t-test, and displays the results.

The values obtained by the DATATEST program are consistent with those calculated in Example Problem 1 above. The slight difference in the calculated t- value stems from the number of decimal places that are used in the computer's calculation. The results from the DATATEST program, i.e., the variances not assumed different and the means assumed different, are consistent with those from Example Problem 1.

DATATEST Screens for the Data from Example Problem 2

The program first asks for the number of values and then allows the user to input the values for the first set of data.

The program then asks for the number of values and then allows the user to input the values for the second set of data.

The program then asks the user to select a level of significance, a.

Finally, the program conducts the F-test and then, based on the F-test results, the appropriate form of the t-test, and displays the results.

The values obtained by the DATATEST program are consistent with those calculated in Example Problem 2 above. The results from the DATATEST program, i.e., the variances assumed different and the means not assumed different, are consistent with those from Example Problem 2.

¹ NOTE: This is for a two-tailed test with the null and alternate hypotheses shown below:

Table 36. Critical Values,t _crit , for the t-test ¹

¹ NOTE: This is for a two-tailed test with the null and alternate hypotheses shown below: