Anthony O'Hagan, Maurice Cox and Liam Wright
School of Mathematics and Statistics, University of Sheffield, UK. and National Physical Laboratory, Teddington, UK
Publication details: Submitted to Journal of Applied Statistics, 2022.
The Welch-Satterthwaite approximation is a simple and well-known approach to the Behrens-Fisher problem of inference about a sum of normal means with unequal variances. It consists of approximating the distribution of the corresponding sum of sample means divided by its estimated standard error by a t distribution, with `effective degrees of freedom' given by the Welch-Satterthwaite formula. This approximation may then be used to construct an approximate confidence interval. However, a paper published by M. Ballico in 2000 shows that the interval can become narrower when one of the variances increases.
Ballico was working and publishing in the field of metrology, where the Welch-Satterthwaite approximation is widely used to construct confidence intervals, but this anomalous behaviour seems to be unremarked in the mainstream statistics literature. We prove that the anomaly can arise whenever one sample size is less than seven. The result has serious implications for metrology, where small sample sizes are common, and we believe it deserves to be more widely known wherever the Welch-Satterthwaite formula is used.