Statistical estimation theory. Statistical inference. Sampling distribution of a statistic. Biased and unbiased estimators. Efficient and inefficient estimates. Point estimates and interval estimates. Confidence interval estimates of population parameters. Probable error.

SolitaryRoad.com

Website owner:  James Miller

[ Home ] [ Up ] [ Info ] [ Mail ]

Statistical inference. An important problem in statistics is to deduce information about a population from samples drawn from it. Conclusions are made about population parameters (such as population mean, standard deviation, etc.) from statistics (mean, standard deviation, etc.) computed for samples of the population. Population parameters, (or briefly parameters), are determined to some level of accuracy from computed sample statistics, (or briefly statistics). The process is called statistical inference.

The statistical inference process utilizes the concept of the sampling distribution of a statistic (such as sampling distribution of the mean, standard deviation, etc) from sampling theory. See Elementary sampling theory. Sampling distribution of a statistic. Sampling distribution of means, standard deviations, proportions, differences and sums.

Def. Sampling distribution of a statistic. Consider all possible samples of size n that can be drawn from a given population (either with or without replacement). For each sample we can compute a statistic, such as the mean, standard deviation, etc. We thus obtain a distribution of the statistic which is called its sampling distribution.

Example. Consider a normal population with a mean μ and variance σ². Assume we repeatedly take samples of a given size from this population and calculate the arithmetic mean for each sample — this statistic is called the sample mean. The distribution of these means is called the “sampling distribution of the sample mean”.

Standard error. The standard deviation of the sampling distribution of a statistic is referred to as the standard error of that quantity.

Thus the sampling distribution of a statistic is the distribution of the statistic for all possible samples of a given sample size from the given population. If, for example the particular statistic used is the sample mean, the distribution is called the sampling distribution of the means or the sampling distribution of the mean. Similarly we could have sampling distributions of standard deviations, variances, medians, proportions, etc.

An important theorem is the following:

Theorem 1. Suppose all possible samples of size n_s are drawn without replacement from a finite population of size n_p where n_p > n_s. Let us denote the mean and standard deviation of the sampling distribution of the mean by and and the population mean and standard deviation by μ_p and σ_p respectively. Then

If the population is infinite or if sampling is with replacement, the above results reduce to

For sample sizes of n ≥30 the sample mean μ_s is a very close approximation to the population mean μ_p and the sample standard deviation σ_s is a very close approximation to the population standard deviation σ_p. In solving problems the population mean μ_p and standard deviation σ_p will generally not be known and the computed values of the sample mean μ_s and standard deviation σ_s are used.

Biased and unbiased estimators. If the mean of the sampling distribution of a statistic equals the corresponding population parameter, the statistic is called an unbiased estimator of the parameter, otherwise it is called a biased estimator. The corresponding values of such statistics are called unbiased or biased estimates respectively.

Example 1. The mean of the sampling distribution of means is given by where μ is the population mean. Thus the sample mean is an unbiased estimate of the population mean μ.

Example 2. The mean of the sampling distribution of variances is given by where σ² is the population variance and n is the sample size. Thus the sample variance s² is a biased estimate of the population variance σ². By use of the modified variance we find so that is an unbiased estimate of σ². However, is a biased estimate of σ.

In the language of expectation we would say that a statistic is unbiased if its expectation equals the corresponding population parameter. Thus and are unbiased since and .

Efficient estimates. If the sampling distribution of two statistics have the same mean (or expectation), the statistic with the smaller variance is called an efficient estimator of the mean while the other statistic is called an inefficient estimator. The corresponding values of the statistics are called efficient or inefficient estimates respectively.

Example. The sampling distribution of the mean and median both have the same mean, namely the population mean. However, the variance of the sampling distribution of means is smaller than for the sampling distribution of medians. See Table 1. Hence the sample mean gives an efficient estimate of the population mean, while the sample median gives an inefficient estimate of it.

Of all statistics estimating the population mean, the sample mean provides the best or most efficient estimate.

Point estimates and interval estimates. Reliability. An estimate of a population parameter given by a single number is called a point estimate of the parameter. An estimate of a population parameter given by two numbers between which the parameter may be considered to lie is called an interval estimate of the parameter.

Interval estimates indicate the precision or accuracy of an estimate and are therefore preferable to point estimates.

Example. If we say that a distance is measured as 5.28 feet, we are giving a point estimate. If, however, we say that the distance is 5.28 + 0.03 feet, i.e. the distance lies between 5.25 and 5.31 feet, we are giving an interval estimate.

A statement of the error or precision of an estimate is often called its reliability.

Confidence interval estimates of population parameters. Let μ_s and σ_s be the mean and standard deviation (standard error) of the sampling distribution of a statistic S. Then if the sampling distribution of S is approximately normal (which is true for many statistics if the sample size n ≥30) we can expect to find an actual sample statistic S lying in the intervals μ_s - σ_s to μ_s + σ_s, μ_s - 2σ_s to μ_s + 2σ_s, orμ_s - 2σ_s to μ_s + 2σ_s about 68.27%, 94.45%, and 99.73% of the time respectively. Some reflection and examination of Fig. 1 will show that the following statement is also true:

We can expect to find, or we can be confident of finding, μ_s in the intervals S - σ_s to S + σ_s, S -

2σ_s to S + 2σ_s, orS - 2σ_s to S + 2σ_s about 68.27%, 94.45%, and 99.73% of the time respectively.

Because of this we call these respective intervals the 68.27%, 94.45%, and 99.73% confidence intervals for estimating μ_s. The end numbers of these intervals (S + σ_s, S + 2σ_s, S + 3σ_s) are then called the 68.27%, 94.45%, and 99.73% confidence limits or, as they are sometimes called, fiducial limits.

Similarly, S + 1.96 σ_s, and S + 2.58 σ_s are 95% and 99% (or .95 and .99) confidence limits for S. The percentage confidence is often called the confidence level. The numbers 1.96, 2.58, etc. in the confidence limits are called confidence coefficients or critical values and are denoted by z_c. From confidence levels we can find confidence coefficients and conversely.

In Table A are given values of z_c corresponding to various confidence levels used in practice. For confidence levels not presented in the table, the values of z_c can be found from the normal curve tables.

Confidence interval estimates for means. If the statistic S is the sample mean , then 95% and 99% confidence limits for estimation of the population mean μ are given by and respectively. More generally, the confidence limits are given by where z_c, which depends on the level of confidence desired, is read from Table A. From Theorem 1 above, the confidence limits for the population mean are given by

if sampling is from an infinite population or if sampling is with replacement from a finite population (where the sample size is n_s) and by

if sampling is without replacement from a population of finite size n_p.

In general the population standard deviation σ_p is unknown so we use the sample standard deviation σ_s which is a very close approximation to the population standard deviation σ_pprovided n ≥30. If n_s < 30 the approximation is poor and small sampling theory must be used.

Example. Problem: Measurements of the diameters of a random sample of 200 ball bearings made by a certain machine during one week showed a mean of 0.824 inches and a standard deviation of 0.042 inches. Find the (a) 95% and (b) 99% confidence limits for the mean diameter of all the ball bearings.

Solution.

(a) The 95% confidence limits are

(b) The 99% confidence limits are

Note that we have assumed the reported standard deviation to be the modified standard deviation . If the standard deviation had been s we would have used which can be taken an for all practical purposes.

Confidence intervals for proportions. If the statistic S is the proportion of “successes” in a sample of size n_s drawn from a binomial population in which p is the proportion of successes (i.e. the probability of success), the confidence limits for p are given by P + z_cσ_p, where P is the proportion of successes in the sample of size n_s. The confidence limits for the population proportion are given by

in case sampling is from an infinite population or is with replacement from a finite population, and by

if sampling is without replacement from a population of finite size n_p. See Table 1.

To compute these confidence limits we can use the sample estimate P for p, which will generally prove satisfactory if n_s ≥30.

Confidence intervals for differences and sums. If S₁ and S₂ are two sample statistics with approximately normal sampling distributions, confidence limits for the differences of the population parameters corresponding to S₁ and S₂ are given by

while confidence limits for the sum of the population parameters are

provided that the samples are independent.

For example, confidence limits for the difference of two population means, in the case where the populations are infinite, are given by

where , σ₁, n₁, , σ₂, n₂ are the respective means, standard deviations and sizes of the two samples drawn from the populations.

Similarly, confidence limits for the difference of two population proportions, where the populations are infinite, are given by

where P₁ and P₂ are the two sample proportions, n₁ and n₂ are the sizes of the two samples drawn from the populations, and p₁ and p₂ are the proportions in the two populations (estimated by P₁ and P₂).

Confidence intervals for standard deviations. The confidence limits for the standard deviation σ of a normally distributed population as estimated from a sample with standard deviation s, are given by

where n_s is the sample size, σ_pis the population standard deviation, and σ_s is the standard error for standard deviations. See Table 1. In computing these confidence limits we use the sample standard deviation (either s or ) as an estimate for σ_p.

Probable error. The 50% confidence limits of the population parameters corresponding to a statistic S are given by S+0.6745σ_s. The quantity 0.6745σ_s is known as the probable error of the estimate.

Much of the above excerpted from Murray R. Spiegel. Statistics. Schaum.

For examples, worked problems, and clarification see Theory and Problems of Statistics by Murray R. Spiegel, Schaum’s Outline Series, Schaum Publishing Co.

References

Murray R Spiegel. Statistics (Schaum Publishing Co.)