In this chapter, we will learn how to test if there are differences between population means. We are really asking, are there differences between the two groups? For example:

• Is there a difference in sales between two franchise locations?
• Does a new medication help reduce patients’ cholesterol levels?
• Did a change in production increase your production levels?

Three Difference Tests

We perform different types of difference tests, depending:

We will step through when to use each test and what the differences are between them below.

How to Perform the Tests?

• We will step through examples of each type of test in the next sections (after this section).
• We will use Excel’s Data Analysis Toolpak to do all of the required calculations.
• The formulas presented in the sections below are just for your reference.

Matched-Pairs

We used the matched-pair differences t-test when we have two related populations:

• Before and after studies on the same person or object.
• Twins, siblings or spouses are matched and one of each is placed in each group.
• People who have been paired together from each group based on certain attributes. This is also a called a type of blocking.

Why is Matched-Pair Best?

This type of test is preferable:

• If there is a ‘match’ between groups.
• This reduces the variability between the two groups
• Which, in effect, helps to better isolate the variability due to an effect you are looking to study.

Example of a Matched Pair Test

• You want to test if a new medication has a helpful effect
• You pair people by similar age, health status, genetic makeup, etc..
• One group receives the new medication
• The other group either receives a placebo or another type of medication (depending what you trying study and the health concern)
• This reduces the variability between groups due to age, health, etc
• And, in effect, then isolates the effect of the new medication.

Pooled Variance t-Test

We can’t always have matched pairs in our studies. Sometimes, there aren’t possible similar individuals we can pair together. In this case, we consider the two samples independent. Note: We should collect random, non-bias samples that are independent of one-another.

In the case where the two independent samples have similar variability, we call this a Pooled Variance t-Test. The variance is ‘similar’ enough if the following is true:

• The standard deviation of one of the samples is less than double the other sample
• The variance of one of the samples is less than four times the variance of the other sample

Pooling the Variance

We call this test ‘pooled’ because we assume that the two samples came from the same population with the same variance. We then ‘pool’ the two samples’ variances together to get a best-guess for the population standard deviation:

\[ S_{p}^2 = \frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2} \]

We did something similar when calculating a ‘pooled’ population proportion estimate in the previous chapter. In this case, Excel’s Data Analysis Toolpak will return this value for us, so this formula is just for your reference. You will not need to use it.

T-test Statistic

We can calculate the test statistic, [latex]t_{test}[/latex] with the following formula:

\[ t_{test} = \frac{\bar{x}_1-\bar{x}_2}{\sqrt{S_{p}^2 \left( \frac{1}{n_1} + \frac{1}{n_2} \right)}} \]

Again, we will have Excel calculate this test statistic. It is also just for your reference.

P-Value

In order to calculate the p-value, we first need the degrees of freedom ([latex]df[/latex]):

\[ df = n_1 + n_2 -2 \]

We use these degrees of freedom and test statistic in Excel’s T.DIST, T.DIST.RT or T.DIST.2T functions. See the hypothesis testing on one mean section for which function to use when.

Unpooled Variance t-Test

In this final case, the unpooled variance test, we use this test if:

• We want to determine if there is a difference two samples
• The samples are independent of one another.
• The standard deviation of one sample is more than double the other sample’s standard deviation.
• The variance of one sample is more than four times the size of the standard deviation sample.

In this case, we cannot assume the two samples are from the same population and can therefore not ‘pool’ their variances.

Test Statistic

The test statistic still follows a t-distribution. When performing the hypothesis test, we assume that the difference is zero or that they both come from the same population. Ie: H₀: μ₁−μ₂ = 0 or H₀: μ₁=μ₂. This simplifies the t_test formula:

\[ t_{test} = \frac{ (\bar{x}_1- \bar{x}_2) – (\mu_1−\mu_2) }{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}} =\frac{(\bar{x}_1-\bar{x}_2)-(0)}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}=\frac{\bar{x}_1-\bar{x}_2}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}} \]

\[\]

Degrees of Freedom

The degrees of freedom formula, which is often fairly simple, becomes quite involved in this case. We first calculate [latex]C[/latex]:

\[C = \frac{\frac{s_1^2}{n_1}}{\frac{s_1^2}{n_1}+\frac{s_1^2}{n_1}}\]

We use this value in the degrees of freedom calculation:

\[ df = \frac{(n_1-1)(n_2-1)}{(n_2-1)C^2+(1-C)^2(n_1-1)} \]

Again, Excel’s Data Analysis Toolpak will calculate the degrees of freedom, test statistic and p-value for us. See the next sections for examples of this.

Required Assumptions

In all of the above cases, we also need the following to be true:

• The samples are randomly collected and non-bias.
• The sample sizes are large enough
• Or, the populations are known to be normally distributed.