{"id":318,"date":"2021-08-03T17:41:15","date_gmt":"2021-08-03T21:41:15","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/statspsych\/?post_type=chapter&p=318"},"modified":"2023-12-30T19:44:03","modified_gmt":"2023-12-31T00:44:03","slug":"chapter-5","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/statspsych\/chapter\/chapter-5\/","title":{"raw":"5. Single Sample Z-test and t-test","rendered":"5. Single Sample Z-test and t-test"},"content":{"raw":"

<\/a>5a. Single Sample Z-test<\/h1>\r\n
video lesson<\/a><\/h6>\r\nNow that you have mastered the basic process of hypothesis testing, you are ready for this: your first real statistical test. First, we will examine the types of error that can arise in the context of hypothesis testing.\r\n\r\nWhich type of error is more serious for a professional?\u00a0 1. I decide that a new mode of transportation, the driverless car, is much safer than current modes of transportation, and recommend to the state that they force conversion to that new mode. But then it turns out after a year that the rates of accidents actually increased.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"584\"]\"Google \"Google Self-Driving Car\" by smoothgroover22 is licensed under CC BY-SA 2.0[\/caption]\r\n\r\nOr... 2. I decide that a new mode of transportation is safer than current modes, but I do not have enough confidence in that margin of safety and therefore do not recommend a sweeping change. After a year it becomes clear that driverless cars are indeed much safer, and a switch would have been beneficial.\r\n\r\nIf you selected the first error as the more serious one, then you chose a [pb_glossary id=\"320\"]Type I error[\/pb_glossary]<\/strong>. This is the type of error that is considered more serious in the realm of statistical decision making, as well.\r\n\r\nIn the process of hypothesis testing, the decision you make is all about probabilities. It is an educated guess. However, there is always room for error. In fact, there are two types of errors you can make in hypothesis testing. Their imaginative labels are Type I<\/strong> and [pb_glossary id=\"321\"]Type II error[\/pb_glossary]<\/strong>. As we saw, Type I error<\/strong> is considered more serious, so the Significance level is typically set with an eye toward the probability of Type I error<\/strong> that is deemed acceptable in that study. \u03b1 = .05 indicates that the researcher accepts a 5% chance of a Type I error<\/strong>. In the decision matrix shown here, you can see how the hypothesis test can play out in four different ways.\r\n\r\n\"Decision\r\n\r\nNote that the real situation, whether the null hypothesis is true or the research hypothesis is true, is unknown. We can never know which is true \u2013 that is why they are hypotheses, after all!\r\n\r\nIf, hypothetically, the null were true, and if we made the decision to reject the null hypothesis, then we would land in this corner of the decision matrix: Type I error<\/strong>. This is what happens if we go out on a limb, reject the null hypothesis, but we are wrong. As researchers we live in fear of making a Type I error<\/strong>. No one wants to make a big thing about their research findings and promote change to a new, different, risky, expensive thing, and then be wrong about it. On the other hand, if the null were true and we did not reject it, then we would be correct.\r\n\r\nHypothetically, let us suppose the research hypothesis were true. If we make the decision to reject the null hypothesis, then we are correct. However, if we decide not to reject the null hypothesis, we would be making a Type II error<\/strong>. This is when we are too conservative, decide there is not enough evidence that the research (alternative) hypothesis could be correct, but we are wrong. Type II error<\/strong> is also not great, so we do try to minimize that probability by doing something called a power analysis. However, researchers generally would rather make a Type II<\/strong> than a Type I<\/strong> error. Keeping the status quo may not do much for to make your scientific career exciting, but objectively speaking it is not such a scary proposition.\r\n\r\nNow, a key thing to remember when you refer back to this decision matrix to determine for which error you are at risk, you never know which column you are in, because the real situation is always unknown. What determines the possible type of error is the thing you have control over: the decision. So you can switch the row you will place yourself in, depending on the observed data, and thus remove yourself from unreasonable risk of Type I error<\/strong>. (The probability of making a Type I error<\/strong> is symbolized as [pb_glossary id=\"580\"]\u03b1[\/pb_glossary]<\/strong><\/em>, so significance levels are often referred to as \u03b1 <\/strong><\/em>.)\r\n

<\/a>Concept Practice: Type I and Type II errors<\/a><\/h4>\r\n \r\n\r\nOkay, we are about to make a connection that is intellectually a bit challenging. So prepare yourself: we will be drawing connections between the central limit theorem, sampling distributions, and the appropriate type of comparison distribution we should use for a hypothesis test where the sample is made up of more than one data point.\r\n\r\nIn the past, we have been using the normal distribution made of up individual scores as our comparison distribution. In the illustration below, if distribution (A) is the distribution of individual scores, (B) is the histogram of a sample of scores drawn from (A). (C) is the [pb_glossary id=\"328\"]distribution of means[\/pb_glossary]<\/strong> from many samples such as the one portrayed in (B).\r\n\r\n\"Introducing\r\n
The shape of a distribution of means<\/strong> is narrower than the distribution of individual scores from which it came. The variance is smaller, because it is divided by N, the sample size:<\/div>\r\n
[latex] \\[\\sigma^{2}_{M} = \\frac{\\sigma ^{2}}{N}\\] [\/latex]<\/div>\r\n
<\/div>\r\n
The standard deviation is smaller, because it is divided by the square root of N:<\/div>\r\n
[latex] \\[\\sigma_{M} = \\frac{\\sigma }{\\sqrt{N}}\\] [\/latex]<\/div>\r\n
<\/div>\r\n
The means are the same for both types of distributions:<\/div>\r\n
\r\n\r\n[latex]\\[\\mu_{M} = \\mu\\] [\/latex]\r\n\r\n<\/div>\r\n \r\n

<\/a>Concept Practice: Distribution of Sample Means<\/a><\/h4>\r\n \r\n\r\nUp until now we worked with individuals as our samples, so we could use our old Z-score formula and compare the sample score to a comparison distribution of individuals. We did this, so that we could avoid adding new symbols to our formulas, and concentrate on the hypothesis testing logic. Now, we want the more realistic situation, so we will use the [pb_glossary id=\"336\"]Z-test[\/pb_glossary]<\/strong> to compare a sample mean to a comparison distribution of sample means<\/strong>. To rephrase, when we had an individual score, we used the distribution of individuals. When we have a sample mean, we need to use the distribution of sample means<\/strong>. The new Z-test<\/strong> statistic formula is shown here:\r\n\r\n[latex] \\[Z=\\frac{M-\\mu _{M}}{\\sigma _{M}}\\] [\/latex]\r\n\r\nIn this formula, M is the sample mean, \u03bcM<\/sub> is the comparison distribution mean, and \u03c3M<\/sub> is the comparison distribution standard deviation.\r\n\r\nWe will be using the terminology \u201cdistribution of means<\/strong>,\u201d because it\u2019s a name that reminds us of how it differs from the distribution of individuals. A term in statistics that means the same thing is \u201csampling distribution.\u201d Why do we need this distribution of means<\/strong>? In reality, we do not compare an individual sample score against a population distribution of individuals. For a real statistical test, we collect a sample and calculate a mean, and compare that sample mean against a comparison\u00a0 distribution of means. We will ground that in an example: If I want to know if my introductory psychology class is collectively performing at the expected level on a test, I do not take their average performance and compare that mean against a set of individual scores, I want to compare it to a set of other class averages. How does this impact our hypothesis test? It matters for Step 2, determining the characteristics of the comparison distribution. Well, it does not change anything about the comparison mean, because the mean of a distribution of means<\/strong> is equal to the mean of the distribution of individuals. But the standard deviation now needs to be converted into the standard deviation of the distribution of means<\/strong>, which is often called the [pb_glossary id=\"334\"]standard error of the mean[\/pb_glossary]<\/strong>. As these equations above show, the variance of a distribution of means<\/strong> is equal to the variance of the distribution of individuals divided by the number of individuals in the sample. And thus, if we take the square root of the variance, the standard deviation of a distribution of means<\/strong> is equal to the standard deviation of the distribution of individuals divided by the square root of the number of individuals in the sample.\r\n\r\nWe will now examine the way the steps of the hypothesis test procedure play out when using the single-sample Z-test<\/strong>. First, we need to formulate the research and null hypotheses. Remember, in such a hypothesis test, we are trying to compare two populations. Population 1, the research population, is the one from which we have drawn a research sample.\r\n\r\n\"\"\r\n\r\nWe do not have access to the actual research population mean, hence the need to calculate a sample mean. That sample mean is our best proxy for the population 1 mean, which we need to compare to the mean of population 2. The nature of the comparison, in particular its directionality, is determined by the researcher\u2019s prediction.\r\n\r\nIn step 2, we need to determine the characteristics of the comparison distribution. This means that we need the mean and standard deviation of the comparison distribution, which represents Population 2. This is where we need to introduce the new distribution of means<\/strong>. The mean is the same as provided, but the standard deviation needs to be converted.\r\n\r\nIf the variance is provided, you can start with this formula to convert to the distribution of means, then take the square root to get the standard deviation:\r\n
[latex] \\[\\sigma^{2}_{M} = \\frac{\\sigma ^{2}}{N}\\] [\/latex]<\/div>\r\nIf the standard deviation is provided, it is easiest to use this formula:\r\n\r\n[latex] \\[\\sigma_{M} = \\frac{\\sigma }{\\sqrt{N}}\\] [\/latex]\r\n\r\nStep 3 is to determine the cutoff sample score, which is derived from two pieces of information. First, we need to know the conventional significance level that applies. Second, we need to know which tail we are looking at. The bottom, the top, or both? This depends on the research hypothesis, so we should always look for directional keywords in problem. If the research hypothesis is directional, we should use a one-tailed test. If the research hypothesis is non-directional, we should use a two-tailed test.\r\n\r\nOnce we have those two pieces of information, we can find the Z-score that forms the border of the shaded tail by looking it up in a table<\/a>. We can then map the cutoff score onto a drawing of the comparison distribution.\r\n\r\n[caption id=\"attachment_348\" align=\"aligncenter\" width=\"458\"]\"\" Example of comparison distribution sketch showing Z-score cutoff and shaded tail representing decision rule[\/caption]\r\n\r\nIn step 4, we need to calculate the Z-test score from the sample mean:\r\n\r\n[latex] \\[Z=\\frac{M-\\mu _{M}}{\\sigma _{M}}\\] [\/latex]\r\n\r\nIn this formula, M represent the research sample mean, \u03bcM<\/sub><\/em> represents the comparison population mean, and \u03c3M<\/sub><\/em> represents the standard deviation of the comparison population. Remember, we are using the distribution of means<\/strong> as our comparison distribution. Once we have calculated the Z-test score, we will mark where it falls on the comparison distribution, and determine whether it falls in the shaded tail or not.\r\n\r\n[caption id=\"attachment_349\" align=\"aligncenter\" width=\"486\"]\"\" Example of mapping sample mean (M) onto comparison distribution based on Z-test calculation[\/caption]\r\n\r\nFinally, it is time to decide whether or not to reject the null hypothesis. If the sample mean falls within the shaded tail, we reject the null hypothesis. If it does not, we do not reject the null hypothesis. In other words if the sample mean is extreme enough relative to the comparison distribution of means, then we reject the null hypothesis.\r\n\r\nOnce we make our decision, we will need to take a close look at what kind of error we might have made as a result of our decision. And remember, from our decision matrix, all we have control over whether we're on the top row or the bottom row.\r\n\r\n\"Decision\r\n\r\nWe do not know the real situation. if we do make the conclusion that we should reject the null hypothesis, then we are at some risk of a Type I error<\/strong>, but as long as we don't exceed our significance level, we accept that risk. If we do not reject the null hypothesis we could be at a risk for Type II error<\/strong>.\r\n\r\nTypically, after conducting a hypothesis test, researchers want to obtain the [pb_glossary id=\"352\"]p-value[\/pb_glossary]<\/strong>, or the probability of the observed sample score or more extreme occurring just by random chance under the comparison distribution. The p-value associated with the sample mean has to be less than the significance level in order to reject the null hypothesis. A common way to find the p-value<\/strong> is to use an online calculator<\/a>.\r\n\r\nOne last exercise I would like you to try is to pretend you are publishing these results in a scientific journal. So you might write this sentence in your results section based on your findings: \"We found that patients with insomnia who received a new drug slept significantly more than patients who did not receive the new drug (p < .05).\"<\/span>\r\n\r\nNotice the term \"significantly\" is used in this example, because the null hypothesis was rejected at the end of the hypothesis test. Also, the p-value<\/strong> is in brackets at the end of this sentence, before the punctuation, just like you would use a citation to back up a factual claim... only here, it is a claim of statistical significance.\r\n\r\n \r\n

<\/a>Concept Practice: Conducting a Z test<\/a><\/h4>\r\n

<\/a>5b. Single-sample T-test<\/h1>\r\n
video lesson<\/a><\/h6>\r\nNow it is time to introduce your second real statistical test: the single-sample [pb_glossary id=\"358\"]t-test[\/pb_glossary]<\/strong>. This test is much like the Z-test<\/strong>, but more commonly used in real life.\r\n\r\nBefore we get started on the t-test<\/strong>, I am curious\u2026 would you have greater confidence in a statistic that comes from a small sample?\r\n\r\n\"\"\r\n\r\nOr in one that comes from an entire population?... If you said that you would have greater confidence in a number that comes from an entire population, then the field of statistics would agree with you.\r\n\r\nSometimes, though, we do not have access to all the data from population that we need, and we have to take a best guess based on a sample. In that case, we probably want to be more conservative in our decision making. That\u2019s what a t-test<\/strong> is for.\r\n\r\nOften, even if we have information regarding the comparison population mean, we do not have access to the population standard deviation (\u03c3<\/em>). In this situation, what we must do is collect a sample reflecting population 1, calculate its mean and standard deviation, and compare that sample mean against a comparison distribution of means<\/strong> using an estimate (S<\/em>M<\/sub>) of the population (\u03c3M<\/sub><\/em>). Note the change in symbol (from \u03c3 <\/em>to S)<\/em> to reflect the fact that we are basing our estimate on sample data, rather than using a known population parameter.\r\n\r\n[latex] \\[S^{2} = \\frac{\\sum (X-M)^{2}}{N-1}\\] [\/latex]\r\n\r\nYou may have noticed that this new formula for calculating variance in a sample has on the bottom \u201cN<\/em> minus 1\u201d rather than just \u201cN<\/em>\u201d as our old calculation formula had. Why \u201cN<\/em> minus 1\u201d? This is a correction, based on sample size, that derives from the concept of [pb_glossary id=\"362\"]degrees of freedom[\/pb_glossary]<\/strong>. We will return to that concept later. For now, let us consider \u201cN<\/em> minus 1\u201d as a correction based on sample size that has the following deliberate and intentional effects on our calculation. If we have a small sample size, in which we should not have a great deal of confidence, subtracting 1 really affects the calculation. For example, if we had an N<\/em> of 2, subtracting off 1 from that sample size will mean that we divide by 1 instead of 2. That has a drastic impact on the variance calculation, in effect doubling our estimated variance. Of course in the context of hypothesis testing, more variance is bad \u2013 in that we are less likely to be able to make conclusions based on the data.\r\n\r\nOn the other hand, with a larger sample size, subtracting 1 will have little impact on our calculation\u2026 dividing by 100 or dividing by 99 will have nearly identical results. Think of it this way \u2013 if we have a small sample size, we are punishing ourselves, like a handicap in a golf game. With a larger sample size, we reward ourselves, boosting our chance of a conclusion statistical decision.\r\n\r\nSubtracting off 1 is like a flat tax \u2013 a flat income tax of 10% for all Canadians would be harder on those with a low income, who would have even less money to meet basic survival needs, whereas the wealthiest Canadians would still have plenty of money for their needs. That is, of course, why democratic governments typically use progressive tax rates, because those who have more can afford to pay a greater proportion of their income and still have enough to cover their basic costs of living. With hypothesis testing, though, the flat tax is desirable, because it is our intent to penalize the smallest sample sizes the most, given the fact that we can have little confidence in the estimates we derive from them.\r\n\r\nThe t-test is designed around the concept of degrees of freedom<\/strong>, which is defined as the number of scores in a given calculation that are free to vary. This is an abstract definition that is difficult to grasp, so it is helpful to consider a concrete example.\r\n
\r\n

Degrees of Freedom Practical Example<\/p>\r\n\r\n<\/header>\r\n

\r\n
First, consider the game of baseball.\u00a0 We understand the field-of-play consists of 9 positions.\u00a0 The coach is \u201cfree\u201d to assign any of the 9 players to any of the 9 positions. Once the 8th player is assigned to the 8th position, the 9th player-position is pre-determined, so to speak.\u00a0 In other words, the coach is not \u201cfree\u201d to pick either the last position or the last player.<\/blockquote>\r\nSource:<\/em> https:\/\/www.isixsigma.com\/ask-dr-mikel-harry\/ask-tools-techniques\/can-you-explain-degrees-freedom-and-provide-example\/\r\n\r\n<\/div>\r\n<\/div>\r\nThe same logic holds when doing a calculation involving all of a sample\u2019s scores. Up until the last score inputted into the formula, the scores are free to vary; but the last score is pre-determined. Thus degrees of freedom<\/strong> in a t-test<\/strong> is N<\/em>-1, or 1 less than the total number of scores in the dataset.\r\n\r\nIn a t-test<\/strong> we will be using the t-distributions<\/strong>, rather than a normal distribution, as our comparison distributions. The t-distributions<\/strong> are a series of distributions, based on the normal distribution, that adjust their shape according to degrees of freedom<\/strong> (which in turn is based on sample size).\r\n\r\nAs mentioned previously, when we do not have the population standard deviation we must estimate it from our sample. This involves a greater risk of error. The risk is related to how large our sample is. So, the fewer the scores in our sample, the fewer the degrees of freedom, and the more conservative the distribution we must use \u2013 one that is wider and shorter (more area in the tails).\r\n\r\n\"\"\r\n\r\nAbove you can see examples of how the shape of the t-distributions changes with reduced degrees of freedom<\/strong>. With degrees of freedom<\/strong> of 100, the shape of the distribution is very close to the normal curve. If we have very few degrees of freedom<\/strong>, like 1, then our shading rejection regions, or goal posts, will move farther out into the tails of the distribution, because there is more area under the tails of the curve. In effect, then, with fewer degrees of freedom<\/strong>, we will find it more difficult to reject the null hypothesis. This is, of course, by design, in order to penalize the hypothesis test if it is based on a very small sample size.\r\n\r\n[caption id=\"attachment_366\" align=\"aligncenter\" width=\"484\"]\"\" Inspiration for this creative illustration came from one of my former students, Andi.\u00a0Blame for artwork quality is entirely on me.<\/em>[\/caption]\r\n\r\nAbove is a less technical visual to see the difference in distribution shape when the sample size is small, versus when the sample size is large. As you can see, when you have a large sample size, the goal posts are very close and easy to reach, but when you have a small sample size, they will really be far away, and this will make it difficult to reject our null hypothesis.\r\n

<\/a>Concept Practice: Sample size and the t-test<\/a><\/h4>\r\n \r\n\r\nNow we are ready to take a look at how a hypothesis test works based on a single-sample t-test<\/strong>. As before, the structure of the test is familiar \u2013 we are comparing the means of Population 1 and Population 2.\r\n\r\n\"\"\r\n\r\nAs usual, we do not have access to the mean of population 1, just the mean of a sample from that population. Furthermore, unlike in a Z-test<\/strong>, we do not have access to one piece of information about population 2: the population standard deviation. Therefore, we will form hypotheses the same way, but when we do steps 2 and 4, we will need to use the sample to generate an estimate of the comparison population standard deviation.\r\n

<\/a>Concept Practice: When to use a Z-test or a t-test<\/a><\/h4>\r\n \r\n\r\nIn step 2, we identify the known comparison population\u2019s mean. However, we will need to use the sample to generate an estimate of the comparison population standard deviation. We calculate the sample variance, using the formula with the \u201cN<\/em>-1\u201d correction,\r\n\r\n[latex] \\[S^{2} = \\frac{\\sum (X-M)^{2}}{N-1}\\] [\/latex]\r\n\r\nand then convert the variance to the distribution of means<\/strong> by dividing by N<\/em>,\r\n\r\n[latex] \\[S^{2}_{M} = \\frac{S ^{2}}{N}\\] [\/latex]\r\n\r\nand finally we square root to get from variance to standard deviation.\r\n\r\n[latex] \\[S_{M} = \\sqrt{S^{2}_{M}}\\][\/latex]\r\n\r\nSM<\/sub><\/em> is what we will use to describe our comparison distribution.\r\n\r\nIn step 3, as before we need to find our cutoff\u00a0 sample score based on the significance level and directionality of the test. Now, however, we also need to use a third piece of information: degrees of freedom<\/strong>. This is found by subtracting one from the sample size. Then we can look in the t-tables<\/a>, identifying the row of the table with the matching degrees of freedom<\/strong>, then looking in the appropriate column based on directionality and significance level. Once we identify the t-score that forms the border of the shaded tail, we can map this onto a drawing of the comparison distribution. For visualization purposes, it is fine to sketch the distribution the same way you did the normal distribution.\r\n

<\/a>Concept Practice: Degrees of Freedom<\/a><\/h4>\r\n \r\n\r\n \r\n\r\nNow for step 4. The t-test<\/strong> formula is just like the Z-test<\/strong> one on top \u2013 sample mean minus comparison population mean. Then you divide by SM<\/sub><\/em>, the estimate of standard deviation that you calculated back in step 2.\r\n\r\n[latex] \\[t = \\frac{M-\\mu _{M}}{S_{M}}\\][\/latex]\r\n\r\nIf your calculated t-test<\/strong> score fell into the shaded tail beyond your cutoff score, then you may reject the null hypothesis. In other words, if the sample mean was extreme enough on the comparison distribution of means, reject the null.\r\n\r\n\"\"\r\n\r\nAfter conducting a hypothesis test, it is helpful to also obtain a report the precise p-value<\/strong> associated with the result. The p-value<\/strong> offers the precise probability of obtaining the t-test<\/strong> result (or more extreme) just by random chance under the comparison distribution. It is found by calculating<\/a> the area under the comparison distribution in the tail beyond the observed t-test<\/strong> score calculated in step 4 (or in both tails, if it was a two-tailed test).\r\n\r\n \r\n

<\/a>Concept Practice: Conduct a t-test<\/a><\/h4>\r\n

<\/a>Concept Practice: Facts about Z- and t-tests<\/a><\/h4>\r\n \r\n\r\n \r\n

Chapter Summary<\/h1>\r\nIn this chapter we learned about the two types of logical errors we can make in hypothesis testing: Type I<\/strong> and Type II errors<\/strong>. We saw that Type I error<\/strong> is considered more serious in statistics, because it represents going out on limb, making a strong conclusion, when that is in fact the wrong decision. For that reason, Type I error<\/strong> risk is strictly controlled in inferential statistics by setting a particular significance level (\u03b1<\/strong><\/em>). We also introduced our first real inferential statistical tests: the single-sample Z-test<\/strong> and the single-sample t-test<\/strong>. In each of these tests, we are comparing the means of two populations, using a sample to estimate the mean of the research population. A Z-test<\/strong> is used when we know the standard deviation of the comparison population (\u03c3<\/strong><\/em>); a t-test<\/strong> is used when we do not have that information and must estimate the standard deviation from the sample (S<\/em><\/strong>).\r\n\r\nKey terms:\r\n\r\n\r\n\r\n\r\n\r\n
Type I error<\/strong><\/td>\r\ndistribution of means<\/strong><\/td>\r\np-value<\/strong><\/td>\r\n<\/tr>\r\n
Type II error<\/strong><\/td>\r\nZ-test<\/strong><\/td>\r\nt-test<\/strong><\/td>\r\n<\/tr>\r\n
\u03b1<\/strong><\/em><\/td>\r\nstandard error of the mean<\/strong><\/td>\r\ndegrees of freedom<\/strong><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Concept Practice<\/h1>\r\n<\/a>[h5p id=\"31\"]\r\n\r\n[h5p id=\"32\"]\r\n\r\n[h5p id=\"33\"]\r\n\r\n[h5p id=\"34\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"37\"]\r\n\r\n[h5p id=\"38\"]\r\n\r\n[h5p id=\"39\"]\r\n\r\n[h5p id=\"40\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"41\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"43\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"44\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"42\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"46\"]\r\n
Return to text<\/a><\/h6>\r\n<\/a>[h5p id=\"45\"]\r\n
Return to text<\/a><\/h6>\r\nReturn to 5a. Single-sample Z test<\/a>\r\n
Try interactive Worksheet 5a<\/a> or download Worksheet 5a<\/h6>\r\nReturn to 5b. Single-sample t test<\/a>\r\n
Try interactive Worksheet 5b\u00a0or download Worksheet 5b<\/h6>\r\n \r\n\r\n<\/a>video 5a1\r\n\r\n[video width=\"1440\" height=\"1080\" mp4=\"https:\/\/pressbooks.bccampus.ca\/statspsych\/wp-content\/uploads\/sites\/1469\/2022\/08\/Chapter-5a1.mp4\"][\/video]\r\n\r\nvideo 5a2\r\n\r\n \r\n\r\n<\/a>video 5b1","rendered":"

<\/a>5a. Single Sample Z-test<\/h1>\n
video lesson<\/a><\/h6>\n

Now that you have mastered the basic process of hypothesis testing, you are ready for this: your first real statistical test. First, we will examine the types of error that can arise in the context of hypothesis testing.<\/p>\n

Which type of error is more serious for a professional?\u00a0 1. I decide that a new mode of transportation, the driverless car, is much safer than current modes of transportation, and recommend to the state that they force conversion to that new mode. But then it turns out after a year that the rates of accidents actually increased.<\/p>\n

\"Google
“Google Self-Driving Car” by smoothgroover22 is licensed under CC BY-SA 2.0<\/figcaption><\/figure>\n

Or… 2. I decide that a new mode of transportation is safer than current modes, but I do not have enough confidence in that margin of safety and therefore do not recommend a sweeping change. After a year it becomes clear that driverless cars are indeed much safer, and a switch would have been beneficial.<\/p>\n

If you selected the first error as the more serious one, then you chose a Type I error<\/a><\/strong>. This is the type of error that is considered more serious in the realm of statistical decision making, as well.<\/p>\n

In the process of hypothesis testing, the decision you make is all about probabilities. It is an educated guess. However, there is always room for error. In fact, there are two types of errors you can make in hypothesis testing. Their imaginative labels are Type I<\/strong> and Type II error<\/a><\/strong>. As we saw, Type I error<\/strong> is considered more serious, so the Significance level is typically set with an eye toward the probability of Type I error<\/strong> that is deemed acceptable in that study. \u03b1 = .05 indicates that the researcher accepts a 5% chance of a Type I error<\/strong>. In the decision matrix shown here, you can see how the hypothesis test can play out in four different ways.<\/p>\n

\"Decision<\/p>\n

Note that the real situation, whether the null hypothesis is true or the research hypothesis is true, is unknown. We can never know which is true \u2013 that is why they are hypotheses, after all!<\/p>\n

If, hypothetically, the null were true, and if we made the decision to reject the null hypothesis, then we would land in this corner of the decision matrix: Type I error<\/strong>. This is what happens if we go out on a limb, reject the null hypothesis, but we are wrong. As researchers we live in fear of making a Type I error<\/strong>. No one wants to make a big thing about their research findings and promote change to a new, different, risky, expensive thing, and then be wrong about it. On the other hand, if the null were true and we did not reject it, then we would be correct.<\/p>\n

Hypothetically, let us suppose the research hypothesis were true. If we make the decision to reject the null hypothesis, then we are correct. However, if we decide not to reject the null hypothesis, we would be making a Type II error<\/strong>. This is when we are too conservative, decide there is not enough evidence that the research (alternative) hypothesis could be correct, but we are wrong. Type II error<\/strong> is also not great, so we do try to minimize that probability by doing something called a power analysis. However, researchers generally would rather make a Type II<\/strong> than a Type I<\/strong> error. Keeping the status quo may not do much for to make your scientific career exciting, but objectively speaking it is not such a scary proposition.<\/p>\n

Now, a key thing to remember when you refer back to this decision matrix to determine for which error you are at risk, you never know which column you are in, because the real situation is always unknown. What determines the possible type of error is the thing you have control over: the decision. So you can switch the row you will place yourself in, depending on the observed data, and thus remove yourself from unreasonable risk of Type I error<\/strong>. (The probability of making a Type I error<\/strong> is symbolized as \u03b1<\/a><\/strong><\/em>, so significance levels are often referred to as \u03b1 <\/strong><\/em>.)<\/p>\n

<\/a>Concept Practice: Type I and Type II errors<\/a><\/h4>\n

 <\/p>\n

Okay, we are about to make a connection that is intellectually a bit challenging. So prepare yourself: we will be drawing connections between the central limit theorem, sampling distributions, and the appropriate type of comparison distribution we should use for a hypothesis test where the sample is made up of more than one data point.<\/p>\n

In the past, we have been using the normal distribution made of up individual scores as our comparison distribution. In the illustration below, if distribution (A) is the distribution of individual scores, (B) is the histogram of a sample of scores drawn from (A). (C) is the distribution of means<\/a><\/strong> from many samples such as the one portrayed in (B).<\/p>\n

\"Introducing<\/p>\n

The shape of a distribution of means<\/strong> is narrower than the distribution of individual scores from which it came. The variance is smaller, because it is divided by N, the sample size:<\/div>\n
\n

  <\/span>   <\/span>\" \[\sigma^{2}_{M} = \frac{\sigma ^{2}}{N}\] \"<\/p>\n<\/div>\n

<\/div>\n
The standard deviation is smaller, because it is divided by the square root of N:<\/div>\n
\n

  <\/span>   <\/span>\" \[\sigma_{M} = \frac{\sigma }{\sqrt{N}}\] \"<\/p>\n<\/div>\n

<\/div>\n
The means are the same for both types of distributions:<\/div>\n
\n

  <\/span>   <\/span>\"\[\mu_{M} = \mu\] \"<\/p>\n<\/div>\n

 <\/p>\n

<\/a>Concept Practice: Distribution of Sample Means<\/a><\/h4>\n

 <\/p>\n

Up until now we worked with individuals as our samples, so we could use our old Z-score formula and compare the sample score to a comparison distribution of individuals. We did this, so that we could avoid adding new symbols to our formulas, and concentrate on the hypothesis testing logic. Now, we want the more realistic situation, so we will use the Z-test<\/a><\/strong> to compare a sample mean to a comparison distribution of sample means<\/strong>. To rephrase, when we had an individual score, we used the distribution of individuals. When we have a sample mean, we need to use the distribution of sample means<\/strong>. The new Z-test<\/strong> statistic formula is shown here:<\/p>\n

  <\/span>   <\/span>\" \[Z=\frac{M-\mu _{M}}{\sigma _{M}}\] \"<\/p>\n

In this formula, M is the sample mean, \u03bcM<\/sub> is the comparison distribution mean, and \u03c3M<\/sub> is the comparison distribution standard deviation.<\/p>\n

We will be using the terminology \u201cdistribution of means<\/strong>,\u201d because it\u2019s a name that reminds us of how it differs from the distribution of individuals. A term in statistics that means the same thing is \u201csampling distribution.\u201d Why do we need this distribution of means<\/strong>? In reality, we do not compare an individual sample score against a population distribution of individuals. For a real statistical test, we collect a sample and calculate a mean, and compare that sample mean against a comparison\u00a0 distribution of means. We will ground that in an example: If I want to know if my introductory psychology class is collectively performing at the expected level on a test, I do not take their average performance and compare that mean against a set of individual scores, I want to compare it to a set of other class averages. How does this impact our hypothesis test? It matters for Step 2, determining the characteristics of the comparison distribution. Well, it does not change anything about the comparison mean, because the mean of a distribution of means<\/strong> is equal to the mean of the distribution of individuals. But the standard deviation now needs to be converted into the standard deviation of the distribution of means<\/strong>, which is often called the standard error of the mean<\/a><\/strong>. As these equations above show, the variance of a distribution of means<\/strong> is equal to the variance of the distribution of individuals divided by the number of individuals in the sample. And thus, if we take the square root of the variance, the standard deviation of a distribution of means<\/strong> is equal to the standard deviation of the distribution of individuals divided by the square root of the number of individuals in the sample.<\/p>\n

We will now examine the way the steps of the hypothesis test procedure play out when using the single-sample Z-test<\/strong>. First, we need to formulate the research and null hypotheses. Remember, in such a hypothesis test, we are trying to compare two populations. Population 1, the research population, is the one from which we have drawn a research sample.<\/p>\n

\"\"<\/p>\n

We do not have access to the actual research population mean, hence the need to calculate a sample mean. That sample mean is our best proxy for the population 1 mean, which we need to compare to the mean of population 2. The nature of the comparison, in particular its directionality, is determined by the researcher\u2019s prediction.<\/p>\n

In step 2, we need to determine the characteristics of the comparison distribution. This means that we need the mean and standard deviation of the comparison distribution, which represents Population 2. This is where we need to introduce the new distribution of means<\/strong>. The mean is the same as provided, but the standard deviation needs to be converted.<\/p>\n

If the variance is provided, you can start with this formula to convert to the distribution of means, then take the square root to get the standard deviation:<\/p>\n

\n

  <\/span>   <\/span>\" \[\sigma^{2}_{M} = \frac{\sigma ^{2}}{N}\] \"<\/p>\n<\/div>\n

If the standard deviation is provided, it is easiest to use this formula:<\/p>\n

  <\/span>   <\/span>\" \[\sigma_{M} = \frac{\sigma }{\sqrt{N}}\] \"<\/p>\n

Step 3 is to determine the cutoff sample score, which is derived from two pieces of information. First, we need to know the conventional significance level that applies. Second, we need to know which tail we are looking at. The bottom, the top, or both? This depends on the research hypothesis, so we should always look for directional keywords in problem. If the research hypothesis is directional, we should use a one-tailed test. If the research hypothesis is non-directional, we should use a two-tailed test.<\/p>\n

Once we have those two pieces of information, we can find the Z-score that forms the border of the shaded tail by looking it up in a table<\/a>. We can then map the cutoff score onto a drawing of the comparison distribution.<\/p>\n

\"\"
Example of comparison distribution sketch showing Z-score cutoff and shaded tail representing decision rule<\/figcaption><\/figure>\n

In step 4, we need to calculate the Z-test score from the sample mean:<\/p>\n

  <\/span>   <\/span>\" \[Z=\frac{M-\mu _{M}}{\sigma _{M}}\] \"<\/p>\n

In this formula, M represent the research sample mean, \u03bcM<\/sub><\/em> represents the comparison population mean, and \u03c3M<\/sub><\/em> represents the standard deviation of the comparison population. Remember, we are using the distribution of means<\/strong> as our comparison distribution. Once we have calculated the Z-test score, we will mark where it falls on the comparison distribution, and determine whether it falls in the shaded tail or not.<\/p>\n

\"\"
Example of mapping sample mean (M) onto comparison distribution based on Z-test calculation<\/figcaption><\/figure>\n

Finally, it is time to decide whether or not to reject the null hypothesis. If the sample mean falls within the shaded tail, we reject the null hypothesis. If it does not, we do not reject the null hypothesis. In other words if the sample mean is extreme enough relative to the comparison distribution of means, then we reject the null hypothesis.<\/p>\n

Once we make our decision, we will need to take a close look at what kind of error we might have made as a result of our decision. And remember, from our decision matrix, all we have control over whether we’re on the top row or the bottom row.<\/p>\n

\"Decision<\/p>\n

We do not know the real situation. if we do make the conclusion that we should reject the null hypothesis, then we are at some risk of a Type I error<\/strong>, but as long as we don’t exceed our significance level, we accept that risk. If we do not reject the null hypothesis we could be at a risk for Type II error<\/strong>.<\/p>\n

Typically, after conducting a hypothesis test, researchers want to obtain the p-value<\/a><\/strong>, or the probability of the observed sample score or more extreme occurring just by random chance under the comparison distribution. The p-value associated with the sample mean has to be less than the significance level in order to reject the null hypothesis. A common way to find the p-value<\/strong> is to use an online calculator<\/a>.<\/p>\n

One last exercise I would like you to try is to pretend you are publishing these results in a scientific journal. So you might write this sentence in your results section based on your findings: “We found that patients with insomnia who received a new drug slept significantly more than patients who did not receive the new drug (p < .05).”<\/span><\/p>\n

Notice the term “significantly” is used in this example, because the null hypothesis was rejected at the end of the hypothesis test. Also, the p-value<\/strong> is in brackets at the end of this sentence, before the punctuation, just like you would use a citation to back up a factual claim… only here, it is a claim of statistical significance.<\/p>\n

 <\/p>\n

<\/a>Concept Practice: Conducting a Z test<\/a><\/h4>\n

<\/a>5b. Single-sample T-test<\/h1>\n
video lesson<\/a><\/h6>\n

Now it is time to introduce your second real statistical test: the single-sample t-test<\/a><\/strong>. This test is much like the Z-test<\/strong>, but more commonly used in real life.<\/p>\n

Before we get started on the t-test<\/strong>, I am curious\u2026 would you have greater confidence in a statistic that comes from a small sample?<\/p>\n

\"\"<\/p>\n

Or in one that comes from an entire population?… If you said that you would have greater confidence in a number that comes from an entire population, then the field of statistics would agree with you.<\/p>\n

Sometimes, though, we do not have access to all the data from population that we need, and we have to take a best guess based on a sample. In that case, we probably want to be more conservative in our decision making. That\u2019s what a t-test<\/strong> is for.<\/p>\n

Often, even if we have information regarding the comparison population mean, we do not have access to the population standard deviation (\u03c3<\/em>). In this situation, what we must do is collect a sample reflecting population 1, calculate its mean and standard deviation, and compare that sample mean against a comparison distribution of means<\/strong> using an estimate (S<\/em>M<\/sub>) of the population (\u03c3M<\/sub><\/em>). Note the change in symbol (from \u03c3 <\/em>to S)<\/em> to reflect the fact that we are basing our estimate on sample data, rather than using a known population parameter.<\/p>\n

  <\/span>   <\/span>\" \[S^{2} = \frac{\sum (X-M)^{2}}{N-1}\] \"<\/p>\n

You may have noticed that this new formula for calculating variance in a sample has on the bottom \u201cN<\/em> minus 1\u201d rather than just \u201cN<\/em>\u201d as our old calculation formula had. Why \u201cN<\/em> minus 1\u201d? This is a correction, based on sample size, that derives from the concept of degrees of freedom<\/a><\/strong>. We will return to that concept later. For now, let us consider \u201cN<\/em> minus 1\u201d as a correction based on sample size that has the following deliberate and intentional effects on our calculation. If we have a small sample size, in which we should not have a great deal of confidence, subtracting 1 really affects the calculation. For example, if we had an N<\/em> of 2, subtracting off 1 from that sample size will mean that we divide by 1 instead of 2. That has a drastic impact on the variance calculation, in effect doubling our estimated variance. Of course in the context of hypothesis testing, more variance is bad \u2013 in that we are less likely to be able to make conclusions based on the data.<\/p>\n

On the other hand, with a larger sample size, subtracting 1 will have little impact on our calculation\u2026 dividing by 100 or dividing by 99 will have nearly identical results. Think of it this way \u2013 if we have a small sample size, we are punishing ourselves, like a handicap in a golf game. With a larger sample size, we reward ourselves, boosting our chance of a conclusion statistical decision.<\/p>\n

Subtracting off 1 is like a flat tax \u2013 a flat income tax of 10% for all Canadians would be harder on those with a low income, who would have even less money to meet basic survival needs, whereas the wealthiest Canadians would still have plenty of money for their needs. That is, of course, why democratic governments typically use progressive tax rates, because those who have more can afford to pay a greater proportion of their income and still have enough to cover their basic costs of living. With hypothesis testing, though, the flat tax is desirable, because it is our intent to penalize the smallest sample sizes the most, given the fact that we can have little confidence in the estimates we derive from them.<\/p>\n

The t-test is designed around the concept of degrees of freedom<\/strong>, which is defined as the number of scores in a given calculation that are free to vary. This is an abstract definition that is difficult to grasp, so it is helpful to consider a concrete example.<\/p>\n

\n
\n

Degrees of Freedom Practical Example<\/p>\n<\/header>\n

\n

First, consider the game of baseball.\u00a0 We understand the field-of-play consists of 9 positions.\u00a0 The coach is \u201cfree\u201d to assign any of the 9 players to any of the 9 positions. Once the 8th player is assigned to the 8th position, the 9th player-position is pre-determined, so to speak.\u00a0 In other words, the coach is not \u201cfree\u201d to pick either the last position or the last player.<\/p><\/blockquote>\n

Source:<\/em> https:\/\/www.isixsigma.com\/ask-dr-mikel-harry\/ask-tools-techniques\/can-you-explain-degrees-freedom-and-provide-example\/<\/p>\n<\/div>\n<\/div>\n

The same logic holds when doing a calculation involving all of a sample\u2019s scores. Up until the last score inputted into the formula, the scores are free to vary; but the last score is pre-determined. Thus degrees of freedom<\/strong> in a t-test<\/strong> is N<\/em>-1, or 1 less than the total number of scores in the dataset.<\/p>\n

In a t-test<\/strong> we will be using the t-distributions<\/strong>, rather than a normal distribution, as our comparison distributions. The t-distributions<\/strong> are a series of distributions, based on the normal distribution, that adjust their shape according to degrees of freedom<\/strong> (which in turn is based on sample size).<\/p>\n

As mentioned previously, when we do not have the population standard deviation we must estimate it from our sample. This involves a greater risk of error. The risk is related to how large our sample is. So, the fewer the scores in our sample, the fewer the degrees of freedom, and the more conservative the distribution we must use \u2013 one that is wider and shorter (more area in the tails).<\/p>\n

\"\"<\/p>\n

Above you can see examples of how the shape of the t-distributions changes with reduced degrees of freedom<\/strong>. With degrees of freedom<\/strong> of 100, the shape of the distribution is very close to the normal curve. If we have very few degrees of freedom<\/strong>, like 1, then our shading rejection regions, or goal posts, will move farther out into the tails of the distribution, because there is more area under the tails of the curve. In effect, then, with fewer degrees of freedom<\/strong>, we will find it more difficult to reject the null hypothesis. This is, of course, by design, in order to penalize the hypothesis test if it is based on a very small sample size.<\/p>\n

\"\"
Inspiration for this creative illustration came from one of my former students, Andi.\u00a0Blame for artwork quality is entirely on me.<\/em><\/figcaption><\/figure>\n

Above is a less technical visual to see the difference in distribution shape when the sample size is small, versus when the sample size is large. As you can see, when you have a large sample size, the goal posts are very close and easy to reach, but when you have a small sample size, they will really be far away, and this will make it difficult to reject our null hypothesis.<\/p>\n

<\/a>Concept Practice: Sample size and the t-test<\/a><\/h4>\n

 <\/p>\n

Now we are ready to take a look at how a hypothesis test works based on a single-sample t-test<\/strong>. As before, the structure of the test is familiar \u2013 we are comparing the means of Population 1 and Population 2.<\/p>\n

\"\"<\/p>\n

As usual, we do not have access to the mean of population 1, just the mean of a sample from that population. Furthermore, unlike in a Z-test<\/strong>, we do not have access to one piece of information about population 2: the population standard deviation. Therefore, we will form hypotheses the same way, but when we do steps 2 and 4, we will need to use the sample to generate an estimate of the comparison population standard deviation.<\/p>\n

<\/a>Concept Practice: When to use a Z-test or a t-test<\/a><\/h4>\n

 <\/p>\n

In step 2, we identify the known comparison population\u2019s mean. However, we will need to use the sample to generate an estimate of the comparison population standard deviation. We calculate the sample variance, using the formula with the \u201cN<\/em>-1\u201d correction,<\/p>\n

  <\/span>   <\/span>\" \[S^{2} = \frac{\sum (X-M)^{2}}{N-1}\] \"<\/p>\n

and then convert the variance to the distribution of means<\/strong> by dividing by N<\/em>,<\/p>\n

  <\/span>   <\/span>\" \[S^{2}_{M} = \frac{S ^{2}}{N}\] \"<\/p>\n

and finally we square root to get from variance to standard deviation.<\/p>\n

  <\/span>   <\/span>\" \[S_{M} = \sqrt{S^{2}_{M}}\]\"<\/p>\n

SM<\/sub><\/em> is what we will use to describe our comparison distribution.<\/p>\n

In step 3, as before we need to find our cutoff\u00a0 sample score based on the significance level and directionality of the test. Now, however, we also need to use a third piece of information: degrees of freedom<\/strong>. This is found by subtracting one from the sample size. Then we can look in the t-tables<\/a>, identifying the row of the table with the matching degrees of freedom<\/strong>, then looking in the appropriate column based on directionality and significance level. Once we identify the t-score that forms the border of the shaded tail, we can map this onto a drawing of the comparison distribution. For visualization purposes, it is fine to sketch the distribution the same way you did the normal distribution.<\/p>\n

<\/a>Concept Practice: Degrees of Freedom<\/a><\/h4>\n

 <\/p>\n

 <\/p>\n

Now for step 4. The t-test<\/strong> formula is just like the Z-test<\/strong> one on top \u2013 sample mean minus comparison population mean. Then you divide by SM<\/sub><\/em>, the estimate of standard deviation that you calculated back in step 2.<\/p>\n

  <\/span>   <\/span>\" \[t = \frac{M-\mu _{M}}{S_{M}}\]\"<\/p>\n

If your calculated t-test<\/strong> score fell into the shaded tail beyond your cutoff score, then you may reject the null hypothesis. In other words, if the sample mean was extreme enough on the comparison distribution of means, reject the null.<\/p>\n

\"\"<\/p>\n

After conducting a hypothesis test, it is helpful to also obtain a report the precise p-value<\/strong> associated with the result. The p-value<\/strong> offers the precise probability of obtaining the t-test<\/strong> result (or more extreme) just by random chance under the comparison distribution. It is found by calculating<\/a> the area under the comparison distribution in the tail beyond the observed t-test<\/strong> score calculated in step 4 (or in both tails, if it was a two-tailed test).<\/p>\n

 <\/p>\n

<\/a>Concept Practice: Conduct a t-test<\/a><\/h4>\n

<\/a>Concept Practice: Facts about Z- and t-tests<\/a><\/h4>\n

 <\/p>\n

 <\/p>\n

Chapter Summary<\/h1>\n

In this chapter we learned about the two types of logical errors we can make in hypothesis testing: Type I<\/strong> and Type II errors<\/strong>. We saw that Type I error<\/strong> is considered more serious in statistics, because it represents going out on limb, making a strong conclusion, when that is in fact the wrong decision. For that reason, Type I error<\/strong> risk is strictly controlled in inferential statistics by setting a particular significance level (\u03b1<\/strong><\/em>). We also introduced our first real inferential statistical tests: the single-sample Z-test<\/strong> and the single-sample t-test<\/strong>. In each of these tests, we are comparing the means of two populations, using a sample to estimate the mean of the research population. A Z-test<\/strong> is used when we know the standard deviation of the comparison population (\u03c3<\/strong><\/em>); a t-test<\/strong> is used when we do not have that information and must estimate the standard deviation from the sample (S<\/em><\/strong>).<\/p>\n

Key terms:<\/p>\n\n\n\n\n\n
Type I error<\/strong><\/td>\ndistribution of means<\/strong><\/td>\np-value<\/strong><\/td>\n<\/tr>\n
Type II error<\/strong><\/td>\nZ-test<\/strong><\/td>\nt-test<\/strong><\/td>\n<\/tr>\n
\u03b1<\/strong><\/em><\/td>\nstandard error of the mean<\/strong><\/td>\ndegrees of freedom<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Concept Practice<\/h1>\n

<\/a><\/p>\n

\n