{"id":1340,"date":"2019-04-24T16:56:03","date_gmt":"2019-04-24T20:56:03","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/simplestats\/?post_type=chapter&p=1340"},"modified":"2019-11-15T19:04:31","modified_gmt":"2019-11-16T00:04:31","slug":"10-2-2-elements-of-the-linear-regression-model","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/simplestats\/chapter\/10-2-2-elements-of-the-linear-regression-model\/","title":{"raw":"10.2.2 Elements of the Linear Regression Model","rendered":"10.2.2 Elements of the Linear Regression Model"},"content":{"raw":"[latexpage]\r\n\r\nThe secret to minimizing the residuals -- and to ensuring the regression line is indeed the best fitting<\/em> (to the data) line -- lies in the way the elements of the line are calculated. The regression\/preditcion line is, after all, created through a<\/em> and b<\/em>, as I explained in Section 10.2:\r\n\r\n \r\n\r\n$$\\hat{y}=a+bx=$$ = predicted values<\/em>\r\n\r\n \r\n\r\nWe can calculate a<\/em> and b<\/em> such that they minimize the residuals through the following formulas:\r\n\r\n \r\n\r\n$$b=\\frac{\\Sigma{(x-\\overline{x})(y-\\overline{y})}}{\\Sigma{(x-\\overline{x})^2}}=\\frac{SP}{SS_x}=$$ = slope<\/em>, or regression coefficient<\/em>\r\n\r\n \r\n\r\n$$a=\\overline{y}-b\\overline{x}=$$ = Y-intercept<\/em>, or constant<\/em>\r\n\r\n \r\n\r\nwhere SP<\/em> is, again, the sum of products, SSx<\/sub><\/em> is the sum of squares for x<\/em>, and $\\overline{x}$ and $\\overline{y}$ are the variable means of x<\/em> and y<\/em>, respectively.\r\n\r\n \r\n\r\nAs with the correlation coefficient r<\/em>, once again, everything revolves around variances (and means)[footnote]So much so that the correlation coefficient r<\/em> and the regression coefficient b<\/em> are related: $b=r\\frac{s_y}{s_x}$ where sy<\/sub><\/em> and sx<\/sub><\/em> are, of course, the standard deviations of y<\/em> and x<\/em>, respectively.[\/footnote].\r\n\r\n \r\n\r\nAn example will serve best to illustrate all this.\r\n\r\n \r\n
\r\n

Example 10.3 Assignment Requirements and Marks<\/em><\/p>\r\n\r\n<\/header>\r\n

\r\n\r\n \r\n\r\nHere I continue with the fictitious data on which Figure 10.2 is based. In a \"sample\" of N<\/em>=11, I have data about the \"respondents\"' completed assignment requirements (x<\/em>) and their assignment marks (y<\/em>). In Table 10.3, I calculate the necessary means, sums of squares, and sum of products.\r\n\r\n \r\n\r\nTable 10.3 Assignment Requirements and Marks: Calculating a and b<\/em>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
$x$<\/td>\r\n$y$<\/td>\r\n\u00a0$(x-\\overline{x})$<\/td>\r\n$(x-\\overline{x})^2$<\/td>\r\n$(y-\\overline{y})$<\/td>\r\n$(y-\\overline{y})^2$<\/td>\r\n\u00a0$(x-\\overline{x})(y-\\overline{y})$<\/td>\r\n<\/tr>\r\n
0<\/td>\r\n10<\/td>\r\n-1.64<\/td>\r\n2.68<\/td>\r\n-41.82<\/td>\r\n1748.76<\/td>\r\n68.43<\/td>\r\n<\/tr>\r\n
1<\/td>\r\n40<\/td>\r\n-0.64<\/td>\r\n0.40<\/td>\r\n-11.82<\/td>\r\n139.67<\/td>\r\n7.52<\/td>\r\n<\/tr>\r\n
2<\/td>\r\n70<\/td>\r\n0.36<\/td>\r\n0.13<\/td>\r\n18.18<\/td>\r\n330.58<\/td>\r\n6.61<\/td>\r\n<\/tr>\r\n
3<\/td>\r\n100<\/td>\r\n1.36<\/td>\r\n1.86<\/td>\r\n48.18<\/td>\r\n2321.49<\/td>\r\n65.70<\/td>\r\n<\/tr>\r\n
1<\/td>\r\n30<\/td>\r\n-0.64<\/td>\r\n0.40<\/td>\r\n-21.82<\/td>\r\n476.03<\/td>\r\n13.88<\/td>\r\n<\/tr>\r\n
1<\/td>\r\n45<\/td>\r\n-0.64<\/td>\r\n0.40<\/td>\r\n-6.82<\/td>\r\n46.49<\/td>\r\n4.34<\/td>\r\n<\/tr>\r\n
2<\/td>\r\n55<\/td>\r\n0.36<\/td>\r\n0.13<\/td>\r\n3.18<\/td>\r\n10.12<\/td>\r\n1.16<\/td>\r\n<\/tr>\r\n
2<\/td>\r\n40<\/td>\r\n0.36<\/td>\r\n0.13<\/td>\r\n-11.82<\/td>\r\n139.67<\/td>\r\n-4.30<\/td>\r\n<\/tr>\r\n
3<\/td>\r\n85<\/td>\r\n1.36<\/td>\r\n1.86<\/td>\r\n33.18<\/td>\r\n1101.03<\/td>\r\n45.25<\/td>\r\n<\/tr>\r\n
3<\/td>\r\n90<\/td>\r\n1.36<\/td>\r\n1.86<\/td>\r\n38.18<\/td>\r\n1457.85<\/td>\r\n52.07<\/td>\r\n<\/tr>\r\n
0<\/td>\r\n5<\/td>\r\n-1.64<\/td>\r\n2.68<\/td>\r\n-46.82<\/td>\r\n2191.94<\/td>\r\n76.61<\/td>\r\n<\/tr>\r\n
\"\\overline{x}\"1.6<\/td>\r\n\"\\overline{y}\"51.8<\/td>\r\n<\/td>\r\nSSx<\/sub><\/em>=12.55<\/strong><\/td>\r\n<\/td>\r\nSSy<\/sub><\/em>=9963.64<\/strong><\/td>\r\nSPxy<\/sub><\/em>=337.27<\/strong><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nThen, I substitute the relevant numbers into the formulas for a<\/em> and b<\/em>:\r\n\r\n \r\n\r\n$$b=\\frac{SP}{SS_x}={337.27}{12.55}=26.88$$\r\n\r\n \r\n\r\n$$a=\\overline{y}-b\\overline{x}=51.8-26.88\\times 1.6=51.8-43.99=7.83$$\r\n\r\n \r\n\r\nThis makes our best-fitting\/regression line<\/strong> this:\r\n\r\n \r\n\r\n$$\\hat{y}=a+bx=7.83+26.88x$$\r\n\r\n \r\n\r\n... which is exactly what SPSS had already told us, if you care to go back to Figure 10.2 in the previous section and check.\r\n\r\n \r\n\r\nYou may or might not be impressed by this, but you certainly need to know how to interpret it. In this case the regression tells us that a student who doesn't complete even one requirement of their assignment is expected to receive 7.83 points <\/strong>(that's the constant, or Y-intercept); further, for every requirement completed, their mark would increase by 26.88 points\u00a0<\/strong>(that's the regression coefficient). That is, the effect of one completed requirement on the assignment mark is 26.88 points.<\/strong>\r\n\r\n \r\n\r\nWe can also calculate the actual predicted values (which form the regression line itself):\r\n