{"id":132,"date":"2018-10-31T18:09:03","date_gmt":"2018-10-31T22:09:03","guid":{"rendered":"https:\/\/pressbooks.bccampus.ca\/simplestats\/?post_type=chapter&p=132"},"modified":"2019-11-15T18:44:39","modified_gmt":"2019-11-15T23:44:39","slug":"10-2-basics-of-linear-regression","status":"publish","type":"chapter","link":"https:\/\/pressbooks.bccampus.ca\/simplestats\/chapter\/10-2-basics-of-linear-regression\/","title":{"raw":"10.2 Basics of Linear Regression","rendered":"10.2 Basics of Linear Regression"},"content":{"raw":"[latexpage]\r\n\r\nYou may find it surprising but you already have<\/em> an idea about linear regression from Section 7.2.3. Again, when describing and examining the association between two continuous variables, we can use the correlation coefficient r<\/em> and a scatterplot plotting the observations in a coordinate system. To visualize the linear relationship between the variables, we could also add a line of best fit<\/em> to the scatterplot. The line of best fit is actually also called a regression line<\/em>; and regression itself is based upon the concepts of correlation and variance, with which you are already familiar.\r\n\r\n \r\n\r\nYou might be asking yourselves at this point what regression adds to the analysis of two continuous variables, or in other words, why do we even need it -- don't we already have Pearson's r<\/em> for that? As you will see in the examples below, linear regression allows us to precisely calculate and predict a change in the dependent<\/em> variable that is due to the independent<\/em> variable<\/strong>.\r\n\r\n \r\n\r\nWhat we say in this case is that the independent variable explains<\/em> a percentage of the variance of the dependent variable<\/strong>. Think about it this way: the dependent variable varies due to arguably many causes (i.e., independent variables), which affect it to a different extent and which each explain some part of its total variance. Through linear regression, we are able to quantify to what extent an independent variable explains the variability of the dependent variable, i.e., to what extent it affects it<\/strong>[footnote]Multivariate regression thus allows for direct comparisons of the size of the independent variables' effects. In the bivariate case, we only focus on the effect of one<\/em> independent variable, without considering and accounting for others -- which is not something you should do in a real-life social science research, especially in terms of causal analysis. Again, the bivariate case serves only as an illustration\/introduction to the expansive topic of regression in general.[\/footnote].<\/strong> To take the example about parental and offspring education from the previous section, doing a regression analysis on these two variables would allow us to predict how much more education a respondent is expected to have for one more year of schooling for the parent[footnote]Or, to put it differently, if one father has one more year of schooling than anther father, how much more schooling the offspring of the first would be expected to have in comparison to the offspring of the second.[\/footnote] (father, in our case), and what percent of respondent's schooling is explained by the years of education of the parent.\r\n\r\n \r\n\r\nHow does linear regression do all that? To put it simply, through the regression line (of best fit), or more precisely, through the way the regression line is created.\r\n\r\n \r\n\r\nThe linear function.<\/strong> How do you draw a line? The simplest method requires exactly two pieces of information: a starting point of the line, and an indicator of slope (so that you know whether the line is straight, sloping upward, or sloping downward). This is captured in the following formula:\r\n\r\n \r\n\r\n$$y=\\alpha + \\beta x$$\r\n\r\n \r\n\r\nwhere\u00a0\u03b1<\/em> is the line's starting point and\u00a0\u03b2<\/em> is the slope of the line. The two variables, x<\/em> and y<\/em>, are the independent and the dependent variable, respectively: we know this because the formula establishes\u00a0y<\/em> as a function<\/em> of x<\/em> (i.e., if we know \u03b1<\/em> and \u03b2<\/em>, we can calculate y<\/em> for any value of x<\/em>).\r\n\r\n \r\n\r\nLet's take a brief example.\r\n\r\n \r\n
\r\n

Example 10.2 Class Assignment Mark<\/em><\/p>\r\n\r\n<\/header>\r\n

\r\n\r\n \r\n\r\nImagine you are given a written, take-home assignment in some class. Your professor has stipulated that there are three part of the assignment, each worth 30 points, and that you would receive 10 points just for turning in your work.\r\n\r\n \r\n\r\nIn this case, your assignment mark is entirely a function of your submitted work. You will be getting 10 points to start with, then 30 points for fulfilling each of the three requirements. The class grades on the submitted assignments could thus be 10 points (0 completed requirements), 40 points (1 completed requirement), 70 points (2 completed requirements), and 100 points (3 completed requirements). Figure 10.1 plots this.\r\n\r\n \r\n\r\nFigure 10.1 Assignment Mark as a Function of Completed Requirements<\/em>\r\n\r\n\"\"\r\n\r\n<\/div>\r\n \r\n
\r\n\r\n \r\n\r\nAs you can see, the relationship between the two variables, assignment requirements completed<\/em> and assignment mark<\/em>, is simply\r\n\r\n \r\n\r\n$$y=10+30x$$\r\n\r\n \r\n\r\nas helpfully shown in the graph itself. This is a summary form of having to write out all the observations:\r\n