Simple (univariate) Linear Regression

Overview of the Concept of Regression

A simple linear regression model attempts to model the relationship between one independent and one dependent variable (Recall: Dependent vs. independent variables). A basic assumption is that the relationship between the variables behaves in a linear, or straight line, fashion. This assumption may, or may not be true. In practice this model assumption needs to be tested. Linear regression is also called line fitting or least squares estimation because the modeling approach uses formulas for finding the y-intercept and slope of a line such that the sum of squares of distances of data points from the line will be at minimum. Because there are many data points relating x and y the regression line attempts to show how on an average their relationship behaves, e.g. linearly increasing or linearly decreasing. Simple linear regression modeling includes also assumptions concerning the average behavior of distances (called errors , estimated by residuals e) from the data points to the regression line when the values of x increase or decrease. It is assumed that the errors remain about the same and are without any predictable patterns. Formally the so called Ordinary Least Squares (OLS) assumptions, which also need to be tested, state that the errors are independent, normally distributed with mean zero and a constant variance.

Learning Objectives

When you have completed this Module you should be able to