Overview of Multiple Linear Regression

A simple linear regression model, Module 2, attempts to model the relationship between one independent and one dependent variable (Recall: Dependent vs. independent variables). As a matter of fact simple linear regression is the core case of multiple linear regression. The only difference is that we have more than one independent variable. Like before, with respect to multiple linear regression models we assume that

• the behavior of a dependent variable y can be explained by a linear, additive relationship between the dependent variable and a set of k independent variables (x_i, i=1 to k), (for simple linear regression k=1),
• the relationship between any of the independent variables x_i and the dependent variable y behaves in a linear, or straight line, fashion,
• the independent variables x_i are independent of each other, and
• the errors are independent, normally distributed with mean zero and a constant variance ((OLS)).

As you already know these assumptions may, or may not be true. In practice all model assumptions need to be tested, and, in practice, there will be no perfect models. Like in simple linear regression we attempt to fit a line, plane or hyperplane to a set of data. (Note: a plane here is synonymous to a surface.) With two independent variables and one dependent variable we are dealing with a 3D space, and therefore the models represent planes, which we can draw if we want. Beyond one dependent- and two independent variables the models, represent what is commonly called, hyperplanes which we cannot draw.

Learning Objectives

When you have completed this Module you should be able to

• understand the concept of multiple linear regression
• explain the concept of multiple linear regression to others
• explain and utilize model assumptions (Ordinary Least Squares assumptions, or OLS)
• estimate parameters
• conduct parameter- and model testing
• conduct a simple residual analysis
• interpret results.