Before you start!

Did you take the Module 3 quiz?!! I hope you did. We are moving forward quite fast. You should have a quite good understanding what linear regression, both simple linear- and multiple linear regression, are all about. It is modeling the relationship between variables, or 'line' fitting. If then the 'line' doesn't really appear to fit, then possibly there is no regression. Please remember that common sense rules here, and in all modeling.

So far we have studied the relationship between one dependent and several independent variables. We have assumed that the relationships are linear and additive. Here we will continue to expand on the concepts covered in the previous modules. We will incorporate polynomial terms into our regression models. However, because we cannot draw beyond three dimensions, we will not be able to study visually the relationships between variables beyond three variables, i.e. one dependent and two independent variables. Therefore, we will here first study polynomial models with one independent variable. We will be able to develop and analyze the models like we did earlier in simple- and multiple linear regression. All concepts discussed in Modules 2 and 3 are still valid. Therefore, it is very important that you go back to those Modules for review of any concept that remains unclear. Please review the concepts and steps for model- and parameter testing, as well as the testing of OLS assumptions.

Note: You should continue to plot variables pair-wise against each other to study and understand their relationship. Most desirably you should plot all possible variable pairs, and then at the end use this information to study e.g. whether variables appear to have relationships, and whether parameter signs (+,-) appear to make sense.

Note: What about the relationship between the independent variables!?! It is very important to note that, in polynomial regression and multiple linear regression the independent variables should be independent of each other. In other words, if you plot two independent variables against each other, then, most desirably, the plot should show no or little association between the variables. The correlation coefficient will be a help, but as you saw earlier in Module 1, it will miss strong non-linear and polynomial relationships. Therefore, the best is still to plot the variables against each other. In practice, however, true independency between variables in multivariate cases is rare.

Note: Please also note the key word in the title of this module. That key word is polynomial. We are extending the relationships between one or more independent variables and a dependent variable to include linear (or straight line) and polynomial (non-linear with respect to independent variables) relationships.

The Electronic Textbook (by StatSoft) is again a good starting point. It has lots of material available, beyond what is covered in this entire course, for your reading and review. From that book, please read through the following chapters and sections:

Please start in the Glossary and look for the word polynomial

Nonlinear Estimation

• General purpose
• Estimating linear and nonlinear models
• Common nonlinear regression models
  • Intrinsically linear regression models
  • Intrinsically nonlinear regression models

You might want to review the below multiple linear regression sections again, now from the perspective of polynomial regression.

Multiple Regression

• General purpose
• Computational approach
  • Least squares
  • The regression equation
  • Unique prediction and partial correlation
  • Predicted and residual scores
  • Residual variance and r-square
  • Interpreting the correlation coefficient
• Assumptions, limitations and practical considerations
  • Assumptions of linearity
  • Normality assumption
  • Limitations
  • Choice of variables
  • Multicollinearity and matrix ill-conditioning

    In particular, please read

  • Fitting centered polynomial models
  • The importance of residual analysis

Note: Like before, extensive use of graphical tools will help you learn to understand the data under consideration, as well as the relationship between variables. Even just for the fun of it, please plot variables against each other always to learn about their relationship and the magnitude of that relationship. Because of the importance of graphical analysis please review again the following:

Graphical Techniques

• Scatterplots
• Regular scatterplots
• Data reduction
• Graphical breakdown
• Special topics in graphical analytical techniques: Brushing
• Brushing defined

Throughout this course reference will be made to these sections.

Also, please remember to turn to the book's very extensive Glossary when you need a short more technical overview of a topic or concepts, or are looking for a definition, and to the Statistical Tables if you need to check critical values for your t- and F-tests with respect to your regression models.