Case 3: Impact of Outliers - Let's continue to use the data from Case 1 and incorporate a small but noticeable change, an outlier data point. From the modeling point of view this change is very significant as can be see in the graph below. Please refer also to Outliers, Scatterplots, Regular Scatterplots, and Data Reduction.

In this example one y value is changed, from y = 3 to y = 15. This change will affect the regression model line significantly (see animation below, and compare to Example 1). Even though this model has the minimum SSE for this data it still does not follow the pattern of the majority of the data points.

In practice data outliers may cause a rapid model deterioration, and consequently, the value of such models as decision making tools suffers. As a general recommendation, one should go back to the original data and check if the outliers are a result of e.g. a recording error, measurement error, or error in the design of the experiment (DOE). Outliers, which lie two or more standard deviations away from the model line should be eliminated. See also the section Confidence interval for a regression line