Nonlinear regression
β¦ the safe use of regression requires a good deal of thought and a good dose of skepticism
Review of linear regression
Many useful models of physical systems are linear equations of the form y = Βx + α. Examples include Ohm's law for electrical resistors and Hooke's law for mechanical springs. Ohm's law models the relationship between the voltage (V) across and the current (I) through a resistor as a line: V = IR. The proportionality constant R is called the resistance. Hooke's law, F = kx, is a good model for the relationship between force (F) across and extension (x) of springs under many conditions. In order to apply these laws to real systems, the proportionality constant must be measured.
One possible way to measure the constant of a spring is to exert a known force on it (by hanging a known mass on it in Earth's gravity, for example) and measure the spring's extension. Two points are needed to define a line. The measurement can be repeated with a different known mass to provide a second datapoint. (In the case of a spring, choosing F = 0 for the second measurement makes the effort rather minimal, since x also equals 0.)
If the two datapoints are known with infinite precision, solving for the exact value of k and π½ is trivial. Unfortunately, all physical measurements include noise. The presence of noise precludes finding the exact values of πΌ and π½.
Linear Regression is a method for finding the magnitude of the relationship between two variables that co-vary.
A simple way to measure R or
The technique assumes that a straight line characterizes the relationship between the two quantities: π¦=π½π₯+πΌ, where π½ is the true slope and πΌ is the true intercept. Some e A simple way to find α and β is to measure the y at two different values of x, giving the datapoints (xi, yi); i = {1,2}.
Measurement noise can be modeled by adding a noise term, εi, to the right side of the model equation: yi=Βix+α+εix. The function of linear regression is to produce estimates of πΌ and π½, denoted by α̂ and β̂, from a sample of N value pairs (xi, yi); i = {1, ..., N} that includes noise in the y-values. The most common regression model assumes that x is known exactly. In practice, regression works well if the relative magnitude of noise in x is much smaller than y.
The most common type of LR minimizes the value of the squared vertical distances between observed and predicted values
Model :
Assumptions:
the independent variable π₯ is known with certainty (or at least very much less error than π¦)
π is an independent, random variable with π=0
The distribution of π is symmetric around the origin
the likelihood of large errors is less than small ones
Uncertainty in slope estimate
The error in slope π=π½Β Μβπ½
Variance of π characterizes slope error
You can calculate a 95% (or other significance level) confidence interval for π½Β Μ
What factors should the uncertainty depend on?
Estimate π^2 (π): π^2 (π)=(ββγπ_π^ γ^2 )/((πβ2)ββγ(π₯_πβπ₯Β Μ
)γ^2 )
N-2 is a βpenaltyβ because regression line minimizes variance of residuals
If the interval contains 0, the null hypothesis that π½=0 cannot be rejected
Step 1: PLOT THE DATA
Examine the residuals
- plot 'em for an informal look
- various tests of residuals exist
Overview of nonlinear regression
βIf your program generates an error message rather than curve fitting results, you probably wonβt be able to make sense of the exact wording of the messageβ
— Fitting Models to Biological Data using Linear and Nonlinar Regression by Motulsky and Shristopoulos
Block diagram of nonlinear regression |