Application to linear models

Everything You Need in One Place

Homework problems? Exam preparation? Trying to grasp a concept or just brushing up the basics? Our extensive help & practice library have got you covered.

Learn and Practice With Ease

Our proven video lessons ease you through problems quickly, and you get tonnes of friendly practice on questions that trip students up on tests and finals.

Instant and Unlimited Help

Our personalized learning platform enables you to instantly find the exact walkthrough to your specific type of question. Activate unlimited help now!

Get the most by viewing this topic in your current grade. Pick your course now.

?
Intros
Lessons
  1. Applications to Linear Models Overview:
  2. Applying Least-Squares Problem to Economics
    • Go from Ax=bAx=b to Xβ=yX\beta=y
    XX → design matrix
    β\beta → parameter vector
    yy → observation vector
  3. Least-Squares Line
    • Finding the best fit line
    • Turning a system of equations into Xβ=yX\beta =y
    • Using the normal equation XTXβ=XTyX^T X\beta=X^T y
    • Introduction of the residual vector
  4. Least-Squares to Other Curves
    • Finding the Best Fit Curve (not a line)
    • Using the normal equation XTXβ=XTyX^T X\beta=X^T y
  5. Least-Squares to Multiple Regressions
    • Multiple Regression → multivariable function
    • Finding a Best Fit Plane
    • Using the normal equation XTXβ=XTyX^T X\beta=X^T y
?
Examples
Lessons
  1. Finding the Least-Squares Line
    Find the equation y=β0+β1xy=\beta_0+\beta_1 x of the least-squares line that best fits the given data points:
    (0,1),(1,2),(2,3),(3,3)(0,1),(1,2),(2,3),(3,3)
    1. Finding the Least-Squares of Other Curves
      Suppose the monthly costs of a product depend on seasonal fluctuations. A curve that approximates the cost is
      y=β0+β1x+β2x2+β3cosy= \beta _0 + \beta _1 x+ \beta _2 x^2 + \beta_3 \cos (2πx12\frac{2 \pi x}{12})

      Suppose you want to find a better approximation in the future by evaluating the residual errors in each data point. Let's assume the errors for each data point to be ϵ1,ϵ2,,ϵn\epsilon_1,\epsilon_2,\cdots,\epsilon_n.
      Give the design matrix, parameter vector, and residual vector for the model that leads to a least-squares fit for the equation above. Assume the data are (x1,y1),,(xn,yn).(x_1,y_1 ),\cdots,(x_n,y_n).
      1. An experiment gives the data points (0,1),(1,3),(2,4),(3,5)(0,1) , (1,3) , (2, 4), (3, 5). Suppose we wish to approximate the data using the equation
        y=A+Bx2y=A+Bx^2

        First find the design matrix, observational vector, and unknown parameter vector. No need to find the residual vector. Then find the least-squares curve for the data.
        1. Finding the Least Squares of Multiple Regressions
          When examining a local model of terrain, we examine the data points to be (1,1,3),(2,2,5),(1,1, 3), (2, 2, 5), and (3,1,3)(3, 1, 3). Suppose we wish to approximate the data using the equation
          y=β0u+β1vy=\beta_0 u+\beta_1 v

          First find the design matrix, observational vector, and unknown parameter vector. No need to find the residual vector. Then find the least-squares curve for the data.
          1. Proof Question Relating to Linear Models
            Show that
            Xβ^2=βTXTy\lVert X \hat{\beta} \rVert^2=\beta^TX^Ty
            Topic Notes
            ?
            In this section, we will apply least square problems to economics.

            Instead of finding the least squares solution of Ax=bAx=b, we will be finding it for Xβ=yX\beta =y where

            XX → design matrix
            β\beta → parameter vector
            yy → observation vector

            Least-Squares Line
            Suppose we are given data points, and we want to find a line that best fits the data points. Let the best fit line be the linear equation
            y=β0+β1xy=\beta_0 + \beta_1 x

            And let the data points be (x1,y1),(x2,y2),,,(xn,yn)(x_1,y_1 ),(x_2,y_2 ),\cdots,,(x_n,y_n ). The graph should look something like this:
            best fit liney=beta_0 + beta_1 x

            Our goal is to determine the parameters β0\beta_0 and β1\beta_1. Let's say that each data point is on the line. Then
            best fit line data points beta_0 + beta_1 x

            This is a linear system which we can write this as:
            linear system

            Then the least squares solution to Xβ=yX\beta=y will be XTXβ=XTyX^T X \beta =X^T y.

            General Linear Model
            Since the data points are not actually on the line, then there are residual values. Those are also known as errors. So we introduce a vector called the residual vector ϵ\epsilon, where
            ϵ=yXβ\epsilon = y - X\beta
            y=Xβ+ϵ y = X\beta + \epsilon

            Our goal is to minimize the length of ϵ\epsilon (the error), so that XβX\beta is approximately equal to yy. This means we are finding a least-squares solution of y=Xβy=X\beta using XTXβ=XTyX^T X\beta=X^T y.

            Least-Squares of Other Curves
            Let the data points be (x1,y1),(x2,y2),,,(xn,yn)(x_1,y_1 ),(x_2,y_2 ),\cdots,,(x_n,y_n ) and we want to find the best fit using the function y=β0+β1x+β2x2y=\beta_0+\beta_1 x+\beta_2 x^2, where β0,β1,β2\beta_0,\beta_1,\beta_2 are parameters. Technically we are using a best fit quadratic function instead of a line now.
            best fit quadratic function

            Again, the data points don't actually lie on the function, so we add residue values ϵ1,ϵ2,,ϵn\epsilon_1,\epsilon_2,\cdots,\epsilon_n where

            add residue values 
 to data points

            Since we are minimizing the length of ϵ\epsilon, then we can find the least-squares solution β\beta using XTXβ=XTyX^T X\beta=X^T y. This can also be applied to other functions.

            Multiple Regression
            Let the data points be (u1,v1,y1),(u2,v2,y2),,,(un,vn,yn)(u_1,v_1,y_1 ),(u_2,v_2,y_2 ),\cdots,,(u_n,v_n,y_n ) and we want to use the best fit function y=β0+β1u+β2vy=\beta_0+\beta_1 u+\beta_2 v, where β0,β1,β2\beta_0,\beta_1,\beta_2 are parameters.
            best fit function Multiple Regression

            Again, the data points don't actually lie on the function, so we add residue values ϵ1,ϵ2,,ϵn\epsilon_1,\epsilon_2,\cdots,\epsilon_n where

            add residue values to data points of the function

            Since we are minimizing the length of ϵ\epsilon, then we can find the least-squares solution β\beta using XTXβ=XTyX^T X\beta=X^T y. This can also be applied to other multi-variable functions.