this ass

Linear Regression Implementation Download the advertising dataset. This is a dataset that gives us the total sales for different products, after marketing them on Television, Radio and Newspaper.
- Implement the Linear regression algorithm to predict the sales value, given three inputs (Television, Radio and Newspaper).
- Calculate the cost function after each iteration, draw cost function as a function of the iteration
- Try the program with various values for the learning rate alpha and number of iteration.’
Solve the problem 1 with normal equation method
Redo exercise 1 with regularization
Redo exercise 2 with regularization

plan

what the fuck is a Linear regression?

it is to my knowledge: fit a thing to a linear equation, with the Minimal Error.

this is the cost function, also loss function:


    E = \frac{1}{2m} \sum^m_{i = 1}\left(h(x_{(i)})- y_{(i)}\right)^2

with h(x) being our model (we are trying to find a and b and make our guess h(x) as close to y as possible)

this comes from the error function here:

e = \frac{1}{2}\left(h(x_{(i)})-y_{(i)}\right)^2

and the cost function from the error function:

E = \frac{1}{m}\sum^m_{i=1}e_{(i)}

as were dealing with linear fitting (h(x) = ax + b) the thing turns to this:


E = \frac{1}{2m} \sum_{i=1}^{m} \left( ax_i + b - y_i \right)^2

chatgpt-ed this shit it calls this strat Mean Squared Error (MSE)

partial derivative of course.

its newton raphson all over again:

you have to change every coefficients so they yk fit the thing every time duh

and how you change goes like this:

a:= a - \alpha \frac{\partial E}{\partial a}

yes. partial deriv that fucking series wrt a. fucking insane innit.

however they did the calculations:

for our case h(x) = ax + b we have


\frac{\partial}{\partial a}{E(a, b)}
\text{ or } \\
\frac{\partial E}{\partial a} = \frac{1}{m}\sum^m_{i=1}(ax_{(i)}+b - y_{(i)})\cdot x_{(i)}

yeah also for the case of b:

b:= b - \alpha \frac{\partial E}{\partial b} \frac{\partial E}{\partial b} = \frac{1}{m}\sum^m_{i=1}(ax_{(i)}+b - y_{(i)})