: predicting a quantitative response $Y$ on the basis of a single predictor variable $X$, assuming there is a linear relationship.
In other words, we assume the relationship between $X$ and $Y$ as
$$ Y\approx\beta_0+\beta_1X $$
where $\beta_0$ and $\beta_1$ are the model coefficients, or parameters,
We train the data to produce coefficient estimates, $\hat\beta_0$ and $\hat\beta_1$, and compute the prediction of $Y$ on the basis of $X=x$ as
$$ \hat y=\hat\beta_0+\hat\beta_1x $$
Find the intercept $\hat\beta_0$ and slop $\hat\beta_1$ such that the resulting line is as close as possible to the observed data points.
→ Closeness can be measured in various methods, but the most common approch involves minimizing the least squares criterion.
$\hat y_i=\hat\beta_0+\hat\beta_1x_i$: the prediction for $Y$ based on the $i$th value of $X$
$e_i=y_i-\hat y_i$: the $i$th residual
$RSS$: Residual Sum of Squares
$=e_1^2+e_2^2+...+e_n^2$
$=(y_1-\hat\beta_0-\hat\beta_1 x_1)^2+(y_2-\hat\beta_0-\hat\beta_1 x_2)^2+...+(y_n-\hat\beta_0-\hat\beta_1 x_n)^2$
We choose $\hat\beta_0$ and $\hat\beta_1$ such that $RSS$ is minimized, as