Regression problems focus on solving the prediction of continuous target variables and are suitable for large data sets containing a large number of numerical features, many of which are correlated. For example, use historical stock market returns to predict future stock price performance, or use a company's historical financial indicators to predict the probability of bond default.

Penalized regression

Classification problems focus on classifying observations. When the dependent variable (target) is a categorical variable, the model that associates the results with independent variables (features) is called a "classifier". Examples include financial fraud or no fraud (two categories), rating assignment (multiple categories, sequential)

Reduce the number of features while retaining the differences in observed results. For example, in the field of investment and risk management, identify the main factors affecting asset prices.

Categorize observational data, such as companies by financial metric characteristics rather than by industry or region.

The evaluation of any ML algorithm focuses on the prediction error on new data rather than the goodness of fit on the data the algorithm was fit to (i.e., the training data).

Algorithms with erroneous assumptions

Unstable models pick up noise.

Base error due to randomness in the data

underfitting means the model does not capture the relationships in the data

Occam’s razor Occam’s razor