Learning patterns from data to make predictions/decisions

Compared to traditional programming (rules vs. learned models)

Samples (instances) and features (variables)

Labels/targets (for supervised learning)

Feature engineering vs. representation learning

Supervised learning (classification, regression)

Unsupervised learning (clustering, dimensionality reduction)

Semi-supervised learning (limited labels)

Self-supervised learning (pretext tasks from unlabeled data)

Reinforcement learning (agents, rewards, policies)

Paradigms are defined mainly by what supervision signal is available and how feedback is obtained.

Hypothesis/model family and parameters

Loss functions (MSE, cross-entropy)

Optimization (gradient descent, stochastic gradient descent)

Regularization (L1/L2, dropout, early stopping)

Training vs. validation vs. test sets

Bias–variance tradeoff

Cross-validation

Classification: accuracy, precision, recall, F1, ROC-AUC

Regression: MAE, MSE/RMSE, R²

Ranking/retrieval: MAP, NDCG

Calibration and threshold selection

Cleaning (missing values, outliers)

Scaling/normalization

Encoding categorical variables (one-hot, embeddings)

Handling imbalance (resampling, class weights)

Experiment tracking and reproducibility

Model deployment (batch, online, edge)

Monitoring (drift, performance, retraining)

Linear regression (ridge, lasso)

Logistic regression

Decision trees

Random forests (bagging)

Gradient boosting (XGBoost, LightGBM, CatBoost)

k-Nearest Neighbors (kNN)

Linear and kernel SVMs

Naive Bayes

Gaussian mixture models (GMM)

Hidden Markov models (HMM)

k-means

Hierarchical clustering

DBSCAN

PCA

t-SNE / UMAP (visualization)

Autoencoders

Multilayer perceptrons (MLP)

Convolutional neural networks (CNN)

Recurrent networks (RNN, LSTM/GRU)

Transformers (attention-based)

Markov decision processes (states, actions, rewards)

Value-based methods (Q-learning, DQN)

Policy gradients (REINFORCE, PPO)

Algorithm families differ by assumptions (linearity, locality, probabilistic structure) and data/compute needs.

Classification vs. regression vs. ranking vs. forecasting

Online vs. offline learning

Simpler baselines first

Capacity vs. data size and noise

Interpretability vs. accuracy tradeoffs

Learning rate, depth, number of estimators, regularization strength

Tuning (grid search, random search, Bayesian optimization)

Global vs. local explanations

Feature importance, SHAP, LIME

Text classification, sentiment analysis

Machine translation, summarization

Information extraction, question answering

Image classification

Object detection, segmentation

OCR and document understanding

Speech recognition

Speaker identification

Audio event detection

Collaborative filtering

Content-based and hybrid recommenders

Ranking and personalization

Demand forecasting

Anomaly detection in logs/sensors

Predictive maintenance

Medical imaging analysis

Risk prediction and triage

Genomics and drug discovery support

Credit scoring

Fraud detection

Algorithmic trading signals

Quality inspection

Supply chain optimization

Robotics and control

Intrusion/malware detection

Phishing and spam filtering

Train-test contamination prevention

Proper temporal splits for time series

Bias and disparate impact

Privacy-preserving learning (differential privacy, federated learning)

Transparency and accountability

Spurious correlations

Distribution shift and concept drift

Adversarial and robustness issues