1.0 Supervised Machine Learning
Types of machine learning algorithms that makes predictions based on given examples (training data).
2.0 Learning Rate
A tuning parameter that is essentially the step size of each optimization iteration while moving toward a minimum of loss function.
3.0 Regularization
A process of encouraging a learning algorithm to shrink the weights (importance) of the parameters without necessarily demanding that the parameter is set to exactly zero. In simpler terms, adding large regularization term to the cost function reduces the weight, w of an algorithm.
4.0 Symbols
5.0 Summary of Comparison Between Regression and Classification
5.1 Definition
5.1.1 Regression
Regression is an algorithm to predict a number from an infinitely many possible numbers (Continuous).
5.1.2 Classification
Classification is an algorithm to predict categories or classes (Discrete).
5.2 Simplest Model
5.2.1 Regression
- Multiple Linear Regression & Vectorized Form
5.2.2 Classification
- Logistic Regression (Sigmoid Function)
5.3 Simple Cost Function
5.3.1 Regression
- Squared Error
- Covex Shape
5.3.2 Classification
- Simplified Loss Function (derived from Maximum Likelihood Estimation)
- Discontinuous Shape
5.4 Modified Cost Function (Regularization)
5.4.1 Regression
5.4.2 Classification
5.5 Gradient Descent
5.5.1 Regression
- Goal: Minimisze J(w,b)
5.5.2 Classification
- Goal: Minimisze J(w,b). Looks the same as regression, just function f definition is different
5.6 Modified Gradient Descent
5.6.1 Regression
5.6.2 Classification
- Looks the same as regression, just function f definition is different
References
[1] “Supervised Machine Learning: Regression and Classification,” Coursera. [Online]. https://www.coursera.org/learn/machine-learning. (Accessed: Oct 28, 2022)
[2] A. E. Yilmaz, “A Taxonomy of Artificial Neural Networks,” M. S. Thesis, Department of Mathematics, Koc University, Instanbul, 2020. [Online]. Available: https://www.researchgate.net/publication/342720152_A_Taxonomy_of_Artificial_Neural_Networks