Hi folks! I just came up with another tag name “JustForFun” + “ML”. So, under the tag, I will try to concisely explain and introduce important concepts of machine learning.
Step 1. Groundwork
To begin with, let’s do some groundwork. Largely, machine learning can be divided into three parts:
1) Supervised learning
2) Unsupervised learning
3) Reinforcement learning
- Supervised learning
We want to estimate a function that maps input x into output y. - Unsupervised learning
The only thing we have is input x. Thereby, we want to estimate the pattern within x. - Reinforcement learning
This is somewhat distinguished from the above, as we have state, action, and most importantly reward in the setting. So that the agent in the setting tries to outperform its previous self, for the sake of achieving the utmost, optimal sequence of actions.
Step 2. Classification — Do you like the Simpsons?
Let’s imagine a world that people living here can only be categorized into two groups: people who love the Simpsons (label: ‘fan’), and people who do not love it(label: ‘not a fan’). Personally I cannot believe there would be anyone who does not love the series, but it is just an imaginary setting, so let’s get going.
Given a person x, we need to put her/him into the labels. If the predicted label y is equal to the real label, then our error rate will be unchanged. However, if the predicted one is not true, then our error rate will get extra points.
To implement a function that measures the accuracy, or error rate in another aspect, we will use indicator variable I.
I can only have 0 or 1. When the predicted label is not equal to the real one, I’s value is 1. When I’s value is 0, we know that the prediction was correct.
Step 3. Bayes Classifier
We can show that the above ‘training error rate’ can be minimized by simply assigning each observation to the most likely class label. This can be summarized in the following conditional probability form.
This shows a probability when Y = j given the observation vector X equals x0. So, there would be a boundary separating people who love the Simpsons and who do not. This is called ‘Bayes decision boundary’.
Step 4. KNN (K-Nearest Neighbors)
Theoretically, it is best to use Bayes Classifier when it comes to predicting categorical values. However, most of the time, we have no idea about the conditional probability distribution of Y given X. Thereby, Bayes Classifier is used as a benchmark to compare each different approach to classify datasets. And one of them is KNN, K-Nearest Neighbors Classifier.
For a given positive integer K and observation x0, KNN Classifier identifies the closest K points around the x0 (K is represented as N0 in the above Fig 3) within the training data. After it estimates the conditional probability by computing the ratio checking how many of them are labeled correctly as j.
K is the choice we make, and this choice of ours affects greatly to the performance of this classifier.
Yes, it would have been far better if I had added some visualizations to this. Perhaps I would come back later with some effective scatter plots with decision boundaries plotted.
Anyhow, this is it for today! Thank you, and see you later! ⛹🏽 🏋🏽 🚴🏽
Or you may be interested in loading images into TensorFlow. Then try this.
