2. Machine Learning Terminology
By Bernd Klein. Last modified: 17 Feb 2022.
Classifier
A program or a function which maps from unlabeled instances to classes is called a classifier.
Live Python training
Enjoying this page? We offer live Python training courses covering the content of this site.
See our Python training courses
Confusion Matrix
A confusion matrix, also called a contingeny table or error matrix, is used to visualize the performance of a classifier.
The columns of the matrix represent the instances of the predicted classes and the rows represent the instances of the actual class. (Note: It can be the other way around as well.)
In the case of binary classification the table has 2 rows and 2 columns.
Example:
| Confusion Matrix |
Predicted classes | ||
|---|---|---|---|
| male | female | ||
| Actual classes |
male | 42 | 8 |
| female | 18 | 32 | |
This means that the classifier correctly predicted a male person in 42 cases and it wrongly predicted 8 male instances as female. It correctly predicted 32 instances as female. 18 cases had been wrongly predicted as male instead of female.
Accuracy (error rate)
Accuracy is a statistical measure which is defined as the quotient of correct predictions made by a classifier divided by the sum of predictions made by the classifier.
The classifier in our previous example predicted correctly predicted 42 male instances and 32 female instance.
Therefore, the accuracy can be calculated by:
accuracy = $(42 + 32) / (42 + 8 + 18 + 32)$
which is 0.72
Let's assume we have a classifier, which always predicts "female". We have an accuracy of 50 % in this case.
| Confusion Matrix |
Predicted classes | ||
|---|---|---|---|
| male | female | ||
| Actual classes |
male | 0 | 50 |
| female | 0 | 50 | |
We will demonstrate the so-called accuracy paradox.
A spam recogition classifier is described by the following confusion matrix:
| Confusion Matrix |
Predicted classes | ||
|---|---|---|---|
| spam | ham | ||
| Actual classes |
spam | 4 | 1 |
| ham | 4 | 91 | |