K-nearest neighbour classfier, k-fold cross validation, leave-one-out

1. K-nearest neighbour classfier
Base on k training instances to classify a new instance into the majority class. The distance between the new instance and each of k training instances is often computed by Euclidean distance.

2. K-fold cross validation
The idea of k-fold cross validation is to try to estimate the true predictive accuracy of a classifier. It is often carried out for a small training dataset. Instead of using a separate small test dataset, we can divide the training dataset into k portions and then use 1 portion for testing and k-1 for training, and then repeat for different portions. Doing that, a more accurate predictive accuracy of a classifier can be achieved.

3. Leave-one-out cross validation
Instead of dividing the training dataset into k different portions, only one instance is kept for testing and the rest is for training.