In this lecture we are going to talk about cross-validation.
So in the previous chapter as we have been talking about linear regression we have been talking about
logistic regression.
We have seen how to split the original data set into a training data set and into a test data set.
So when dealing with credit scoring in the previous lectures we use these train tasks split matter that
split the regional dataset into our training dataset and to a past dataset the task size 0.3 defined
at 70 percent of the original dataset is for training purposes and 30 percent of the original data set
is the task set.
OK so we have split the regional dataset into a training dataset and into a task data set.
We feed the model to the training data set and we test the model on the task later sat.
Which means that we only have information how the model performs to our in sample data.
But we would like to see the accuracy when dealing with new data sets.
And before talking about cross-validation Let's talk a bit about overfitting and underfeeding overfitting
means that the model has range to veil on the training data set which means that it is very accurate
on the training data set but yields poor results on the test set.
It is usually because the model is too complex.
So as you can see we have the data points like this and the fitted model is this Arat curve as you can
see it is rather complex.
This is called overfitting.
When the model has trained to veil which means that the model learns to noise instead of the actual
relationships between the variables in the data it is a problem because this noise is not present in
the past dataset.
So that's why overfit data models have good accuracy on the training data set but performs very very
poorly on the test data set.
OK what about underfeeding under 15.
Is quite on the contrary the model hasn't been fitted well to the training data set.
So as you can see we have the same data points and instead of a very very complex model we have a very
very simple model just a simple linear regression which misses the trends in the training data set.
This is usually the case when we use two simple models for the problem.
So overfitting when we use two complex models and under-15 when we use two simple models.
OK.
So this is why we have to consider K for cross-validation which helps to avoid under affecting and overfitting
by the way it helps to avoid overfitting more than under effecting.
But this is how we can deal with these problems.
The aim is to be able to generalize the model to new data sets with the same accuracy.
OK and what's extremely important that we use all the data set for training.
So instead of splitting the dataset into a training data set and test data set we are going to use all
the data set for training purposes as well as for test purposes.
So what do we have the original training data set and we are going to split the data into K falls in
this case.
Cazy close to 5.
So we have 1 2 3 4 5 subset that we want at case separate learning experience k minus 1 Fords for training
and one for testing purposes in this case.
The first full is the test data set and all the other forwards are the training data sets.
Okay then in the next iteration This is the task data set and all the other forwards are the training
data set up in the next iteration.
This is the test data set and all the other forwards are the training examples in the next iteration.
This is that tath sat and in the final iteration this is that test set.
So we wanted case separate learning experience k minus one for for training and one forth for testing
purposes we average the results from this K experience and it was the it went.
The advantage that all observations as far as that data set is concerned are used for both training
and validation plus each observations are used for validation.
Exactly wons.
OK so instead of splitting the data into a training data set and test data set every single sample in
the original dataset we a training example in a given iteration and it's going to be a test sample in
another one.
OK so we use K falls and we run it case separate learning experience.
That's why K for cross-validation is a computational have the operation in the sense data idea agreed
and will take more time to finish.
But anyways this is how we are able to deal with underfeeding and overfitting.
We just have to use K words.
Hence the name K-4 is cross-validation in the next lecture we are going to take a look at a concrete
example of how to use K4 course validation.
Thanks for watching.