In the previous lecture we have been talking about long short term memory or Alice DMH and we have come
to the conclusion that okay it is working fine but there are lots of lots of way to matrices hands in
the training procedure.
We have to make lots of lots of updates and that's why they to be current units or Groos came to be
because these units are a simplified Alice T.M. block.
So all the gates are included in a single Aberdare gate.
OK so if you may recall for honesty YEM there has been three gates forget gate input gate and out to
the gate as far as gate it will be current units are concerned.
We just have a single update gate.
OK.
Again one easy to revisable to use Drewes in order to cope with the vanishing gradient problem.
So this is why we have to deal with ls TGM.
This is why we have to deal with grues in order to cope with the banishing gradient problem.
What's very important that gayeties with current unit control the flow of information light unless the
unit but without having to use a memory unit.
So in this case we don't have memory source at all.
It just exposes the full hidden content without any control and because it is a simple model.
It is more efficient because the architecture is simple.
There's not going to be several educates to update.
And that's why the training procedure is going to be much faster.
So as far as the architecture is concerned no memory sounds at all.
As you can see and if you just had an update gate that controls the imported South okay as far as the
input is concerned there's remember a gate that controls how much previous stops impact the new stop.
But anyways the most relevant gate is the update gate that's going to decide what input to use and what's
not.
Basically what are the important features in the input and what's not.
So this update gate is the most important part of gated recurrent units.
OK by the way lots of lots of applications relied heavily on Al SDM because we can train in our own
network with the help of the GP.
So graphical processing unit and it doesn't matter that we have three gates with lots of lots of educates.
We are able to solve this problem of computational power in the 21st century.
Quite cheap we don't care about the lots of lots of adulate we are able to train this architecture.
We have both parallelization and the half of the graphical processing unit.
So basically that's why there is no good solution.
Getas with current units are working quite fine as well as long short term memory.
They are very very similar.
The gate is recurrent you need is just the simplified ASTM.
So that's all about the main architecture of our ASTM and grew.
And this is how we can cope with it.
Vanishing gradient problem in the next chapter we are going to take a look at concrete examples of how
to use recurrent out of the networks.
And we will come to the conclusion that it is working quite fine.
Thanks for watching.