Additional training of an already-trained network

General
1

Hi there,

I was wondering whether it is possible to load a network that has been trained online for, say, 100 epochs, and give it additional training over a few more epochs (for instance, because the loss function has not reached an asymptode after the initial 100 epochs)? A question has come up about cases where the original training was interrupted prematurely, in which case the answer was that the learning rate at the point of interruption should be approximated and then training could be resumed ( Question about ordered model output and context variables - #9 by KLDivergence ). I was wondering whether a similar logic applies to cases where the original training has finished successfully, but perhaps was not sufficient for practical purposes, or whether this is a fundamentally different case?

Many thanks in advance!

2

Hi Erik, while we are thinking about enabling saving and loading of optimizers (@LarsKue), you could apply the same logic to your case. This would not be much different than standard “fine-tuning” in other areas of deep learning.