Simple CL methods for large models

Hi, I have a model for layout understanding task, the model is pretty large and has complex architecture. My problem is that I have to use some Continual Learning technique with it to control its forgetting.

Can you recommend some simple yet effective methods which can be used with such architectures? I mean, e.g. Experience Replay is a good example, because we can think of our model like about black-box and do not engage into its architecture. A simple implementation would be a great benefit.

1 Like

Yes, I also suggest to go with replay first!
If your model is developed in Pytorch it should be relatively easy to port it to Avalanche and use the replay strategies already implemented there!