text: I quite liked the revival of the dual memory system ideas and the cognitive (neuro) science inspiration.  The paper is overall well written and tackles serious modern datasets, which was impressive,  even though it relies on a pre-trained, fixed ResNet (see point below). My only complaint is that I felt I couldn’t understand why the model worked so well.  A better motivation for some of the modelling decisions would be helpful.  For instance, how much the existence (and training) of a BLA network really help  — which is a central new part of the paper, and wasn’t in my view well motivated.  It would be nice to compare with a simpler baseline, such as a HC classifier network with reject option.  I also don’t really understand why the proposed pseudorehearsal works so well.  Some formal reasoning, even if approximate, would be appreciated. Some additional comments below: - Although the paper is in general well written,  it falls on the lengthy side  and I found it difficult at first to understand the flow of the algorithm.  I think it would be helpful to have a high-level pseudocode presentation of the main steps. - It was somewhat buried in the details that the model actually starts with a fixed, advanced feature pre-processing stage (the ResNet, trained on a distinct dataset, as it should).  I’m fine with that,  but this should be discussed.  Note that there is evidence that the neuronal responses in areas as early as V1 change as monkeys learn to solve discrimination tasks.  It should be stressed that the model does not yet model end-to-end learning in the incremental setting. - p. 4, Eq. 4, is it really necessary to add a loss for the intermediate layers, and not only for the input layer?  I think it would be clearer to define the \mathcal{L} explictily somewhere.  Also, shouldn’t the sum start at j=0?