text: This paper investigates the effect of replacing identity skip connections with trainable convolutional skip connections in ResNet.  The authors find that in their experiments, performance improves.  Therefore, the power of skip connections is due to their linearity rather than due to the fact that they represent the identity. Overall, the paper has a clear and simple message and is very readable.  The paper contains a good amount of experiments,  but in my opinion not quite enough to conclude that identity skip connections are inherently worse.  The question is then: how non-trivial is it that tandem networks work?  For someone who understands and has worked with ResNet and similar architectures, this is not a surprise.  Therefore, the paper is somewhat marginal but, I think, still worth accepting. Why did you choose a single learning rate for all architectures and datasets instead of choosing the optimal one for each archtitecture and dataset?  Was it a question of computational resources?  Using custom step sizes would strenghten your experimental results significantly.  In the absence of this, I would still ask that you create an appendix where you specify exactly how hyperparameters were chosen. Other comments:- "and that it’s easier for a layer to learn from a starting point of keeping things the same (the identity map) than from the zero map" I don't understand this comment.  Networks without skip connections are not initialized to the zero map but have nonzero, usually Gaussian, weights. - in section 2, reason (ii), you seem to imply that it is a good thing if a network behaves as an ensemble of shallower networks.  In general, this is a bad thing.  Therefore, the fact that ResNet with tandom networks is an ensemble of shallower networks is a reason for why it might perform badly, not well.  I would suggest removing reason (ii). - in section 3, reason (iii), you state that removing nonlinearities from the skip path can improve performance.  However, using tandom blocks instead of identity skip connections does not change the number of nonlinearity layers.  Therefore, I do not see how reason (iii) applies to tandem networks. - "The best blocks in each challenge were competitive with the best published results for their numbers of parameters; see Table 2 for the breakdown."  What are the best published results?  I do not see them in table 2.