text: The authors provide a novel, interesting, and simple algorithm capable of training with limited memory.   The algorithm is well-motivated and clearly explained,  and empirical evidence suggests that the algorithm works well.   However, the paper needs additional examination in how the algorithm can deal with larger data inputs and outputs.   Second, the relationship to existing work needs to be explained better. Pro: The algorithm is clearly explained, well-motivated, and empirically supported. Con: The relationship to stochastic gradient markov chain monte carlo needs to be explained better.   In particular, the update form was first introduced in [1],  the annealing scheme was analyzed in [2],  and the reflection step was introduced in [3].   These relationships need to be explained clearly. The evidence is presented on very small input data.   With something like natural images, the parameterization is much larger and with more data, the number of total parameters is much larger.   Is there any evidence that the proposed algorithm could continue performing comparatively as the total number of parameters in state-of-the-art networks increases?  This would require a smaller ratio of included parameters. [1] Welling, M. and Teh, Y.W., 2011. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning (ICML-11)(pp. 681-688). [2] Chen, C., Carlson, D., Gan, Z., Li, C. and Carin, L., 2016, May. Bridging the gap between stochastic gradient MCMC and stochastic optimization. In Artificial Intelligence and Statistics(pp. 1051-1060). [3] Patterson, S. and Teh, Y.W., 2013. Stochastic gradient Riemannian Langevin dynamics on the probability simplex. In Advances in Neural Information Processing Systems (pp. 3102-3110).