text: The authors present confidence-based autodidactic returns, a Deep learning RL method to adjust the weights of an eligibility vector in TD(lambda)-like value estimation to favour more stable estimates of the state.  The key to being able to learn these confidence values is to not allow the error of the confidence estimates propagate back though the deep learning architecture. However, the method by which these confidence estimates are refined could be better described.  The authors describe these confidences variously as: "some notion of confidence that the agent has in the value function estimate" and "weighing the returns based on a notion of confidence has been explored earlier (White & White, 2016; Thomas et al., 2015)".  But the exact method is difficult to piece together from what is written.  I believe that the confidence estimates are considered to be part of the critic  and the w vector to be part of the theta_c parameters.  This would then be captured by the critic gradient for the CAR method that appears towards the end of page 5.  If so, this should be stated explicitly. There is another theoretical point that could be clearer.  The variation in an autodidactic update of a value function (Equation (4)) depends on a few things,  the in variation future value function estimates themselves being just one factor.  Another two sources of variation are: the uncertainty over how likely each path is to be taken, and the uncertainty in immediate rewards accumulated as part of some n-step return.  In my opinion, the quality of the paper would be much improved by a brief discussion of this,  and some reflection on what aspects of these variation contribute to the confidence vectors and what isn't captured. Nonetheless, I believe that the paper represents an interesting and worthy submission to the conference.  I would strongly urge the authors to improve the method description in the camera read version though.  A few additional comments are as follows:  • The plot in Figure 3 is the leading collection of results to demonstrate the dominance of the authors' adaptive weight approach (CAR) over the A3C (TD(0) estimates) and LRA3C (truncated TD(lambda) estimates) approaches.  However, the way the results are presented/plotted, namely the linear plot of the (shifted) relative performance of CAR (and LRA3C) versus A3C, visually inflates the importance of tasks on which CAR (and LRA3C) perform better than A3C, and diminishes the importance of those tasks on which A3C performs better.  It would be better kept as a relative value and plotted on a log-scale so that positive and negative improvements can be viewed on an equal setting.   • On page 3, when Gt is first mentioned, Gt should really be described first, before the reader is told what it is often replaced with.   • On page 3, where delta_t is defined (the j step return TD error, I think the middle term should be $gamma^j V(S_{t+j})$   • On page 4 and 5, when describing the gradient for the actor and critic, it would be better if these were given their own terminology, but if not, then use of the word respectively in each case would help.