text: The paper presents an off-policy actor-critic method for learning a stochastic policy with entropy regularization.  It is a direct extension of maximum entropy reinforcement learning for Q-learning (recently called soft-Q learning), and named soft actor-critic (SAC).  Empirically SAC is shown to outperform DDPG significantly in terms of stability and sample efficiency, and can solve relatively difficult tasks that previously only on-policy (or hybrid on-policy/off-policy) method such as TRPO/PPO can solve stably.  Besides entropy regularization, it also introduces multi-modal policy parameterization through mixture of Gaussians that enables diverse, on-policy exploration.      The main appeal of the paper is the strong empirical performance of this new off-policy method in continuous action benchmarks.  Several design choices could be the key,  so it is encouraged to provide more ablation studies on these, which would be highly valuable for the community.  In particular, - Amortization of Q and \pi through fitting state value function - On-policy exploration vs OU process based off-policy exploration - Mixture vs non-mixture-based stochastic policy - SAC vs soft Q-learning Another valuable discussion to be had is the stability of off-policy algorithm comparing Q-learning versus actor-critic method. Pros: - Simple off-policy algorithm that achieves significantly better performance than existing off-policy baseline algorithms - It allows on-policy exploration in off-policy learning, partially thanks to entropy regularization that prevents variance from shrinking to 0.  It could be considered a major success of off-policy algorithm that removes heuristic exploration noise. Cons: - Method is relatively simple extension from existing work in maximum entropy reinforcement learning.  It is unclear what aspects lead to significant improvements in performance due to insufficient ablation studies.  Other question: - Above Eq. 7 it discusses that fitting a state value function wrt Q and \pi is shown to improve the stability significantly.  Is this comparison with directly estimating state value using finite samples?  If so, is the primary instability due to variance of the estimate, which can be avoided by drawing a lot of samples or do full integration (still reasonably tractable for finite mixture model)?  Or, is the instability from elsewhere?  By having SGD-based fitting of state value function, it appears to simulate slowly changing target values (similar role as target networks).  If so, could a similar technique be used with DDPG and get more stable performance?