text: The authors propose to speed up RL techniques, such as DQN, by utilizing expert demonstrations.  The  expert demonstrations are sequences of consecutive states that do not include actions, which is closer to a real setting of imitation learning.  The goal of this process is to extract a function that maps any given state to a subgoal.  Subgoals are then used to learn different Q-value functions, one per subgoal.  To learn the function that maps states into subgoals, the authors propose a surrogate reward model that corresponds to the angle between: the difference between two consecutive states (which captures velocity or direction) and a given subgoal.  A von Mises- Fisher distribution policy is then assumed to be used by the expert to generate actions that guide the agent toward the subgoal.  Finally, the mapping function state->subgoal is learned by performing a gradient descent on the expected total cost (based on the surrogate reward function, which also has free parameters that need to be learned). Finally, the authors use the DQN platform to learn a Q-value function using the learned  surrogate reward function that guides the agent to specific subgoals, depending on the situation. The paper is overall well-written,  and the proposed idea seems interesting.  However, there are rather little explanations provided to argue for the different modeling choices made, and the intuition behind them.  From my understanding, the idea of subgoal learning boils down to a non-parametric (or kernel) regression where each state is mapped to a subgoal based on its closeness to different states in the expert's demonstration.  It is not clear how this method would generalize to new situations.  There is also the issue of keeping tracking of a large number of demonstration states in memory.  This technique reminds me of some common methods in learning from demonstrations, such as those using GPs or GMMs,  but the novelty of this technique is the fact that the subgoal mapping function is learned in an IRL fashion, by tacking into account the sum of surrogate rewards in the expert's demonstration.  The architecture of the action value estimator does not seem novel,  it's basically just an extension of DQN with an extra parameter (subgoal g). The empirical evaluation seems rather mixed.  Figure 3 shows that the proposed method learns faster than DQN,  but Table I shows that the improvement is not statistically significant, except in two games, DefendCenter and PredictPosition.  Are these the results after all agents had converged?  Overall, this is a good paper,  but focusing on only a single game (Doom) is a weakness that needs to be addressed  because one cannot tell if the choices were tailored to make the method work well for this game.  Since the paper does not provide significant theoretical or algorithmic contribution,  at least more realistic and diverse experiments should be performed.