text: This paper considers the following model of a signal x = W^T h + b, where h is an m-dimensional random sparse vector, W is an m by n matrix, b is an n dimensional fixed bias vector.  The random vector h follows an iid sparse signal model,  each coordinate independently have some probability of being zero,  and the remaining probability is distributed  among nonzero values according to some reasonable pdf/pmf.  The task is to recover h, from the observation x via the activation functions like Sigmoid or ReLU.  For example, \hat{h} = Sigmoid(W^T h + b). The authors then show that, under the random sparsity model of h, it is possible to upper bound the probability P(||h-\hat{h}|| > \delta. m) in terms of the parameters of the distribution of h and W and b.  In some cases noise can also be tolerated.  In particular, if W is incoherent (columns being near-orthonormal), then the guarantee is stronger.  As far as I understood, the proofs make sense  - they basically use Chernoff-bound type argument.  It is my impression that a lot of conditions have to be satisfied for the recovery guarantee to be meaningful.  I am unsure if real datasets will satisfied so many conditions.  Also, the usual objective of autoencoders is to denoise  - i.e. recover x, without any access to W.  The authors approach in this vein seem to be only empirical.  Some recent works on associative memory also assume the sparse recovery model  - connections to this literature would have been of interest.  It is also not clear why compressed sensing-type recovery using a single ReLU or Sigmoid would be of interest:  are their complexity benefits?