That paper was 10 months ago. There have been many RL papers in the meantime, but sparsity is only a problem with respect to reward, not state or action, from what I can see.