Can you recommend a cheap/lower end Nvidia GPU for training for someone who just wants to play around with NNs a bit and isn't interested in peak performance? Could I get something for $100 that would be better than my CPU?
While you can in theory run CUDA on an Nvidia 1030 GPU (currently around $100) it's really not worth it. The cheapest card actually worth buying for ML is the 1050 Ti, which can be had for around $200.
If your budget is $100 I'd take that money and hunt around for various cloud based solutions. Most have introductory offers and/or cheap/free solutions for hobbyists with modest needs. $100 will go a long way on these services if you're careful. Once you've used up your $100 you'll have a much better idea of what, if anything, you actually need.
