Great book for learning concepts and for getting a generic overview (but goes deep enough that you can jump straight into implementation if you want). I recommend it highly.
Nice list, but there are too many papers like this and it is easy to get stuck in theory. I would suggest to grab some simple neural network (Darknet is great for that) code and read that first. If something does not make sense, find the theory from papers.
This approach is dangerous in statistics and machine learning. Things often work on your toy examples but it may be for totally different reasons than you think and it can bite you later. Since it looks like it works to you, you will never look into the necessary theory. With the approach 'theory first' you might not get nice results as fast but I would bet that your understanding would be much deeper.
I am not saying that you need to read all the papers before coding a line, but the approach of coding your way into it will give you more shallower knowledge if you don't follow up with the theory.
I agree with posts above. Diving straight into 'practice' in statistics (and other fields e.g. cryptography being the most notorious) leaves you open to a great many of pitfalls. Best case you will be inefficient with your approaches.
Your examples may work, a couple of testing sets giving you high confidence, and then you attempt to use it in the wild and everything falls apart.
At the same time machine learning is a lot about data cleaning, bootstrapping, picking the right algorithm with mininum iteration, minimizing your iteration cycle as much as possible etc which you don't gain until you actually mess around and get your hands dirty. Plus there are little implementation tidbits specific to each project.
That might work for some simpler machine learning algorithms, but in deep learning I think you'll have an even harder time correctly figuring out intuitively what's going on than you would learning the math.
These papers are not very theoretical at all. You can get lucky if everything you need has been implemented in your library of choice, but most deep learning papers are highly practical engineering-driven affairs and brushing them off as unnecessary theory is just doing yourself a disservice.
You don't really have to know anything. And the current free course does not have anything to do with neural nets. They use monte carlo, kalman filters, particle filters, PID, etc. You should feel comfortable with probability theory and if you know some linear algebra, the kalman filter example is going to make a lot more sense.
It starts on October 31st, I took it a while ago and learned a lot. The course is gravitating towards the theoretical end of the spectrum. If you want something thats a bit easier to get into, I can recommend the course by Hugo Larochelle:
I find the problem is actually how to select what content to study given a time constraint like if you had 5 hours or 20 hours or 200 hours - what should you read?
Like an exhaustive list is great but it's an optimization problem - how do I maximise understanding subject to a time constraint - which resources do I select to maximise learning in x hours/days/years?
> which resources do I select to maximise learning in x
> hours/days/years?
Pick a particular open-source implementation of something you're interested in, say the Google image captioning technology, read the papers necessary to understand that, and play with that implementation.
The main problem with the plan implied by the OP's bibliography is that it's not practical at all. You'll end up with a catholic knowledge of the theory and no idea about how to use it.
Is there a link to the book mentioned in item 1 (1.0 Book) as a PDF? The closest I could find is http://www.deeplearningbook.org/ which says I cannot get a PDF of the book, though clearly one existed at some point.
I don't think it's clear that there was ever an official PDF provided.
There was some discussion of making a PDF from the HTML in a previous thread dedicated to that book. https://news.ycombinator.com/item?id=11446164
I would skim these to select something basic and then try to experiment with a real system with those documents as a guide/reference.
Try to actually learn one level at a time.
That is my approach right now. I have read a lot but without being 100% on the fundamentals it mostly goes over my head so I am backing up. I plan to try Tensorflow examples also but I expect it will be pretty shaky until my high-level practical surface knowledge can meet in the middle with my fundamental knowledge if I can keep progressing.
Great book for learning concepts and for getting a generic overview (but goes deep enough that you can jump straight into implementation if you want). I recommend it highly.