I list some helpful resources for learning deep neural networks based on my learning experience. Will update from time to time.

lectures & textbook

The standford cs231n is a good start to learn CNNs. The cs224n is about natural language processing with deep learning.
CS229: Machine Learning, knowledge about machine learning (not deep) is also important.

I also recommend the MIT deep learning book.

If you are interested in reinforcement learning, David Silver is a good teacher. The videos of his RL cource is available at YouTube.

blogs

Blogs usually simplified concepts to make them easier to understand. Thus, they are more friendly to beginners.

Here are some high-quality blogs I have read about CNNs.

A Beginner’s Guide To Understanding Convolutional Neural Networks. PS: the author was only 19 when he wrote this high-quality blog.

A 2017 Guide to Semantic Segmentation with Deep Learning

A Year in Computer Vision is a good report about DNNs in CV.

Review of Deep Learning Algorithms for Object Detection

Surveys

two survey papers I like:
Recent Advances in Recurrent Neural Networks
Recent Advances in Convolution Neural Networks

There is a github repo that collects the state-of-the-art results for machine learning problems.

DNN Genealogy, an interactive visualization tool developed by our group, summarizes the milestone models in the development of DNNs.

The Caffe Model Zoo is also a good place to look for DNN models.

Framework

Don’t start with tensorflow!
Even though tensorflow is the most famous and popular machine learning framework, it is definitely not friendly to beginners.

I would recommend Keras and Pytorch.

I like Keras because:
a) it supports multiple backend engines, including TensorFlow, CNTK, Theano, and MXNet;
b) it offers consistent and simple APIs that minimizes the number of actions required for common use cases.

Keras offers offical examples, implementations of popular models, and a number of pre-trained models.