README.md

TensorFlow and Deep Learning Tutorials

Google's Deep Learning Tutorials

• TensorFlow Official Deep Learning Tutorial [中文].
• MLP with Dropout TensorFlow [中文] TensorLayer [中文]
• Autoencoder TensorLayer [中文]
• Convolutional Neural Network TensorFlow [中文] TensorLayer [中文]
• Recurrent Neural Network TensorFlow [中文] TensorLayer [中文]
• Deep Reinforcement Learning TensorLayer [中文]
• Sequence to Sequence TensorFlow TensorLayer[中文]
• Word Embedding TensorFlow [中文] TensorLayer [中文]

Deep Learning Reading List

• MIT Deep Learning Book
• Karpathy Blog
• Stanford UFLDL Tutorials
• Colah's Blog - Word Embedding [中文]
• Colah's Blog - Understand LSTN [门函数]

Tutorial index

0 - Prerequisite

• Introduction to Machine Learning (notebook)
• Introduction to MNIST Dataset (notebook)

1 - Introduction

• Hello World (notebook) (code)
• Basic Operations (notebook) (code)

2 - Basic Models

• Nearest Neighbor (notebook) (code)
• Linear Regression (notebook) (code)
• Logistic Regression (notebook) (code)

3 - Neural Networks

• Multilayer Perceptron (notebook) (code)
• Convolutional Neural Network (notebook) (code)
• Recurrent Neural Network (LSTM) (notebook) (code)
• Bidirectional Recurrent Neural Network (LSTM) (notebook) (code)
• Dynamic Recurrent Neural Network (LSTM) (code)
• AutoEncoder (notebook) (code)

4 - Utilities

• Save and Restore a model (notebook) (code)
• Tensorboard - Graph and loss visualization (notebook) (code)
• Tensorboard - Advanced visualization (code)

5 - Multi GPU

• Basic Operations on multi-GPU (notebook) (code)

Dataset

Some examples require MNIST dataset for training and testing. Don't worry, this dataset will automatically be downloaded when running examples (with input_data.py). MNIST is a database of handwritten digits, for a quick description of that dataset, you can check this notebook.

Official Website: http://yann.lecun.com/exdb/mnist/

Selected Repositories

• jtoy/awesome-tensorflow
• nlintz/TensorFlow-Tutoirals
• adatao/tensorspark
• ry/tensorflow-resnet

Tricks

• Tricks to use TensorLayer

Examples

Basics

• Multi-layer perceptron (MNIST) - Classification task, see tutorial_mnist_simple.py.
• Multi-layer perceptron (MNIST) - Classification using Iterator, see method1 and method2.

Computer Vision

• Denoising Autoencoder (MNIST). Classification task, see tutorial_mnist.py.
• Stacked Denoising Autoencoder and Fine-Tuning (MNIST). Classification task, see tutorial_mnist.py.
• Convolutional Network (MNIST). Classification task, see tutorial_mnist.py.
• Convolutional Network (CIFAR-10). Classification task, see tutorial_cifar10.py and tutorial_cifar10_tfrecord.py.
• VGG 16 (ImageNet). Classification task, see tutorial_vgg16.py.
• VGG 19 (ImageNet). Classification task, see tutorial_vgg19.py.
• InceptionV3 (ImageNet). Classification task, see tutorial_inceptionV3_tfslim.py.
• Wide ResNet (CIFAR) by ritchieng.
• More CNN implementations of TF-Slim can be connected to TensorLayer via SlimNetsLayer.
• Spatial Transformer Networks by zsdonghao.
• U-Net for brain tumor segmentation by zsdonghao.
• Variational Autoencoder (VAE) for (CelebA) by yzwxx.
• Variational Autoencoder (VAE) for (MNIST) by BUPTLdy.
• Image Captioning - Reimplementation of Google's im2txt by zsdonghao.

Natural Language Processing

• Recurrent Neural Network (LSTM). Apply multiple LSTM to PTB dataset for language modeling, see tutorial_ptb_lstm.py and tutorial_ptb_lstm_state_is_tuple.py.
• Word Embedding (Word2vec). Train a word embedding matrix, see tutorial_word2vec_basic.py.
• Restore Embedding matrix. Restore a pre-train embedding matrix, see tutorial_generate_text.py.
• Text Generation. Generates new text scripts, using LSTM network, see tutorial_generate_text.py.
• Chinese Text Anti-Spam by pakrchen.
• Chatbot in 200 lines of code for Seq2Seq.
• FastText Sentence Classification (IMDB), see tutorial_imdb_fasttext.py by tomtung.

Adversarial Learning

• DCGAN (CelebA). Generating images by Deep Convolutional Generative Adversarial Networks by zsdonghao.
• Generative Adversarial Text to Image Synthesis by zsdonghao.
• Unsupervised Image to Image Translation with Generative Adversarial Networks by zsdonghao.
• Improved CycleGAN with resize-convolution by luoxier
• Super Resolution GAN by zsdonghao.
• DAGAN: Fast Compressed Sensing MRI Reconstruction by nebulaV.

Reinforcement Learning

• Policy Gradient / Network (Atari Ping Pong), see tutorial_atari_pong.py.
• Deep Q-Network (Frozen lake), see tutorial_frozenlake_dqn.py.
• Q-Table learning algorithm (Frozen lake), see tutorial_frozenlake_q_table.py.
• Asynchronous Policy Gradient using TensorDB (Atari Ping Pong) by nebulaV.
• AC for discrete action space (Cartpole), see tutorial_cartpole_ac.py.
• A3C for continuous action space (Bipedal Walker), see tutorial_bipedalwalker_a3c*.py.
• DAGGER for (Gym Torcs) by zsdonghao.
• TRPO for continuous and discrete action space by jjkke88.

Miscellaneous

• Distributed Training. mnist and imagenet by jorgemf.
• Merge TF-Slim into TensorLayer. tutorial_inceptionV3_tfslim.py.
• Merge Keras into TensorLayer. tutorial_keras.py.
• Data augmentation with TFRecord. Effective way to load and pre-process data, see tutorial_tfrecord*.py and tutorial_cifar10_tfrecord.py.
• Data augmentation with TensorLayer, see tutorial_image_preprocess.py.
• TensorDB by fangde see here.
• A simple web service - TensorFlask by JoelKronander.
• Float 16 half-precision model, see tutorial_mnist_float16.py

Useful Links

• Tricks to use TensorLayer