Spinning Up a Pong AI With Deep Reinforcement Learning

Note: These are the Jupyter Notebooks for my article Spinning Up a Pong AI With Deep Reinforcement Learning on FloydHub.

Within a few years, Deep Reinforcement Learning will completely transform robotics, an industry with the potential to automate 64 percent of manufacturing. Hard-to-engineer behaviors will become a piece of cake, so long as there are enough Deep RL practitioners to implement them.

The following notebooks (file ending with .ipynb) detail how to train a model in Keras that plays the game of Pong using simple Policy Gradient techniques:

Files overview

#Main notebooks
  |train.ipynb                             #Main notebook to train the Pong agent in Keras
  |train-with-log.ipynb                    #Same as above but with tensorflow logs and model saving/loading

#Demo notebooks
  |demo.ipynb                              #Simple notebook to generate a gif with a random model
  |demo_black_and_white.ipynb              #Same as above but generate the gif for the pre-processed frame diff

#Utils
  |support.py                              #To support the gif generation
  |easy_tf_log.py                          #Tool to easily plot variable
  |karpathy.py                             #Functions used in karpathy's original RL Pong post

#FloydHub specific  
  |floyd.yml                               #Tells what to execute when launching a job
  |floyd_requirements.txt                  #Necessary packages to install
  
#Trained weights
  |my_model_weights.h5                     #Saved weights after training
  

Demo of random agent

Gif generated with demo.ipynb:

Gif generated with demo_black_and_white.ipynb:

Pre-processed frames in black and white

Run the code on FloydHub

Click the above blue button to open a Workspace on FloydHub where you will find the same environment.

Local

pip install keras gym jupyter

git clone https://github.com/mtrazzi/spinning-up-a-Pong-AI-with-deep-RL
cd spinning-up-a-Pong-AI-with-deep-RL
jupyter notebook

Go do the desired notebook, files that end with '.ipynb'. To run the model, go to the menu then click on Cell > Run all

Credits

• Karpathy's code and blogpost for the neural network architecture, pre-processing and training methods
• mrahtz's easy-tf-log and tensorflow pong implementation
• FloydHub's gym-retro template for the gif generation