Action Detection is ipython notebook contains all steps of training action recognition model with optical flow and single person tracking
Requirements
- MERL_Shopping_Dataset
the implementation of the model was based on on A Multi-Stream Bi-Directional Recurrent Neural Network for Fine-Grained Action Detection paper
the model consist of three stages
- person detection and tracking
- optical flow
- action detection
for this part we peruse pytorch-ssd repo. we faced some problems cause of the view angle of the camera, some time model didn't notice the person as person or mistake the dimensions of him but we solved by those steps:
- take highest object's score as a person, and ignoring the detection label of it
- make fixed box instead of dynamic one
- for the missing person frames, keep the previous one as the current
for this part we peruse flowiz repo.
the problem w faced at this point was the output of the repo wasn't good enough when we use two consecutive frames so that we decided to take frame[n] and frame[n-6] to calculate the optical flow in frame n
for this part we peruse Action Recognition repo. model consist of:
- vgg16 net as features extractor (encoder)
- lstm net with attention mechanism (decoder)
trained by make prediction every 6 frames
after download this repo and it's requirements, you have to download pytorch-ssd repo inside action recognition directory , and put crop.py file into pytorch-ssd repo directory and enjoy with playing by the notebook