SLRZoo: Sign Language Recognition with deep learning methods

Usage

Requirements

Create a new virtual environment

conda create -n slrzooenv
conda activate slrzooenv

and install requirements:

pip install -r requirements.txt

Training

To train the model(s) for Continuous Sign Language Recognition run this command:

python train_cslr.py  

To train the model(s) for Isolated Sign Language Recognition run this command:

python train_islr.py  

Evaluation

To evaluate my model on ImageNet, run:

python eval.py --pretrained-cpkt mymodel.pth 

Download

You can run download.sh which automatically downloads datasets (except CSL-Daily, whose downloading needs an agreement submission), pretrained models, keypoints and place them under corresponding locations. Or you can download these files separately as follows.

Datasets

Download datasets from their websites and place them under the corresponding directories in data/

• Phoenix-2014
• Phoenix-2014T
• CSL-Daily
• CSL-500
• GSL
• MS-ASL

Implemented methods

1. Camgoz, N. C., Hadfield, S., Koller, O., & Bowden, R. (2017, October). Subunets: End-to-end hand shape and continuous sign language recognition. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 3075-3084). IEEE.

2. Cui, Runpeng, Hu Liu, and Changshui Zhang. "A deep neural framework for continuous sign language recognition by iterative training." IEEE Transactions on Multimedia 21.7 (2019): 1880-1891.

3. H. Zhou, W. Zhou and H. Li, "Dynamic Pseudo Label Decoding for Continuous Sign Language Recognition," 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China, 2019, pp. 1282-1287.

4. Pu, Junfu, Wengang Zhou, and Houqiang Li. "Iterative alignment network for continuous sign language recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.