IMITECIO

Table of Contents

• About the Project
  • Tech Stack
  • File Structure
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Results and Demo
• Future Work
• Troubleshooting
• Contributors
• Acknowledgements and Resources
• License

About The Project

Pose Estimation (& Replication) Based Game

• Pose Estimation has been for a while in the deep learning industry. Considering the various applications of it, we aimed at creating a simple game at first. Some few things we intend to do further are mentioned at the end. Refer this documentation

---

---

Tech Stack

• The Posenet model trained by Google is used.
• For the Posenet Js to Python port this repo is used.

The rest is openCV for webcam input, Unity for NotEndlessRunner game and keyboard for keypresses. Although pynput is used , we prefer pywin32 for windows.

• OpenCV
• Unity

File Structure

IMITECIO
├───docs                                                                       ## Documents and Images
│   └───images
├───NotEndlessRunner                                                           ## Game Files
│   |
│   ├───Assets
│   │   ├───Animation
│   │   ├───Materials
│   │   ├───Prefab
│   │   ├───Raw Mocap Data
│   │   ├───Scenes
│   │   ├───Scripts
│   │   ├───Sprites
│   │   └───TextMesh Pro
│   │       ├───Documentation
│   │       ├───Fonts
│   │       ├───Resources
│   │       └───Sprites
│   ├───Library
|   |
│   ├───Logs
│   ├───obj
│   ├───Packages
│   └───ProjectSettings
├───OsX.app                                          ## Game for MacOs
│   └───Contents
│       ├───Frameworks
│       ├───MacOS
│       ├───MonoBleedingEdge
│       └───Resources
└───posenet                                          ## Main Python Files
    ├───MonoBleedingEdge
    │   ├───EmbedRuntime
    │  
    ├───NotEndlessRunner_Data
    │   ├───Managed
    │   └───Resources
    ├───posenet
    │   └───converter
    └───models                                       ## You may need to download the models initially.
        └───checkpoints

Getting Started

Prerequisites & Installation

• Tested on Windows

git clone https://github.com/SAtacker/IMITECIO.git
cd IMITECIO

In your Virtual Environment

pip install -r "requirements.txt"
python posenet\webcam_demo.py

Results and Demo

• On Manipulating the code according to your requirements, you could use it to control most of the games.

• We have tried playing with SpaceInvaders

Future Work

• Posenet tflite to Python - Single Person detection
• Posenet Based Game
• Pose Estimation in 3d
• Socket Code instead of Keyboard Presses
• C++ Port for faster recognition

Troubleshooting

• You may need to install tensorflow 1.15.0 or tensorflow 1.13 on your own.
• There could be an issue only in the tensorflow installation.
• However one small thing to remember is that multithreading does not guarentee the Game to run first . Meaning that the OpenCV window might appear later and gamethread before. We dont't have a fix for this. Rather press Alt + Tab to overcome this thing.

Contributors

• Shreyas Atre
• Dhruvi Doshi
• Sarah Nisar
• Arnav

Acknowledgements and Resources

• SRA VJTI Eklavya 2020
• Refered this for PoseNet Python
• PoseNet
• OpenPose
• Pytorch 3D pose
• Paper
• Paper
• Paper
• Paper
• Space Invaders

License

License