Skip to content

Latest commit

 

History

History
256 lines (230 loc) · 10.9 KB

README.md

File metadata and controls

256 lines (230 loc) · 10.9 KB
Raw

Contributors Forks License Stargazers Downloads Issues Poke-Pi-Dex
Raspberry Pi Python nVIDIA Keras TensorFlow Open CV


Poké-Pi-Dex

Poké-Pi-Dex is our deep learning / computer vision related project for nostalgic poké-weebs.

We recreated a Pokédex clone, which recognizes pictures of Pokémon from the first generation, using a Convolutional Neural Network. It's built on Raspberry Pi4 with LCD display, PiCamera, speaker and some other components attached.
The case is made of recycled cardboard. 🌱

View Report · Presentation · Project Work · Request Feature|Report Bug · Italiano



Table of Contents
  1. Demo
  2. Features
  3. Usage
  4. Tools
  5. Resources
  6. Roadmap
  7. License
  8. Contacts

Demo

Watch our demo on YouTube!

Demo Video
Demo video

Features

  • Main menu with functionalities and About panel.
  • Pokémon-image acquisition from app.
  • Pokémon prediction of captured pic, with an excellent accuracy, using a Convolutional Neural Network.
  • Different objects recognition, such as cards, plushies, figures.
  • Pokémon info (name, id, types, description and stats) and the previous and following evolution stages of Pokémon recognized.
  • Audio description of the predicted Pokémon.
  • Play Pokémon cry.
  • Settings view where users can edit parameters, such as language (English and Italian), fullscreen, volume.
  • Debug mode to check the values of the prediction.
  • Easter egg, try and find it!

Usage

To use the application follow these steps: TO-DO

Prerequisites

TO-DO

Installation

TO-DO

Tools

Resources

Roadmap

  • Dataset
    • find a decent dataset for the neural network
    • fix (cut pictures) and extend it
  • Classifier
    • CNN with 3 conv layers and 2 FC layers
    • data augmentation (random flip, rotation, contrast, brightness)
    • try dropout
    • try batch norm
    • loss and accuracy graphs
    • test real life pics
    • improve old CNN
  • Application
    • Pokémon repository
      • find .json file and load it into a dictionary
      • check and fix it
      • create Pokémon class
    • video input
      • create a separate class
      • make a function that can take a frame from the picamera (to test)
      • display the image inside a canvas
    • GUI structure
      • create a main menu
      • create an about panel
      • create a main app view divided in 2 (left-side for video input, right-side for Pokémon details)
      • create a settings panel
    • button to get the current frame
    • labels and entry for the Pokémon details (stats with dynamic bars and different colors)
    • add buttons to scroll between multiple evolutions (example: Eevee has different evolutions)
    • change the "type(s)" entry (from text to image)
    • button to play cry
      • collect cry audio files
    • description voice reading
      • collect description audio files with a bot
    • different language update
    • debug mode
  • Raspberry Setup
    • buy components
      • LDC display
      • PiCamera
      • power supply (powerbank)
      • speaker
      • push buttons
      • type-C elbow adapter
      • A/D converter (ADS1115)
    • integrate components
      • LCD display
      • PiCamera
      • power supply
      • speaker
      • buttons
      • analog joystick
    • prepare OS (disable password, enable interfaces, ...)
  • App Deployment
    • prepare environment (install python3 and required packages)
    • clone the repo
    • test the app
  • Case Prototype
    • project
    • cardboard cutout
    • painting
  • Report
    • setup a LaTeX document
    • draft a possible subdivision into chapters
    • write the report
  • Presentation
  • Demo Video
  • Extra & Future Developments
    • use a more comples neural network with the new dataset
    • use new form of data augmentation
    • add an amplifier to speaker
    • insert one or more white LEDs near the camera lens
    • add settings option to enable/disable the video freezing after taking a picture
    • finish the 3D model and print it
    • extend the Pokédex to the following Pokémon generations
    • porting of the application to mobile systems (Android, iOS) - Flutter + Firebase

License

Distributed under the GPLv3 License. See LICENSE for more information.

Contacts