This project aims to build a hands-free digital image browsing system for use in a sterile setting in a medical environment.
This project was evaluated on an incremental basis at multiple stages by an Academic Mentor from SSN College of Engineering, Chennai, India, and an Industry Mentor from IBM, as part of an Innovation, Employability and Entrepreneurship program for students.
- Artificial Intelligence
- Deep Learning
- Image Processing
- Web Development
- Cloud Database
Surgeons and doctors often require reference imaging results whilst operating on patients. Furthermore, they may need to manipulate the image view through basic tranformative operations such as panning, scaling, and rotation among others. However, in doing so, computer peripheral devices need to be accessed through touch. To avoid compromising the sterility of their instruments and operating hand, they presently instruct surgical assistants to operate the viewing media to adjust it to their requirement. The granularity in reaching their required state of view is limited to the effectiveness of the verbal communication, and can often take several attempts before reaching the required state.
To eliminate the need for such an assistant, and to make the manipulation operations more granular and completely in control of the surgeon/doctor, our system proposes to use gestures as the instruction media. The getures are captured through a camera attached to the viewing system; the digital frames are processed to recognize the gesture using deep learning; each gesture is mapped to a corresponding transformations on the image; and the transformation is effected on the image. This makes for a hands-free sterility-preserving view manipulation mechanism.
In practicality, the system's machine learning backend may be trained to recognize any number of gestures. For this proof-of-concept project, we implement 6 different gestures, and map them to specific operations on a medical image. The entire system is deployed on a web-server, with user-management, real-time gesture capturing, recognition and change effecting using peripherals commonly found on a typical personal computer: web-camera, LCD display and trackpad/mouse.
The following operations are mapped to each gesture in the proof-of-concept implementation,
| Gesture Image | Gesture Name | Action |
|---|---|---|
 |
Zero | Annotate the image with a rectangle |
 |
One | Resize the image to resolution (200, 200) |
 |
Two | Rotate the image |
 |
Three | Blur the image |
 |
Four | Resize the image to resolution (400, 400) |
 |
Five | Convert the image to grayscale |
- Tensforflow/Keras
- OpenCV
- NumPy
- Flask
- NodeRED
- Team ID: PNT2022TMID53060
- Project ID: IBM-Project-21974-1659800381
| Role | Name | Register Number |
|---|---|---|
| Team Leader | Karthik D | 195001047 |
| Team Member 1 | A Anirudh | 195001015 |
| Team Member 2 | Nestor Ingarshal J | 195001069 |
| Team Member 3 | Pugalarasu K | 195001306 |
This project was evaluated on an incremental basis at multiple stages by Academic Experts from SSN College of Engineering, Chennai, India, and an Industry Experts from IBM, as part of an Innovation, Employability and Entrepreneurship program for students.
| Role | Name |
|---|---|
| Academic Mentor | Dr. Lakshmi Priya S |
| Academic Evaluator | Dr. Kanchana R |
| Industry Mentor and Evaluator | Ms. Pradeepthi |