Debaleen Das Spandan

I designed and developed a prototype web-based interface that enabled users to control connected devices over the internet through gestures, voice commands, and Human Interface Devices (HID). Built with React.js and TensorFlow.js, the system incorporated a custom gesture recognition model that I trained, optimized, and converted to TensorFlow Lite for seamless deployment in the browser. For voice recognition, I integrated Google Speech-to-Text, allowing users to issue pre-defined commands naturally alongside gesture-based input. HID support provided an additional interaction layer, making the system adaptable to different accessibility needs. This project showcased the potential of multimodal IoT interaction.

This is a prototype I desigend and built during my short-stint as a Research Engineer at National Institute of Technology, Durgapur in 2022.

Short video showcasing the gesture recognition in the prototype