Prompt: The task this week is to dive into the world of single-board micro-computers by making something with the Raspberry Pi. “With” being the keyword here. You could take it to mean “build a system with the device as the platform running code to perform some action” similar to working with an Arduino or you might consider just using the Raspberry Pi environment to try out a new piece of software running on it and have the goal of making something using the built-in tools available.
Goal: Create a webcam UI, allowing users to manipulate the interact with the browser using their body.
Honestly, this project was a failure, well not a complete failure, but not a success.
I was interested in utilizing the human body as a user interface by mapping with a machine learning library and then using body position coordinates to initiate events. To make things extra spicy, I was hoping to accomplish this on a Raspberry Pi with a camera module. For non-technical people, I wanted you to be able to click a button by moving your hand or head.
The project was off to a great start initially, with the webcam running smoothly on a local server and even a playful photo booth. However, upon introducing the machine learning library (ML5JS with the PoseNet model), everything ground to a halt. The Raspberry Pi didn’t have the processing power or was not configured to handle such a load, resulting in the super slow video, and terrible body position tracking. I was never really able to resolve this issue. Upon further research, it seems that most of the successful machine learning projects on RPIs generally involved made use of python and OpenCV. With zero knowledge of python and a time constraint, learning, and implementing these tools wasn’t feasible for this project.
However, I wanted to ensure some proof of concept and using P5.JS, ML5.JS, and my laptop was able to control turn on and off two filters using the position of my nose. While the interaction was not as engaging as I had sought, it did give me hope and reason to continue exploring this space.
