A gesture-controlled embedded "info hub" that uses inertial sensing and on-device machine learning to trigger contextual information displays. The system allows a user to perform simple hand gestures to retrieve weather, stock prices, quotes, and jokes, all rendered on an OLED screen in real time.
The device uses an IMU to capture fixed-rate accelerometer data (AX/AY) during short, touch-triggered recording windows. Each gesture is recorded over a deterministic time window, converted into a feature vector, and classified on-device using a model trained and deployed with Edge Impulse.
Four gestures were trained and mapped to system actions: W (Weather), S (Stock price), Q (Quote), and J (Joke). A capacitive touch pin is used to explicitly gate recording, ensuring clean data collection and preventing accidental triggers.
After establishing Wi-Fi connectivity, the system prefetches and caches network data (weather and stock prices) to avoid repeated API calls during runtime. Quotes and jokes are fetched live on demand. Results are rendered on a 128×64 OLED display, with output persisting until the next user interaction, creating a clear and intuitive user experience.
A gesture-controlled embedded "info hub" that uses inertial sensing and on-device machine learning to trigger contextual information displays. The project runs entirely on an ESP32 microcontroller and combines sensor data acquisition, real-time inference, network communication, and UI rendering within a tightly constrained embedded environment.