Smart connected sensors platform for multi-nodal sensor fusion, gesture recognition, and full-body motion tracking

Abstract:
While many consumer electronics devices formerly could only rely on inertial sensing using only a single IMU (inertial measurement unit), more and more use cases emerge that require the fusion of data sampled by several sensor nodes simultaneously. Examples of such applications include joint angle monitoring for rehabilitation, complex gesture recognition for sports exercises and gaming, and full-body motion tracking. As many applications require the sensors to be mounted to different locations on the human body, wireless connectivity must be established between all sensor nodes. With this connectivity, it is possible to exchange and fuse data of multiple sensor nodes without needing a physical connection between nodes. Since most of the end-user devices for such applications are battery-powered, there is a clear requirement for low power consumption of each sensor node. This work describes a ready-to-use smart connected sensor system platform for collecting, synchronizing, and fusing sensor data of up to 8 sensor nodes, with each node containing a smart sensor with an integrated IMU, a Bluetooth Low Energy (BLE) microcontroller as well as other sensors relevant for full body motion analysis, such as magnetometers and pressure sensors. As the aforementioned use cases vary in requirements on the number of nodes, sensor output types, and sampling rates, a flexible platform concept for rapid use-case adaption is a core feature of the described concept. Key challenges of designing solutions for connected sensor applications are highlighted and discussed using an example system. Furthermore, the benefits of pushing sensor data fusion and machine learning algorithms to the edge are elaborated. Finally, a wearable reference hardware design, as well as a software platform for rapid use-case implementation, will be presented.