You don’t know what to do with all your data? Or maybe you have an idea for a motion analysis tool, but don’t know exactly how to realize it? Maybe the following sensor applications can give you the final necessary inspiration or input for system implementation. And then – just jump into the cold water and program what you want to. Rome wasn’t build in a day either…

Sensor Applications Using Movement Sonification

Often, movement sonification is used in rehabilitation, for example to re-learn fundamental motor skills after stroke or any other illness. You can find much information on this in respective publications. But since I assume you hit this page because of your love and passion for sport (like me), I want to give you two ideas for an application there. A general design for auditive feedback sensor applications would be to transfer real-time acoustical motion information via wireless headphones to the sensor-equipped athlete. Its specific design then depends on the intended purpose, as well as principle characteristics of the sport and is up to your own creativity. But let me remind you – before you would apply any sonification in real training, you would first need to validate its efficiency and use…

Ski Jumping

One of the key aspects in the execution of a ski jump is the take-off. Approaching the end of the in-run slope with a speed of approximately 25\frac{m}{s}, the time interval for an optimal take-off is very small. However, good timing decisively affects the flight curve and the overall jumping distance: early or late take-off initiation can already reduce jump length by 5 metres or more. Additional acoustic information during the in-run phase might consequently be useful to support the athlete in the execution of an ideal take-off.

One reasonable sonification could be the auditive display of the distance to the take-off table and end of the in-run slope. Knowing the velocity of the athlete-ski system and the in-run length, global sound features can represent distal information, for example as an increase or decrease in sound linear to the approach of the take-off table. Another idea would be to increase the attack frequency of a continuously generated tone as it is often employed in proximity sensors used for parking of automobiles.

Speed Skating

Another, more feasible sonification scenario is the support of motor synchronization and consistency in training of long-distance speed skating. Here, the idea is to sonify the main motions of the distal limb. Assuming that the human brain can perceive deceleration and asymmetries of the general motion sound pattern within a race or training run, a sonification should support the athlete in keeping the initial rhythm and speed despite exhaustion.

Wrist and ankle joints appear especially suitable to sonify the primary skating motion pattern of continuously swinging arms and legs. For synchronization of two or more extremities and the maintenance of skating speed, the two kinematic parameters velocity and distance to the body center should be sufficient to display all necessary information. One possible setting would be to encode velocity by any kind of spectral modulation (for example brightness or pitch) and the radial distance by volume. By this mapping, both the counter movement of the weaving left and right extremities as well as eventual decrease in speed are represented in an intuitive way. Whereas the positional parameters would be encoded in the same way for all four limbs, every joint would be represented by its own, clearly distinguishable sound timbre. This ensures that motion information of all limbs can be perceived simultaneously and then be discriminated by the athlete. The continuity in sound mapping on the other hand keeps the dimensionality of the sound mapping small, so that the implicit motion information is understood immediately.

sensor applications sonification

Sensor Applications Using Machine Learning

Several sensor applications of various level of detail are possible for the analysis of motion information obtained from machine learning methods. Here, I want to shortly discuss two scenarios that are related to the previously described retrieval of relevant performance properties for the assessment of motor style and performance quality. Since the main data collection of my PhD was based on inertial ski jump measurements, both are described in the context of ski jumping here. However, they can be used in a similar way for many other sports.

In contrast to the provision of auditory feedback, style feedback based on artificial motion knowledge is acquired by the user in a self-controlled way. This means that timing, distribution and frequency of the provision of respective motion information are left to the individual decision and preferences of a user.

Athlete Style Training

A wearable, automated framework for detailed assessment and training of motor style can be created from previously learned motion knowledge on jump style and errors.Two points are important to ensure good usability of such training system: (a) an intuitive graphical user interface (GUI) and (b) a direct and unambiguous man-machine communication and feedback provision. The following process design is one possibility to ensure the compliance to both criteria.

First, incoming sensor data of a current motion performance is received, processed and classified under the learned style criteria feature design. Once the basic system computation is done, the athlete can ask for specific information on motion parts or motion properties by sending retrieval requests. These are predefined in the GUI via radio buttons, checkboxes or any other form of graphical selection and control element. To fulfill criteria (b), their related search criteria and communication phrases should be held precise and intuitive. The respective information is then retrieved and lastly delivered to the user as text or sound output.

Internally, every search query is associated to one of the learned style criteria for information retrieval. A possible query in the user front end of ski jumping could for example be whether the arms have been held parallel during flight. In the back end this information would be labeled with its related style criteria (here A3), and the respective error recognition result be used to display an either positive or negative output feedback.

sensor applications retrieval 1

Judging Support

The second sample application does not address athletes, but a different group of information recipients – judges. When developing a motion analysis system for such a different information user group, one usually follows a clear defined concrete intention. This information is much more specific than the support of motor skill acquisition and often got inspired by problems and constraints that occurred in daily situations. Having determined the existence of errors in style and motor execution, the present application is obviously closely related to the judging of motor style and all its related problems of fraud and bias.

The general idea for a performance scoring scenario of ski jumping is to implement an executable computer program file that awards point scores to a motion performance on the base of the previously learned motion knowledge. Incoming sensor data is transferred to the computer in either offline or — in case that stable data transmission via wireless network connections is enabled — online mode. For the former, this data transfer could be executed from a standard file load dialogue. For the latter, data could be sent out by the sensors and fetched on the fly by a receiving computer located in the judge’s tower.

Designed and implemented with a common option window and dialog framework, the judging system executable should be intuitive and easy to use for any experienced computer user. The evaluation of an incoming data stream is then simply started as full motion analysis including (1) the derivation of body kinematics, (2) the computation of error points per style criteria and (3) the determination of the resulting summed output score. In the last step, the resulting feedback is then presented to the user respectively judge.

sensor applications retrieval 2

One can see that the judging feedback system offers considerably less dialog options than the athlete style training system. This system efficiency is due to the fact that the principal function of the system (determination of the final score) is already very clear and precisely formulated. As a result, the communication between system and user is less ambiguous and does not require internal translations to increase usability. Furthermore, it is not dependent on self-controlled system use mechanisms. Instead, it is continuously employed for all situations that require a solution to the underlying problem, meaning competitions.