RESEARCH ON KINECT BASED TRAINING SYSTEM FOR SPORTS OBSTACLE ASSESSMENT

PESQUISA NO SISTEMA DE TREINAMENTO BASEADO NO KINECT PARA AVALIAÇÃO DE OBSTÁCULOS ESPORTIVOS

INVESTIGACIÓN SOBRE EL SISTEMA DE ENTRENAMIENTO BASADO EN KINECT PARA LA EVALUACIÓN DE OBSTÁCULOS DEPORTIVOS

ABSTRACT

Objective:

The study draws attention to designing a dyskinesia assessment system using a Kinect sensor to improve the efficiency of rehabilitation training.

Methods:

The login page design. Step 2: System functions setting. Relevant movement guidance content and rehabilitation evaluation content are incorporated in the system to make rehabilitation training efficient and orderly. Comprehensive data processing, evaluation, and export functions are necessary to reference rehabilitation physicians in diagnosis and treatment. Step 3: System modules design. Based on the system settings, corresponding functional modules have been designed and developed. With each module realizing its specific functions, it must be ensured that there is a certain degree of correlation between the modules. Step 4: The system function framework design.

Results:

A simple and comfortable login page is designed; 2. The system is capable of rehabilitation training and data management; 3. Specifically designed modules include sports collection module, rehabilitation training module, rehabilitation evaluation module, and information management module; 4. After logging in, the patient should first search for the rehabilitation plan in the rehabilitation training function module and then perform rehabilitation training regarding the rehabilitation plan. Kinect synchronously obtains patient sports information throughout the training process, and patients can obtain rehabilitation assessment information and automatically save it in the information management module.

Conclusions:

The Kinect-based dyskinesia assessment and training system designed in this study can heighten the efficiency of rehabilitation training for patients with dyskinesia, and it is highly suggested in clinical practice. Level of evidence II; Therapeutic studies - investigation of treatment results.

Keywords:
Rehabilitation; Information management; Dyskinesias

RESUMO

Objetivo:

O estudo chama a atenção para o projeto de um sistema de avaliação de discinesia usando sensor Kinect para melhorar a eficiência do treinamento de reabilitação.

Métodos:

O design da página de login. Etapa 2: configuração das funções do sistema. Conteúdo de orientação de movimento relevante e conteúdo de avaliação de reabilitação são incorporados ao sistema para tornar o treinamento de reabilitação eficiente e ordenado. Funções abrangentes de processamento, avaliação e exportação de dados são necessárias para fornecer referência para médicos de reabilitação em diagnóstico e tratamento. Etapa 3: Projeto dos módulos do sistema. Com base nas configurações do sistema, os módulos funcionais correspondentes foram projetados e desenvolvidos. Com cada módulo realizando suas funções específicas, deve-se garantir que haja um certo grau de correlação entre os módulos. Etapa 4: O design da estrutura de funções do sistema.

Resultados:

Uma página de login simples e confortável é projetada; 2. O sistema é capaz de treinamento de reabilitação e gerenciamento de dados; 3. Módulos especificamente projetados incluem módulo de coleta de esportes, módulo de treinamento de reabilitação, módulo de avaliação de reabilitação e módulo de gerenciamento de informações; 4. Após o login, o paciente deve primeiro pesquisar o plano de reabilitação no módulo de função de treinamento de reabilitação e, em seguida, realizar o treinamento de reabilitação com referência ao plano de reabilitação. O Kinect obtém de forma síncrona informações sobre esportes do paciente durante o processo de treinamento, e os pacientes podem obter informações de avaliação de reabilitação e salvá-las automaticamente no módulo de gerenciamento de informações.

Conclusões:

O sistema de avaliação e treinamento de discinesia baseado no Kinect projetado neste estudo pode aumentar a eficiência do treinamento de reabilitação para pacientes com discinesia, e é altamente sugerido na prática clínica. Nível de evidência II; Estudos terapêuticos- investigação dos resultados do tratamento.

Descritores:
Reabilitação; Gestão da informação; Discinesias

RESUMEN

Objetivo:

El estudio llama la atención para el diseño de un sistema de evaluación de la discinesia utilizando el sensor Kinect para mejorar la eficiencia del entrenamiento de rehabilitación.

Métodos:

Diseño de la página de inicio de sesión. Paso 2: Configuración de las funciones del sistema. El contenido de la guía de movimiento relevante y el contenido de evaluación de la rehabilitación se incorporan en el sistema para hacer que la capacitación en rehabilitación sea eficiente y ordenada. Las funciones integrales de procesamiento, evaluación y exportación de datos son necesarias para proporcionar referencia a los médicos rehabilitadores en el diagnóstico y el tratamiento. Paso 3: Diseño de los módulos del sistema. Sobre la base de la configuración del sistema, se han diseñado y desarrollado los módulos funcionales correspondientes. Con cada módulo realizando sus funciones específicas, debe asegurarse que existe un cierto grado de correlación entre los módulos. Paso 4: El diseño del marco de la función del sistema.

Resultados:

Se diseña una página de inicio de sesión simple y cómoda; 2. El sistema es capaz de entrenamiento en rehabilitación y manejo de datos; 3. Los módulos específicamente diseñados incluyen el módulo de recolección de deportes, el módulo de capacitación en rehabilitación, el módulo de evaluación de la rehabilitación y el módulo de gestión de la información; 4. Después de iniciar sesión, el paciente debe buscar primero el plan de rehabilitación en el módulo de función de entrenamiento de rehabilitación y luego realizar el entrenamiento de rehabilitación con referencia al plan de rehabilitación. Kinect obtiene de forma sincrónica la información deportiva del paciente durante todo el proceso de entrenamiento, y los pacientes pueden obtener información sobre la evaluación de la rehabilitación y guardarla automáticamente en el módulo de gestión de información.

Conclusiones:

El sistema de evaluación y entrenamiento de discinesia basado en Kinect diseñado en este estudio puede aumentar la eficiencia del entrenamiento de rehabilitación para pacientes con discinesia, y está altamente recomendado en la práctica clínica. Nivel de evidencia II; Estudios terapéuticos- investigación de los resultados del tratamiento.

Descriptores:
Rehabilitación; Gestión de la información; Discinesias

INTRODUCTION

Research background

With the development of economy, people's living standard has improved in recent years.¹ However, the number of in patients with dyskinesia increases rapidly due to various causes such as stroke, cerebral thrombosis, accident injury, etc.² Clinical studies have found that the central nervous system of the human brain has a strong of plasticity. After the initial treatment, patients with sports disorders can recover slowly as long as they recover repeatedly during the recovery period.^3,4 The traditional and most common model of rehabilitation is a one-on-one rehabilitation. Because the subjective judgment of the rehabilitation doctors plays an important role in the evaluation of the training effect, there is a lack of exercise data information^5,6 that can objectively reflect the therapeutic effect. In the field of rehabilitation medicine, virtual reality technology and human-computer interaction technology are the focus of research. The Kinect sensor developed by Microsoft is an advanced somatosensory technology with the of speech and gesture recognition.^7,8

The basis and the importance of this research

Kinect sensor has a variety of functions such as collecting human body motion information, voice recognition, gesture recognition, and fusion of multiple discipline technologies. It has found broad applications in the medical field resulting from its high potential social value.^9–11 If Kinect sensor is applied to the dyskinesia assessment training system, the patient's rehabilitation training is not restricted by external factors such as the activity site, indicating that the patient can carry out rehabilitation training conveniently in the family or the community. Further, the patient's exercise data information during the training process can be displayed in a real-time manner.¹²

Based on the comprehensive analysis of the previous sports rehabilitation system and the demands of dyskinesia patients, a Kinect-based dyskinesia assessment training system is designed, which is not restricted by external environmental factors.

METHODS

The login page design of the system strives to be simple and comfortable for the user's daily operation. When the user uses the training evaluation system for the first time, he needs to register an ID and set a login password. To input the ID account and the password is a premise to log in and use the system normally every time.

System function settings

Rehabilitation training: due to various factors, patients have physical dyskinesia with varying degrees. Repeated rehabilitation training will help activate muscles, nerves, and other tissue, accelerating the recovery process of normal body functions. Therefore, this system focuses on the patient's experience to simplify the system operation interface as much as possible to facilitate its daily use in the rehabilitation plan. Relevant movement guidelines should be set to help patients to operate in accordance with during rehabilitation training. Meanwhile, relevant rehabilitation evaluation content needs to be included in the system, so that training suggestions can be made according to the individual training plans and training effects of different patients. Furthermore, the rehabilitation training effects are updated in time to make rehabilitation training more efficient and orderly.

Data management: except for personal factors, the rehabilitation training is also inseparable from the correction guidance of the rehabilitation physicians so that the patients can carry out the following rehabilitation training scientifically and effectively. For patients with dyskinesia, rehabilitation training in the family or the community is convenient. This relies on the advance formulation of a rehabilitation plan and requires rehabilitation physicians to have an in-depth understanding of the patient's condition and rehabilitation. On this basis, the system needs to have complete data processing, evaluation, and export functions. These intuitive data will provide reference for the diagnosis and treatment of rehabilitation physicians.

System module design

Based on the system settings, corresponding functional modules have been designed and developed. With each module realizing their specific functions, it must be ensured that there is a certain degree of correlation between the modules. The system is specifically divided into the following functional modules, namely sports collection, rehabilitation training, rehabilitation evaluation, and information management module. Although these modules differ in functions, they are inherently interconnected.

System function framework

After logging in, the patient first searches for the rehabilitation plan in the rehabilitation training function module, and then conducts rehabilitation training by referring to the standard training movement and videos in the rehabilitation plan. During the training process, Kinect will synchronously obtain the patient's movement data information, and the patient can obtain the rehabilitation assessment information which is automatically saved it in the information management module.

RESULTS

System login interface (Figure 1)

Figure 1
System login page design.

Motion acquisition module

The event model is used to collect bone data information, and a callback function is added to the SkeletonFrameReady event by programming, and the entered motion data information is calculated in the callback function. All data information is acquired by the Kinect sensor and then processed by the system. After the initial processing, the data will continue to be transmitted to each functional module for secondary processing. Figure 2 is the flow chart of the motion acquisition module. (Figure 3)

Figure 2
Flow chart of motion acquisition module.

Figure 3
Motion acquisition module interface.

Rehabilitation training module

When the patient logs in and selects the rehabilitation training module, the system interface will play the corresponding rehabilitation training standard movement according to the established rehabilitation plan, and the patient needs to make similar movement within the prescribed time (90S). If the time is beyond the prescribed time, the interface will show that it has not been completed and needs to be trained again. If the movement by the patient does not meet the standard, the movement rehearsal is required until the movement reaches the standard (90% similarity rate is considered standard). When the movement in a training plan is successfully completed, the interface will display the end of training. Figure 4 is the flow chart of the rehabilitation training module.

Figure 4
The flow chart of rehabilitation training module.

Rehabilitation assessment module

When the patient logs in and selects the rehabilitation evaluation module, the system interface will display a set of standard movement videos, and the patient needs to act in accordance with the video simultaneously. During evaluation, the system will extract the key feature values of the training movement and the standard movement, and use the DTW algorithm to calculate the similarity between the two selected key feature values. During the training process, the interface will also display the different joint movement angles of the standard movement and the patient's movement in the form of a line chart, helping the patient to complete the training in a standardized manner. After all the training movement is completed, the patient's current rehabilitation training report will be generated in the system in real time. As shown in Figure 5, it is a schematic diagram of the joint movement angle of the training movement and the standard movement. Figure 6 is the flow chart of the rehabilitation assessment module.

Figure 5
The joint movement angle of the standard movement and training movement.

Figure 6
The flow chart of rehabilitation evaluation module.

Information Management Module

The system contains a large amount of data and information about the patient's rehabilitation training, such as basic patient information, the completion of the rehabilitation plan, and the results of the rehabilitation evaluation. Hence, to realize the efficient management of the information and facilitate the daily operation and use, the SQL Server 2008 database server is used to manage the training data information of patients with dyskinesia.

When a patient logs into the system for the first time, it is necessary to create a user basic information statistical table Trainee Basic Infornation.dbo, which can save all the data information of the patient in the rehabilitation training. Field name Traineeld, data type int, data meaning is the ID of the user name; field name TraineeName, data type Nvarchar(50), data meaning is user name; field name TrainingType, data type Nvarchar(50), data meaning is the initial training movement; Field name ActualTrainingTimes, data type int, data meaning is the total number of training words; field name FinishedPercentage, data type float, data meaning is the progress of the training plan; field name TrainingScore, data type Nvarchar(50), data meaning is the most recent training results; field name LastTrainingTime, data type Nvarchar(50), data meaning is the time of the last training.

After the patient completes the rehabilitation training each time, the user dynamic data statistics table Trainee Dynamic Data Table.dbo needs to be created. This table is used to record and update the patient's completion of each rehabilitation training. Field Name Traineeld, data type int, data meaning is the ID of the user name; field name LoginName, data type Nvarchar(50), data meaning is user login name; field name LoginPwd, data type Nvarchar(20), data meaning is user login password; field name TrainineeGender. data type Nvarchar(10), the data meaning is the patient's gender; field name TrainineeAge, data type int, the data meaning is the patient's age; field name TrainineeTel, data type Nvarchar(15), the data meaning is the patient's contact information; field name TrainineelniJoint, Data type Nvarchar(50), data meaning is the patient's initial training joint points; field name TrainineelniMoint, data type Nvarchar(50), data meaning is the patient's initial training movement; field name TrainineeTimes, data type int, data meaning is the number of patient training plans; Field name TrainineeSets, data type int, data meaning is the number of training plan sets. (Figure 7)

Figure 7
System function framework flowchart.

DISCUSSION

We designed a Kinect-based dyskinesia assessment and training system that combines Kinect somatosensory sensors with virtual reality technology.^13,14 Through human-computer interaction technology to collect real-time human motion data information, and carry out related algorithm calculation, so that patients can easily carry out autonomous rehabilitation training, obtain rehabilitation training evaluation results, greatly improve the efficiency of rehabilitation.

In terms of information management, an information management module is designed using SQL Server 2008 database server¹⁵ to manage the training data information. The system contains a large amount of data information about the patient's rehabilitation training, such as basic patient information and rehabilitation plan information, completion of the rehabilitation plan, and rehabilitation evaluation results, realizing the efficient management of the information and convenient for users’ daily operations.

CONCLUSION

Although the preset system functions are realized in this research, there are still many shortcomings. The design for recording completion degree of the patient is not perfect, and the optimal plan can be designed according to the needs of different patients in the following research. What's more, the rationality of the algorithm needs to be further verified to meet the actual application requirements of this evaluation and training system.

ACKNOWLEDGEMENTS

This work was supported by 2020 Provincial Quality Engineering Teaching Research Key Projects of Colleges and Universities: Teaching Research on Amateur Training Sports Club System in Higher Vocational Colleges.

REFERENCES

• ¹
  Keszler MS, Heckman JT, Kaufman GE, Morgenroth DC. Advances in prosthetics and rehabilitation of individuals with limb loss. Phys Med Rehabil Clin N Am. 2019;30(2):423-37.
• ²
  Ferreira Dos Santos L, Christ O, Mate K, Schmidt H, Krüger J, Dohle C. Movement visualisation in virtual reality rehabilitation of the lower limb: a systematic review. Biomed Eng Online. 2016;15(Suppl 3):144.
• ³
  Mikołajczyk T, Kłodowski A, Mikołajewska E, Walkowiak P, Berjano P, Villafañe JH, et al. Design and control of system for elbow rehabilitation: Preliminary findings. Adv Clin Exp Med. 2018;27(12):1661-9.
• ⁴
  Zhao Y, Liang C, Gu Z, Zheng Y, Wu Q. A new design scheme for intelligent upper limb rehabilitation training robot. Int J Environ Res Public Health. 2020;17(8):2948.
• ⁵
  Kim WS, Cho S, Park SH, Lee JY, Kwon S, Paik NJ. A low cost kinect-based virtual rehabilitation system for inpatient rehabilitation of the upper limb in patients with subacute stroke: A randomized, double-blind, sham-controlled pilot trial. Medicine (Baltimore). 2018;97(25):e11173.
• ⁶
  Kairy D, Veras M, Archambault P, Hernandez A, Higgins J, Levin MF, et al. Maximizing post-stroke upper limb rehabilitation using a novel telerehabilitation interactive virtual reality system in the patient's home: study protocol of a randomized clinical trial. Contemp Clin Trials. 2016;47:49-53.
• ⁷
  Guzsvinecz T, Szucs V, Sik-Lanyi C. Suitability of the Kinect sensor and leap motion controller-a literature review. Sensors (Basel). 2019;19(5):1072.
• ⁸
  Shih MC, Wang RY, Cheng SJ, Yang YR. Effects of a balance-based exergaming intervention using the Kinect sensor on posture stability in individuals with Parkinson's disease: a single-blinded randomized controlled trial. J Neuroeng Rehabil. 2016;13(1):78.
• ⁹
  Denève A, Moughamir S, Afilal L, Zaytoon J. Control system design of a 3-DOF upper limbs rehabilitation robot. Comput Methods Programs Biomed. 2008;89(2):202-14
• ¹⁰
  Webster A, Poyade M, Rea P, Paul L. The co-design of hand rehabilitation exercises for multiple sclerosis using hand tracking system. Adv Exp Med Biol. 2019;1120:83-96.
• ¹¹
  Han J, Shao L, Xu D, Shotton J. Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans Cybern. 2013;43(5):1318-34.
• ¹²
  Pöhlmann ST, Harkness EF, Taylor CJ, Astley SM. Evaluation of Kinect 3D Sensor for Healthcare Imaging. J Med Biol Eng. 2016;36(6):857-70.
• ¹³
  Escamilla-Nunez R, Michelini A, Andrysek J. Biofeedback systems for gait rehabilitation of individuals with lower-limb amputation: a systematic review. Sensors (Basel). 2020;20(6):1628.
• ¹⁴
  Li C, Rusák Z, Horváth I, Ji L. Influence of complementing a robotic upper limb rehabilitation system with video games on the engagement of the participants: a study focusing on muscle activities. Int J Rehabil Res. 2014;37(4):334-42.
• ¹⁵
  Taheri B, Haddadpoor A, Mirkhalafzadeh M, Mazroei F, Aghdak P, Nasri M, et al. Modification of infant hypothyroidism and phenylketonuria screening program using electronic tools. J Educ Health Promot. 2017;6:4.

Publication Dates

• Publication in this collection
  26 Nov 2021
• Date of issue
  2021

History

• Received
  28 Apr 2021
• Accepted
  10 May 2021