A Residual Transformer Framework for Biomechanics-Aware Yoga Posture Recognition and Feedback

Correct yoga posture execution ensures optimal effect from the therapy and excludes musculoskeletal injuries resulting from incorrect performance. The vast majority of approaches for yoga postures analysis that use computer vision algorithms classify postures statically using rule-based models, which are not generic enough owing to inability to deal with complicated dynamic movements and give specific instructions on how to fix postures. This paper introduces the Residual Biomechanical Transformer Network (RBTNet), a deep learning-based approach for temporal classification of yoga postures and biomechanical deviations detection. RBTNet utilizes YOLOv8m-Pose for 17 keypoints detection, 74-dimensions biomechanical features extraction, sliding window temporal sequence construction (window length = 30), Transformer Encoder with multi-head self-attention, attention pooling mechanism and cascaded residual MLP blocks for feature processing. Deviations of joint angles in specific postures serve as the source for corrective feedback generation without a need of user-profile database or additional information about age and flexibility. The proposed method was tested using an eight-class yoga posture dataset (8 classes: chair, cobra, dog, no pose, shoulder stand, triangle, tree, warrior). With the help of early stopping strategy, RBTNet achieved 98% accuracy with a weighted F1-score of 0.98% in just 21 epochs.