pegasama/YogaPoseClassify

Yoga Pose Classification Toolkit

This repository contains a small, script-first pipeline to prepare data, extract pose landmarks with MediaPipe, train machine‑learning pose classifiers, and run a real‑time webcam demo.

The sections below explain what each Python script in the project root does and how to use it on macOS (zsh). For dependencies, see requirements.txt.

Prerequisites

• Python 3.x
• Install Python packages:

pip install -r requirements.txt

Optional but recommended: create and activate a virtual environment before installing.

Typical end‑to‑end workflow

1. (Optional) Extract raw images from the included Parquet dataset into train/test folders using extract_images.py.
2. Run pose_detection.py to generate per‑image pose landmark JSON files under PoseData/label_*.
3. Train and evaluate a classifier with ml_pose_classifier.py. Optionally export to ONNX or TFLite.
4. Run the webcam demo with realtime_pose_classifier.py using your saved model.

---

Script: extract_images.py

Purpose

• Extract images and labels from the provided Parquet files (in YogaDataSet/data/) and save them into folders by label for training and testing.

Inputs/Outputs

• Input: YogaDataSet/data/train-00000-of-00001.parquet, YogaDataSet/data/test-00000-of-00001.parquet
• Output: Images under TrainData/train/label_* and/or TrainData/test/label_*

Usage

# Process both train and test (default behavior)
python extract_images.py

# Train only
python extract_images.py --train --output TrainData

# Test only to a custom folder
python extract_images.py --test --output MyOutputDir

Notes

• The script creates label_0, label_1, … subfolders and writes image files with their original extensions.

---

Script: pose_detection.py

Purpose

• Run MediaPipe Pose on your labeled image folders and save normalized landmark coordinates to JSON files for training.

Preprocessing

• Uses the nose as the head reference point and applies: position = (pos − headPos) × 100, rounded to 2 decimals. This matches the training pipeline.

Inputs/Outputs

• Input: An images root (default TrainData/train) organized as label_*/*.jpg|png|…
• Output: JSON files under PoseData/label_*/<image_name>.json

Usage

# Process images from default input into PoseData
python pose_detection.py

# Custom input and output
python pose_detection.py --input TrainData/train --output PoseData --batch-size 100

Tips

• Supported image extensions: .jpg, .jpeg, .png, .bmp, .tiff
• Requires a working OpenCV + MediaPipe install (see requirements.txt).

---

Script: ml_pose_classifier.py

Purpose

• Train, evaluate, and export pose classifiers from landmark JSONs. Supports Random Forest, SVM, Gradient Boosting, Logistic Regression, and a knowledge‑distilled RF→MLP variant.

Data expectation

• Directory structure like:
  • PoseData/label_0/*.json
  • PoseData/label_1/*.json
  • …

Common options

• --data/-d Pose JSON root (default: PoseData)
• --model/-m Model type: random_forest (default), svm, gradient_boost, logistic, distilled_rf
• --test-size/-t Test split ratio (default: 0.2)
• --save-model/-s Path to save the trained model (.pkl via joblib)
• --load-model/-l Path to load an existing model
• --predict/-p Predict a single JSON file
• --evaluate/-e Evaluate a folder of JSON files
• --export-onnx Export the trained model to ONNX (tree models or distilled MLP)
• --export-model-type Controls which model flavor to export
• --export-tflite Export distilled student MLP to TFLite (requires extra deps)

Typical commands

# 1) Train a Random Forest and save it
python ml_pose_classifier.py \
    --data PoseData \
    --model random_forest \
    --test-size 0.2 \
    --save-model models/pose_classifier_random_forest.pkl

# 2) Evaluate a saved model on a held‑out folder (e.g., TestData)
python ml_pose_classifier.py \
    --model random_forest \
    --load-model models/pose_classifier_random_forest.pkl \
    --evaluate TestData

# 3) Export to ONNX (Random Forest or distilled MLP)
python ml_pose_classifier.py \
    --model random_forest \
    --load-model models/pose_classifier_random_forest.pkl \
    --export-onnx models/pose_classifier_random_forest.onnx

# 4) Knowledge distillation: train RF teacher + MLP student
python ml_pose_classifier.py \
    --data PoseData \
    --model distilled_rf \
    --save-model models/pose_classifier_distilled_rf.pkl

# 5) Export the student MLP to TFLite (extra packages required)
python ml_pose_classifier.py \
    --model distilled_rf \
    --load-model models/pose_classifier_distilled_rf.pkl \
    --export-tflite models/pose_classifier_distilled_mlp.tflite

Notes

• ONNX export depends on skl2onnx and onnx. TFLite export additionally needs onnx-tf and tensorflow.
• Linear classifiers (svm, logistic) are not supported by Unity Barracuda. Prefer random_forest or the distilled MLP for deployment.

---

Script: realtime_pose_classifier.py

Purpose

• Run live pose classification from your webcam using a previously trained model. Draws the skeleton, highlights the used joints, and overlays prediction + confidence.

Model loading

• If --model is not provided, the script auto‑searches common filenames in the project root:
  • pose_classifier_random_forest.pkl
  • pose_classifier_logistic.pkl
  • pose_classifier_distilled_rf.pkl

Usage

# Auto‑detect a model and open the default camera (0)
python realtime_pose_classifier.py

# Specify a model file and camera index
python realtime_pose_classifier.py \
    --model models/pose_classifier_random_forest.pkl \
    --camera 0

Keyboard controls

• Q: Quit
• L: Toggle landmark keypoints
• C: Toggle pose connections
• R: Reset prediction history (smoothing window)

Notes

• Uses the same preprocessing as training (nose‑relative coordinates ×100, 2‑decimal rounding, StandardScaler).
• For smoother predictions, a small history window is used to compute a stable label and average confidence.

---

Useful folders and artifacts

• YogaDataSet/data/ — Parquet files used by extract_images.py.
• TrainData/train|test/label_*/ — Image folders produced by extraction.
• PoseData/label_*/ — Landmark JSONs generated by pose_detection.py.
• models/ — Example trained/exported models and label mappings.
• confusion_matrix_*.png — Saved confusion matrix plots (when enabled in training script).

Troubleshooting

• MediaPipe install issues on macOS: ensure you’re using a supported Python version and the latest pip; try reinstalling mediapipe and opencv-python.
• Camera cannot open: try a different --camera index, close other apps using the camera, or allow camera permissions for Python in macOS Privacy settings.
• Model not found in real‑time script: pass --model with an explicit path to your .pkl file.