Active Mentor: Pose-Detection Coach
Standalone tablet fitness coach with real-time pose detection, 55+ exercises, and on-device ML inference. React Native shell, native Android intelligence, no backend.
A tablet fitness coach that watches your form during a workout and corrects it mid-rep. Everything runs on-device, no cloud, no login, no waiting on a trainer to review the video.
A Camera2 frame lands as YUV bytes in an ImageReader, wraps into an InfoBlob with pose metadata, and passes to the ExerciseClient. The ExerciseClient runs the frame through ModelManager, which owns whichever TFLite or MediaPipe model this exercise picked. The pose (17 to 33 keypoints) is handed to an AbstractExercise subclass like SquatsExercise, which runs its own state machine: track hip depth, cross the DOWN line, cross the UP line, increment the score. Skeleton and form feedback draw on an OverlayView on top of the camera preview.
TensorFlow Lite inference and Canvas overlay drawing at 30fps can't sit on the JS bridge. The React Native side is a shell — exercise menu, camera setup, dashboard for past sessions. The bridge (android/app/src/main/java/com/jogo/NativeBridge.java) exposes only three ReactMethods: startPortraitActivity(exerciseSetting, index) at :46, startLandscapeActivity(exerciseSetting, index) at :56, and readSessions(promise) at :66. The moment you tap Start, an intent hands off to a native Activity and the RN side goes silent until the exercise finishes and writes a session JSON. The Dashboard calls readSessions on next mount to reload the history.
Different exercises need different tradeoffs. android/app/src/main/assets/ ships 10 model files across 4 families: Thunder (Thunder.tflite + Thunder-int8.tflite quantized variant), Lightning (lightning.tflite), MoveNet (movenet_lightning_v4.tflite + movenet_thunder_v4.tflite), MediaPipe (pose_landmarker_lite.task + full.task + heavy.task), and YOLO11 pose (yolo11n-pose_int8.tflite + yolo11s-pose_int8.tflite for exercises that need an object detector like dribbling to spot the ball). Squats want joint stability (MoveNet Thunder). High knees need frame rate over precision (PoseNet Lightning). MediaPipe's 33-keypoint pose is only worth loading for exercises where hand or foot landmarks matter. Each exercise in exercise_data.js declares its preferred model; ModelManager instantiates lazily and holds the reference for the session.
Calibration measures the athlete's nose-hip and hip-knee distances at the start (android/app/src/main/java/com/exercises/PersonExercises/squats/SquatsExercise.java). During the exercise, the current hip Y coord is normalized against those calibrated distances. DECISIONLINE_HIGH = 0.7 (SquatsExercise.java:24), DECISIONLINE_LOW = 0.5 (:23). Decision line positions computed at :92 (HIGH: kneeLocationY − calibratedDistanceHipKneeY × 0.7 × scaleRatio) and :96 (LOW: same with 0.5). State DOWN when hip crosses LOW, UP when it crosses HIGH. A full DOWN → UP cycle counts one rep. Below a keypoint confidence threshold, the frame is skipped rather than guessed. Bad form (back angle out of range, knee tracking off) decrements the score with a red overlay flash.
No login, no cloud, no user accounts. Session JSON files live in the app's private storage. This was intentional: trainers wanted athletes to be able to open the app and start moving without friction. The tradeoff is no cross-device sync and no coach dashboard.