--- title: Live Fitness Trainer emoji: 💪 colorFrom: blue colorTo: green sdk: docker app_file: app.py health_check_path: /healthz app_port: 8080 # Or the port Gunicorn is set to in your Dockerfile CMD pinned: false --- # Fitness Trainer with Pose Estimation An AI-powered web application that tracks exercises using computer vision and provides real-time feedback. This version now includes **real-time feedback via WebSockets** for an enhanced live training experience. ## Features - Real-time pose estimation using MediaPipe. - Multiple exercise types: Squats, Push-ups, and Hammer Curls. - Customizable sets and repetitions (via UI or API parameters). - Exercise form feedback. - Progress tracking (if database features are enabled). - Web interface for easy access, including a live test page using WebSockets. - HTTP API for single frame analysis and stateful exercise tracking. - **WebSocket API for low-latency, interactive exercise tracking.** ## Installation 1. Clone the repository: ```bash git clone https://github.com/yourusername/fitness-trainer-pose-estimation.git # Replace with actual repo URL cd fitness-trainer-pose-estimation ``` 2. Install dependencies: ```bash pip install -r requirements.txt ``` This will install all necessary Python packages, including Flask, Flask-SocketIO, Gunicorn, eventlet, OpenCV, and MediaPipe. 3. Set up the static folder structure (if not already present): ```bash mkdir -p static/images ``` 4. Add exercise images to the `static/images` folder (if using the original UI templates): - `squat.png` - `push_up.png` - `hammer_curl.png` ## Usage ### Running Locally (Development) 1. Start the Flask-SocketIO development server: ```bash python app.py ``` This runs the app with Flask's built-in server, suitable for development. It will typically be available at `http://127.0.0.1:5000`. 2. Access the application: * **Main UI (if developed):** Open a web browser and navigate to `http://127.0.0.1:5000/`. * **Live WebSocket Test Page:** 1. To use `live_test.html`, it's recommended to serve it via a Flask route. You can add a simple route to `app.py` like this: ```python @app.route('/live') def live_test_page(): return render_template('live_test.html') ``` And move `live_test.html` into the `templates` folder. Then access it via `http://127.0.0.1:5000/live`. 2. Alternatively, if you prefer to keep `live_test.html` in the root and test without modifying `app.py` for serving it, you might be able to open the `live_test.html` file directly in your browser. However, ensure the Flask server (`python app.py`) is running, as the JavaScript in `live_test.html` needs to connect to the WebSocket server at `ws://127.0.0.1:5000`. Relative paths to static assets like CSS or JS within the HTML might not work as expected if opened as a direct file, which is why serving it via Flask is more robust. 3. Using the Live Test Page (`live_test.html`): * Select an exercise type from the dropdown. * Click "Start Trainer". * Allow webcam access if prompted. * Position yourself in front of your camera so that your full body is visible. * Observe the real-time feedback (reps, stage, messages) updated via WebSockets. * Click "Stop Trainer" to end the session. ## API Usage The application offers two primary ways to interact with its pose estimation capabilities: a traditional HTTP API and a real-time WebSocket API. ### 1. HTTP API This API is suitable for scenarios where you want to send individual frames or manage exercise sessions via HTTP requests. #### a. Stateless Single-Frame Analysis (`/api/analyze_frame`) Analyzes a single image frame for pose landmarks without maintaining state. * **URL:** `http://127.0.0.1:5000/api/analyze_frame` * **Method:** `POST` * **Payload (JSON):** * `image` (string): Base64 encoded image. * `exercise_type` (string, optional): "squat", "push_up", "hammer_curl". Defaults to "squat". * **Success Response (JSON):** `{"success": true, "landmarks": [...]}` * **Error Response (JSON):** `{"error": "Error message"}` #### b. Stateful HTTP Exercise Tracking (`/api/track_exercise_stream` and `/api/end_exercise_session`) These endpoints allow tracking an exercise over a series of frames, maintaining state on the server. * **`/api/track_exercise_stream` (POST):** * **Payload (JSON):** * `session_id` (string): Unique ID for the session. * `exercise_type` (string): Exercise type. * `image` (string): Base64 encoded image of the current frame. * `frame_width` (int): Width of the video frame. * `frame_height` (int): Height of the video frame. * **Success Response (JSON):** Contains `success`, `landmarks_detected`, and exercise-specific `data`. * **`/api/end_exercise_session` (POST):** * **Payload (JSON):** `session_id` (string). * **Success Response (JSON):** Confirmation message. (For detailed request/response structures of the HTTP API, refer to the previous README version or inspect the `app.py` code.) ### 2. WebSocket API for Live Training This API provides a low-latency, bidirectional communication channel for real-time exercise tracking, as used by `live_test.html`. * **Endpoint Connection:** The client connects to the server's root URL, e.g., `ws://127.0.0.1:5000/` (or `wss://your-deployed-app.com/`). Flask-SocketIO handles this on the default namespace. * **Client-side Setup:** 1. Include the Socket.IO client library in your HTML: ```html ``` 2. Connect to the server: ```javascript const socket = io(); // Or io.connect('http://your-server-address'); ``` * **Key WebSocket Events:** * **Client to Server:** * `socket.emit('start_exercise_session', { exercise_type: 'squat' });` * Call this to initiate a new exercise session. * `exercise_type` can be 'squat', 'push_up', or 'hammer_curl'. * `socket.emit('process_frame', { image: 'base64img_data', exercise_type: 'squat', frame_width: width, frame_height: height });` * Send a video frame for processing. * `image`: Base64 encoded string of the JPEG frame. * `exercise_type`: The current exercise. * `frame_width`, `frame_height`: Dimensions of the original video frame. * **Server to Client:** * `socket.on('session_started', (data) => { ... });` * Confirmation that the session has started. * `data` typically includes: `{ session_id: 'server_assigned_sid', exercise_type: 'squat' }`. The `request.sid` on the server is used as the session identifier for WebSocket communications. * `socket.on('exercise_update', (data) => { ... });` * Provides feedback for the processed frame. * `data` structure: ```json { "success": true, "landmarks_detected": true, // or false "data": { // Structure depends on the exercise_type (matches HTTP API's track_exercise_stream response data) // Example for 'squat': // "counter": 0, "stage": "down", "angle_left": 85.0, "feedback": "Good depth!" // Example for 'push_up': // "counter": 0, "stage": "up", "angle_body_left": 170.0, "feedback": "Keep body straight." // Example for 'hammer_curl': // "counter_left": 1, "stage_left": "up", "angle_left_curl": 45.0, "feedback_left": "Good curl!", ... (and right side) }, "message": "Optional message, e.g., 'No landmarks detected.'" // if landmarks_detected is false } ``` * `socket.on('session_error', (data) => { ... });` * Sent if there's an error starting a session (e.g., invalid exercise type, server busy). * `data`: `{ error: "Error message description" }` * `socket.on('frame_error', (data) => { ... });` * Sent if there's an error processing a specific frame (e.g., bad image data, unexpected server error). * `data`: `{ error: "Error message description" }` * `socket.on('disconnect', () => { ... });` * Standard Socket.IO event. The client is disconnected. Server-side, the `disconnect` handler in `app.py` cleans up the `active_exercise_sessions` entry for that client (`request.sid`). ## Project Structure - `app.py` - Main Flask application with HTTP routes and SocketIO event handlers. - `templates/` - HTML templates (e.g., `index.html`, `live_test.html` if moved here). - `static/` - CSS, client-side JavaScript (e.g., `websocket_trainer.js`), and images. - `pose_estimation/` - Pose estimation modules. - `exercises/` - Exercise tracking classes. - `feedback/` - User feedback modules. - `utils/` - Helper functions and utilities. - `Dockerfile` - For building the application container. - `requirements.txt` - Python dependencies. - `live_test.html` - Client-side example for testing WebSocket functionality (currently in root, consider moving to `templates/` and serving via a route). ## Technologies Used - Flask - Web framework. - Flask-SocketIO - For WebSocket communication. - Gunicorn + eventlet - For concurrent WSGI server suitable for WebSockets. - OpenCV - Computer vision tasks. - MediaPipe - Pose estimation. - HTML/CSS/JavaScript - Frontend. ## Deployment Containerization with Docker is highly recommended for deployment. ### Dockerfile The provided `Dockerfile` sets up the Python environment, installs dependencies, and configures Gunicorn to run the application. **Key change for WebSockets:** The Dockerfile now uses `eventlet` as the worker class for Gunicorn, which is essential for Flask-SocketIO: ```dockerfile # ... (other Dockerfile instructions) ... # Run app.py when the container launches using Gunicorn WSGI server CMD ["gunicorn", "--worker-class", "eventlet", "-w", "1", "--bind", "0.0.0.0:${PORT}", "--timeout", "0", "app:app"] ``` * `--worker-class eventlet`: Uses the asynchronous eventlet worker. * `-w 1`: Starts 1 worker process. For eventlet, a single worker can handle many concurrent connections. * `--timeout 0`: Disables worker timeouts, important for long-lived WebSocket connections. ### Hosting Options #### 1. Hugging Face Spaces - **Setup:** Push your code (including the `Dockerfile` and `app.py`) to a Hugging Face Space repository. Select "Docker" as the Space SDK. - **WebSocket Support:** Hugging Face Spaces generally supports WebSockets. The application should be accessible. - **Port Configuration:** Ensure your `CMD` in the Dockerfile uses a port variable like `${PORT}` (which HF Spaces sets, often to 7860 or 8080) or a fixed port that HF Spaces can map. The `app_port: 8080` in the README metadata is a common choice. #### 2. Google Cloud Run - **Setup:** Build your Docker container and push it to Google Container Registry (GCR) or Artifact Registry. Deploy the image from there to Cloud Run. - **WebSocket Support & Session Affinity:** - Cloud Run supports WebSockets. - **Session Affinity:** For applications using Flask-SocketIO with its default in-memory session management (`active_exercise_sessions` in this app, keyed by `request.sid`), session affinity is crucial if you scale to more than one instance. Session affinity directs requests from a specific client to the same container instance. Enable it in your Cloud Run service settings. - **Statelessness Consideration:** While session affinity helps, be aware that `active_exercise_sessions` are still tied to individual instances. If an instance restarts or you scale down and up, that specific instance's memory is lost. For more robust session management across multiple instances or instance restarts, an external store (like Redis via Memorystore) for `active_exercise_sessions` would be necessary. For the current implementation, session affinity is the primary mechanism to make WebSockets work with multiple instances. - **Gunicorn Command:** The `CMD` in the Dockerfile is already configured for Cloud Run (binds to `0.0.0.0:${PORT}`). #### 3. Other PaaS/VPS - Ensure the platform supports Python and Docker (or allows manual setup of Python, eventlet, Gunicorn). - Configure your reverse proxy (like Nginx) correctly to handle WebSocket connections (Upgrade and Connection headers) if you place one in front of Gunicorn. ### Note on Session Management (General) The current application uses an in-memory Python dictionary (`active_exercise_sessions`) to store user session data, keyed by the WebSocket `request.sid`. This is simple but has limitations: * **Scalability:** Data is not shared across multiple Gunicorn worker processes (if you were to increase `-w` beyond 1, though with `eventlet` one worker is standard) or multiple server instances (e.g., when load balancing or on platforms like Cloud Run with multiple instances). * **Persistence:** Sessions are lost if the server or worker restarts. For production environments requiring high availability or scalability, consider implementing session management using an external store like Redis or Memcached. ## Contributing Contributions are welcome! Please feel free to submit a Pull Request. Ensure your contributions align with the project's structure and coding style. If adding new features, please update relevant documentation and consider adding tests. ## License This project is licensed under the MIT License - see the `LICENSE` file for details (if one exists in the repository).