README.md

metadata

title: Live Fitness Trainer
emoji: 💪
colorFrom: blue
colorTo: green
sdk: docker
app_file: app.py
health_check_path: /healthz
app_port: 8080
pinned: false

Fitness Trainer with Pose Estimation

An AI-powered web application that tracks exercises using computer vision and provides real-time feedback. This version now includes real-time feedback via WebSockets for an enhanced live training experience.

Features

• Real-time pose estimation using MediaPipe.
• Multiple exercise types: Squats, Push-ups, and Hammer Curls.
• Customizable sets and repetitions (via UI or API parameters).
• Exercise form feedback.
• Progress tracking (if database features are enabled).
• Web interface for easy access, including a live test page using WebSockets.
• HTTP API for single frame analysis and stateful exercise tracking.
• WebSocket API for low-latency, interactive exercise tracking.

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/fitness-trainer-pose-estimation.git # Replace with actual repo URL
   cd fitness-trainer-pose-estimation
   

2. Install dependencies:

   pip install -r requirements.txt
   

   This will install all necessary Python packages, including Flask, Flask-SocketIO, Gunicorn, eventlet, OpenCV, and MediaPipe.

3. Set up the static folder structure (if not already present):

   mkdir -p static/images
   

4. Add exercise images to the static/images folder (if using the original UI templates):

   • squat.png
   • push_up.png
   • hammer_curl.png

Usage

Running Locally (Development)

1. Start the Flask-SocketIO development server:

   python app.py
   

   This runs the app with Flask's built-in server, suitable for development. It will typically be available at http://127.0.0.1:5000.

2. Access the application:

   • Main UI (if developed): Open a web browser and navigate to http://127.0.0.1:5000/.
   • Live WebSocket Test Page:
     1. To use live_test.html, it's recommended to serve it via a Flask route. You can add a simple route to app.py like this:

        @app.route('/live')
        def live_test_page():
            return render_template('live_test.html')
        

        And move live_test.html into the templates folder. Then access it via http://127.0.0.1:5000/live.
     2. Alternatively, if you prefer to keep live_test.html in the root and test without modifying app.py for serving it, you might be able to open the live_test.html file directly in your browser. However, ensure the Flask server (python app.py) is running, as the JavaScript in live_test.html needs to connect to the WebSocket server at ws://127.0.0.1:5000. Relative paths to static assets like CSS or JS within the HTML might not work as expected if opened as a direct file, which is why serving it via Flask is more robust.

3. Using the Live Test Page (live_test.html):

   • Select an exercise type from the dropdown.
   • Click "Start Trainer".
   • Allow webcam access if prompted.
   • Position yourself in front of your camera so that your full body is visible.
   • Observe the real-time feedback (reps, stage, messages) updated via WebSockets.
   • Click "Stop Trainer" to end the session.

API Usage

The application offers two primary ways to interact with its pose estimation capabilities: a traditional HTTP API and a real-time WebSocket API.

1. HTTP API

This API is suitable for scenarios where you want to send individual frames or manage exercise sessions via HTTP requests.

a. Stateless Single-Frame Analysis (/api/analyze_frame)

Analyzes a single image frame for pose landmarks without maintaining state.

• URL: http://127.0.0.1:5000/api/analyze_frame
• Method: POST
• Payload (JSON):
  • image (string): Base64 encoded image.
  • exercise_type (string, optional): "squat", "push_up", "hammer_curl". Defaults to "squat".
• Success Response (JSON): {"success": true, "landmarks": [...]}
• Error Response (JSON): {"error": "Error message"}

b. Stateful HTTP Exercise Tracking (/api/track_exercise_stream and /api/end_exercise_session)

These endpoints allow tracking an exercise over a series of frames, maintaining state on the server.

• /api/track_exercise_stream (POST):
  • Payload (JSON):
    • session_id (string): Unique ID for the session.
    • exercise_type (string): Exercise type.
    • image (string): Base64 encoded image of the current frame.
    • frame_width (int): Width of the video frame.
    • frame_height (int): Height of the video frame.
  • Success Response (JSON): Contains success, landmarks_detected, and exercise-specific data.
• /api/end_exercise_session (POST):
  • Payload (JSON): session_id (string).
  • Success Response (JSON): Confirmation message.

(For detailed request/response structures of the HTTP API, refer to the previous README version or inspect the app.py code.)

2. WebSocket API for Live Training

This API provides a low-latency, bidirectional communication channel for real-time exercise tracking, as used by live_test.html.

• Endpoint Connection: The client connects to the server's root URL, e.g., ws://127.0.0.1:5000/ (or wss://your-deployed-app.com/). Flask-SocketIO handles this on the default namespace.

• Client-side Setup:

  1. Include the Socket.IO client library in your HTML:

     <script src="https://cdn.socket.io/4.7.2/socket.io.min.js"></script>
     

  2. Connect to the server:

     const socket = io(); // Or io.connect('http://your-server-address');
     

• Key WebSocket Events:

  • Client to Server:

    • socket.emit('start_exercise_session', { exercise_type: 'squat' });
      • Call this to initiate a new exercise session.
      • exercise_type can be 'squat', 'push_up', or 'hammer_curl'.
    • socket.emit('process_frame', { image: 'base64img_data', exercise_type: 'squat', frame_width: width, frame_height: height });
      • Send a video frame for processing.
      • image: Base64 encoded string of the JPEG frame.
      • exercise_type: The current exercise.
      • frame_width, frame_height: Dimensions of the original video frame.

  • Server to Client:

    • socket.on('session_started', (data) => { ... });
      • Confirmation that the session has started.
      • data typically includes: { session_id: 'server_assigned_sid', exercise_type: 'squat' }. The request.sid on the server is used as the session identifier for WebSocket communications.
    • socket.on('exercise_update', (data) => { ... });
      • Provides feedback for the processed frame.
      • data structure:

        {
            "success": true,
            "landmarks_detected": true, // or false
            "data": {
                // Structure depends on the exercise_type (matches HTTP API's track_exercise_stream response data)
                // Example for 'squat':
                // "counter": 0, "stage": "down", "angle_left": 85.0, "feedback": "Good depth!"
                // Example for 'push_up':
                // "counter": 0, "stage": "up", "angle_body_left": 170.0, "feedback": "Keep body straight."
                // Example for 'hammer_curl':
                // "counter_left": 1, "stage_left": "up", "angle_left_curl": 45.0, "feedback_left": "Good curl!", ... (and right side)
            },
            "message": "Optional message, e.g., 'No landmarks detected.'" // if landmarks_detected is false
        }
        

    • socket.on('session_error', (data) => { ... });
      • Sent if there's an error starting a session (e.g., invalid exercise type, server busy).
      • data: { error: "Error message description" }
    • socket.on('frame_error', (data) => { ... });
      • Sent if there's an error processing a specific frame (e.g., bad image data, unexpected server error).
      • data: { error: "Error message description" }
    • socket.on('disconnect', () => { ... });
      • Standard Socket.IO event. The client is disconnected. Server-side, the disconnect handler in app.py cleans up the active_exercise_sessions entry for that client (request.sid).

Project Structure

• app.py - Main Flask application with HTTP routes and SocketIO event handlers.
• templates/ - HTML templates (e.g., index.html, live_test.html if moved here).
• static/ - CSS, client-side JavaScript (e.g., websocket_trainer.js), and images.
• pose_estimation/ - Pose estimation modules.
• exercises/ - Exercise tracking classes.
• feedback/ - User feedback modules.
• utils/ - Helper functions and utilities.
• Dockerfile - For building the application container.
• requirements.txt - Python dependencies.
• live_test.html - Client-side example for testing WebSocket functionality (currently in root, consider moving to templates/ and serving via a route).

Technologies Used

• Flask - Web framework.
• Flask-SocketIO - For WebSocket communication.
• Gunicorn + eventlet - For concurrent WSGI server suitable for WebSockets.
• OpenCV - Computer vision tasks.
• MediaPipe - Pose estimation.
• HTML/CSS/JavaScript - Frontend.

Deployment

Containerization with Docker is highly recommended for deployment.

Dockerfile

The provided Dockerfile sets up the Python environment, installs dependencies, and configures Gunicorn to run the application. Key change for WebSockets: The Dockerfile now uses eventlet as the worker class for Gunicorn, which is essential for Flask-SocketIO:

# ... (other Dockerfile instructions) ...

# Run app.py when the container launches using Gunicorn WSGI server
CMD ["gunicorn", "--worker-class", "eventlet", "-w", "1", "--bind", "0.0.0.0:${PORT}", "--timeout", "0", "app:app"]

• --worker-class eventlet: Uses the asynchronous eventlet worker.
• -w 1: Starts 1 worker process. For eventlet, a single worker can handle many concurrent connections.
• --timeout 0: Disables worker timeouts, important for long-lived WebSocket connections.

Hosting Options

1. Hugging Face Spaces

• Setup: Push your code (including the Dockerfile and app.py) to a Hugging Face Space repository. Select "Docker" as the Space SDK.
• WebSocket Support: Hugging Face Spaces generally supports WebSockets. The application should be accessible.
• Port Configuration: Ensure your CMD in the Dockerfile uses a port variable like ${PORT} (which HF Spaces sets, often to 7860 or 8080) or a fixed port that HF Spaces can map. The app_port: 8080 in the README metadata is a common choice.

2. Google Cloud Run

• Setup: Build your Docker container and push it to Google Container Registry (GCR) or Artifact Registry. Deploy the image from there to Cloud Run.
• WebSocket Support & Session Affinity:
  • Cloud Run supports WebSockets.
  • Session Affinity: For applications using Flask-SocketIO with its default in-memory session management (active_exercise_sessions in this app, keyed by request.sid), session affinity is crucial if you scale to more than one instance. Session affinity directs requests from a specific client to the same container instance. Enable it in your Cloud Run service settings.
  • Statelessness Consideration: While session affinity helps, be aware that active_exercise_sessions are still tied to individual instances. If an instance restarts or you scale down and up, that specific instance's memory is lost. For more robust session management across multiple instances or instance restarts, an external store (like Redis via Memorystore) for active_exercise_sessions would be necessary. For the current implementation, session affinity is the primary mechanism to make WebSockets work with multiple instances.
• Gunicorn Command: The CMD in the Dockerfile is already configured for Cloud Run (binds to 0.0.0.0:${PORT}).

3. Other PaaS/VPS

• Ensure the platform supports Python and Docker (or allows manual setup of Python, eventlet, Gunicorn).
• Configure your reverse proxy (like Nginx) correctly to handle WebSocket connections (Upgrade and Connection headers) if you place one in front of Gunicorn.

Note on Session Management (General)

The current application uses an in-memory Python dictionary (active_exercise_sessions) to store user session data, keyed by the WebSocket request.sid. This is simple but has limitations:

• Scalability: Data is not shared across multiple Gunicorn worker processes (if you were to increase -w beyond 1, though with eventlet one worker is standard) or multiple server instances (e.g., when load balancing or on platforms like Cloud Run with multiple instances).
• Persistence: Sessions are lost if the server or worker restarts. For production environments requiring high availability or scalability, consider implementing session management using an external store like Redis or Memcached.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. Ensure your contributions align with the project's structure and coding style. If adding new features, please update relevant documentation and consider adding tests.

License

This project is licensed under the MIT License - see the LICENSE file for details (if one exists in the repository).