README.md

metadata

title: Rotation Human Eval
emoji: 📉
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

Rotation human evaluation (HF Space)

Web form for the stitched human_eval stimuli.

Question: If I rotate the first image, can I get the second image?
Answers: Yes / No (multiple choice)

Features

• Random question order per username (fixed on first start; same username resumes where you left off)
• Logs response time (seconds) per image
• Saves one JSON file per participant: responses/<username>.json
• JSON keys are image filenames (e.g. armenian_character13_character33_diff_90degree_pair.png)

Example response JSON

{
  "username": "alice",
  "created_at": "2026-05-23T12:00:00+00:00",
  "updated_at": "2026-05-23T12:15:00+00:00",
  "order": ["armenian_character13_character33_diff_90degree_pair.png", "..."],
  "responses": {
    "armenian_character13_character33_diff_90degree_pair.png": {
      "answer": "No",
      "time_seconds": 4.231,
      "question_index": 2,
      "question_folder": "Q2",
      "answered_at": "2026-05-23T12:01:00+00:00"
    }
  }
}

Deploy to Hugging Face Spaces

1. Copy stimuli into this folder (required before push):

   rsync -av /projectnb/ivc-ml/xthomas/RESEARCH/VLM_BLIND/human_eval/ \
     /projectnb/ivc-ml/xthomas/RESEARCH/VLM_BLIND/human_eval_hf_space/human_eval/
   

2. Create a new Space on huggingface.co/spaces → Docker SDK.

3. Push this directory:

   cd /projectnb/ivc-ml/xthomas/RESEARCH/VLM_BLIND/human_eval_hf_space
   git init
   git add app.py Dockerfile requirements.txt README.md human_eval/
   git commit -m "Add rotation human eval Space"
   git remote add origin https://huggingface.co/spaces/<user>/<space-name>
   git push -u origin main
   

4. Enable Persistent storage on the Space (Settings) so /data/responses/ keeps JSON files across restarts.

Local test

cd human_eval_hf_space
pip install -r requirements.txt
# ensure ./human_eval/ exists (see rsync above)
python app.py

Responses are written to ./responses/ locally (or /data/responses on HF with persistent disk).

Environment variables

Variable	Default	Description
HUMAN_EVAL_DIR	./human_eval	Root folder with Q1/, Q2/, …
RESPONSES_DIR	/data/responses	Per-user JSON output directory
PORT	7860	Gradio port