license: mit
language: en
library_name: transformers
tags:
- action-recognition
- kth
- cnn
- video
datasets:
- kth-action-recognition
metrics:
- accuracy
model-index:
- name: move
results:
- task:
type: video-classification
dataset:
name: KTH
type: kth-action-recognition
split: test
metrics:
- type: accuracy
value: 0.5046
move
A small action classifier over KTH video: sixteen 64 by 64 grayscale frames in, one of six actions out. Each frame passes a two-conv CNN, the sixteen representations max-pool over time, and a linear head predicts walking, jogging, running, boxing, handwaving or handclapping. Trained for ten epochs with Adam at learning rate 1e-3, batch size 32, seed 0. Test accuracy 0.5046 on 216 clips from held-out subjects. CPU training.
No transformer here. The PreTrainedModel wrapper exists only so the weights serialize as config.json plus model.safetensors and load through AutoModel (including trust_remote_code via auto_map), the same arrangement as wear, tone, pole and fuse.
Usage
import numpy as np
from transformers import AutoModel
from modeling_move import MoveTemporal # registers the architecture
model = AutoModel.from_pretrained("harpertoken/move")
model.eval()
clip = np.zeros((16, 64, 64), dtype=np.float32)
print(model.predict_label(clip))
predict_label takes sixteen 64 by 64 float frames in range 0 to 1 and returns the action. Frames are uniformly sampled across the clip and converted to grayscale at 64 by 64, matching training exactly. Needs torch and transformers.
Training
KTH actions via a public mirror, subjects 01 to 16 for training (383 clips) and 17 to 25 for testing (216 clips), so no person appears in both sets. One sequence is absent from the mirror, hence 383 rather than 384.
The gate for publishing this model was temporal gain over a single-frame baseline using the identical encoder and preprocessing, differing only in aggregation:
| Model | Seed 0 | Seed 1 |
|---|---|---|
| Single middle frame | 0.4074 | 0.3704 |
| Temporal mean-pool | 0.4398 | not run |
| Temporal max-pool | 0.5046 | 0.4907 |
Mean-pooling gained 0.032 on 216 clips against a standard error near 0.034, which is not significant, so it was not published. Max-pooling gains 0.097 and 0.120 across two seeds, replicating in direction and magnitude. The published weights are the seed-0 max-pool run.
Two things went wrong before it worked. A concat-style fusion of frame features sat at chance; the max over time learned, because the most distinctive pose matters more than the average. And per-sample disk reads dominated wall time until clips were preloaded into RAM.
The wrapper was checked for exact equivalence: identical predictions on test batches in eval mode, after catching that dropout made train-mode outputs differ.
No dataset is published alongside this model. KTH already exists canonically and re-hosting it would add a duplicate, so the card cites the source instead.
Limitations
0.5046 is modest, and the card states it plainly. Six balanced classes put chance at 0.167, so the model has learned real motion distinctions, but fine confusions (jogging versus running, handwaving versus handclapping) dominate the errors. Anything outside short grayscale clips of single actors is out of scope.