๐Ÿ™ GitHub ๐Ÿ“„ Paper: MC3 ๐Ÿ’ฝ Dataset: HMDB51

MC3-18 HMDB51 (Kinetics-400 Init)

MC3-18 (Mixed Convolution 3D) fine-tuned on HMDB51 split 1, initialized from Kinetics-400 pretrained weights, trained as part of the video pipeline in human-action-classification. A reference baseline, not a state-of-the-art result -- published HMDB51 split-1 methods using ensembles, test-time augmentation, and multi-crop evaluation reach roughly 70-75%. A sibling model initialized from UCF-101 weights instead of Kinetics-400 is also available; see Related Resources below.


Task Architecture Pretrained on Kinetics-400
Accuracy F1 Score Params
License Source

Performance

Metric Value
Accuracy (Top-1) 56.34%
Precision (macro) 55.14%
Recall (macro) 56.34%
F1 Score (macro) 54.72%
Parameters 11.5M
Best epoch 114 / 150

Precision, recall and F1 are computed with macro averaging over HMDB51's 51 (equally-sized, 30 videos each) test classes -- macro recall equals accuracy here because the test split is exactly class-balanced.

Overfitting

Train Acc Val Acc Gap
This model ~75% 56.34% ~19%

A ~19-point train/validation gap is expected here: HMDB51 has only ~70 training videos per class, and MC3-18's 11.5M parameters are enough to memorize a set that size even with augmentation. See the UCF-101-initialized sibling model for a comparison with a smaller gap at similar accuracy.


Evaluation Protocol

Metrics above come from VideoTrainer.validate() in hac.video.training.train, run on HMDB51 split 1's test set (1,530 videos, 51 classes), at the checkpoint's best-performing epoch. Each clip: 8 frames sampled uniformly (no frame skipping), resized preserving aspect ratio to roughly 128x171, center-cropped to 112x112, normalized with Kinetics-400 statistics -- a single center clip per video, no test-time augmentation or multi-crop averaging.

Why 8 frames instead of 16? HMDB51 contains many short videos (some 10-20 frames total). An 8-frame window with no frame-skipping avoids the frame-repetition/tiling that a longer window would force on those clips, at the cost of a shorter temporal receptive field than the UCF-101 models in this project use.


Usage

Install Dependencies

Not yet published on PyPI -- install from source:

git clone https://github.com/dronefreak/human-action-classification
cd human-action-classification
pip install -e .

Load the Model from Hugging Face

import json
import torch
from huggingface_hub import hf_hub_download
from hac.video.models.classifier import Video3DCNN

config_path = hf_hub_download(repo_id="dronefreak/mc3-18-hmdb51-kinetics", filename="config.json")
weights_path = hf_hub_download(
    repo_id="dronefreak/mc3-18-hmdb51-kinetics",
    filename="mc3-18-hmdb51-kinetics.pth",
)

with open(config_path) as f:
    config = json.load(f)

model = Video3DCNN(
    num_classes=config["num_classes"],  # 51
    model_name=config["model_type"],
    pretrained=False,
)

checkpoint = torch.load(weights_path, map_location="cpu", weights_only=False)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

Run Inference on a Video

The repo's VideoPredictor wraps frame sampling, transforms, and the forward pass end-to-end (pass num_frames=8 to match this model's training configuration):

from hac.video.inference.predictor import VideoPredictor

predictor = VideoPredictor(model_path=weights_path, num_frames=8, device="cpu")
result = predictor.predict_video("path/to/video.mp4", top_k=5)

print(result["top_class"], result["top_confidence"])

Note: VideoPredictor's built-in class list defaults to UCF-101's 101 classes -- for HMDB51 you'll want to pass/override the 51 class names listed below rather than relying on the predictor's default.


Training Configuration

Setting Value Source
Dataset HMDB51 split 1 (3,570 train / 1,530 test videos, 51 classes) HMDB51 split files
Architecture MC3-18 (torchvision.models.video.mc3_18) checkpoint config
Pretrained init Kinetics-400 checkpoint config
Optimizer SGD (momentum=0.9, nesterov=False) checkpoint optimizer state
Initial learning rate 0.0003 checkpoint optimizer state
Weight decay 0.003 checkpoint optimizer state
LR schedule StepLR (step_size=50, gamma=0.1) checkpoint scheduler state
Epochs trained 150 (best at epoch 114) checkpoint + training history
Frames per clip 8 (frame_interval=1) training script default
Spatial resolution 112x112 (aspect-preserving resize + random crop) training script default
Batch size not recorded in checkpoint --
Augmentation MixUp (alpha=0.6), CutMix (alpha=1.0), label smoothing (0.15), RandomHorizontalFlip, ColorJitter, RandomGrayscale training script default (unconfirmed exact values for this run)

Rows marked "checkpoint ..." are read directly out of the optimizer/scheduler state and config dict stored inside mc3-18-hmdb51-kinetics.pth. Rows marked "training script default" reflect hac.video.training.train's CLI defaults/flags at the time of training but weren't independently re-derived from the checkpoint for this exact run -- no separate run-config file was saved alongside it.


Kinetics-400 vs. UCF-101 Initialization

This project also ships an MC3-18/HMDB51 model initialized from UCF-101 weights instead of Kinetics-400 -- see mc3-18-hmdb51-ucf-transfer for that model's own card and verified numbers before drawing any conclusions from comparing the two; don't assume the two cards' headline framing is still in sync with each other.

  • Kinetics-400 init (this model): larger, more diverse pretraining corpus; uses 8-frame clips to avoid tiling short HMDB51 videos.
  • UCF-101 init: domain-closer to HMDB51 (similar YouTube/movie sources, overlapping action categories); uses 16-frame clips to match its own UCF-101 pretraining configuration, which causes frame tiling on HMDB51's shorter videos.

HMDB51 Classes

The model predicts 51 action classes: brush_hair, cartwheel, catch, chew, clap, climb, climb_stairs, dive, draw_sword, dribble, drink, eat, fall_floor, fencing, flic_flac, golf, handstand, hit, hug, jump, kick, kick_ball, kiss, laugh, pick, pour, pullup, punch, push, pushup, ride_bike, ride_horse, run, shake_hands, shoot_ball, shoot_bow, shoot_gun, sit, situp, smile, smoke, somersault, stand, swing_baseball, sword, sword_exercise, talk, throw, turn, walk, wave.


Known Limitations

  • Overfits on HMDB51's small per-class sample size (~19-point train/val gap) despite MixUp/CutMix/label smoothing.
  • Single model, no ensembling; no test-time augmentation (multi-crop, multi-clip temporal sampling).
  • Optimized for 8-frame clips; accuracy on substantially longer or shorter temporal windows is unverified.
  • Trained and evaluated on HMDB51 split 1 only -- performance on splits 2/3 is unverified.

Repository Contents

mc3-18-hmdb51-kinetics.pth
config.json
README.md

config.json doubles as the Hub's download-count query file: since this repo has no library_name integration the Hub recognizes, it falls back to counting requests against config.json (per Hugging Face's download-stats docs) -- the loading snippet above fetches it as part of normal usage, so downloads register.


Related Resources


Citation

If you use this model, please consider citing the HMDB51 dataset, the MC3 architecture, and the training framework:

@inproceedings{kuehne2011hmdb,
  title={HMDB: a large video database for human motion recognition},
  author={Kuehne, Hildegard and Jhuang, Hueihan and Garrote, Est{\'\i}baliz and Poggio, Tomaso and Serre, Thomas},
  booktitle={2011 International Conference on Computer Vision},
  pages={2556--2563},
  year={2011},
  organization={IEEE}
}
@inproceedings{tran2018closer,
  title={A Closer Look at Spatiotemporal Convolutions for Action Recognition},
  author={Tran, Du and Wang, Heng and Torresani, Lorenzo and Ray, Jamie and LeCun, Yann and Paluri, Manohar},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2018}
}
@misc{saksena2025mc3hmdbkinetics,
  author = {Saumya Saksena},
  title = {{MC3-18 HMDB51 (Kinetics-400 Init)}},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dronefreak/mc3-18-hmdb51-kinetics}},
  note = {Trained with the human-action-classification framework, Top-1 Accuracy: 56.34\%}
}
@software{saksena2026hac,
  author       = {Saumya Saksena},
  title        = {{Human Action Classification: Pose-based and Video-based Models}},
  year         = 2026,
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/dronefreak/human-action-classification}}
}

License

Apache-2.0

Downloads last month
37
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Paper for dronefreak/mc3-18-hmdb51-kinetics

Evaluation results