File size: 2,730 Bytes
9e466c9 5639aeb 9e466c9 5639aeb a71cb1b 5639aeb a71cb1b 5639aeb a71cb1b 5639aeb a71cb1b 5639aeb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | ---
tags:
- model_hub_mixin
- pytorch_model_hub_mixin
- vision
- perceiver
- adaptive-computation
license: mit
datasets:
- timm/imagenet-12k-wds
---
# AdaPerceiver (Logit + Feature Distilled from ViT-H CLIP)
This repository hosts the **logit + feature distilled AdaPerceiver model**, introduced in
**“AdaPerceiver: Transformers with Adaptive Width, Depth, and Tokens”**.
📄 Paper: https://arxiv.org/abs/2511.18105
📦 Code: https://github.com/pjajal/AdaPerceiver
📚 Model Collection: https://huggingface.co/collections/pjajal/adaperceiver-v1
This model is distilled from [ViT-H CLIP model](https://huggingface.co/timm/vit_huge_patch14_clip_224.laion2b_ft_in12k).
---
## Model Description
**AdaPerceiver** is a Perceiver-style transformer architecture designed for **runtime-adaptive computation**.
A single trained model can dynamically trade off **accuracy and compute** by adjusting:
- the **number of latent tokens**,
- the **effective depth**, and
- the **embedding dimension**.
This specific checkpoint corresponds to the **logit + feature distilled AdaPerceiver model**, trained on **ImageNet-12K** using a ViT-H teacher. It exposes both:
- classification **logits**, and
- **feature representations**
---
## Training Details
- **Training Data:** ImageNet-12K
- **Training Objective:** Logit distillation + feature distillation
- **Teacher Model:** [ViT-H/14 CLIP model](https://huggingface.co/timm/vit_huge_patch14_clip_224.laion2b_ft_in12k).
- **Architecture:** Adaptive Perceiver with block-masked attention and Matryoshka FFNs
- **Adaptivity Axes:** Tokens, Depth, Width
For full training details, see Appendix D of the paper.
---
## How to Use
This model can be loaded using the AdaPerceiver Hub-compatible class.
```python
import torch
from hub.networks.adaperceiver_distill import DistillAdaPerceiver
model = DistillAdaPerceiver.from_pretrained("pjajal/adaperceiver-v1")
# forward(
# x: input image tensor (B, C, H, W)
# num_tokens: number of latent tokens to process (optional)
# mat_dim: embedding dimension (optional)
# depth: early-exit depth (optional)
# token_grans: block-mask granularities (optional)
# )
out = model(
torch.randn(1, 3, 224, 224),
num_tokens=256,
mat_dim=128,
depth=12,
)
print(out.logits.shape, out.features.shape)
```
## Reference
If you use this models please cite the AdaPerceiver paper:
```bibtex
@article{jajal2025adaperceiver,
title={AdaPerceiver: Transformers with Adaptive Width, Depth, and Tokens},
author={Jajal, Purvish and Eliopoulos, Nick John and Chou, Benjamin Shiue-Hal and Thiruvathukal, George K and Lu, Yung-Hsiang and Davis, James C},
journal={arXiv preprint arXiv:2511.18105},
year={2025}
}
```
|