VIT Adapter

[aselimc]

Hello,

I propose adding ViT adapter option in TimmUniversalEncoder for ViTs that does not have multi-scale features, so that SMP can now support a whole new set of backbones. The following links can be used as a reference/starting point.

DINOv3_Adapter
DINOv2_Adapter
Original VIT Adapter Repo
AdaptersHub
Adapters HF Version

Or perhaps these type of adapters might actually better be on timm? @rwightman @qubvel

[ogencoglu]

+1