MODEL_CHANGELOG.md

Towhee Model Change Log

All notable updates to Towhee models will be documented in this file.

[Preview] Latest

[0.8.0] Aug. 16, 2022

Add 3 SOTA models

• nnfp

  • page: audio-embedding/nnfp
  • paper: Neural Audio Fingerprint for High-specific Audio Retrieval based on Contrastive Learning

• RepMLPNet

  • paper: Hierarchical Vision MLP with Re-parameterized Locality

• Wave-ViT

  • paper: Unifying Wavelet and Transformers for Visual Representation Learning

[0.7.3] Jul. 27, 2022

Add 5 SOTA models

• CoCa

  • paper: CoCa

• CoFormer

  • paper: CoFormer

• TransRAC

  • paper: TransRAC

• CVNet

  • paper: CVNet

• MaxViT

  • paper: MaxViT

[0.7.1] Jul. 01, 2022

Add 1 vision transformer backbone, 1 text-image retrieval model, 2 video retrieval models

• MPViT

  • page: image-embedding/mpvit
  • paper: MPViT : Multi-Path Vision Transformer for Dense Prediction

• LightningDOT

  • page: image-text-embedding/lightningdot
  • paper: LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

• BridgeFormer

  • page: video-text-embedding/bridge-former
  • paper: Bridging Video-text Retrieval with Multiple Choice Questions

• collaborative-experts

  • page: video-text-embedding/collaborative-experts
  • paper: TEACHTEXT: CrossModal Generalized Distillation for Text-Video Retrieval

[0.7.0] Jun. 24, 2022

Add 6 video understanding/classification models

• Video Swin Transformer

  • page: action-classification/video-swin-transformer
  • paper: Video Swin Transformer

• TSM

  • page: action-classification/tsm
  • paper: TSM: Temporal Shift Module for Efficient Video Understanding

• Uniformer

  • page: action-classification/uniformer
  • paper: UNIFORMER: UNIFIED TRANSFORMER FOR EFFICIENT SPATIOTEMPORAL REPRESENTATION LEARNING

• OMNIVORE

  • page: action-classification/omnivore
  • paper: OMNIVORE: A Single Model for Many Visual Modalities

• TimeSformer

  • page: action-classification/timesformer
  • paper: Is Space-Time Attention All You Need for Video Understanding?

• MoViNets

  • page: action-classification/movinet
  • paper: MoViNets: Mobile Video Networks for Efficient Video Recognition

Add 4 video retrieval models

• CLIP4Clip

  • page: video-text-embedding/clip4clip
  • paper: CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval

• DRL

  • page: video-text-embedding/drl
  • paper: Disentangled Representation Learning for Text-Video Retrieval

• Frozen in Time

  • page: video-text-embedding/frozen-in-time
  • paper: Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval

• MDMMT

  • page: video-text-embedding/mdmmt
  • paper: MDMMT: Multidomain Multimodal Transformer for Video Retrieval

[0.6.1] May. 13, 2022

Add 3 text-image multimodal models

• CLIP

  • page: image-text-embedding/clip
  • paper: Learning Transferable Visual Models From Natural Language Supervision

• BLIP

  • page: image-text-embedding/blip
  • paper: BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

• LightningDOT

  • page: image-text-embedding/lightningdot
  • paper: LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

Add 6 video understanding/classification models

• I3D (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

• C2D (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: Non-local Neural Networks

• Slow (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: SlowFast Networks for Video Recognition

• SlowFast (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: SlowFast Networks for Video Recognition

• X3D (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: X3D: Expanding Architectures for Efficient Video Recognition

• MViT (from PyTorchVideo)

  • page: action-classification/pytorchvideo
  • paper: Multiscale Vision Transformers