Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perceiver] Add calculation of decoders/preprocessors output channels #26

Open
wants to merge 99 commits into
base: modeling_perceiver
Choose a base branch
from
Open

[Perceiver] Add calculation of decoders/preprocessors output channels #26

wants to merge 99 commits into from

Commits on Oct 11, 2021

  1. First draft

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    fc77bd6 View commit details
    Browse the repository at this point in the history

  2. Style and remove mlm

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    177ad35 View commit details
    Browse the repository at this point in the history

  3. Make forward pass work

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    c9b0064 View commit details
    Browse the repository at this point in the history

  4. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    df5be95 View commit details
    Browse the repository at this point in the history

  5. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    66a42ee View commit details
    Browse the repository at this point in the history

  6. Fix bug

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4d25043 View commit details
    Browse the repository at this point in the history

  7. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    381e0d1 View commit details
    Browse the repository at this point in the history

  8. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    77bf795 View commit details
    Browse the repository at this point in the history

  9. Add PerceiverTokenizer first draft

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    697af73 View commit details
    Browse the repository at this point in the history

  10. Improve conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    b98b1df View commit details
    Browse the repository at this point in the history

  11. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    fef9208 View commit details
    Browse the repository at this point in the history

  12. Make conversion script work for the encoder

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    2e6055a View commit details
    Browse the repository at this point in the history

  13. Make conversion script work with local pickle files

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0a10005 View commit details
    Browse the repository at this point in the history

  14. Style & quality, fix-copies

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    3b7e9ec View commit details
    Browse the repository at this point in the history

  15. Add dummy input to conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    25de5e0 View commit details
    Browse the repository at this point in the history

  16. Add absolute position embeddings to TextPreProcessor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    b223379 View commit details
    Browse the repository at this point in the history

  17. Make forward pass of encoder work

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    6d37915 View commit details
    Browse the repository at this point in the history

  18. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ebb49a1 View commit details
    Browse the repository at this point in the history

  19. Move text preprocessor to separate script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    43417c2 View commit details
    Browse the repository at this point in the history

  20. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    d429cfd View commit details
    Browse the repository at this point in the history

  21. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4188df2 View commit details
    Browse the repository at this point in the history

  22. Add post processor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    b15241d View commit details
    Browse the repository at this point in the history

  23. Make MLM model work

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    5179d57 View commit details
    Browse the repository at this point in the history

  24. Style

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    87f24c8 View commit details
    Browse the repository at this point in the history

  25. Add PerceiverForMaskedLM

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    db9ff20 View commit details
    Browse the repository at this point in the history

  26. Add PerceiverImagePreprocessor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ab0981b View commit details
    Browse the repository at this point in the history

  27. Make style

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0ccdea9 View commit details
    Browse the repository at this point in the history

  28. Make PerceiverForImageClassification work

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0685088 View commit details
    Browse the repository at this point in the history

  29. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    089f9a0 View commit details
    Browse the repository at this point in the history

  30. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    e32cde7 View commit details
    Browse the repository at this point in the history

  31. Use tokenizer in conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a49bf66 View commit details
    Browse the repository at this point in the history

  32. Use PerceiverForMaskedLM in conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    6ab614c View commit details
    Browse the repository at this point in the history

  33. Define custom PerceiverModelOutput

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a5e83e7 View commit details
    Browse the repository at this point in the history

  34. Improve PerceiverAttention to make it work for both MLM and image cla… 

    …ssification
    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    6a0b8ff View commit details
    Browse the repository at this point in the history

  35. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    7557b09 View commit details
    Browse the repository at this point in the history

  36. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    1852416 View commit details
    Browse the repository at this point in the history

  37. More improvements to the conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    113f0ef View commit details
    Browse the repository at this point in the history

  38. Make conversion script work for both MLM and image classification

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0d5666a View commit details
    Browse the repository at this point in the history

  39. Add PerceiverFeatureExtractor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ef9dcdf View commit details
    Browse the repository at this point in the history

  40. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    70fabe1 View commit details
    Browse the repository at this point in the history

  41. Style and quality

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    24014aa View commit details
    Browse the repository at this point in the history

  42. Add center cropping

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    50ea770 View commit details
    Browse the repository at this point in the history

  43. Fix bug

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    9f8d2ad View commit details
    Browse the repository at this point in the history

  44. Small fix

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    758b912 View commit details
    Browse the repository at this point in the history

  45. Add print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    f04a1d3 View commit details
    Browse the repository at this point in the history

  46. Fix bug in image preprocessor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    59260bf View commit details
    Browse the repository at this point in the history

  47. Fix bug with conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    f9c65e9 View commit details
    Browse the repository at this point in the history

  48. Make output position embeddings an nn.Parameter layer instead of nn.E… 

    …mbedding
    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    7269ce0 View commit details
    Browse the repository at this point in the history

  49. Comment out print statements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    436e467 View commit details
    Browse the repository at this point in the history

  50. Add position encoding classes

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a0881ca View commit details
    Browse the repository at this point in the history

  51. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    b63ffb5 View commit details
    Browse the repository at this point in the history

  52. Use position_encoding_kwargs

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    3eb3c11 View commit details
    Browse the repository at this point in the history

  53. Add PerceiverForImageClassificationFourier

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    505d6cd View commit details
    Browse the repository at this point in the history

  54. Make style & quality

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4b08c19 View commit details
    Browse the repository at this point in the history

  55. Add PerceiverForImageClassificationConvProcessing

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    e89774e View commit details
    Browse the repository at this point in the history

  56. Style & quality

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    1867093 View commit details
    Browse the repository at this point in the history

  57. Add flow model

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    df1b14b View commit details
    Browse the repository at this point in the history

  58. Move processors to modeling file

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    7399cc8 View commit details
    Browse the repository at this point in the history

  59. Make position encodings modular

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    9f18885 View commit details
    Browse the repository at this point in the history

  60. Make basic decoder use modular position encodings

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ddf0e01 View commit details
    Browse the repository at this point in the history

  61. Add PerceiverForOpticalFlow to conversion script

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4012143 View commit details
    Browse the repository at this point in the history

  62. Add AudioPreprocessor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    64099ac View commit details
    Browse the repository at this point in the history

  63. Make it possible for the basic decoder to use Fourier position embedd… 

    …ings
    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    5663621 View commit details
    Browse the repository at this point in the history

  64. Add PerceiverForMultimodalAutoencoding

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    3a66549 View commit details
    Browse the repository at this point in the history

  65. Improve model for optical flow

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0cf0c46 View commit details
    Browse the repository at this point in the history

  66. Improve _build_network_inputs method

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    03bd7e1 View commit details
    Browse the repository at this point in the history

  67. Add print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4489c1e View commit details
    Browse the repository at this point in the history

  68. Fix device issue

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    f98a69d View commit details
    Browse the repository at this point in the history

  69. Fix device of Fourier embeddings

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    b0fbc00 View commit details
    Browse the repository at this point in the history

  70. Add print statements for debugging

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    87457b8 View commit details
    Browse the repository at this point in the history

  71. Add another print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    99befc5 View commit details
    Browse the repository at this point in the history

  72. Add another print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ce41b86 View commit details
    Browse the repository at this point in the history

  73. Add another print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    371fb00 View commit details
    Browse the repository at this point in the history

  74. Add another print statement

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    9cfb336 View commit details
    Browse the repository at this point in the history

  75. Improve PerceiverAudioPreprocessor

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    1c2fb11 View commit details
    Browse the repository at this point in the history

  76. Improve conversion script for multimodal modal

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    3be52ae View commit details
    Browse the repository at this point in the history

  77. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    47f23d9 View commit details
    Browse the repository at this point in the history

  78. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    1cbb13b View commit details
    Browse the repository at this point in the history

  79. Improve multimodal model

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    ffa25df View commit details
    Browse the repository at this point in the history

  80. Make forward pass multimodal model work

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    4f0404b View commit details
    Browse the repository at this point in the history

  81. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    57da1cc View commit details
    Browse the repository at this point in the history

  82. Improve tests

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a292f21 View commit details
    Browse the repository at this point in the history

  83. Fix some more tests

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    8aa3144 View commit details
    Browse the repository at this point in the history

  84. Add output dataclasses

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    6ffdf39 View commit details
    Browse the repository at this point in the history

  85. Make more tests pass

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    0c9b51b View commit details
    Browse the repository at this point in the history

  86. Add print statements for debuggin

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    2e1af16 View commit details
    Browse the repository at this point in the history

  87. Add tests for image classification

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a77d9fd View commit details
    Browse the repository at this point in the history

  88. Add PerceiverClassifierOutput

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    49ab608 View commit details
    Browse the repository at this point in the history

  89. More improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    96fe9d0 View commit details
    Browse the repository at this point in the history

  90. Make more tests pass for the optical flow model

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    82f9f45 View commit details
    Browse the repository at this point in the history

  91. Make style & quality

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    9543cc4 View commit details
    Browse the repository at this point in the history

  92. Small improvements

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    d6f928c View commit details
    Browse the repository at this point in the history

  93. Don't support training for optical flow model for now

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    a690266 View commit details
    Browse the repository at this point in the history

  94. Fix _prepare_for_class for tests

    @NielsRogge
    NielsRogge committed Oct 11, 2021
    Configuration menu
    Copy the full SHA
    85079ec View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2021

  1. Make more tests pass, add some docs

    @NielsRogge
    NielsRogge committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    4f8cb33 View commit details
    Browse the repository at this point in the history

  2. Add multimodal model to tests

    @NielsRogge
    NielsRogge committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    dd6c5ce View commit details
    Browse the repository at this point in the history

  3. Added output_size calculation for preprocessors

    @esceptico
    esceptico committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    87cd3a9 View commit details
    Browse the repository at this point in the history

  4. Fix tensor shape in OneHotPreprocessor test

    @esceptico
    esceptico committed Oct 12, 2021
    Configuration menu
    Copy the full SHA
    1fbb1e9 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2021

  1. Added num_query_channels property for decoders

    @esceptico
    esceptico committed Oct 17, 2021
    Configuration menu
    Copy the full SHA
    7e89ecd View commit details
    Browse the repository at this point in the history