Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add intermediate_size to GPT-NeoX models #1212

Merged
merged 11 commits into from
Sep 7, 2024
Merged

Add intermediate_size to GPT-NeoX models #1212

merged 11 commits into from
Sep 7, 2024

Commits on May 10, 2024

  1. Update transformer.py -> Add intermediate_size

    @dtamayo-nlp
    dtamayo-nlp authored May 10, 2024
    Configuration menu
    Copy the full SHA
    6c6a46b View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. add support for rwkv and mamba and add todos about swiglu

    @jahatef
    jahatef committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a80e302 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. refactor activations and mlps

    @jahatef
    jahatef committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    22149f0 View commit details
    Browse the repository at this point in the history

  2. change llama config to swiglu

    @jahatef
    jahatef committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    148a86f View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. fixes gelu fusion

    @jahatef
    jahatef committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    63e8677 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. pre-commit run

    @jahatef
    jahatef committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ae544cf View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. add assert message to mamba linear

    @Quentin-Anthony
    Quentin-Anthony committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    1170078 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Update 1-3B.yml 

    revert accidental change
    @jahatef
    jahatef authored Sep 4, 2024
    Configuration menu
    Copy the full SHA
    048826f View commit details
    Browse the repository at this point in the history

  2. Update 1-3B.yml

    @jahatef
    jahatef authored Sep 4, 2024
    Configuration menu
    Copy the full SHA
    8f5d152 View commit details
    Browse the repository at this point in the history

  3. fixes various issues

    @jahatef
    jahatef committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    0da5ac8 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2024

  1. add back swiglu check

    @jahatef
    jahatef committed Sep 7, 2024
    Configuration menu
    Copy the full SHA
    3400c43 View commit details
    Browse the repository at this point in the history