Code

Basic Attention attribution#148

Code

Merged

lsickert merged 35 commits intomaininseq-team/inseq:mainfrom attention-attributioninseq-team/inseq:attention-attributionCopy head branch name to clipboardJan 16, 2023

+1,461-615

Commits

Commits on Sep 23, 2022

• added jupyterlab dependency (for easier testing)

  lsickert

  committed
  b9ccbf2

  Copy full SHA for b9ccbf2

Commits on Oct 17, 2022

• initial commit attention methods, added output_attentions parameter to model initialization

  lsickert

  committed
  08d5dbd

  Copy full SHA for 08d5dbd

Commits on Oct 19, 2022

• Merge branch 'main' into attention-attribution

  lsickert

  committed
  57ae54a

  Copy full SHA for 57ae54a

Commits on Oct 24, 2022

• added basic attention method stubs\n added attention method registry

  lsickert

  committed
  544321c

  Copy full SHA for 544321c

Commits on Nov 21, 2022

• reverted changes to output generation (forward pass done inside attribution function now)

  lsickert

  committed
  d0e859f

  Copy full SHA for d0e859f

• first working version of basic attention methods

  lsickert

  committed
  a2a2021

  Copy full SHA for a2a2021

Commits on Nov 22, 2022

• fixed rounding of values in cli output

  lsickert

  committed
  7b3c4fd

  Copy full SHA for 7b3c4fd

• added documentation to most methods and generalized functions

  lsickert

  committed
  13fc9f3

  Copy full SHA for 13fc9f3

Commits on Nov 23, 2022

• Merge branch 'main' into attention-attribution\n\nNeeded to downgrade torch to ~1.12.1 again because of platform issue: pytorch/pytorch#88826

  lsickert

  committed
  bb13036

  Copy full SHA for bb13036

Commits on Nov 24, 2022

• removed python 3.11 build target

  lsickert

  committed
  9340343

  Copy full SHA for 9340343

• fix safety warnings

  lsickert

  committed
  3cfd706

  Copy full SHA for 3cfd706

Commits on Nov 25, 2022

• set correct python version in pyproject.toml

  lsickert

  committed
  2765d63

  Copy full SHA for 2765d63

• regenerated requirements without 3.11

  lsickert

  committed
  4dd442f

  Copy full SHA for 4dd442f

Commits on Dec 9, 2022

• Merge branch 'main' into attention-attribution, quick fix for mps issue

  lsickert

  committed
  b14407c

  Copy full SHA for b14407c

Commits on Dec 12, 2022

• Merge branch 'main' into attention-attribution

  lsickert

  committed
  6a72166

  Copy full SHA for 6a72166

Commits on Jan 2, 2023

• merge branch 'main' into attention-attribution

  lsickert

  committed
  6535b09

  Copy full SHA for 6535b09

• update deps after merge

  lsickert

  committed
  624435e

  Copy full SHA for 624435e

• include 3.11 as build target

  lsickert

  committed
  06f89a8

  Copy full SHA for 06f89a8

• fix different attribution_step argument formatting

  lsickert

  committed
  7bcbe92

  Copy full SHA for 7bcbe92

• added basic decoder-only support

  lsickert

  committed
  b2fc73c

  Copy full SHA for b2fc73c

Commits on Jan 3, 2023

• fixed output error for decoder only models

  lsickert

  committed
  b044b4c

  Copy full SHA for b044b4c

• removed unnecessary convergence delta references in attention attribution call

  lsickert

  committed
  8c344b7

  Copy full SHA for 8c344b7

Commits on Jan 4, 2023

• allow negative indices when selecting a specific attention head for attribution

  lsickert

  committed
  f51cf25

  Copy full SHA for f51cf25

• added missing negation to head checking

  lsickert

  committed
  c6a9e70

  Copy full SHA for c6a9e70

• fixed last_layer_attention attribution

  lsickert

  committed
  6c9cfae

  Copy full SHA for 6c9cfae

Commits on Jan 9, 2023

• use custom format_attribute_args function for attention methods

  lsickert

  committed
  b78bcc1

  Copy full SHA for b78bcc1

• always use decoder_input_embeds in forward output

  lsickert

  committed
  d27f1c3

  Copy full SHA for d27f1c3

• reworked LastLayerAttention to work with any single layer and allow a range of layers to be specified for averaging in AggregatedAttention

  lsickert

  committed
  cacaa31

  Copy full SHA for cacaa31

Commits on Jan 10, 2023

• Minor bugfixes and version bumps

  gsarti

  committed
  a8d5264

  Copy full SHA for a8d5264

• Generalized attention attribution

  gsarti

  committed
  966f63c

  Copy full SHA for 966f63c

Commits on Jan 13, 2023

• updated documentation and added 'min' aggregation function

  lsickert

  committed
  1301a02

  Copy full SHA for 1301a02

Commits on Jan 14, 2023

• Tests, typing fix, additional checks

  gsarti

  committed
  914ee8f

  Copy full SHA for 914ee8f

• Fix style

  gsarti

  committed
  7c825ad

  Copy full SHA for 7c825ad

Commits on Jan 15, 2023

• added tests for attention utils

  lsickert

  committed
  f6f0a64

  Copy full SHA for f6f0a64

Commits on Jan 16, 2023

• classmethod -> staticmethod where possible

  gsarti

  committed
  f40f63b

  Copy full SHA for f40f63b