Update main with development in preparation of 1.2.0 release
#67
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Given a linear operator `A`, the `SubmatrixLinearOperator` class allows constructing the linear operator for `A[row_idxs, :][:, col_idxs]`. The sub-matrix can be changed after initialization using the `set_submatrix` method.
* [REF] Allow for partial specification of boundaries * [FIX] Use correct order * [ADD] Test approximate_boundaries and approximate_boundaries_abs * [DEL] Remove print statement * Apply black * Fix flake8 config for python 3.8
* [ADD] Implement `NeumannInverseLinearOperator` and test on toy problem * [ADD] Test Neumann inverse on damped GGN * [ADD] Modify truncation default, improve error message
Co-authored-by: Felix Dangel <[email protected]>
* [ADD] Implement and test `_adjoint` * [FIX] Use column dimension to create random vector * [ADD] Implement model Jacobian * [DOC] Mention Jacobian in README * [REQ] Deprecate python 3.7 to use functorch from inside torch * [DOC] Use python 3.8 to build the docs * [FIX] darglint * [REQ] Use torch>=2 for built-in functorch * [REF] Implement Jacobian via base class, remove print statements * [FIX] Documentation and rename --------- Co-authored-by: Felix Dangel <[email protected]>
#34) * [ADD] Implement transposed model Jacobian * [FIX] Correct math in docstrings --------- Co-authored-by: Felix Dangel <[email protected]>
* [ADD] Adjoints of Jacobians * [FIX] Annotation cycle --------- Co-authored-by: Felix Dangel <[email protected]>
* [REF] Use API of built-in ``functorch`` * [FIX] Return functorch gradient as tuple --------- Co-authored-by: Felix Dangel <[email protected]>
* [DOC] Add Fisher-weighted model merging demo * [FIX] `system_packages` deprecation of RTD * [FIX] flake8
* [ADD] Hutchinson trace estimator * [DOC] Add Hutchinson estimator to documentation * [REF] Move example, polish rst * [RTD] Try fixing deprecated `build.image` https://blog.readthedocs.com/use-build-os-config/#use-build-os-instead-of-build-image-on-your-configuration-file * [FIX] Precision of doctest
* [ADD] Implement Hutch++ * [ADD] Test Hutch++ * [DOC] Specify comparability with Hutchinson * [FIX] Bug: Use span of `A @ S`, not span of `S`
* [REF] Use same code in trace tests, extract random vector generation * [ADD] Hutchinson-style diagonal estimation * [DOC] Add diagonal estimator to documentation * [DOC] Short summary for each trace/diagonal estimation method
* [ADD] Prototype for KFAC linear operator * [DOC] Progress on documentation * [DOC] Describe KFAC and its limitations * [FIX] Name of fixture * [FIX] Darglint * [FIX] Darglint See terrencepreilly/darglint#53 * [DOC] Show supported layers in error message * [DOC] Improve correctness
* [ADD] Prototype for KFAC linear operator * [DOC] Progress on documentation * [DOC] Describe KFAC and its limitations * [FIX] Name of fixture * [FIX] Darglint * [FIX] Darglint See terrencepreilly/darglint#53 * [DOC] Show supported layers in error message * [ADD] Support arbitrary order of parameters
* [ADD] Prototype for KFAC linear operator * [DOC] Progress on documentation * [DOC] Describe KFAC and its limitations * [FIX] Name of fixture * [FIX] Darglint * [FIX] Darglint See terrencepreilly/darglint#53 * [ADD] Support `CrossEntropyLoss` in KFAC
* Add test for one datum KFAC EF exactness * Add fisher_type argument and support empirical Fisher in KFAC * Minor docstring fixes * Clarify fisher_type docstring
* [ADD] Support treating only biases of a layer * [ADD] Support layers without bias * [REF] Use a mapping from parameter ids to module names * [DOC] Remove outdated doc
* [ADD] Attempt supporting treating weights and biases jointly in KFAC * [DOC] Update limitation * [DEL] Remove `assert`s * [FIX] Make tests work * [REF] More verbose id
* [ADD] Support modules with inplace activations in KFAC * [FIX] Use correct device in test
* Add test for MSELoss type-2 KFAC * Implement type-2 KFAC for MSELoss * Add docstrings * Fix black * Fix docstring * Add test for CrossEntropyLoss type-2 KFAC * Implement type-2 KFAC for CrossEntropyLoss * Fix auto-merge issue * Fix comment * [REF] Refactor type-2 using Hessian matrix square root * [DEL] Remove unused imports * [FIX] Darglint * [FIX] Function name in docs * [REF] Improve function name * [REF] Rename `num_classes` into `output_dim` --------- Co-authored-by: Felix Dangel <[email protected]>
* Add Rearrange module util * Add test for KFAC with >2d model outputs * Add support for >2d linear layer inputs (expand) and model outputs * Fix black * Address review feedback * Remove superfluous if statement
* Add Rearrange module util * Add test for KFAC with >2d model outputs * Add support for >2d linear layer inputs (expand) and model outputs * Fix black * Add WeightShareModel KFAC test util * Add weight-sharing KFAC test cases * Add KFAC with weight-sharing exactness test * Add support for KFAC-expand and reduce for linear modules * Fix isort * Remove unused line * Add reference to KFAC docstring * Fix KFAC docstring * Refactor num_loss_terms for readability * Minor test fixes * Switch from warning to ValueError when inconsistent loss_average is used * Fix tests
* [ADD] Draft for activation Hessian * [REF] Move activation Hessian to ``experimental`` and add Hessian test * [FIX] flake8 * [ADD] Add activation Hessian to API and documentation * [DOC] Add missing exception * [DOC] Add usage example for activation Hessian * [DOC] Use double backtick * [DOC] Document `_preprocess`
* Add Rearrange module util * Add test for KFAC with >2d model outputs * Add support for >2d linear layer inputs (expand) and model outputs * Fix black * Add WeightShareModel KFAC test util * Add weight-sharing KFAC test cases * Add KFAC with weight-sharing exactness test * Add support for KFAC-expand and reduce for linear modules * Fix isort * Add Conv2dModel KFAC test util * Add Conv2d KFAC test cases * Modify KFAC weight-sharing test for Conv2dModel * Add extract_patches and extract_averaged_patches utils * Add support for KFAC for Conv2d modules * Fix docstring * Fix black and different test input shape * Change rtol for KFAC weight-sharing test from 1e-5 to 1e-4 * Improve readability of patch_extractor_fn selection * Fix docstring
…0` (#66) * [DOC] Update changelog & contributors, fix PyPI upload, prepare `1.2.0` * [FIX] Incorporate feedback
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.