Skip to content

Commit

Permalink
Merge pull request #76 from andreped/andreped-patch-2
Browse files Browse the repository at this point in the history
README technique order update [no ci]
  • Loading branch information
andreped authored May 3, 2023
2 parents 94e3569 + 5330dfe commit d024a7c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ Our implementations enable theoretically **infinitely large batch size**, with *

| Technique | Usage |
| - | - |
| `Adaptive Gradient Clipping` | `model = GradientAccumulateModel(accum_steps=4, agc=True, inputs=model.input, outputs=model.output)` |
| `Batch Normalization` | `layer = AccumBatchNormalization(accum_steps=4)` |
| `Adaptive Gradient Clipping` | `model = GradientAccumulateModel(accum_steps=4, agc=True, inputs=model.input, outputs=model.output)` |
| `Mixed precision` | `model = GradientAccumulateModel(accum_steps=4, mixed_precision=True, inputs=model.input, outputs=model.output)` |

* As batch normalization (BN) is not natively compatible with GA, we have implemented a custom BN layer which can be used as a drop-in replacement.
Expand Down

0 comments on commit d024a7c

Please sign in to comment.