L-electron-Rare/mlx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added Adafactor (#415)

Browse Source 
* Added adafactor

* Added Adafactor and ran pre-commit

* modified operations

* Added docstrings

* Switched two ops to fix a bug

* added underscore for internal functions and removed the plus sign in the last return statment

* Removed parameter rms from the optimizer state because its not needed

* Added simple MNIST test for Adafactor and temporary training log

* remove test files

* nits in docs

* comment nit

---------

Co-authored-by: Awni Hannun <[email protected]>
This commit is contained in:
Hazem Essam
2024-01-23 15:11:27 -08:00
committed by GitHub
co-authored by Awni Hannun
parent 755dcf6137
commit 37fc9db82c
3 changed files with 164 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/src/python/optimizers.rst
+1
View File
@@ -40,6 +40,7 @@ model's parameters and the **optimizer state**.
SGD
RMSprop
Adagrad
Adafactor
AdaDelta
Adam
AdamW