8 Commits (develop)

Author	SHA1	Message	Date
Zhang Ting	e97905c5fa	improve performance of momentum (#30881)	5 years ago
Zhen Wang	7f7dfccf20	Support pure fp16 training for AMP API. (#29544) ... * add cast ops before and after unsupported fp16 ops. * Keep partial net in FP32 pattern. * Support check_finite_and_unscale and update_loss_scaling for FP16 calculation mode. * Add fp16 support for adam op. * add multi precision attr for adam. * Fix the bug of test_multi_precision_fp16_train UT. * Code format for CI. * Fix the redefine error about MPTypeTrait on windows. * fix bugs of the _create_accumulators func in Momentum. * fix bug when inserting post cast op. * Add the update_loss_scaling op in allow_set of UnusedVarCheck. * Update for ci coverage. * Add some doc for OptimizerWithMixedPrecision. * Fix the code style. * Imporve the doc of `amp_init`. * Change for fp16 testing if users have the infer program defined in separate way.	5 years ago
wanghuancoder	88e6dc4ac5	optimize momentum to speedup dygraph, a little, test=develop (#30099)	5 years ago
huangxu96	a29006d128	Optimizer trans momentum (#29597) ... * merge amp related function in Momentum from paddle.fluid.contrib.optimizer into paddle.optimizer. * Add unittest for 2.0 Momentum API. * fix some bugs in weight_decay.	5 years ago
Jiawei Wang	b11ab12787	Fix doc (adadelta, sgd, momentum) (#29212) ... * fix 3 doc * fix 3 doc * Update adadelta.py	5 years ago
Jiawei Wang	a5d13d593c	Momentum Velocity init in Momentum.__init__() (#29223) ... * add lamb optimizer and unittest * fix momentum resume training * fix momentum acc	5 years ago
Leo Chen	3815d7aa40	Upgrade string literals to raw string (#28989) ... * upgrade comment string to raw string * fix string in * fix string with ' ' * revert update on comments * upgrade only necessary * fix sample code checker * fix comments with '''	5 years ago
Jiawei Wang	a1b99fae07	Adadelta Optimizer (#26590) ... * add doc; notest * fix doc; notest * update doc; notest * refine optimizer && adam * refine optimizer; notest * add adam * fix doc * fix doc && add adamw; notest * add error message * bug fix * refine rmsprop && adamax * fix ci * buf fix * update comment * unify arguments place; notest * fix ut, test=develop * bug fix * fix conflicts, test=develop * add examples code * bug fix * fix comments * fix sample code * add sample code for Optimizer * add adamax ut, test=develop * fix rmsprop ut, test=develop * add ut for optimizer.py and adamw.py * first commit of adadelta optimizer * fix learning rate * fix adadelta doc and add sgd momentum * remove unused fluid * fix codestyle * Update test_adam_op.py * Update test_adam_op.py * fix SGD in 2 unittests * fix SGD in 2 unittests * fix ci * fix ut Co-authored-by: MRXLT <xlt2024@gmail.com> Co-authored-by: mapingshuo <mps2012@yeah.net>	6 years ago