16 Commits (87197f8c2e4d002fc39027c3d4ee99f4ead0ba2c)

Author	SHA1	Message	Date
Zhen Wang	be3777a50a	Add pure fp16 training with master weights. (#27712) ... * add the weight decay func for the momentum op * Add the multi_precision function in Momentum Optimizer. * Make sure that the initial value of master weights are same with the fp16 weights. * add static loss scaling. * add the rescale_grad function in the pure fp16 training. * use the original momentum updating method. * Polish some codes, such as variable names. * add docstring for apis. * update the var creation details of _create_master_weight. * not modify codes about imperative momentum updating. * Fix the error of test_dist_sparse_tensor_load_momentum UT. * add unit test for multi precision fp16 training. * add more unit tests for CI. * Use lower threshold values for allclose comparing in test_multi_precision_fp16_train UT. * For CI Coverage Checking.	5 years ago
Jiawei Wang	a5d13d593c	Momentum Velocity init in Momentum.__init__() (#29223) ... * add lamb optimizer and unittest * fix momentum resume training * fix momentum acc	5 years ago
Zhou Wei	c0a991c874	accumulate gradient for leaf tensor with previous graph and expose leaf tensor concept (#28429) ... * The leaf tensor concept is exposed and the gradient accumulation of leaf tensor * The leaf tensor concept is exposed and the gradient accumulation of leaf tensor * fix coverage * fix api doc * fix CI unittest * fix CI unittest * fix unitest * empty tensor does’t need inner_var_ * fix some error message	5 years ago
furnace	8ff3550658	refactor momentum op to combine weight (#27414) ... * refactor momentum op to combine weight_decay (scale op and sum op)	5 years ago
Jiawei Wang	a1b99fae07	Adadelta Optimizer (#26590) ... * add doc; notest * fix doc; notest * update doc; notest * refine optimizer && adam * refine optimizer; notest * add adam * fix doc * fix doc && add adamw; notest * add error message * bug fix * refine rmsprop && adamax * fix ci * buf fix * update comment * unify arguments place; notest * fix ut, test=develop * bug fix * fix conflicts, test=develop * add examples code * bug fix * fix comments * fix sample code * add sample code for Optimizer * add adamax ut, test=develop * fix rmsprop ut, test=develop * add ut for optimizer.py and adamw.py * first commit of adadelta optimizer * fix learning rate * fix adadelta doc and add sgd momentum * remove unused fluid * fix codestyle * Update test_adam_op.py * Update test_adam_op.py * fix SGD in 2 unittests * fix SGD in 2 unittests * fix ci * fix ut Co-authored-by: MRXLT <xlt2024@gmail.com> Co-authored-by: mapingshuo <mps2012@yeah.net>	5 years ago
Wu Yi	856f0da0fe	Fp16 training (#14992) ... * wip * wip * wip * wip for test * add fp16 tests test=develop * fix cpu build test=develop * fix test=develop * fix py3 tests test=develop * fix lr_scheduler dtype test=develop * fix test=dvelop * test fix ci compile test=develop * fix build and merge test=develop * fallback momentumop change to general test=develop * make fp16 lr schedule simple test=develop * fix ut test=develop * fix tests test=develop * remove fp16 learning rate cast test=develop	7 years ago
typhoonzero	da87f7a698	Revert "[Feature] Fp16 training for resnet50 (#14850)" ... This reverts commit 3d750f9c5a.	7 years ago
Wu Yi	3d750f9c5a	[Feature] Fp16 training for resnet50 (#14850) ... * wip * wip * wip * wip for test * add fp16 tests test=develop * fix cpu build test=develop * fix test=develop * fix py3 tests test=develop * fix lr_scheduler dtype test=develop * fix test=dvelop * test fix ci compile test=develop * fix build and merge test=develop * fallback momentumop change to general test=develop	7 years ago
Wu Yi	26200f2e42	[1.1] [project] train imagenet using large batch size (#13766) ... * fix nccl2 lars dist support * put lars in momentum op * add tests lars * fix ci * fix cpu kernel * soft warning * remove lars in test_recognize_digits.py * move to another op * add file * update api.spec test=develop * update test=develop * fix api.spec test=develop * wip * wip, finish grad merge ops * wip, finish graph build * wip test running * work on 1 gpu * workable version * update * fix tests * fuse broadcast op * fix compile failed * refine * add batch merge test mnist * fix CI test=develop * fix build * use independent bn params for batch merge test=develop * update api.spec * follow comments and for test * wip * refine tests test=develop * follow comments test=develop * remove startup bn modify test=develop * follow comments test=develop * fix merge test=develop	7 years ago
dzhwinter	00e8791f66	fix compile in cpu error. test=develop	7 years ago
dzhwinter	8329a1f139	add sparse update momentum. test=develop	7 years ago
minqiyang	99d3f08920	Add print_function for all python files	7 years ago
minqiyang	9fc13fde18	Remove python3 relative import of unittest	7 years ago
minqiyang	35e6abd7bb	Change iter_parameters back and port unittests code to Python3	7 years ago
qingqing01	873a50ce35	Fix serious bug in nesterov momentum optimizer. (#12231) ... * Fix serious bug in nesterov momentum optimizer.	7 years ago
Luo Tao	b11956a0b5	move Fluid API code out of V2 API code	7 years ago