12 Commits (be3777a50a08fa06f0a700f1fd5bead38ac47e1b)

Author	SHA1	Message	Date
Zhen Wang	be3777a50a	Add pure fp16 training with master weights. (#27712) ... * add the weight decay func for the momentum op * Add the multi_precision function in Momentum Optimizer. * Make sure that the initial value of master weights are same with the fp16 weights. * add static loss scaling. * add the rescale_grad function in the pure fp16 training. * use the original momentum updating method. * Polish some codes, such as variable names. * add docstring for apis. * update the var creation details of _create_master_weight. * not modify codes about imperative momentum updating. * Fix the error of test_dist_sparse_tensor_load_momentum UT. * add unit test for multi precision fp16 training. * add more unit tests for CI. * Use lower threshold values for allclose comparing in test_multi_precision_fp16_train UT. * For CI Coverage Checking.	5 years ago
furnace	8ff3550658	refactor momentum op to combine weight (#27414) ... * refactor momentum op to combine weight_decay (scale op and sum op)	5 years ago
Chengmo	d014e29fc6	fix error message (#27318) ... * fix sgd/momentum/dpsgd/rmsprop error message	5 years ago
Chen Weihang	26cc1fe508	Replace risky GetInputType method with secure IndicateVarDataType interface (#20668) ... * replace part of the old implementation, test=develop * restore concat op, test=develop * update all ops implemention & delete GetDataTypeOfVar func, test=develop	6 years ago
WangXi	250e72d254	Fix DGC algorithm flow to make it the same as paper (#20758)	6 years ago
Chen Weihang	8cb54ede8c	Add user-friendly error message in optimizer ops to give a hint about the position sensitive problem of run(startup_program) (#19605) ... * add extra error message hint in optimizer ops * polish format & delete useless change, test=develop * extract init judue from shape compare, test=develop	6 years ago
sneaxiy	f0d108f589	fix const_cast ... test=develop	6 years ago
sneaxiy	c1f7e54f62	merge develop ... test=develop	7 years ago
Wu Yi	3d750f9c5a	[Feature] Fp16 training for resnet50 (#14850) ... * wip * wip * wip * wip for test * add fp16 tests test=develop * fix cpu build test=develop * fix test=develop * fix py3 tests test=develop * fix lr_scheduler dtype test=develop * fix test=dvelop * test fix ci compile test=develop * fix build and merge test=develop * fallback momentumop change to general test=develop	7 years ago
sneaxiy	ae6f46a1a9	rewrite variable type ... test=develop	7 years ago
minqiyang	53433d7f2e	Revert the changes of VLOG ... test=develop	7 years ago
Wu Yi	a2d9b34417	Refine operator cmake (#14413) ... * wip simplify operator framework * wip * wip * done test=develop * clean test=develop * fix test=develop * fix deps test=develop * fix cpu build test=develop * fix tensorrt build test=develop * fix tests test=develop * fix test=develop * fix cpu build test=develop	7 years ago