13 Commits (8c19d7aa2f89a38b3a68e53c73d88af16a3de8ce)

Author	SHA1	Message	Date
WangXi	31ed9c9eed	Fleet distributed strategy support pure fp16 (#30754)	5 years ago
Zhen Wang	7f7dfccf20	Support pure fp16 training for AMP API. (#29544) ... * add cast ops before and after unsupported fp16 ops. * Keep partial net in FP32 pattern. * Support check_finite_and_unscale and update_loss_scaling for FP16 calculation mode. * Add fp16 support for adam op. * add multi precision attr for adam. * Fix the bug of test_multi_precision_fp16_train UT. * Code format for CI. * Fix the redefine error about MPTypeTrait on windows. * fix bugs of the _create_accumulators func in Momentum. * fix bug when inserting post cast op. * Add the update_loss_scaling op in allow_set of UnusedVarCheck. * Update for ci coverage. * Add some doc for OptimizerWithMixedPrecision. * Fix the code style. * Imporve the doc of `amp_init`. * Change for fp16 testing if users have the infer program defined in separate way.	5 years ago
WangXi	619c62bb48	fix adamw apply gradient (#30130)	5 years ago
Leo Chen	a4b9daf97c	fix optimizer dtype (#29917)	5 years ago
ShenLiang	e2d01eb650	Support dynamic graph distributed (#28997) ... * add reducer * refine envent for memorycopy * add concat&split for allreduce * apply concat & split for fuse tensor * fix nccl dep * fix the untest, compile problem and ddp initialize problem * fix untest for mac & add some comments & solve the repeated param in sublayers * fix untest for windows & fix document	5 years ago
Leo Chen	3815d7aa40	Upgrade string literals to raw string (#28989) ... * upgrade comment string to raw string * fix string in * fix string with ' ' * revert update on comments * upgrade only necessary * fix sample code checker * fix comments with '''	5 years ago
Zhou Wei	e122e16456	fix english doc, unittest, and remove useless alias of 2.0 lr_scheduler (#27686) ... * fix doc and unittest of 2.0 lr_scheduler * fix doc of 2.0 lr_scheduler * fix unittest * fix english doc of lr_scheduler * fix api name of lr scheduler * fix api name of lr scheduler	5 years ago
MRXLT	84d8e49de8	refine adam/strided_slice && fix doc for rmsprop/unstack (#27740) ... * refine parameters order && doc * update rmsprop doc * refine adam/transpose/unstack/stride_slice * fix bug && doc * fix doc * bug fix * bug fix * fix doc * fix doc * fix doc * fix doc * depercate old strided_slice * update doc * set default value for name * update doc	5 years ago
Chen Weihang	dec53a9c79	Remove DataParallel.scale_loss & apply_collective_grads (#27603) ... * remove data parallel scale loss & apply collective_grads * move apply in minimize * fix failed unittests	5 years ago
MRXLT	f936adbd2d	fix adam (#27343) ... * fix adam * rmsprop support double	5 years ago
MRXLT	9166307315	add check for sparse parameters with weight_decay (#27141) ... * add check for sparse parameters with weight_decay * move sparse check to adam.py	5 years ago
MRXLT	1f36d3cdcb	update optimizer (#26711) ... * update doc * update doc * fix optimizer sample code * add default value for adamw weight_decay * fix adamw * change LearningRateDecay to _LRScheduler * fix adamw;notest * fix load;notest * remove file * bug fix * fix code style * bug fix * add ut * adamw support weight_decay=0 * fix ut * fix set_lr doc * fix doc * change parameters place	5 years ago
MRXLT	eeda90d674	[WIP] update optimizer for 2.0 (#26288) ... refine Optimizer/Adam/Admax/RMSProp && add Admw * buf fix * update comment * unify arguments place; notest * fix ut, test=develop * bug fix * fix conflicts, test=develop * add examples code * bug fix * fix comments * fix sample code * add sample code for Optimizer * add adamax ut, test=develop * fix rmsprop ut, test=develop * add ut for optimizer.py and adamw.py * remove TestAdamOptimizerBetaVariable * update api && add ut * update doc && fix ut * add ut Co-authored-by: mapingshuo <mps2012@yeah.net>	5 years ago