Paddle

Commit Graph

Author	SHA1	Message	Date
Zhang Ting	906e7f921e	add fuse_bn_act op (#27230 ) * add fused_bn_add_relu op	4 years ago
Zhen Wang	d708b21074	Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for static graph amp training. (#26240 ) * update amp_check_finite_and_scale_op for static_amp. * use amp_check_finite_and_scale in static graph amp. * update grads to zero when grads own infinite values(as for amp_checkout_finite_and_scale op). * add update_loss_scaling op in cpp. * add update_loss_scaling_op unit test. * update the doc of the check_finite_and_unscale op * Update the process of gradients updating skipping if the gradients have infinite values. * update the way to zero grads. * update test_update_loss_scaling_op.py * add log info when find infinite grads. * add the unit test for UpdateLossScaling Layer.	5 years ago
Zhen Wang	bcdbac1753	fix some cast error. (#26884 )	5 years ago
mapingshuo	f0e743f136	fix AMP and recompute (#23551 ) * allow amp and recompute working together	5 years ago
gongweibao	5e07db15e6	fix init scaling value test=develop (#22145 )	5 years ago
Zhen Wang	be2e3e67d9	Fix some typos in AMP. (#21354 ) * fix some typos in AMP. test=develop * delete useless codes. test=develop	5 years ago
gongweibao	3255fe69bb	Add custom black variable name set in amp interface. (#20875 ) * add custom black varname test=develop * fix dtype test=develop * fix num test=develop * fix ut test=develop * fix coverage test=develop * fix blackvar names test=develop	5 years ago
gongweibao	1d82025e89	Add interface so user can get scaled loss when they use customized loss. (#20571 )	5 years ago
gongweibao	7b9e33972a	delete backward return list test=develop (#20294 )	5 years ago
Jie Fang	d9db94d752	Optimize amp for multi-gpu to enable FP16 gradients transfer across gpus. (#19714 ) Optimize amp for multi-gpu to enable FP16 gradients transfer across gpus	6 years ago
gongweibao	6c2bc29cc0	Fix float16 optimizer. (#19682 ) Fix float16 optimizer	6 years ago
Jie Fang	c6a598a276	init new amp, optimize inserting cast op for batchnorm (#18596 ) init new amp, optimize inserting cast op for batchnorm	6 years ago
gongweibao	abaf87be2b	Change backward_guard to optimize_guard to maximize the allreduce overlap. (#19506 ) Change backward_guard to optimize_guard to maximize the allreduce overlap	6 years ago
Zeng Jinle	5dce1da680	remove reset recordio usage (#19519 )	6 years ago
Jie Fang	2b4ef509ea	init custom black white list (#18377 ) test=develop	6 years ago
Jie Fang	172c2facef	init black/white lists (#17847 ) test=develop	6 years ago
Jie Fang	30e178fa2c	init auto loss scaling (#17194 ) * init auto loss scaling test=develop * change API.spec * change ifelse to switch and use reduce_sum to optimize checking isfinite test=develop * Remove redundant code test=develop	6 years ago
Yibing Liu	beda78258f	Init mixed precision training interface (#16856 ) * Init mixed precision training interface * Add fp16 test script test=develop * All initializers support float16 test=develop * Code cleanup & add more code annotations test=develop * Update API spec test=develop * Add usage example in doc test=develop	6 years ago

18 Commits (b38e4f2840ebc4ee0195ab8de789bf5b8d54ef37)