5 Commits (8f2656ef5ca4ab16f06d94b8ca9392d3f0f760ae)

Author	SHA1	Message	Date
Zhen Wang	365c2c9c89	fix error message showing in UpdateLossScalingOp (#27596)	5 years ago
Zhen Wang	d708b21074	Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for static graph amp training. (#26240) ... * update amp_check_finite_and_scale_op for static_amp. * use amp_check_finite_and_scale in static graph amp. * update grads to zero when grads own infinite values(as for amp_checkout_finite_and_scale op). * add update_loss_scaling op in cpp. * add update_loss_scaling_op unit test. * update the doc of the check_finite_and_unscale op * Update the process of gradients updating skipping if the gradients have infinite values. * update the way to zero grads. * update test_update_loss_scaling_op.py * add log info when find infinite grads. * add the unit test for UpdateLossScaling Layer.	5 years ago
Leo Chen	2d95280e1f	Feature/Enable Auto-Mixed-Precision in dynamic graph (#24903) ... * add auto_cast, test=develop * add loss scaler, test=develop * add comments, test=develop * refine code, test=develop * refine code, test=develop * do not set flags automatically, test=develop * fix custom op bug, test=develop * add more test, test=develop * refine enable logic, test=develop * enable amp test with GPU, test=develop * add unittest * add test for found_inf * follow comments * follow comments * remove global variable, use singleton * add some notes * update comments * update comments * update comments * add use_dynamic_loss_scaling argument * refine found_inf * refine found_inf	5 years ago
Leo Chen	a7cb97a1a5	Fix/isfinite on windows (#24927) ... * refine isfinite, test=develop * use namespace std of isfinite, test=develop, test=win_gpu	5 years ago
Leo Chen	1e818158f5	Feature/add amp_checkout_finite_and_scale op (#24875) ... * add amp_check_finite_and_scale op, test=develop * add cpu kernel, test=develop * use bool, test=develop * follow comments, test=develop	5 years ago