You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Zhen Wang be3777a50a Add pure fp16 training with master weights. (#27712) ... * add the weight decay func for the momentum op * Add the multi_precision function in Momentum Optimizer. * Make sure that the initial value of master weights are same with the fp16 weights. * add static loss scaling. * add the rescale_grad function in the pure fp16 training. * use the original momentum updating method. * Polish some codes, such as variable names. * add docstring for apis. * update the var creation details of _create_master_weight. * not modify codes about imperative momentum updating. * Fix the error of test_dist_sparse_tensor_load_momentum UT. * add unit test for multi precision fp16 training. * add more unit tests for CI. * Use lower threshold values for allclose comparing in test_multi_precision_fp16_train UT. * For CI Coverage Checking.		6 years ago
..
__init__.py	init custom black white list (#18377)	7 years ago
amp_nn.py	Update amp_check_finite_and_scale_op and add an updating_loss_scaling op for static graph amp training. (#26240)	6 years ago
decorator.py	optimizer amp, all use fp16 communication, overlap last comm and compute (#28957)	6 years ago
fp16_lists.py	Layer norm fp16 (#29169)	6 years ago
fp16_utils.py	Add pure fp16 training with master weights. (#27712)	6 years ago