Commit Graph

10 Commits (87197f8c2e4d002fc39027c3d4ee99f4ead0ba2c)

Author SHA1 Message Date
WangXi 0a1862d1d2
fleet combine amp dgc recompute meta optimizer (#27643)
5 years ago
mapingshuo 3e2a348886
add string variable support for RecomputeOptimizer (#25728)
5 years ago
Chen Weihang 25a233e46d
Simplify Program printing code to improve debugging efficiency (#23918)
5 years ago
Zhou Wei 66dc8e30f0
move the initialize position of grad_clip to optimizer(__init__),and speed up clip (#23782)
5 years ago
WangXi f2265d9ffd
Fix problem use recompute and dgc same time (#23010)
5 years ago
WangXi 62fd3209e1
Fix dgc param regularizer, test=develop (#22888)
5 years ago
WangXi 768f9242e9 Fix dgc clip & rampup step, test=develop (#21491)
6 years ago
WangXi 8ac7687e36 Fix dgc accuracy by mv regularization to local (#21278)
6 years ago
WangXi de5d3ff688 Fix dgc buffer illegal & reuse velocity (#21012)
6 years ago
WangXi 250e72d254 Fix DGC algorithm flow to make it the same as paper (#20758)
6 years ago