Paddle

Commit Graph

Author	SHA1	Message	Date
MRXLT	55098b975e	fleet support paddle.optimzier (#28026 ) fleet support paddle.optimzier * bug fix * fix fleet_base * bug fix * fix coverage	5 years ago
gongweibao	a7c5210051	Fix test_hdfs bug. (#26068 ) * fix merge3 test=develop	5 years ago
gongweibao	0067a2e4ec	Save checkpoint automatically (#25917 )	5 years ago
gongweibao	80f1c50738	Fix typo in interface. (#24779 )	5 years ago
mapingshuo	f0e743f136	fix AMP and recompute (#23551 ) * allow amp and recompute working together	5 years ago
gongweibao	24a063f6ac	Add fleet checkpoint on local fs and remote fs(such as hdfs) for EDL (#22586 )	5 years ago
tianshuo78520a	d2ba91aad1	fix typo words (#22653 )	5 years ago
WangXi	3ec289a6a3	fix sync_batch_norm hang in fleet (#21838 )	6 years ago
lilong12	da75ac8b6c	bugfix: construct a DistributedStrategy instance if the passed one is None (#21545 )	6 years ago
lilong12	53148e0696	modify the implementation of save_persistables and save_inference_model for fleet collective mode (#20802 ) * modify the implementation of save_persistables and save_inference_model functions for fleet collective, test=develop * add ut, test=develop	6 years ago
WangXi	cadc6a9704	fix dgc test and bug when not set trainers_endpoints_, test=develop (#20617 )	6 years ago
mapingshuo	f55d1c6867	Fleet: deal with special case: strategy is None (#20359 ) * special case: strategy is None	6 years ago
mapingshuo	9901f69677	Forward recompute3 (#19913 ) * add recompute based checkpoints methods for large batch training test=develop * add append_backward_with_forward_recomputation test=develop * refine optimizer test=develop * update backward and optimizer test=develop * make Variable usable test=develop * add recompute code * refine optimizer test=develop * refine addup _append_backward_ops_with_checkpoints_ 1) for recompute part, just cache the grad_op_desc without appending to block 2) before appending grad_op_desc to backward part, addup_repetitive_vars, remove unused branch test=develop * make method private * add recompute strategy into DistributedStrategy test=develop * checkpoint version3 test=develop * remove some print information test=develop * remove unused sumop test=develop * try to fix recompute with graph building modules * add input names to vars should be held * add memory debug tool * backup backward * Fix bugs * add backward desc for op not in any segments * add exception info for sub_block test=develop * modify code style test=develop * modify code style test=develop * remove print functions test=develop * add API spec test=develop test=document_preview * make Recompute a child class of Optimizer test=develop test=document_preview * add API spec test=develop test=document_preview * modify API spec test=develop test=document_preview * add document for Recompute test=develop test=document_preview * change API doc of Rcompute test=develop test=document_preview * code cleaning test=develop test=document_preview * modify API spec * fix bugs when segments hold no element * add testcase for Recompute Optimizer test=develop test=document_preview * add test for apply_gradient, and code cleaning test=develop test=document_preview * add test case for load function * enable CI test=develop test=document * add test case test=develop test=document_preview * add sample code for 4 function of recompute optimizer test=develop test=document_preview	6 years ago
gongweibao	e8d3745c0f	change _origin_program test=develop (#19863 ) change _origin_program test=develop	6 years ago
gongweibao	6c2bc29cc0	Fix float16 optimizer. (#19682 ) Fix float16 optimizer	6 years ago
Yi Liu	4ef6b8457a	adapte fleet api for localsgd and support nccl comm configuration in executor (#19443 ) test=develop	6 years ago
gongweibao	86f0591175	Remove node_num function. (#19167 ) node_num is not needed for users, so remove them and fix the bugs about it!	6 years ago
gongweibao	29d8781240	Polish fleet API to support cuda collective mode and nccl2 mode. (#18966 ) Polish fleet API to support cuda collective mode and nccl2 mode	6 years ago
guru4elephant	9c17a899d7	upgrade collective fleet api (#18533 ) * upgrade collective fleet api	6 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175 ) * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	6 years ago
tangwei12	101f74cb19	fix save/load in fleet (#17675 ) * fix save/load in Fleet * add UT framework of Fleet	6 years ago
Qiao Longfei	58f7695ab2	Async exe support communicator (#17386 ) Async exe support communicator	6 years ago
tangwei12	565d309501	Reformat fleet API (#17135 ) * fix some logic in distributed transpiler, test=develop * reformat fleet API, test=develop	6 years ago
tangwei12	1a4a51db2b	Fleet unify distributed training (#16791 ) * implement distributed transpiler with fleet	6 years ago

24 Commits (0073f9bdb0b43a8d298346e28a2b403fe351bac3)