139 Commits (e6177072295dd9e54b7968d82327a0cbee68d332)

Author	SHA1	Message	Date
ShenLiang	01e2874a0e	Support multi-stream communication for dynamic graph distributed (#29525) ... * fix fleet for multi-stream * fix memcpy for ncclid * use sync to solve move operation	4 years ago
WangXi	9cbcc6cadc	fleet sync build strategy, test=develop (#29732)	4 years ago
JZ-LIANG	d33d468f02	[Sharding] add hybrid-dp feature (#29518) ... * Sharding add hybrid-dp feature * update sharding in distributed_strategy * update sharding unitest * revise code format for sharding	4 years ago
ShenLiang	2ef9e0e23c	Rebuild group automatically in dynamic graph distributed (#29255) ... * add tensor_indices in AssignGroupBySize * add rebuild group in reducer	4 years ago
lilong12	b122d0bb76	Fix bug in gloo that gloo initialization hangs (#29447) ... * update, test=develop	4 years ago
ShenLiang	4064354a01	support dp run single card (#29358)	4 years ago
gongweibao	96de8b008f	cleanup enum test=develop (#29294)	4 years ago
ShenLiang	2d6aa1a5bb	fix warning of fleet (#29317)	4 years ago
ShenLiang	2cd0bf5764	Fix doc of fleet api (#29282) ... * fix doc, test=document_fix	4 years ago
ShenLiang	46b73e6cd9	Change the api of DataParallel and Fleet (#29224)	4 years ago
123malin	cc9c619679	test=develop, fix doc (#29200) ... * fix fleet api doc	4 years ago
WangXi	0c2a51d240	optimizer amp, all use fp16 communication, overlap last comm and compute (#28957)	4 years ago
123malin	92817f8005	test=develop, rm pathlib (#28658) ... * test=develop, rm pathlib	4 years ago
ShenLiang	e2d01eb650	Support dynamic graph distributed (#28997) ... * add reducer * refine envent for memorycopy * add concat&split for allreduce * apply concat & split for fuse tensor * fix nccl dep * fix the untest, compile problem and ddp initialize problem * fix untest for mac & add some comments & solve the repeated param in sublayers * fix untest for windows & fix document	4 years ago
Chen Long	d576d6ddeb	fix some docs test=develop;test=document_fix (#29159)	4 years ago
lilong12	216e085605	update, test=develop (#29139)	4 years ago
lilong12	a1add716bc	Add a flag to control whether to initialize gloo (#29150)	4 years ago
ShenLiang	cddc70964d	fix InMemoryDataset doc (#28688) ... * add Inmemorydataset	4 years ago
JZ-LIANG	0dadacc4eb	[sharding] doc, api, bug fixed (#28983) ... * add lars to fleet meta optimizer * add lamb to proto * add lamb to fleet meta optimizer * fixed syntax bug * fixed syntax bug * fixed syntax error in lamb, add config setter of lamb in distributed_strategy * trigger unitest to rerun * add new unitest func for lamb * revise unitest for lars and lamb * revise dgc meta unitest * revise lars document in distribute_strategy * revise lars lamb document in distributed_strategy.py * revise lars lamb document in distributed_strategy.py * add weight decay exclude logic to lars * restore optimzier.py * restore optimizer.py as develop except lars * add epsilon and exclude fn to distributed_sttrategy * add lars epsilon * revise unitest for fleet lars and lamb * revise lars lamb unitest for CI coverage * revise lars argument api * revise lars argument api * revise lars argument api * revise api doc of lars * fix op role * add sharding save and add_sync_comm_for_test function * add comm_analyse to utlis * revise sharding_utils * add sharding saving unittest * revise sharding utils for unittest * revise sharding en doc * update sharding utils api * add doc for sharding * fixed bug in sharding var size count * update varsize count in sharding * fix sharding num_nccl_comm * Revert "fix sharding num_nccl_comm" This reverts commit d51587c15e9323acf226ddd36154275f0d1daf76.	4 years ago
lilong12	2a864c70c4	fix the bug in gloo (#29112) ... * update, test=develop	4 years ago
WangXi	e931c7baf9	Fix multi nccl comm & wait server ready (#28663)	4 years ago
gongweibao	1358397e97	Clean up the redundant files and unify the launch interface. (#28928)	4 years ago
Chen Weihang	bb16c2515d	Polish parallel api impl & doc details (#28980) ... * polish parallel api impl & doc details * add unittest for coverage * remove spawn test in py2.7 * add parallel api into white list	4 years ago
Leo Chen	3815d7aa40	Upgrade string literals to raw string (#28989) ... * upgrade comment string to raw string * fix string in * fix string with ' ' * revert update on comments * upgrade only necessary * fix sample code checker * fix comments with '''	4 years ago
123malin	fbf9564f6b	【paddle.distributed.fleet】Optimize ParameterServer's Async Mode (#28442) ... * test=develop, optimize global_step	4 years ago
lilong12	f77a78cdee	enable pipeline to run with Executor.run() (#28373) ... * update, test=develop	4 years ago
Chen Weihang	bff4179cc7	lazily init global group in collective (#28780)	4 years ago
JZ-LIANG	5a9f6889c1	[Sharding] add new features (#28568) ... * add lars to fleet meta optimizer * add lamb to proto * add lamb to fleet meta optimizer * fixed syntax bug * fixed syntax bug * fixed syntax error in lamb, add config setter of lamb in distributed_strategy * trigger unitest to rerun * add new unitest func for lamb * revise unitest for lars and lamb * revise dgc meta unitest * revise lars document in distribute_strategy * revise lars lamb document in distributed_strategy.py * revise lars lamb document in distributed_strategy.py * add weight decay exclude logic to lars * restore optimzier.py * restore optimizer.py as develop except lars * add epsilon and exclude fn to distributed_sttrategy * add lars epsilon * revise unitest for fleet lars and lamb * revise lars lamb unitest for CI coverage * revise lars argument api * revise lars argument api * revise lars argument api * revise api doc of lars * fix op role * add sharding save and add_sync_comm_for_test function * add comm_analyse to utlis * revise sharding_utils * add sharding saving unittest * revise sharding utils for unittest	4 years ago
lilong12	e4f9415338	update doc, test=document_fix (#28498)	4 years ago
danleifeng	a24d186814	fix nccl init failed in parallel dygraph mode (#28497)	4 years ago
Chengmo	4dc8c44ba1	【Paddle.Fleet】Fix fleetrun heter (#28252) ... * fix fleetrun heter ps on paddlecloud	4 years ago
mapingshuo	81244fbfab	add sharding strategy in fleet(#27900) ... * add sharding	4 years ago
WangXi	11acbfae06	refine auto strategy, test=document_fix (#28211)	4 years ago
MRXLT	55098b975e	fleet support paddle.optimzier (#28026) ... fleet support paddle.optimzier * bug fix * fix fleet_base * bug fix * fix coverage	4 years ago
lilong12	5bb348a1c2	add doc for ReduceOp (#28051) ... * add doc, test=document_fix	4 years ago
WangXi	fb641c915e	【paddle.fleet】fleet add _get_applied_meta_list and _get_applied_graph_list (#27952)	4 years ago
lilong12	ff0ebefc1e	put gloo initialization log to file (#27969) ... * update, test=develop	4 years ago
tangwei12	202bfab1be	Feature/large scale kv save base/delta (#27470) ... * add size method for large scale * add large scale UT * add ut for checkpoint	4 years ago
123malin	aa3b4ed717	【paddle.fleet】geo send sparse optimize (#27719) ... * test=develop, fix geo sgd communicator * test=develop, gloo_init_method * test=develop, bug fix for gloo http_init	4 years ago
danleifeng	8d7908f3fd	【paddle.fleet】raise error when using multi-cards in fleet non_distributed mode (#27854) ... * raise error if use multi-cards in fleet non_distributed mode; test=develop	4 years ago
chentianyu03	d05058d268	Remove and reorganize the alias of APIs (#27717) ... * modify cond while_loop to paddle.static.nn.cond * modify crop_tensor to paddle.crop * modify Variable to paddle.static.Variable * remove nn.beam_search, nn.beam_search_decode, nn.gather_tree * remove bpr_loss, center_loss, rank_loss, smooth_l1, teacher_student_sigmoid_loss, edit_distance, sampled_softmax_with_cross_entropy in nn.functional * remove apis in nn.functional.learn_rate.py * remove pool2d, pool3d, adaptive_pool2d, adaptive_pool3d in nn.functional * remove apis in nn.functional.vision * remove erf, soft_relu in nn.functional.activation * remove apis in nn.functional.extension * remove nn.functional.rnn * remove hash from nn.functional.lod * remove row_conv from nn.functional.extension * remove one_hot, pad2d, pad_constant_like from nn.functional.common * remove nn.gather_tree, nn.BilinearTensorProduct, nn.Pool2D, nn.Pad2D * remove apis from optimizer.__init * remove tensor.creation.fill_constant * remove elementwise_mul in nn.functional.common and modify to paddle.multiply * remove tensor.stat.reduce_mean * remove reduce_all, reduce_any in tensor.logic * remove apis in tensor.math * remove apis in tensor.__init__ * remove has_inf, has_nan in tensor.search * remove apis in framework.__init__ * remove apis in paddle.__init__ * remove apis in nn.functional.__init__ * modify removed alias apis to raw api in doc and unittests * fix remove grid_sample bug * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * delete alias api relastions in doc * reserve paddle.compat, paddle.sysconfig * remove unittest for paddle.reduce_all, paddle.reduce_any * modify removed alias apis to raw api in doc and unittests * recover paddle.save and paddle.load * resolve conflicts * fix sample code missing paddle.enable_static() bug * fix sample code missing paddle.enable_static() bug * fix to_string sample code error	4 years ago
Chengmo	328cb289ed	【paddle.fleet】fix sparse load (#27680) ... * add sparse tensor load method	4 years ago
123malin	a4f850748a	【paddle.fleet】bug fix for parameter_recv (#27838) ... * test=develop, bug fix for parameter_recv * test=develop, for unittest, test_fleet_rolemaker_new	4 years ago
Chen Weihang	ed31dac6eb	remove scale loss and coll grads, test=document_fix (#27874)	4 years ago
WangXi	50619cd842	use floyd algorithm to find meta optimizer max path, test=develop (#27867)	4 years ago
mapingshuo	8d2cb14f98	support gradient merge with recompute, test=develop (#27834) ... * support gradient merge with recompute, test=develop test=develop	4 years ago
Chengmo	c5f2802d56	【paddle.fleet】Update fleetrun & ps-heter (#27472) ... * refine fleetrun.ps_launch * update fleet run for multi device support * ps_graph support ps-gpu * fix heter save * add heter save unittest * fix unittest & simple code * update fleetrun * fix fleetrun * fix launch barrier * fix role maker * add paddlecloud rolemaker unittest * rename heter_worker_device_guard	4 years ago
WangXi	0a1862d1d2	fleet combine amp dgc recompute meta optimizer (#27643)	4 years ago
danleifeng	a01bc6b31d	【paddle.fleet】fleet support non_distributed training in dygraph mode (#27714) ... * fleet support non_distributed training in dygraph mode; test=develop	4 years ago
lilong12	742cbe6660	[bug fix] avoiding multiple initialization of gloo for fleet in dygraph mode (#27706) ... * add double grad for expand, test=develop	4 years ago