75 Commits (develop)

Author	SHA1	Message	Date
ShenLiang	c3634c6b0a	fix amp bug of fleet (#31532)	5 years ago
lilong12	dc8dfba35b	align the default value of some configuration for fleet to that of single cards (#30740) ... * update, test=develop	5 years ago
123malin	16b4260b2f	test=develop, save/load, shrink (#30625) ... * test=develop, save/load, shrink Co-authored-by: seiriosPlus <tangwei12@baidu.com>	5 years ago
WangXi	31ed9c9eed	Fleet distributed strategy support pure fp16 (#30754)	5 years ago
Zhen Wang	4a9de931a2	Fix the bug in fleet amp_init. (#30606) ... * Fix the bug in fleet amp_init. * Fix the amp_init unit test.	5 years ago
huangxu96	138620084c	Add fleet amp_init() (#30572) ... * add fleet amp.init() * add unittest for fleet_amp_init	5 years ago
JZ-LIANG	75936d838f	Recompute Offload (#30233)	5 years ago
Chengmo	d479ae1725	【Paddle.Fleet】Support local save sparse param (#30175) ... * add save tensor support Co-authored-by: seiriosPlus <tangwei12@baidu.com>	5 years ago
lilong12	b0bd93de00	Disable gloo by default (#29805) ... * update, test=develop	5 years ago
tangwei12	032414ca2a	[Feature] one ps (3/4) (#29604) ... * oneps (3/4) Co-authored-by: MrChengmo <cmchengmo@163.com> Co-authored-by: malin10 <malin10@baidu.com> Co-authored-by: chengmo <chengmo@baidu.com>	5 years ago
ShenLiang	01e2874a0e	Support multi-stream communication for dynamic graph distributed (#29525) ... * fix fleet for multi-stream * fix memcpy for ncclid * use sync to solve move operation	5 years ago
ShenLiang	2ef9e0e23c	Rebuild group automatically in dynamic graph distributed (#29255) ... * add tensor_indices in AssignGroupBySize * add rebuild group in reducer	5 years ago
lilong12	b122d0bb76	Fix bug in gloo that gloo initialization hangs (#29447) ... * update, test=develop	5 years ago
ShenLiang	4064354a01	support dp run single card (#29358)	5 years ago
ShenLiang	2d6aa1a5bb	fix warning of fleet (#29317)	5 years ago
ShenLiang	2cd0bf5764	Fix doc of fleet api (#29282) ... * fix doc, test=document_fix	5 years ago
ShenLiang	46b73e6cd9	Change the api of DataParallel and Fleet (#29224)	5 years ago
123malin	cc9c619679	test=develop, fix doc (#29200) ... * fix fleet api doc	5 years ago
ShenLiang	e2d01eb650	Support dynamic graph distributed (#28997) ... * add reducer * refine envent for memorycopy * add concat&split for allreduce * apply concat & split for fuse tensor * fix nccl dep * fix the untest, compile problem and ddp initialize problem * fix untest for mac & add some comments & solve the repeated param in sublayers * fix untest for windows & fix document	5 years ago
Chen Long	d576d6ddeb	fix some docs test=develop;test=document_fix (#29159)	5 years ago
JZ-LIANG	0dadacc4eb	[sharding] doc, api, bug fixed (#28983) ... * add lars to fleet meta optimizer * add lamb to proto * add lamb to fleet meta optimizer * fixed syntax bug * fixed syntax bug * fixed syntax error in lamb, add config setter of lamb in distributed_strategy * trigger unitest to rerun * add new unitest func for lamb * revise unitest for lars and lamb * revise dgc meta unitest * revise lars document in distribute_strategy * revise lars lamb document in distributed_strategy.py * revise lars lamb document in distributed_strategy.py * add weight decay exclude logic to lars * restore optimzier.py * restore optimizer.py as develop except lars * add epsilon and exclude fn to distributed_sttrategy * add lars epsilon * revise unitest for fleet lars and lamb * revise lars lamb unitest for CI coverage * revise lars argument api * revise lars argument api * revise lars argument api * revise api doc of lars * fix op role * add sharding save and add_sync_comm_for_test function * add comm_analyse to utlis * revise sharding_utils * add sharding saving unittest * revise sharding utils for unittest * revise sharding en doc * update sharding utils api * add doc for sharding * fixed bug in sharding var size count * update varsize count in sharding * fix sharding num_nccl_comm * Revert "fix sharding num_nccl_comm" This reverts commit d51587c15e9323acf226ddd36154275f0d1daf76.	5 years ago
lilong12	2a864c70c4	fix the bug in gloo (#29112) ... * update, test=develop	5 years ago
Leo Chen	3815d7aa40	Upgrade string literals to raw string (#28989) ... * upgrade comment string to raw string * fix string in * fix string with ' ' * revert update on comments * upgrade only necessary * fix sample code checker * fix comments with '''	5 years ago
mapingshuo	81244fbfab	add sharding strategy in fleet(#27900) ... * add sharding	6 years ago
WangXi	11acbfae06	refine auto strategy, test=document_fix (#28211)	6 years ago
MRXLT	55098b975e	fleet support paddle.optimzier (#28026) ... fleet support paddle.optimzier * bug fix * fix fleet_base * bug fix * fix coverage	6 years ago
WangXi	fb641c915e	【paddle.fleet】fleet add _get_applied_meta_list and _get_applied_graph_list (#27952)	6 years ago
tangwei12	202bfab1be	Feature/large scale kv save base/delta (#27470) ... * add size method for large scale * add large scale UT * add ut for checkpoint	6 years ago
123malin	aa3b4ed717	【paddle.fleet】geo send sparse optimize (#27719) ... * test=develop, fix geo sgd communicator * test=develop, gloo_init_method * test=develop, bug fix for gloo http_init	6 years ago
danleifeng	8d7908f3fd	【paddle.fleet】raise error when using multi-cards in fleet non_distributed mode (#27854) ... * raise error if use multi-cards in fleet non_distributed mode; test=develop	6 years ago
123malin	a4f850748a	【paddle.fleet】bug fix for parameter_recv (#27838) ... * test=develop, bug fix for parameter_recv * test=develop, for unittest, test_fleet_rolemaker_new	6 years ago
Chen Weihang	ed31dac6eb	remove scale loss and coll grads, test=document_fix (#27874)	6 years ago
WangXi	50619cd842	use floyd algorithm to find meta optimizer max path, test=develop (#27867)	6 years ago
Chengmo	c5f2802d56	【paddle.fleet】Update fleetrun & ps-heter (#27472) ... * refine fleetrun.ps_launch * update fleet run for multi device support * ps_graph support ps-gpu * fix heter save * add heter save unittest * fix unittest & simple code * update fleetrun * fix fleetrun * fix launch barrier * fix role maker * add paddlecloud rolemaker unittest * rename heter_worker_device_guard	6 years ago
WangXi	0a1862d1d2	fleet combine amp dgc recompute meta optimizer (#27643)	6 years ago
danleifeng	a01bc6b31d	【paddle.fleet】fleet support non_distributed training in dygraph mode (#27714) ... * fleet support non_distributed training in dygraph mode; test=develop	6 years ago
lilong12	742cbe6660	[bug fix] avoiding multiple initialization of gloo for fleet in dygraph mode (#27706) ... * add double grad for expand, test=develop	6 years ago
lilong12	5132f5129d	terminate http server used by gloo for fleet after init (#27698)	6 years ago
lilong12	bbc2add703	Initialize gloo for low level collective apis (#27672) ... * add gloo initializer, test=develop	6 years ago
lilong12	36c0410223	Revert "Initialize gloo for low level collective apis (#27356)", test=document_fix (#27665)	6 years ago
123malin	6822307745	test=develop, rm netifaces (#27581) ... * test=develop, rm netifaces	6 years ago
lilong12	fa73e4a284	Initialize gloo for low level collective apis (#27356) ... * add gloo initializer, test=develop	6 years ago
Dong Daxiang	4e8f18ab25	Get final strategy (#27602) ... * add get final strategy for user to print final strategy	6 years ago
Chengmo	0e101c4f6f	Fix test dist fleet heter ctr (#27513) ... * fix test_dist_fleet_heter_ctr & peformance update	6 years ago
WangXi	e550fc02ae	fleet2.0 add fp16 grad compression (#27480)	6 years ago
123malin	32ad4f90a4	【paddle.fleet】 Usages Change: from fleet.util() to fleet.util (#27468) ... * test=develop, bug fix	6 years ago
danleifeng	0721767ba9	fix server_num bug;test=develop (#27442)	6 years ago
tangwei12	d6b54de467	【paddle.fleet】Fix/role maker api fix (#27326) ... * fix fleet util and gloo * fix worker endpoints * fix * fix UT * fix gloo * fix gloo * update gloo * update gloo * update gloo * update gloo * update gloo * fix gloo wrapper for hdfs * add file gloo and UT * fix UT * fix UT * fix UT * hide public method of RoleMaker * fix UT * GPU fleetrun support gloo * parameterserver fleetrun support gloo * add UT * add UT * fix UT * fix get server endpoint * fix get server endpoint * fix UT * hide public method of rolemaker * hide public method of rolemaker * hide public method of rolemaker * Update test_fleet_rolemaker_new.py * hide public method of rolemaker * hide public method of rolemaker	6 years ago
tangwei12	99626502f7	【paddle.fleet】gloo and util (#27213) ... * fix worker endpoints * fix gloo wrapper for hdfs * GPU fleetrun support gloo * parameterserver fleetrun support gloo * fix get server endpoint	6 years ago
123malin	f36b9a7f79	【Fleet2.0 Util】 add documents (#26698) ... * test=develop, util documents	6 years ago