175 Commits (develop)

Author	SHA1	Message	Date
Chen Long	d576d6ddeb	fix some docs test=develop;test=document_fix (#29159)	5 years ago
lilong12	216e085605	update, test=develop (#29139)	5 years ago
lilong12	a1add716bc	Add a flag to control whether to initialize gloo (#29150)	5 years ago
ShenLiang	cddc70964d	fix InMemoryDataset doc (#28688) ... * add Inmemorydataset	5 years ago
JZ-LIANG	0dadacc4eb	[sharding] doc, api, bug fixed (#28983) ... * add lars to fleet meta optimizer * add lamb to proto * add lamb to fleet meta optimizer * fixed syntax bug * fixed syntax bug * fixed syntax error in lamb, add config setter of lamb in distributed_strategy * trigger unitest to rerun * add new unitest func for lamb * revise unitest for lars and lamb * revise dgc meta unitest * revise lars document in distribute_strategy * revise lars lamb document in distributed_strategy.py * revise lars lamb document in distributed_strategy.py * add weight decay exclude logic to lars * restore optimzier.py * restore optimizer.py as develop except lars * add epsilon and exclude fn to distributed_sttrategy * add lars epsilon * revise unitest for fleet lars and lamb * revise lars lamb unitest for CI coverage * revise lars argument api * revise lars argument api * revise lars argument api * revise api doc of lars * fix op role * add sharding save and add_sync_comm_for_test function * add comm_analyse to utlis * revise sharding_utils * add sharding saving unittest * revise sharding utils for unittest * revise sharding en doc * update sharding utils api * add doc for sharding * fixed bug in sharding var size count * update varsize count in sharding * fix sharding num_nccl_comm * Revert "fix sharding num_nccl_comm" This reverts commit d51587c15e9323acf226ddd36154275f0d1daf76.	5 years ago
lilong12	2a864c70c4	fix the bug in gloo (#29112) ... * update, test=develop	5 years ago
WangXi	e931c7baf9	Fix multi nccl comm & wait server ready (#28663)	5 years ago
gongweibao	1358397e97	Clean up the redundant files and unify the launch interface. (#28928)	5 years ago
Chen Weihang	bb16c2515d	Polish parallel api impl & doc details (#28980) ... * polish parallel api impl & doc details * add unittest for coverage * remove spawn test in py2.7 * add parallel api into white list	5 years ago
Leo Chen	3815d7aa40	Upgrade string literals to raw string (#28989) ... * upgrade comment string to raw string * fix string in * fix string with ' ' * revert update on comments * upgrade only necessary * fix sample code checker * fix comments with '''	5 years ago
123malin	fbf9564f6b	【paddle.distributed.fleet】Optimize ParameterServer's Async Mode (#28442) ... * test=develop, optimize global_step	5 years ago
lilong12	f77a78cdee	enable pipeline to run with Executor.run() (#28373) ... * update, test=develop	5 years ago
Chen Weihang	bff4179cc7	lazily init global group in collective (#28780)	5 years ago
JZ-LIANG	5a9f6889c1	[Sharding] add new features (#28568) ... * add lars to fleet meta optimizer * add lamb to proto * add lamb to fleet meta optimizer * fixed syntax bug * fixed syntax bug * fixed syntax error in lamb, add config setter of lamb in distributed_strategy * trigger unitest to rerun * add new unitest func for lamb * revise unitest for lars and lamb * revise dgc meta unitest * revise lars document in distribute_strategy * revise lars lamb document in distributed_strategy.py * revise lars lamb document in distributed_strategy.py * add weight decay exclude logic to lars * restore optimzier.py * restore optimizer.py as develop except lars * add epsilon and exclude fn to distributed_sttrategy * add lars epsilon * revise unitest for fleet lars and lamb * revise lars lamb unitest for CI coverage * revise lars argument api * revise lars argument api * revise lars argument api * revise api doc of lars * fix op role * add sharding save and add_sync_comm_for_test function * add comm_analyse to utlis * revise sharding_utils * add sharding saving unittest * revise sharding utils for unittest	5 years ago
lilong12	e4f9415338	update doc, test=document_fix (#28498)	5 years ago
danleifeng	a24d186814	fix nccl init failed in parallel dygraph mode (#28497)	5 years ago
Chengmo	4dc8c44ba1	【Paddle.Fleet】Fix fleetrun heter (#28252) ... * fix fleetrun heter ps on paddlecloud	5 years ago
mapingshuo	81244fbfab	add sharding strategy in fleet(#27900) ... * add sharding	6 years ago
WangXi	11acbfae06	refine auto strategy, test=document_fix (#28211)	6 years ago
MRXLT	55098b975e	fleet support paddle.optimzier (#28026) ... fleet support paddle.optimzier * bug fix * fix fleet_base * bug fix * fix coverage	6 years ago
lilong12	5bb348a1c2	add doc for ReduceOp (#28051) ... * add doc, test=document_fix	6 years ago
WangXi	fb641c915e	【paddle.fleet】fleet add _get_applied_meta_list and _get_applied_graph_list (#27952)	6 years ago
lilong12	ff0ebefc1e	put gloo initialization log to file (#27969) ... * update, test=develop	6 years ago
tangwei12	202bfab1be	Feature/large scale kv save base/delta (#27470) ... * add size method for large scale * add large scale UT * add ut for checkpoint	6 years ago
123malin	aa3b4ed717	【paddle.fleet】geo send sparse optimize (#27719) ... * test=develop, fix geo sgd communicator * test=develop, gloo_init_method * test=develop, bug fix for gloo http_init	6 years ago
danleifeng	8d7908f3fd	【paddle.fleet】raise error when using multi-cards in fleet non_distributed mode (#27854) ... * raise error if use multi-cards in fleet non_distributed mode; test=develop	6 years ago
chentianyu03	d05058d268	Remove and reorganize the alias of APIs (#27717) ... * modify cond while_loop to paddle.static.nn.cond * modify crop_tensor to paddle.crop * modify Variable to paddle.static.Variable * remove nn.beam_search, nn.beam_search_decode, nn.gather_tree * remove bpr_loss, center_loss, rank_loss, smooth_l1, teacher_student_sigmoid_loss, edit_distance, sampled_softmax_with_cross_entropy in nn.functional * remove apis in nn.functional.learn_rate.py * remove pool2d, pool3d, adaptive_pool2d, adaptive_pool3d in nn.functional * remove apis in nn.functional.vision * remove erf, soft_relu in nn.functional.activation * remove apis in nn.functional.extension * remove nn.functional.rnn * remove hash from nn.functional.lod * remove row_conv from nn.functional.extension * remove one_hot, pad2d, pad_constant_like from nn.functional.common * remove nn.gather_tree, nn.BilinearTensorProduct, nn.Pool2D, nn.Pad2D * remove apis from optimizer.__init * remove tensor.creation.fill_constant * remove elementwise_mul in nn.functional.common and modify to paddle.multiply * remove tensor.stat.reduce_mean * remove reduce_all, reduce_any in tensor.logic * remove apis in tensor.math * remove apis in tensor.__init__ * remove has_inf, has_nan in tensor.search * remove apis in framework.__init__ * remove apis in paddle.__init__ * remove apis in nn.functional.__init__ * modify removed alias apis to raw api in doc and unittests * fix remove grid_sample bug * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * modify removed alias apis to raw api in doc and unittests * delete alias api relastions in doc * reserve paddle.compat, paddle.sysconfig * remove unittest for paddle.reduce_all, paddle.reduce_any * modify removed alias apis to raw api in doc and unittests * recover paddle.save and paddle.load * resolve conflicts * fix sample code missing paddle.enable_static() bug * fix sample code missing paddle.enable_static() bug * fix to_string sample code error	6 years ago
Chengmo	328cb289ed	【paddle.fleet】fix sparse load (#27680) ... * add sparse tensor load method	6 years ago
123malin	a4f850748a	【paddle.fleet】bug fix for parameter_recv (#27838) ... * test=develop, bug fix for parameter_recv * test=develop, for unittest, test_fleet_rolemaker_new	6 years ago
Chen Weihang	ed31dac6eb	remove scale loss and coll grads, test=document_fix (#27874)	6 years ago
WangXi	50619cd842	use floyd algorithm to find meta optimizer max path, test=develop (#27867)	6 years ago
mapingshuo	8d2cb14f98	support gradient merge with recompute, test=develop (#27834) ... * support gradient merge with recompute, test=develop test=develop	6 years ago
Chengmo	c5f2802d56	【paddle.fleet】Update fleetrun & ps-heter (#27472) ... * refine fleetrun.ps_launch * update fleet run for multi device support * ps_graph support ps-gpu * fix heter save * add heter save unittest * fix unittest & simple code * update fleetrun * fix fleetrun * fix launch barrier * fix role maker * add paddlecloud rolemaker unittest * rename heter_worker_device_guard	6 years ago
WangXi	0a1862d1d2	fleet combine amp dgc recompute meta optimizer (#27643)	6 years ago
danleifeng	a01bc6b31d	【paddle.fleet】fleet support non_distributed training in dygraph mode (#27714) ... * fleet support non_distributed training in dygraph mode; test=develop	6 years ago
lilong12	742cbe6660	[bug fix] avoiding multiple initialization of gloo for fleet in dygraph mode (#27706) ... * add double grad for expand, test=develop	6 years ago
lilong12	5132f5129d	terminate http server used by gloo for fleet after init (#27698)	6 years ago
lilong12	bbc2add703	Initialize gloo for low level collective apis (#27672) ... * add gloo initializer, test=develop	6 years ago
Qinghe JING	1539a23822	Fix bugs in hdfs download (#27344) ... * set default value to strategy in distributed_optimizer test=develop	6 years ago
yaoxuefeng	780140599f	【paddle.distributed.fleet】add data_generator in distributed.fleet.dataset (#27345)	6 years ago
lilong12	36c0410223	Revert "Initialize gloo for low level collective apis (#27356)", test=document_fix (#27665)	6 years ago
123malin	6822307745	test=develop, rm netifaces (#27581) ... * test=develop, rm netifaces	6 years ago
lilong12	fa73e4a284	Initialize gloo for low level collective apis (#27356) ... * add gloo initializer, test=develop	6 years ago
Dong Daxiang	4e8f18ab25	Get final strategy (#27602) ... * add get final strategy for user to print final strategy	6 years ago
Chengmo	0e101c4f6f	Fix test dist fleet heter ctr (#27513) ... * fix test_dist_fleet_heter_ctr & peformance update	6 years ago
WangXi	e550fc02ae	fleet2.0 add fp16 grad compression (#27480)	6 years ago
123malin	32ad4f90a4	【paddle.fleet】 Usages Change: from fleet.util() to fleet.util (#27468) ... * test=develop, bug fix	6 years ago
tangwei12	bc5f0246a8	large scale kv speedup (#26510) ... * rename communicator meet->BatchesCounter * fix parame recv for sparse * geo sparse init from pserver * optimize init from pserver * add large scale optimizer fuse(SGD/ADAM) * rectification init_worker and exe.run startup program	6 years ago
danleifeng	0721767ba9	fix server_num bug;test=develop (#27442)	6 years ago
danleifeng	905e2346ac	add endpoints log;test=develop (#27439)	6 years ago
danleifeng	fc61efd736	fix port env bug(int);test=develop (#27405)	6 years ago
tangwei12	d6b54de467	【paddle.fleet】Fix/role maker api fix (#27326) ... * fix fleet util and gloo * fix worker endpoints * fix * fix UT * fix gloo * fix gloo * update gloo * update gloo * update gloo * update gloo * update gloo * fix gloo wrapper for hdfs * add file gloo and UT * fix UT * fix UT * fix UT * hide public method of RoleMaker * fix UT * GPU fleetrun support gloo * parameterserver fleetrun support gloo * add UT * add UT * fix UT * fix get server endpoint * fix get server endpoint * fix UT * hide public method of rolemaker * hide public method of rolemaker * hide public method of rolemaker * Update test_fleet_rolemaker_new.py * hide public method of rolemaker * hide public method of rolemaker	6 years ago
tangwei12	99626502f7	【paddle.fleet】gloo and util (#27213) ... * fix worker endpoints * fix gloo wrapper for hdfs * GPU fleetrun support gloo * parameterserver fleetrun support gloo * fix get server endpoint	6 years ago
123malin	f36b9a7f79	【Fleet2.0 Util】 add documents (#26698) ... * test=develop, util documents	6 years ago
danleifeng	8d05c00c67	fix paddle.fleet en-doc for apis in dynamic mode (#27354) ... * fix fleet dynamic-mode en-doc;test=develop	6 years ago
ShenLiang	746a8ded29	fix comment of adaptive lsgd (#27362)	6 years ago
gongweibao	11bcf0e21c	Cleanup redundant code files (#27319)	6 years ago
ShenLiang	54b81fa32c	add adaptivelsgd in meta_optimizer (#27289) ... * add adaptivelsgd * Todo fix the code to avoid the conflict.	6 years ago
yaoxuefeng	c67c391682	refine fleet dataset class api (#27133)	6 years ago
danleifeng	389a9a7e0e	fix ports conflict when use paddlecloud to launch analogue multi-nodes (#26191) ... * fix ports conflict when launching multi-nodes in paddlecloud;test=develop * add DISTRIBUTED_TRAINER_ENDPOINTS env for cloud;test=develop	6 years ago
mapingshuo	9dedafa0df	fix strategy, test=develop (#27323) ... * fix strategy, test=develop * fix can_apply	6 years ago
ShenLiang	2b6a5793fe	remove auto mode from localsgd optimizer (#27237) ... * rm auto from localsgd	6 years ago
123malin	60c3ef3ab8	【paddle.fleet】parameter_server_optimizer support auto_strategy (#27181) ... * parameter_server_optimizer support auto_strategy	6 years ago
JZ-LIANG	5d039f4086	modified the implement of Lars optimizer (#26733) ... add lars to fleet meta optimizer	6 years ago
Dong Daxiang	f7d08b7db8	【paddle.fleet】refine launch and distributed repr string for print (#27093) ... * refine launch and distributed repr string for print	6 years ago
123malin	f2d68d3ed5	【paddle.fleet】parameter_server_optimizer support auto_strategy (#26838) ... * test=develop, add ps auto	6 years ago
ShenLiang	aca450f6fb	fix the localsgd optimizer (#27094) ... * fix the localsgd	6 years ago
Dong Daxiang	0443b480b8	【paddle.fleet】add auto parallel L1 implementations (#27090) ... * add auto parallel L1 implementation test=develop	6 years ago
Chengmo	a72752263b	support heter-xpu-ps (#27018) ... support heter-xpu-ps	6 years ago
mapingshuo	9e4fe92303	fix strategy example (#26856) ... * fix doc * test=develop * test=develop * test=develop * test=develop * test=develop * test=develop * test=develop * test=develop * fix doc, test=develop * update localsgd doc test=develop * update localsgd doc test=develop * fix fleet dgc amp doc, test=develop * fix, test=develop * fix async configs Co-authored-by: liuyi05 <gavin1332@gmail.com> Co-authored-by: WangXi <wangxi16@baidu.com> Co-authored-by: seiriosPlus <tangwei12@baidu.com>	6 years ago
danleifeng	6b4ca0d7f1	【paddle.fleet】distributed_optimizer supports dygraph (#26541) ... paddle.distributed.fleet supports dynamic graph execution.	6 years ago
danleifeng	3a2a711681	【paddle.fleet】simplify fleetrun log infos (#26888) ... * print detailed and clear log infos; test=develop	6 years ago
danleifeng	e35ad3eee8	【paddle.fleet】support running python train.py for fleet tasks (#26249) ... * support running python train.py for fleet-task; test=develop	6 years ago
lilong12	030b298e82	fix sample codes in collective.py (#26787) ... * fix sample codes, test=develop	6 years ago
Chengmo	d0962abd20	supplement bug fix of parameter server (#26217) ... * fix fluid.embedding	6 years ago
Chen Weihang	28cb653145	Remove backend argument of init_parallel_env (#26773) ... * remove backend argument of init_parallel_env * remove keep name table in transformer * add cpu version check * add skip unittest for init_parallel_env * polish doc: remove func use & update example	6 years ago
tangwei12	9ded7565ec	【paddle.fleet】FleetAPI 2.0 (#26772) ... * add FleetAPI doc Co-authored-by: liuyuhui <liuyuhui@baidu.com>	6 years ago
Chengmo	7f2aa2db3c	【paddle.fleet】Support Heter Parameter Server (#25998) ... * Support Heter Parameter Server	6 years ago
Dong Daxiang	994217ea05	【paddle.fleet】fix api documents (#26777) ... * fix api document	6 years ago
Chen Weihang	31f422ae5e	Add interface to launch parallel dygraph by multiprocessing (#26044) ... * add dygraph parallel run interface * polish implement & unified env property name * add print config arg * refactor init_parallel_env function * Compatible with multiprocessing and launch modes * set default trainer start port * support run in python 2 * polish python2 support code * remove python2 support * refine launch import * polish dome design details * refactor api implemention & path * use new method _set_expected_place * add spawn unittest framework & mnist test * add more unittests & doc * fix unittest failed * polish english doc * self review and polish details * refactor code by reviewer's comments * fix unittest failed * fix parallel_env unittest * fix several typos * fix error introduced when fixing typos * add unpublic note for start_processes * polish details by xiaoguang's comment * verify correctly when spawn nprocs=-1 * refactor spawn & init_parallel_env design * polish doc details * open spawn unittests * try to fix doc compile error * try to fix unknown doc format error * add skip unittest when not gpu	6 years ago
Yi Liu	2024ef69a2	【paddle.fleet】add comments about localsgd in DistributedStrategy (#26709) ... * modified timeout value on windows and mac (#26690) * add Local SGD algorithm referenced paper test=develop	6 years ago
lilong12	1c68138327	[api 2.0] add collective op for cpu using gloo and paddle.distributed.* apis (#26552) ... add collective op for cpu using gloo and paddle.distributed.* apis	6 years ago
JZ-LIANG	958d7212c7	【paddle.fleet】Document refine lars & lamb (#26533)	6 years ago
Dong Daxiang	08d736ad78	【paddle.fleet】add cudnn related strategies to DistributedStrategy (#26598) ... * add cudnn related strategies to DistributedStrategy	6 years ago
WangXi	7ff197d3ba	Add fleet dgc amp doc, test=document_fix (#26608)	6 years ago
liuyuhui	66596bd20f	【paddle.fleet】solve the initial configuration about fleet and rolemaker (#26368) ... * solve the initial configuration about fleet and rolemaker Co-authored-by: seiriosPlus <tangwei12@baidu.com>	6 years ago
Dong Daxiang	7d3e46e1d5	【paddle.fleet】Document refine (#26526) ... * add documentation for DistributedStrategy	6 years ago
Dong Daxiang	83cd185947	【paddle.fleet】Meta from optimizer (#26392) ... * consider the combination of different strategies to work together	6 years ago
123malin	57d434df5d	add save/load for parameter server (#26235) ... * add save/load for parameter server	6 years ago
mapingshuo	cd48bdad31	add feature to fleet2.0 role_maker, distribute_strategy, test=develop (#26267) ... * add feature to fleet2.0 role_maker, distribute_strategy, test=develop	6 years ago
Dong Daxiang	4ec51e0205	【paddle.fleet】Clear disable (#26334) ... * add check approval test=develop	6 years ago
Yi Liu	3b2c580a66	【paddle.fleet】make fleet_localsgd_meta_optimizer work (#26213) ... * make fleet_localsgd_meta_optimizer work * fix bug in localsgd meta optimizer	6 years ago
Qinghe JING	d549a9b1fe	【paddle.fleet】Set default value to strategy in distributed_optimizer (#26246) ... * set default value to strategy in distributed_optimizer test=develop	6 years ago
liuyuhui	935da32d25	【paddle.fleet】upgrade fleet: modify role_maker (#26038) ... * add unittest for paddlerolemaker with gloo	6 years ago
Dong Daxiang	50a5bcfc9d	【paddle.fleet】paddle.fleet -> paddle.distributed.fleet. (#26186) ... * move paddle.fleet to paddle.distributed.fleet	6 years ago
Yi Liu	f45f8363eb	records the offset of log when creating by paddle.distributed.launch (#25725) ... test=develop	6 years ago
gongweibao	80f1c50738	Fix typo in interface. (#24779)	6 years ago
Yi Liu	b0f18947f3	fix the compatibility of PY2 and PY3 in paddle.distributed.launch (#25304) ... * fix the compatibility of PY2 and PY3 in paddle.distributed.launch test=develop * only pull log of local rank 0 test=develop * log exception if UnicodeEncodeError occurs when pulling log in paddle.distributed.launch test=develop Co-authored-by: SunGaofeng <peakbee@gmail.com>	6 years ago
Yi Liu	5209b9a510	cat log to stdout when setting log_dir in launch (#25147) ... test=develop	6 years ago
mapingshuo	9388a6381c	fix popen error (#24767) ... replace join to terminate	6 years ago