67 Commits (b639a882c3d20131131456f9ccf7292c39bf4fec)

Author	SHA1	Message	Date
123malin	20cdff0e02	Optimize decay (#20816) ... * update pserver decay blocks * update distributed notify handler	5 years ago
hong	8c4573a3cb	GradMaker for dygraph (#19706) ... * refactor dygraph,test=develop * fix failed unittest,test=develop * polish code,test=develop * check windows ci error,test=develop try to fix windows ci error by np.allclose,test=develop * polish vlog and profiler, test=develop * try to fix preceding ops order,test=develop * test transformer in windows ci, test=develop * use python c-api to speed up tracer.trace,test=develop * test=develop, fix docker with paddle nccl problem * test=develop, add ut for debug string and gradient_accumulator * test=develop, add tests for layer/gradient_accumulator/prepared_op * test=develop, fix complie error for test_prepared_op * test=develop, add more ut for dygraph * test=develop, create API.spec for dygraph api change * optimize grad maker; test=develop * optimize grad maker * test * grad make optim; test=develop * fix unittest bugs; test=develop * add dygraph grad op maker and split_op * grad op maker refactor; test=develop * add dygraph grad maker; test=develop * fix op deformable_conv_v1_op bug; test=develop * fix deformable_conv prroi pool bugs; * fix new op grad op maker bug; test=develop * fix split by ref bug; test=develop * fix dygraph auto prune bug; test=develop * fix test_trace bug; test=develop * fix fused emb seq pool bug; test=develop * remove useless code in op_desc file; test=develop * remove useless code, StrVarBaseNode; test=develop * fix review issues; test=develop * fix rank_loss grad maker; test=develop * remove flag in VarBase; test=develop * fix distributed_notify_op compile bug ; test=develop * fix reshape op double grad; test=develop * fix expand as op; test=develop * add impertive type_defs.h for demo_train; test=develop * fix inference lib cmake; test=develop * fix inference lib; test=develop * fix infernce_lib; test=develop * fix inference cmake; test=develop * fix inference lib; test=develop * fix inference lib; test=develop * remove condition dygraph grad maker, modify local name; test=develop * fix split grad maker bug; test=develop * fix pyramid_op bug; test=develop * change travis time out limit; test=develop * restore travis; test=develop * change timeout limit; test=develop	5 years ago
Chen Weihang	26cc1fe508	Replace risky GetInputType method with secure IndicateVarDataType interface (#20668) ... * replace part of the old implementation, test=develop * restore concat op, test=develop * update all ops implemention & delete GetDataTypeOfVar func, test=develop	5 years ago
Chengmo	940c6ff1c8	Fix communicator slow bug & fix communicator stop bug (#20366) ... * test=develop,Fix communicator slow bug * test=develop, delete if() in stop_worker() * test=develop * fix UT, test=develop * fix bug in fetch handler, test=develop * fix bug in fetch handler, test=develop * test=develop, fix fetch barrier bug * test=develop, bug fix * test=develop, bug fix * test=develop, fix bug	5 years ago
123malin	b4a3b75002	bug fix: invalid learning rate decay in pserver async mode (#20325) ... * bug fix: invalid learning rate decay in pserver async mode	5 years ago
tangwei12	b5a410466c	Trainer heartbeat for async mode (#19600) ... Heartbeat for distributed async training.	5 years ago
Chengmo	728ec1b43d	Add GEO-SGD distribute training algorithm (#20018) ... * refector geo sgd & communicator	5 years ago
tangwei12	8f0b3c0516	the integrated communicator (#19849) ... * add a base class for the Communicator * add AsyncCommunicator Impl for async distributed training	5 years ago
gongweibao	57f0f0f2dc	Delete pserver complete file before executor running. (#19468)	6 years ago
tangwei12	65c7368400	Fix the correctness of async mode at distributed training (#18863) ... * fix correctness of the communicator * fix a bug in send thread when sending var context is empty, test=develop * add lookup_table_prefetch_op and prefetch optimize, test=develop * remove remote prefetch GPU supported * word2vec force with CPU, test=develop * test dist remote lookup table force with CPU, test=develop	6 years ago
tangwei12	19dac67e9f	fix distribute transpiler GRPC error code 4, RPC Deadline (#18984) ... * fix sync mode hang in transpiler * remove sync mode in send/recv * replace PADDLE_ENFORCE with PADDLE_ENFORCE_NE	6 years ago
zhang wenhui	539c870753	add fl_listen_and_serv &fl_transpiler,test=develop (#19091) ... add fl_listen_and_serv op for Federated_learning and fl_distribute_transpiler add this op to pserver program . This op just listen the endpoint and sum&scale.	6 years ago
gongweibao	fd4b15a2f6	Unset unittests http_proxy env to avoid timeout. (#19269) ... Unset unittests http_proxy env to avoid timeout.	6 years ago
Leo Chen	80eab822c1	Remove unused DefaultGradOpDescMaker in REGISTER_OPERATOR() (#19166) ... * remove unused DefaultGradOpDescMaker in REGISTER_OPERATOR(), test=develop * remove SplitIdsOpGradMaker since it is buggy and not tested, update spec file, test=develop	6 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175) ... * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	6 years ago
gongweibao	0d561ef442	fix 2dconn test=develop (#17681)	6 years ago
gongweibao	65bbf950ee	Add multi-ncclcomm and 2D ncclallreduce support. (#17263)	6 years ago
chengduo	b5f4d5ed0e	Add broadcast operators (#17503) ... * This PR adds broadcast for multi-process. And it could be used in dynamic graph to broadcast parameters.	6 years ago
Qiao Longfei	58f7695ab2	Async exe support communicator (#17386) ... Async exe support communicator	6 years ago
Yan Xu	0217555530	polish parallel dygraph code (#17164) ... * add var grad hook test=develop	6 years ago
Yan Xu	0b07eef118	ParallelDyGraph with GPU collective mode (#16827) ... implement dygraph.parallel.DataParallel to hook reduce op.	6 years ago
Qiao Longfei	0e663d7f51	fix split_byref_op infer shape ... test=develop	6 years ago
Qiao Longfei	8b8a0487c7	fix compile test=develop	6 years ago
Qiao Longfei	a541c25ab6	fix cpplint test=develop	6 years ago
Qiao Longfei	0608f8ca56	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async_sparse_param_update_recorder	6 years ago
Qiao Longfei	fb6cc3a1bd	follow commnet, optimize code and add comment test=develop	6 years ago
Qiao Longfei	b542639dc0	code clean test=develop	6 years ago
Qiao Longfei	30618409db	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async-ssa-graph-executor-communicator	6 years ago
Qiao Longfei	be0c482304	update trainer_id	6 years ago
Qiao Longfei	de65398cb8	update transpiler and listen and serv op	6 years ago
Wu Yi	6382b62f6b	Collective ops (#15572) ... * wip allreduce in op * wip * wip * wip * wip adding test * wip for conflict with mp mode * fix tests test=develop * fix cpu build test=develop * fix travis clang format test=develop * fix cpu build test=develop * update api.spec test=develop * delete comment test=develop * fix cpplint test=develop * fix test=develop * follow comment test=develop * add file test=develop * fix build test=develop * update test=develop * to be compatible with sync_bn, and fix mp mode in develop test=develop	6 years ago
minqiyang	b40e41fbd1	Polish code style ... test=develop	6 years ago
Qiao Longfei	ea0df4e8a2	add some check	6 years ago
minqiyang	db0c970823	Polish code ... test=develop	6 years ago
minqiyang	438bca9c3d	Implement Runtime Var Type Inference ... test=develop	6 years ago
minqiyang	ca392c7e97	Implement infer var type context	6 years ago
Qiao Longfei	065b68b6ca	clean code	6 years ago
Qiao Longfei	63cd70a8b8	fix blocking problem	6 years ago
Qiao Longfei	c0e5941e31	add commnet for recv do_not_run	6 years ago
Qiao Longfei	ff8054c5a7	can run	6 years ago
Qiao Longfei	255b36dad2	can run	6 years ago
Qiao Longfei	c2cce6bafa	simplify parameter send and recv	6 years ago
Qiao Longfei	3c6b733d14	remove exe context	6 years ago
Qiao Longfei	9573d610ef	use rpc common in parameter send and recv	6 years ago
Qiao Longfei	02425b2f64	fix compile	6 years ago
Qiao Longfei	fbd186bd5d	complete recv op	6 years ago
Qiao Longfei	a0585d08ed	init parameter recv	6 years ago
Qiao Longfei	4356f186b4	complete parameter_send	6 years ago
Qiao Longfei	657a4f9430	code can compile	6 years ago
Qiao Longfei	c7e3868007	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-communicator	6 years ago