Paddle

Commit Graph

Author	SHA1	Message	Date
Yi Liu	a873fa84ce	supports collective training with programs (#18392 ) 1. Since allreduce op has 4 reduce types, We split these four reduce types into four ops 2. We also refined the collective op code, e.g. we separated the collective op kernel into CPUKernel and CUDAKernel, and remove the device specified DeviceContext parameter in template as we already knew the target DeviceContext 3. We remove the newly added Collective op role to reduce the complexity of program and graph analysis	6 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175 ) * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	6 years ago
Sylwester Fraczek	543e53db05	fix typo releated->related	6 years ago
peizhilin	a6f5ceee74	add the python callstack for debug support test=develop	6 years ago
peizhilin	2388d0e7d6	Revert "cherry-pick the #12759" test=develop This reverts commit `7f6d8acecb`.	6 years ago
peizhilin	7f6d8acecb	cherry-pick the #12759 test=develop	6 years ago
chengduozh	acec4cb8ca	[1.1]fix op_role value test=release/1.1	6 years ago
Wu Yi	9da9b1926b	[1.1] fix graph num hang (#14072 ) * fix graph num hang test=develop * re-enable tests test=develop * re-enable graph num check test=develop * fix multi device pass role check test=develop	6 years ago
Xin Pan	d5d09672c8	better fix test=develop	6 years ago
Xin Pan	8f2116d8fa	clean up after the changes have been stopped for so long. test=develop	6 years ago
Yu Yang	186b2b138d	Revert "Merge pull request #13201 from reyoung/revert_callstack" (#13697 ) This reverts commit `21bb9e91fc`, reversing changes made to `3fa68dc101`. test=develop	6 years ago
Yu Yang	7119d6c3cf	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into revert_callstack	6 years ago
Wu Yi	29c63d180f	[Feature] dist op role and lr op role, to support memory optimize with dist training (#13220 ) * wip * clean up * should fix running with memopt * add ut * mark lr schedule op role * hide lr_schedule_guard * use op_role_var instead of ufind * unify dist test name * wip for py3 support * fix var deref * fix python3 mem_opt order * remove comments	6 years ago
Yu Yang	cda7842e26	Revert "Revert "Add Python Callstacks when Op::Run error (#12759 )"" This reverts commit `1f270275a6`.	7 years ago
Xin Pan	51ef0ad766	allow to use name_scope for debugging and visiualization	7 years ago
guochaorong	1f270275a6	Revert "Add Python Callstacks when Op::Run error (#12759 )" This reverts commit `b2df17003f`.	7 years ago
Yu Yang	b2df17003f	Add Python Callstacks when Op::Run error (#12759 ) * Add Python Callstacks when Op::Run error * Skip op with sub-block * refactor: refine callstack info's format * Reshape only support matrix * Polish Python code * Fix UT * Fix Py3	7 years ago
dzhwinter	595a2c83ae	explicit gradient of elementwise_add/elementwise_sub (#11970 ) * "add gradient register" * "make some enhance" * "better format" * "fix typo" * "fix reuse" * "fix get expected kernel" * "change the mkldnn code" * "fix mkldnn" * "fix mkldnn failed test" * "add comment"	7 years ago
dzhwinter	bfa3fd6f15	add inplace attribute to op_proto_maker (#10665 ) * "add inplace attribute" * "register inplace attribute" * "change se-next model for memory-reuse" * "fix typo" * repick * fix merge conflict * "fix stupid error"	7 years ago
Yancey1989	20c24c05aa	singleton rpc_client	7 years ago
yuyang18	c978259027	Add default value of op_role	7 years ago
yuyang18	017bba1664	Add op role	7 years ago
reyoung	a0c413833d	Polish Dctor	7 years ago
Yu Yang	0e78cb69fb	Clean OpProtoAndCheckerMaker Do not use ctor * Reduce line of codes. * We can use virtual function for Maker now. * The implementation does not care what maker holds, it is easier to refactor later.	7 years ago
Abhinav Arora	cbbf08aee9	Fix CPPLint errors in some framework files	7 years ago
qingqing01	24509f4af9	Fix the grammar in copyright. (#8403 )	7 years ago
Yi Wang	fc374821dd	Correct #include path	7 years ago
Yi Wang	90648f336d	Move file to fluid/; Edit CMakeLists.txt	7 years ago

28 Commits (bfb6ac816e8b5cbd848ccb9e4470a818fc55e015)