325 Commits (280a8784f7073619180d5e8af1b982fec3b14580)

Author	SHA1	Message	Date
Yi Liu	a873fa84ce	supports collective training with programs (#18392) ... 1. Since allreduce op has 4 reduce types, We split these four reduce types into four ops 2. We also refined the collective op code, e.g. we separated the collective op kernel into CPUKernel and CUDAKernel, and remove the device specified DeviceContext parameter in template as we already knew the target DeviceContext 3. We remove the newly added Collective op role to reduce the complexity of program and graph analysis	6 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175) ... * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	6 years ago
chengduo	e06c69c788	Fix default value of fluid.memory_optimize (#18295) ... * fix default value of fluid.memory_optimize test=develop * fix api.spec test=develop	6 years ago
tangwei12	659b72a97c	fix document of python api get_startup_program() (#17764) ... * add example to get_startup_program() * fix example to get_startup_program()	6 years ago
yaoxuefeng	ac92e4c066	fix distributed_transpiler.py api test=develop (#17668)	6 years ago
gongweibao	0d561ef442	fix 2dconn test=develop (#17681)	6 years ago
tangwei12	0d3c48e0a8	fix doc in transpiler, test=develop (#17313) ... * fix doc in transpiler, test=develop	6 years ago
gongweibao	65bbf950ee	Add multi-ncclcomm and 2D ncclallreduce support. (#17263)	6 years ago
Michał Gallus	0c39b97b4e	[MKL-DNN] Add Fully Connected Op for inference only(#15226) ... * fuse mul and elementwise add to fc * Reimplement the FC forward operator * Fix FC MKLDNN integration by transposing weights * Add FC MKLDNN Pass test=develop * FC MKLDNN Pass: change memcpy to std::copy * Fix MKLDNN FC handling of mismatch input and weights dims * Lower tolerance for MKL-DNN in resnet50 test test=develop * Adjust FC to support MKLDNN Op placement test=develop * Adjust Placement Op to set use_mkldnn attribute for graph test=develop * MKLDNN FC: fix weights format so that gemm version is called test=develop * FC MKLDNN: Remove tolerance decrease from tester_helper * FC MKL-DNN: Refactor the code, change input reorder to weight reorder * MKL-DNN FC: Introduce operator caching test=develop * FC MKL-DNN: Fix the tensor type in ExpectedKernelType test=develop * FC MKL-DNN: fix style changes test=develop * FC MKL-DNN: fallback to native on non-supported dim sizes test=develop * FC MKLDNN: fix CMake paths test=develop * FC MKLDNN: Refine placement pass graph mkldnn attribute test=develop * Fix Transpiler error for fuse_conv_eltwise test=develop * Fix missing STL includes in files test=develop * FC MKL-DNN: Enable new output size computation Also, refine pass to comply with newest interface. test=develop * FC MKL-DNN: enable only when fc_mkldnn_pass is enabled * FC MKL-DNN: Allow Weights to use oi or io format * FC MKL-DNN: Adjust UT to work with correct dims test=develop * Enable MKL DEBUG for resnet50 analyzer test=develop * FC MKL-DNN: Improve Hashing function test=develop * FC MKL-DNN: Fix shape for fc weights in transpiler * FC MKL-DNN: Update input pointer in re-used fc primitive * Add log for not handling fc fuse for unsupported dims test=develop * FC MKL-DNN: Move transpose from pass to Op Kernel test=develop * FC MKL-DNN: Disable transpose in unit test test=develop * FC MKL-DNN: Remove fc_mkldnn_pass from default list * Correct Flag for fake data analyzer tests test=develop * FC MKL-DNN: Add comment about fc mkldnn pass disablement test=develop * FC MKL-DNN: Disable fc in int8 tests test=develop	6 years ago
Qiao Longfei	92e7d5d7cc	fix distribute doc test=develop (#17318) ... * fix distribute doc	6 years ago
Qiao Longfei	58f7695ab2	Async exe support communicator (#17386) ... Async exe support communicator	6 years ago
liuwei1031	f82e4d75e7	improve the doc of paddle.fluid.memory_optimize, test=develop (#17473) ... * improve the doc of paddle.fluid.memory_optimize, test=develop * fix typo, test=develop	6 years ago
liuwei1031	6a53fa95e7	improve the API Sample of DataFeeder, memory_optimize and release_memory (#17374) ... * improve the API Sample of DataFeeder, memory_optimize and release_memory, test=develop * update API.spec, test=develop, test=document_preview * tweak the code format of feed API, test=develop * update API.spec, test=develop * improve doc for DataFeeder and default_main_program, test=develop	6 years ago
tangwei12	7330cd639c	truncated_gaussian_random supported in distributed training, test=develop (#17091)	6 years ago
tangwei12	1a4a51db2b	Fleet unify distributed training (#16791) ... * implement distributed transpiler with fleet	6 years ago
Qiao Longfei	0608f8ca56	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async_sparse_param_update_recorder	6 years ago
Qiao Longfei	d640c6cfa9	fix pylint	6 years ago
Qiao Longfei	542b52fac3	fix trainer_id	6 years ago
Qiao Longfei	de65398cb8	update transpiler and listen and serv op	6 years ago
Qiao Longfei	fab1b54d99	Merge branch 'add-communicator' of ssh://github.com/jacquesqiao/Paddle into add-async-ssa-graph-executor-communicator	6 years ago
Xin Pan	0c277ac6e9	polish ... test=develop	6 years ago
Xin Pan	840cf780e4	add deprecation warning. ... test=develop	6 years ago
Qiao Longfei	b8491bfd4e	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-communicator	6 years ago
tangwei12	971f3bc9b0	fix params with only 1 dim (#15828) ... * fix params with only 1 dim * test=develop	6 years ago
dzhwinter	84f067be94	update. test=develop ... test=develop	6 years ago
dzhwinter	d453b0dcf7	add details. test=develop	6 years ago
Qiao Longfei	8bda4ab213	parameter recv can run	6 years ago
Qiao Longfei	fbd186bd5d	complete recv op	6 years ago
Qiao Longfei	4356f186b4	complete parameter_send	6 years ago
dzhwinter	9c9ad7d40b	Merge remote-tracking branch 'origin/develop' into feature/ir_inplace_pass ... test=develop	6 years ago
dzhwinter	0a63234c85	follow comments. test=develop	6 years ago
dzhwinter	8b97a3a44f	rerun ci. test=develop	6 years ago
tangwei12	90df7ff378	transpiler.py code clean (#15555) ... * move var strusted to vars_distributed.py, add optimizer's block name, test=develop * rename optimzier's seems complex, revert it, test=develop * replace * with details, test=develop	6 years ago
dzhwinter	a26a6bc728	add flag. test=develop	6 years ago
gongweibao	fe8f28c957	Add GetVariableNoBarrier on brpc. (#15488)	6 years ago
Wu Yi	22db82c053	fix tangwei merge issue test=develop (#15506)	6 years ago
tangwei12	8b50ad80ff	checkpoint at distributed training (#14854) ... checkpoint for distributed training.	6 years ago
Qiao Longfei	810439a993	fix style test=develop	6 years ago
Qiao Longfei	b16e832d4d	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix-dist-sparse-decay	6 years ago
乔龙飞 Qiao Longfei	e1679b8847	Merge pull request #14893 from JiabinYang/feature/add_prefech_hs ... Feature/add prefech hs	6 years ago
Qiao Longfei	49cce3fd0e	fix dist sparse l2 decay ... test=develop	6 years ago
haowang101779990	66ea718452	en api improve format Dec 27 ... test=develop	6 years ago
sneaxiy	3e917a934a	add scope_pool ... add module cleanup test=develop	6 years ago
JiabinYang	3b7b2e1ded	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_prefech_hs	6 years ago
JiabinYang	b2f789c66d	add test transpiler dist test, test=develop	6 years ago
Xin Pan	c89a1fb287	Merge pull request #14879 from panyx0718/clean ... clean parallel do	6 years ago
dzhwinter	7cd24b1318	add ir memory optimize. (#14530) ... * follow comments. test=develop * Fix typo * fix compile error. test=develop * merge develop branch. test=develop * Remove set_equal * Polish code * Delete unused functions test=develop * polish code. test=develop * follow comment * polish code. * fix windows compile error. test=develop * fix op handle. * rerun ci. test=develop * rerun ci. test=develop * rerun macci. test=develop * polish code. test=develop * rewrite sort code. test=develop * remove unused code. test=develop * fix tests. test=develop * fix conflict. test=develop * follow comment. test=develop * merge develop branch. test=develop * fix tests. test=develop * remove ToTypeIndex. test=develop * rerun ci. test=develop	6 years ago
dzhwinter	acd4b75923	skip_opt_set support list (#14845) ... * test=develop * fix tests. test=develop	6 years ago
Xin Pan	47ea2534fb	clean parallel do ... test=develop	6 years ago
JiabinYang	c35fdf1581	Merge branch 'add_prefetch_in_nce' of https://github.com/seiriosPlus/Paddle into feature/add_prefech_hs	6 years ago