Commit Graph

582 Commits (c34b24ede782612464bc4c7cad47c40661616e9d)

Author SHA1 Message Date
sneaxiy d231e55065 merge develop
6 years ago
Yu Yang fdc689142c Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation
6 years ago
minqiyang 87450b9ad4 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_vlog
6 years ago
chengduo c5b6573a5a
Fix input<tensor> (#14208)
6 years ago
minqiyang 3db9fad764 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_vlog
6 years ago
Xin Pan ff6c809bfc
Merge pull request #14251 from panyx0718/fix
6 years ago
minqiyang 0c3227a523 Change the origin VLOG level to 10 times
6 years ago
Xin Pan 59c66532e7 add more logs and comments
6 years ago
Yu Yang 057a682ee9 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation
6 years ago
Yu Yang c28beb8a3c
test(Pe): add dry run tests for pe (#14254)
6 years ago
Xin Pan cb2d33a851 resolve conflict
6 years ago
Xin Pan 8c11d3fed6 clean up
6 years ago
Xin Pan 0a89650507 fix more tests
6 years ago
Xin Pan a3b27e3237 fix
6 years ago
Xin Pan f25eb9a71d fix some tests.
6 years ago
Xin Pan adf5615e54 clean kGraphOp
6 years ago
Xin Pan 2e14999942 clean1
6 years ago
Xin Pan 34b401fc6c clean up a global graph attr.
6 years ago
Zeng Jinle 8ac2242b6e
Merge pull request #14075 from sneaxiy/remove_some_locks_in_pe
6 years ago
sneaxiy faac8a76ce remove unnecessary codes
6 years ago
Wu Yi 306236c2c0
feature/DC asgd (#12722)
6 years ago
sneaxiy 7ff320f8cc merge develop
6 years ago
sneaxiy 203027ca86 test=develop
6 years ago
sneaxiy cf1944af2a test=develop
6 years ago
Yan Xu d10b8efcc0
Merge pull request #14152 from Yancey1989/add_fused_broadcast_unittest
6 years ago
Yu Yang c21597cf07
fix(PE): use shared_ptr<BlockingQueue> for cross thread communication (#14136)
6 years ago
chengduo ed087f8232
refine op_handle (#14178)
6 years ago
Yancey1989 6bfa6a0a33 add fused broadcast op unit test, test=develop
6 years ago
sneaxiy 2494ca83ab test=develop
6 years ago
sneaxiy f2eed667c0 test=develop
6 years ago
sneaxiy cef8cc81db merge develop
6 years ago
Qiao Longfei 3d4e050802 fix compile, optimize code test=develop
6 years ago
Yu Yang c01696f8c2 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation
6 years ago
Qiao Longfei d26ff8cb2d Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into cpu-for-1.1-merge-with-shape
6 years ago
Wu Yi 26200f2e42
[1.1] [project] train imagenet using large batch size (#13766)
6 years ago
sneaxiy 2414f92f54 test=develop
6 years ago
sneaxiy 45559d042c move to pass
6 years ago
sneaxiy a314a80cdb merge develop
6 years ago
seiriosPlus c34610f86d Fix lookup table at CPU Reduce strategy, test=develop
6 years ago
Qiao Longfei fad42fe7cc broadcast handle not inited parameter
6 years ago
sneaxiy 1af3fe8c35 test=develop
6 years ago
Xin Pan d5d09672c8 better fix
6 years ago
sneaxiy 5be6f762d0 remove_lock_in_some_ops
6 years ago
Yu Yang 461f71a90b Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation
6 years ago
chengduozh 82d2903b63 Fix fast ParallelExe bug
6 years ago
sneaxiy fb6201e93e test=develop
6 years ago
sneaxiy d87569134c test=develop
6 years ago
sneaxiy 4c672ab1a2 Merge reyoung:rewrite_allocation
6 years ago
sneaxiy 9606b37ce4 test=develop
7 years ago
Yu Yang 58ed412f68 refactor(memory): rewrite memory allocation and make it extentable
7 years ago
Xin Pan 7cd2761736
Merge pull request #13416 from panyx0718/ir
7 years ago
Yu Yang 0be1582df0
Merge pull request #13525 from reyoung/fix_mixed_vector
7 years ago
Xin Pan f117feab0c modify comments
7 years ago
Xin Pan c2b3838cf5 add some comments
7 years ago
Xin Pan 0d9ee0dced fix resolve conflicts
7 years ago
Xin Pan b43e49fa31 fix
7 years ago
Xin Pan 36c2a9af27 pass builder allow cutomize pass in python.
7 years ago
Zeng Jinle f855c05fff
Merge pull request #13520 from sneaxiy/enhance_eager_delete
7 years ago
Yu Yang 3043f51b3a
Merge pull request #13511 from reyoung/fix_ce
7 years ago
Wu Yi 29c63d180f
[Feature] dist op role and lr op role, to support memory optimize with dist training (#13220)
7 years ago
Yu Yang e1913bc5d8 Fix MixedVector
7 years ago
sneaxiy 0a36ef3c18 enhance eager deletion
7 years ago
Yu Yang 6d2c6f96f1 Revert "Revert "Merge pull request #13431 from chengduoZH/refine_lod""
7 years ago
Yu Yang a6c8d6b9a2 Revert "Merge pull request #13431 from chengduoZH/refine_lod"
7 years ago
chengduo d402234ba8
Feature/op_fuse_pass (#12440)
7 years ago
Xin Pan 23ba766252
Merge pull request #13475 from panyx0718/ir5
7 years ago
Xin Pan 0bd7a67eaf avoid creating dangling ir::Node.
7 years ago
chengduoZH 6534f8527a Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refine_lod
7 years ago
Xin Pan a83a4fab5c
Merge pull request #13441 from panyx0718/ir2
7 years ago
Xin Pan e5b322051b clean
7 years ago
chengduoZH def2a8b1b0 Async memcpy
7 years ago
sneaxiy 612e1a3155 modification
7 years ago
sneaxiy d0b2453ecd merge develop
7 years ago
sneaxiy 24ea39c4c6 feature/eager_delete_tensor
7 years ago
Yancey1989 757f9683ab update comment text
7 years ago
Yancey1989 9ee1b7bc04 add some comments
7 years ago
Yancey1989 bad4ea192e update by comment
7 years ago
Yancey1989 1664899b63 update
7 years ago
Yancey1989 1e1b6622fd update by comment
7 years ago
Yancey1989 5ce1a960a5 move bcast op into pass
7 years ago
Xin Pan 5ae052bd14 remove some unused codes
7 years ago
dzhwinter ab1097cd8e
Feature/template (#13093)
7 years ago
Yu Yang 819af27d63
Merge pull request #13079 from reyoung/feature/fix_bug
7 years ago
Wu Yi 0b8067c0dc
fix dist train reduce mode (#13068)
7 years ago
yuyang18 555f03d8f0 Fix bug when loss@GRAD is reused.
7 years ago
Wu Yi 0ee6fed05b
Refine dist rpc deps (#12899)
7 years ago
Wu Yi c6f212a339
Merge pull request #12847 from typhoonzero/add_scope_info_in_graphviz
7 years ago
Wu Yi b8da70c370
Resovle multi gpu async deps (#12828)
7 years ago
typhoonzero dd7a79158b add scope info in graphviz debug
7 years ago
yuyang18 b8029fd650 Follow comments
7 years ago
yuyang18 05cadf1b24 Add FastExecutor
7 years ago
yuyang18 c6eb7a89ff Merge branch 'feature/better_exception_holder' into feature/fast_executor
7 years ago
yuyang18 aac80ef4cc Polish API of exception holder
7 years ago
yuyang18 d49763a87d Stash
7 years ago
Xin Pan 626abfc33a code clean up and renaming
7 years ago
Yu Yang c4f8afa258 Fix bug when memopt optimize loss.grad and use ParallelExecutor
7 years ago
chengduo 97a77512b4
Fix the order of sum (#12562)
7 years ago
Xin Pan caf10b474f make profiler use thread_id from g_thread_id
7 years ago
fengjiayi bc1b7b96ec Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into dev_exception_holder
7 years ago
qiaolongfei c8c8c01a23 fix mac build of graph_executor
7 years ago
fengjiayi 3e4083ed1f Make exception handling of threaded_ssa_graph_executor an independent class
7 years ago
Xin Pan 4b8ae523c4
Merge pull request #12367 from panyx0718/ir_pass
7 years ago
Xin Pan 398cfb47b1 disable dist_se_resnext since it's not stable yet.
7 years ago
Xin Pan ab72d28a5e clean up and correctness check
7 years ago
Xin Pan aa1085ddc5 all passes
7 years ago
Xin Pan e4d7d7ae8f pass refactoring
7 years ago
Xin Pan 142e832d21 pass registration
7 years ago
qiaolongfei 64e7902530 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into parallel-executor-support-prefetch
7 years ago
qiaolongfei 754e96a30c distribute lookup table work with parallel executor
7 years ago
Xin Pan 5173a53c8a fix reorder issue.
7 years ago
Xin Pan 21a45420f0 polish and test
7 years ago
Xin Pan 93355cc0d2 fix control deps
7 years ago
Xin Pan f6d99d1f73 polish
7 years ago
Xin Pan c3f6e0e8a2 add namespace to Graph
7 years ago
Xin Pan 0b3465d215 better
7 years ago
Xin Pan dcaf183daa builder SSA graph at the beginning.
7 years ago
Yu Yang b06309381b
Merge pull request #12149 from reyoung/feature/combine_open_files_and_double_buffer
7 years ago
chengduo a3ac54b672
Fix Reduce functor (#12262)
7 years ago
Tao Luo 6fd75edae5 Fix var_handle CMake dependence (#12261)
7 years ago
qiaolongfei 0e30c9d6fb fix mac build
7 years ago
yuyang18 7268760fbc
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/combine_open_files_and_double_buffer
7 years ago
Xin Pan da5efa735a fix
7 years ago
Xin Pan 5daad16218 polish
7 years ago
Xin Pan ff5a7b67ed polish
7 years ago
Xin Pan 9c9e28b57b fix program to graph
7 years ago
Xin Pan 64eaa4c829 clean
7 years ago
Xin Pan 10786a243e polish graph
7 years ago
Xin Pan 2fa8df1caf separate graph building pass and graph-based pe builder
7 years ago
Xin Pan 37e514432b op compose node and update nodes.
7 years ago
Xin Pan 9605fcd124 all graphs
7 years ago
Xin Pan af79b19207 add a simple program to graph
7 years ago
Xin Pan 68aa500451 polish attrs
7 years ago
Xin Pan 9b9603306c graph attrs
7 years ago
Xin Pan 2eeaa8d5cf Graph in ParallelExecutor Builder
7 years ago
Xin Pan 7781297c70 variants
7 years ago
yuyang18 b8975d6842
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/combine_open_files_and_double_buffer
7 years ago
chengduo 86b0a72576
Refine multi thread cpu parallel exe (#11406)
7 years ago
yuyang18 8c2eba71e2
Refine demo
7 years ago
fengjiayi 335e4b9d52 fix a dead lock bug
7 years ago
yuyang18 c822d0309b
Refine code
7 years ago
yuyang18 0a445da631
Make scope_buffered_ssa_graph_executor Exception safe
7 years ago
yuyang18 8e86721fe7
Fix data balance on single GPU
7 years ago
fengjiayi ce16b40b04
Merge pull request #11891 from JiayiFeng/dev_eof_exp
7 years ago
yuyang18 2d0e5592b5
Use std::map for Place <--> DeviceContext
7 years ago
fengjiayi 3fab4f65a4 Add EOFException to represent EOF in C++ reader
7 years ago
fengjiayi ff4317cee9 follow comments
7 years ago
fengjiayi 4b950951d3 Add unittests and fix a few bugs
7 years ago
fengjiayi 077434c26c fix CPU compile error
7 years ago
fengjiayi 47388020a2 fix bugs
7 years ago
fengjiayi 2e320079d3 fix bugs
7 years ago
fengjiayi 5b4f283069 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into dev_data_balance
7 years ago
fengjiayi b6dc3a59f1 Add DataBalanceOpHandle to MultiDeviceSSAGragh
7 years ago
yi.wu 88cb47bd86 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_paraexe_bcast
7 years ago
yi.wu 3d69a82b83 fix dist train broadcasting bug
7 years ago
chengduo bb29800aaa
small refine (#11460)
7 years ago
yi.wu 254154a9bf fix sparse paraexe dist train
7 years ago
fengjiayi e45a55523b
Merge pull request #11639 from JiayiFeng/fix_mac_compile
7 years ago
chengduo da556ed6d4
enhance ParallelExecutor stable (#11637)
7 years ago
fengjiayi 964f515e9a fix mac compile
7 years ago
Yu Yang 9b3f48d7e6
Merge pull request #11616 from chengduoZH/fix_parallel_exe
7 years ago
chengduoZH c99fca5f90 Add No Mutex
7 years ago
chengduoZH 13de72388d Fix broadcast
7 years ago
Xin Pan 0151e4eb3d
Merge pull request #11608 from panyx0718/doc
7 years ago
chengduoZH 28a86aebc3 Fix Parallel Exe(VarHandel's version)
7 years ago
Xin Pan df31926fcf small thread-safety fix and doc improvements.
7 years ago
Yancey1989 3d875b693f Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_memcpy_with_dist
7 years ago
Yancey1989 f52d78d189 update by comment
7 years ago
Yancey1989 6d752bafd8 use get_appropriate_dev to schedule rpc op
7 years ago
Yancey1989 4444e79e46 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_memcpy_with_dist
7 years ago
qiaolongfei 16658f7b59 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refine-prefetch
7 years ago
qiaolongfei 83a577e8ce fix build problem
7 years ago
chengduoZH 173d72b481 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into enable_cpu_on_pe
7 years ago
chengduoZH aadaadf735 replace use_event with use_cuda, because use_event means the program running with CUDA, so use_cuda maybe more intuitive.
7 years ago
gongweibao 627d7a64f8
Clean `sendop` `recv` operator. (#11309)
7 years ago
chengduo fa29ef0b0d
Merge pull request #11277 from chengduoZH/check_ssa_graph
7 years ago
chengduoZH 961fbce8e2 follow comments
7 years ago
chengduoZH 7b723839ef Add cpu test for parallel_executor_crf executor_fetch_feed, and enable these tests
7 years ago
chengduoZH d24e046c1e fix allReduce bug
7 years ago
chengduoZH a57e8a4338 add cpu test
7 years ago
chengduoZH 495368c243 ADD CPU_NUM
7 years ago
chengduoZH 27073c284d nccl_all_reduce_op_handle => all_reduce_op_handle
7 years ago
chengduoZH 2d94697a82 code refine
7 years ago
chengduoZH 5a3c8bf813 fix in c++ side
7 years ago
fengjiayi d745840a6e fix a small compile error on Mac
7 years ago
chengduoZH 0c851cab22 add SSA graph checker
7 years ago
Yancey1989 d5a88b9340 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_memcpy_with_dist
7 years ago
chengduoZH 1076e85135 refine logic
7 years ago
Yancey 0aa9546eed
fix dist train error (#11281)
7 years ago
chengduoZH 8291b916d6 replace graph_builder_factory with ssa_graph_builder_factory
7 years ago
chengduoZH 9ac785be39 check graph's validation
7 years ago
fengjiayi a1e046bfc0
Merge pull request #11270 from JiayiFeng/fix_a_error_on_max
7 years ago
fengjiayi 2f5e310167 fix a compile error
7 years ago
yuyang18 8149b0a9aa Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add_fuse_var_op_handle
7 years ago
Yancey1989 23433def4b Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_memcpy_with_dist
7 years ago
yuyang18 d9af153232 SSA Graph Builder Factory
7 years ago
chengduoZH a584bc86dd add fuse var op handle
7 years ago
Yancey1989 e533a4b4ab Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_memcpy_with_dist
7 years ago
Yancey1989 82d741c4b9 fix op name typo
7 years ago
Yancey1989 6d69ae0c6e code cleanup
7 years ago
Yancey1989 93401c98e1 overlap rpc op memcpy in distributed training
7 years ago
yuyang18 86a61c177f Add ScopeBufferedSSAGraphExecutor
7 years ago
chengduoZH e330cd032e balance parameter update
7 years ago
Yancey1989 20c24c05aa singleton rpc_client
7 years ago
Yancey1989 28596a3386 add gflag ssa_graph_path
7 years ago
Yancey1989 ad6c0142c4 clean up codes
7 years ago
Yancey1989 268e9dc1c6 polish code
7 years ago
Yancey1989 0aa6f9e934 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into overlap_send_op
7 years ago
Yu Yang d406c76a30
Merge pull request #10744 from reyoung/feature/refine_parallel_executor
7 years ago
Yancey1989 540b45350d use req_count as atomic type
7 years ago
yuyang18 ad6a8715ff Disable unstable unittests
7 years ago
Yancey1989 6debbcd9f9 connect fetch barrier and concat op
7 years ago
yuyang18 3b04f0099c Fix bug
7 years ago
yuyang18 39a44e3834 Add Debug log
7 years ago
yuyang18 fb370f4411 Refine code
7 years ago
yuyang18 03e4da6d04 Fix bug
7 years ago
Yancey1989 147d54ba62 update
7 years ago
Yancey1989 e9abc66910 fix pe
7 years ago
yuyang18 0e5a278c23 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/support_op_role
7 years ago
Yancey1989 62af10d440 support multiple devices
7 years ago
Yancey 7ae03ec08b
fix lack split_byref op in parallel executor (#10770)
7 years ago
Yu Yang 14248a64d7 Fix hang when input is duplicated (#10709)
7 years ago
yuyang18 3923d40992 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/support_op_role
7 years ago
yuyang18 624caee56b Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_fetch_op_handle
7 years ago
chengduoZH e383ea20dc fix fetch op handle
7 years ago
yuyang18 999d0fdbef By default is all reduce
7 years ago
yuyang18 5895989a4f Add ref
7 years ago
yuyang18 a77d1bc65e Add debug code
7 years ago
yuyang18 44c52a8c1a Polish op_proto_maker
7 years ago
yuyang18 017bba1664 Add op role
7 years ago
yuyang18 08295f9877 Add build strategy
7 years ago
yuyang18 e5281b3c2d Clean code & add execution strategy
7 years ago
yuyang18 54cbf799d6 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/change_pe_strategy
7 years ago
chengduoZH 97cb5479ae change PE strategy
7 years ago
chengduo 18f527bfa3
Merge pull request #10586 from jacquesqiao/fix-compile
7 years ago
Yu Yang 046405e091
Merge pull request #10486 from reyoung/feature/clean_op_maker
7 years ago
qiaolongfei 3c2a6109ec fix build computation_op_handle
7 years ago
qiaolongfei 07d94bdd82 fix compile fetch_op_handle on mac
7 years ago
chengduoZH a89cd46700 Wait VarDummyHandle generated
7 years ago
reyoung b0ca371f11 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/clean_op_maker
7 years ago
chengduoZH 830532213a extract method from broadcast::RunImpl
7 years ago
chengduoZH 9eec2c7509 refine pe
7 years ago
chengduoZH f4851f14b8 clean code
7 years ago
Yu Yang 0e78cb69fb Clean OpProtoAndCheckerMaker
7 years ago
chengduoZH 676dfd1875 follow comments
7 years ago
chengduoZH aff8a26d71 check generated_op_
7 years ago
chengduoZH 2e5d44f102 fix fetch op
7 years ago
chengduoZH 881e063ee2 follow comments
7 years ago
chengduoZH ff599b9218 use Reduce and Broadcast
7 years ago
chengduoZH 0441c2cc45 fix ci
7 years ago
chengduoZH f9c680c43e Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/update_sparse_parameter
7 years ago