Commit Graph

614 Commits (add9ed3388d0082fcf26615a0408536d6932cecf)

Author SHA1 Message Date
Yu Yang eb2e4eeade Debug
7 years ago
Yu Yang 64c139e854 Using constructor for VarHandle
7 years ago
chengduoZH 4abef50123 code refine
7 years ago
Yu Yang b4aaa00a8a Polish logic of ParallelExecutor
7 years ago
Yu Yang e9e27e0f32 Revert
7 years ago
Yu Yang 7289e75836 Udpate
7 years ago
Yancey1989 ed6241cd44 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into auto_grwon_sparse_table
7 years ago
Yu Yang 4999f85f6a Clean RunDelayedOp method
7 years ago
Yu Yang c0f4e0332c Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/simplify_delay_logic
7 years ago
Yang Yang dafe06af46 first commit
7 years ago
chengduoZH 3301d4402e Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_reduce_op_handle
7 years ago
chengduoZH 3c5bbf42c4 make unit test to work
7 years ago
chengduoZH 2aaa75ecf8 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/refine_gather_reduce
7 years ago
chengduoZH e39adc8600 add reduce op handle
7 years ago
guru4elephant a097d082c7
Merge pull request #9938 from guru4elephant/fix_testcase
7 years ago
Yu Yang 5b84c9b59c CreateOpHandleIOs
7 years ago
dongdaxiang fb873713e7 update program_desc_test testcase
7 years ago
Yu Yang d0342f12aa Simplify DelayOps Logic
7 years ago
chengduoZH 690cd1f748 refine gather and broadcast
7 years ago
chengduo b43d87c917
Merge pull request #9825 from chengduoZH/feature/add_gather_and_BCast_op_handle
7 years ago
chengduoZH 384d6ee8ac follow comments
7 years ago
Yu Yang ed2d7d7d26 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/mix_cpu_gpu_op
7 years ago
Yu Yang 925c17aba7
Merge pull request #9895 from reyoung/feature/fix_transformer_hang
7 years ago
Yu Yang 4452ff76b7 Fix CPU compile
7 years ago
chengduoZH 02842cfc25 enhance broadcast_op_handle and gather_op_handle
7 years ago
Yu Yang 79be06045c Support CPU/GPU mixture for ParallelExecutor
7 years ago
Yancey1989 ac7cb949d0 auto-grown sparse table
7 years ago
Yu Yang 7083c2a6b3 Merge branch 'feature/fix_transformer_hang' into feature/mix_cpu_gpu_op
7 years ago
Yu Yang cedade9494 Stash
7 years ago
Wu Yi 1bdc7261a5
Merge pull request #9578 from typhoonzero/threadpool_for_io
7 years ago
Tao Luo 2c552d4e61
Merge pull request #9630 from Xreki/core_inference_prepare
7 years ago
Yu Yang 6b20b35589 Fix Transformer Hang Problem
7 years ago
typhoonzero a08bf76f74 refine name
7 years ago
chengduoZH b0267ac93a refine broadcast op
7 years ago
chengduoZH 8b597d9ddc Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_gather_and_BCast_op_handle
7 years ago
Liu Yiqun 449bdde58a Correct some typos.
7 years ago
chengduoZH e26c6d78ad code refine
7 years ago
Qiao Longfei 4c55a6022a
Dist transpiler support prefetch (#9714)
7 years ago
typhoonzero 26cfc634b9 multi stream thread pool
7 years ago
Liu Yiqun 2762959f79 Merge branch 'develop' into core_inference_prepare
7 years ago
Liu Yiqun 339be6254e Refine the order of arguments.
7 years ago
Yu Yang ad73b331c7 Eagerly drop local scope in iteration (#9838)
7 years ago
Yang Yang(Tony) a64edbf14e
delete backward.cc related code on the python side (#9854)
7 years ago
Yang Yang(Tony) b26f505002
remove net op and cond_op (#9663)
7 years ago
chengduoZH ae5923e205 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_gather_and_BCast_op_handle
7 years ago
chengduoZH e7684911fd add gather op handle
7 years ago
Liu Yiqun bf485999f4 Merge branch 'develop' into core_inference_prepare
7 years ago
typhoonzero dfc60251bf Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multigpumultinode
7 years ago
fengjiayi 90084a25d2
Merge pull request #9743 from JiayiFeng/modify_readers_to_fit_parallel_executor
7 years ago
chengduoZH 124c93081d remove ContextMap
7 years ago
typhoonzero e2ca8e1097 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into threadpool_for_io
7 years ago
typhoonzero d1e63a1d92 fix ci
7 years ago
typhoonzero 16a9dfe480 finish
7 years ago
JiayiFeng 273f4892b2 update recordio unittest
7 years ago
chengduoZH 6db96ec23c follow comments
7 years ago
chengduoZH 8eaec5dd7c add BCast and Gather
7 years ago
Tao Luo 0ad892a0d4
Merge pull request #9816 from luotao1/remove_op
7 years ago
typhoonzero ec6976810d Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multigpumultinode
7 years ago
fengjiayi a84b81502c Remove Readers' HasNext()
7 years ago
wanghaoshuang 19c1a68ee9 Fix lost of LoD while splitting tensor in parallel executor.
7 years ago
typhoonzero ce08dc8751 have stream removed error
7 years ago
typhoonzero 0bf799a523 wip testing
7 years ago
chengduo e0babe7c3e
Merge pull request #9780 from chengduoZH/feature/fix_batch_size_is_littler_than_gpu_count
7 years ago
typhoonzero b9c28df922 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multigpumultinode
7 years ago
Luo Tao e7467d94a0 add remove_op, remove_var in Python end
7 years ago
JiayiFeng ee178d5aeb fix bugs
7 years ago
Yu Yang 161344bf47
Merge pull request #9774 from reyoung/feature/simplify_data_structures
7 years ago
Xin Pan 326b434ab5
Merge pull request #9699 from panyx0718/test
7 years ago
typhoonzero 22f03a1b3b Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multigpumultinode
7 years ago
Yu Yang 17bfe3f6be Change to avoid GCC4.8 bug
7 years ago
Liu Yiqun e24172eb54 Simplify the inference unittest of fit a line and add some comment.
7 years ago
Liu Yiqun bdb21f6bc3 Merge branch 'develop' into core_inference_multi_thread
7 years ago
chengduoZH 7e7611d067 when the number of samples of current batch is less than the count of devices, let it crash.
7 years ago
Yu Yang ad40faaafb Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/simplify_data_structures
7 years ago
Yu Yang f051c768e5 Simplify DataStructure in SSAGraph
7 years ago
Liu Yiqun a9855e4afd Merge branch 'develop' into core_inference_fix_run
7 years ago
qingqing01 2b7e5bd366
Support testing during training by ParallelExecutor. (#9738)
7 years ago
Yi Wang 25ad6884bb Merge branch 'develop' of http://github.com/paddlepaddle/paddle into cpplint-memory-detail
7 years ago
typhoonzero baea2cf178 wip
7 years ago
Yancey1989 9eaf445809 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into prefech_prog_on_server
7 years ago
Liu Yiqun a9e826ed49 Add the check of has_feed/fetch_operators back.
7 years ago
Liu Yiqun 7b40f7ce4a Merge branch 'develop' into core_inference_prepare
7 years ago
Liu Yiqun 208fcf5225 Merge branch 'develop' into core_inference_multi_thread
7 years ago
typhoonzero 01c6618de9 first wip commit
7 years ago
Yi Wang eebb205324 Update CMakeLists
7 years ago
Yi Wang e185502ebe
Fix cpplint errors with paddle/fluid/platform/dynload (#9715)
7 years ago
Xin Pan b138d29c38 Avoid init_p2p all the times
7 years ago
Yi Wang 797a7184ac
Unify Fluid code to Google C++ style (#9685)
7 years ago
Abhinav Arora b5b7ea12fa
Fix CPPLint issues in tuple.h (#9670)
7 years ago
chengduo 3d3e5aeb9c
Merge pull request #9648 from chengduoZH/feature/expose_CUDAPinnedPlace_to_python
7 years ago
chengduoZH 4ff237f93c follow comments
7 years ago
Lei Wang 09b4a1a361 Build: generate all the build related files into one directory. (#9512)
7 years ago
Qiao Longfei 424dd2fc43
Merge pull request #9597 from jacquesqiao/sgd-support-update-selected-rows
7 years ago
Abhinav Arora 7bf82f82b1
Fix CPPlint errors in channel.h, channel_impl.h and channel_test.cc (#9628)
7 years ago
qiaolongfei ff4208e676 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into sgd-support-update-selected-rows
7 years ago
qiaolongfei e66bd4cb73 add GetDataTypeOfVar
7 years ago
chengduoZH 17842e33c3 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/expose_CUDAPinnedPlace_to_python
7 years ago
Xin Pan 043c23080c
Merge pull request #9637 from panyx0718/feed
7 years ago
chengduoZH 8e4e155c52 add PyCUDAPinnedTensorSetFromArray
7 years ago
Tao Luo 5eb9cecc4a
Merge pull request #9607 from luotao1/remove_var
7 years ago
Liu Yiqun fecbf7ff73 Merge branch 'develop' into core_inference_fix_run
7 years ago
Yancey1989 abfd9fe798 Merge branch 'develop' of github.com:PaddlePaddle/Paddle into prefech_prog_on_server
7 years ago
Liu Yiqun 3a82578278 Merge branch 'develop' into core_inference_prepare
7 years ago
Yancey1989 1842758117 prefetch prog run on new scope
7 years ago
Xin Pan 4bbfa9eccb Add feed to ParallelExecutor
7 years ago
Yu Yang bc8f436061
Merge pull request #9634 from reyoung/feature/fix_leaf_ops
7 years ago
Liu Yiqun 9cba062252 Add inferface to change the feed/fetch_holder_name.
7 years ago
chengduo c14305f0f6
Merge pull request #9380 from chengduoZH/feature/add_CUDAPinnedPlace
7 years ago
Yu Yang 27533b6423 Fix Leaf Ops in Graph
7 years ago
Liu Yiqun fbd3604cad Split Executor.Run to Executor.Prepare and Executor.RunPreparedContext for inference.
7 years ago
chengduoZH e099b18045 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_CUDAPinnedPlace
7 years ago
Luo Tao 09b53c086d add remove_var from c++ end
7 years ago
qiaolongfei 31e8d807d9 optimize code
7 years ago
typhoonzero 1f6e0448bc Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into prepare_pserver_executor
7 years ago
Xin Pan b123ce88a1 Add enable/disable for delayed ops
7 years ago
Xin Pan be1373dcf9 Polish
7 years ago
Xin Pan 46f3a39e91 polish and add comments.
7 years ago
Xin Pan d0ac92531d Improve ParallelExecutor performance
7 years ago
chengduo dd75fbde81
Merge pull request #9560 from chengduoZH/feature/fix_parallel_exe
7 years ago
qiaolongfei e64dda7e44 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into sgd-support-update-selected-rows
7 years ago
Liu Yiqun 8137f58c06 Merge branch 'develop' into core_inference_fix_run
7 years ago
qiaolongfei 2669aea67f sgd_op support optimize SelectedRows
7 years ago
Liu Yiqun 2a2e22e35f Merge branch 'develop' into core_inference_fix_run
7 years ago
typhoonzero b851c0739f update compile
7 years ago
chengduoZH 494bee5135 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/fix_parallel_exe
7 years ago
qiaolongfei 13ecb5e50a Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into lookup_table_support_SelectedRows_as_parameter
7 years ago
typhoonzero f43be75b82 multi stream thread pool
7 years ago
chengduoZH 30adc0b5f8 add notation
7 years ago
typhoonzero 75bfdb3a3c refine
7 years ago
qiaolongfei 3b3d210c3e lookuptable support SelectedRows as table parameter
7 years ago
qiaolongfei 9a101cfc08 clean code
7 years ago
qiaolongfei 997e9a1fd2 fix mac compile
7 years ago
chengduoZH de5e56bee8 add og has been broadcasted
7 years ago
typhoonzero fc4bcdd753 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into prepare_pserver_executor
7 years ago
Abhinav Arora ffcc760478 Fix deadlock in channel_test (#9544)
7 years ago
chengduo 81d93514d6
Merge pull request #9522 from chengduoZH/feature/refine_parallel_exe
7 years ago
Qiao Longfei 23bab34ca3
Fix data transform when inplace (#9450)
7 years ago
chengduoZH 60d0a0594e refine parallel
7 years ago
Yu Yang fa21436d0d
Merge pull request #9080 from reyoung/cpp_parallel_executor
7 years ago
Abhinav Arora 5f9da86ba5
Fix the order of reads and write from buffered channel (#9423)
7 years ago
Yang Yu af230d9bef Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into cpp_parallel_executor
7 years ago
Abhinav Arora f5da16e51b
Disabling channel test to debug issue (#9491)
7 years ago
Yu Yang e868950e5f Add comments
7 years ago
Tao Luo 857a8997de
Merge pull request #9384 from luotao1/removeVar
7 years ago
Yu Yang 38b53b37b4 Remove Pop method
7 years ago
Yu Yang ce2f096372 Merge branch 'cpp_parallel_executor' of github.com:reyoung/Paddle into cpp_parallel_executor
7 years ago
Yu Yang 7da1ea07a2 Use PopAll
7 years ago
Yang Yu b0775588c0 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into cpp_parallel_executor
7 years ago
Yu Yang 084cdd1f4f Rename code
7 years ago
Luo Tao 7f4012247e adjust remove rule for variables
7 years ago
Yu Yang 201f79d039 Use Extend method
7 years ago
Yu Yang dcf7bd2d92 Add initP2P
7 years ago
Yu Yang 50f71f5005 Using blocking queue
7 years ago
Yu Yang 7dcb217e31 Refine allreduce op
7 years ago
Yu Yang c0c2e15920 NCCL AllReduce
7 years ago
Yu Yang 3f88fad08c Fix merge op
7 years ago
Yu Yang 5b92dd4026 Remove dev sync
7 years ago
Yu Yang 52dd8ff09a Force sync dev
7 years ago
Yu Yang dfb8680018 Early drop fetch op
7 years ago
Yu Yang 9af870854e Use heap variables
7 years ago
Yu Yang 222763296f Change fetch op
7 years ago
Yu Yang 76570c2e96 Wait fetch op
7 years ago
Yu Yang b6ca3711b4 Get error
7 years ago
Yu Yang 55e2cc3d87 FetchOp Force sync
7 years ago
Yu Yang 5a02739ce9 Throw error
7 years ago
Yu Yang f385228f05 Add Paddle Enforce
7 years ago
Yu Yang 833e522d16 Enhance drop kids
7 years ago
Yu Yang aba46f077b Disable P2P
7 years ago
chengduoZH ab601c19c3 Add CUDAPinnedPlace
7 years ago
Abhinav Arora 65534c4762
Fluid channels should match the semantics of Go Channels (#9265)
7 years ago
chengduoZH 158d6c4d19 add unit test
7 years ago
Luo Tao ccfec1bcb1 remove vars when remove ops
7 years ago
chengduoZH 18eb77303d add CUDAPinnedPlace
7 years ago
typhoonzero 1ab4fcb5e7 prepare pserver executor
7 years ago
chengduoZH a0e2cf03e4 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into feature/add_pinned_memory
7 years ago
Yu Yang 9dd64d83f3 WMT Model
7 years ago
chengduoZH 39004080f4 replace use_pinned with is_pinned
7 years ago
Yu Yang cb40c33137 Update unittest
7 years ago
Yu Yang 3aa2a8ffcf Follow comments
7 years ago
Yu Yang 02aaecca35 Fix CPU compile
7 years ago
Yu Yang 54bd17fe7b Complete Flowers
7 years ago
Yu Yang 50e7e25db3 Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into cpp_parallel_executor
7 years ago
Yu Yang 5c7a523326 Add Graphviz output
7 years ago
Qiao Longfei 8ccc61f334
support empty tensor (#9338)
7 years ago
Yu Yang edfd741e3a Add simple python wrapper for ParallelExecutor
7 years ago
Yu Yang a7b0d5bd26 Clean code
7 years ago
Yu Yang e3144393e3 Extract Executors to indie modules
7 years ago
Yu Yang c70b60dd70 Make executor steal graph inside
7 years ago
Yu Yang 4c3361cda8 Extract GraphExecutor
7 years ago
Yu Yang b123e43bf9 extract multi devices graph builder
7 years ago
Varun Arora 76ae540f8e
Move Select to concurrency.py; incorporate outputs (#9136)
7 years ago
dzhwinter 13f1050ab0
"fix mixed_vector bug" (#9319)
7 years ago
Liu Yiqun 5419da6e7a Fix bug caused by block_id.
7 years ago
Yu Yang dd73d18bb7 Extract SSAGraph
7 years ago
Yu Yang 79989c9025 Add SSA builder
7 years ago
Yu Yang 64d7a30271 Extract SSAGraph
7 years ago
Yu Yang 8dec4ad7a1 Use int not Place for vars
7 years ago
Yu Yang 3181501013 Rerange code
7 years ago
Yu Yang f28ae6e4b1 Reorganize Code
7 years ago
Yu Yang 5c333e4143 Add dctor for dev_ctx
7 years ago
Liu Yiqun 0968753454 Enable the test of not creating variables every time.
7 years ago
Yu Yang 15f5f10ed5 AddInput/AddOutput for OpHandle
7 years ago
Yu Yang 5368e50d84 Reorganize code
7 years ago
Yu Yang fe7ed285d1 Extract NCCLCtxMap
7 years ago
Yu Yang 6ebc6bf533 ReorganizeCode
7 years ago
Yiqun Liu 7bb4ea9c13
Add an argument in Executor.Run to allow users to choose whether to create and destroy variables every time. (#9242)
7 years ago
Yu Yang a478a11e0b NCCL Guard for bcast
7 years ago
Yu Yang f2685bed81 Clean code
7 years ago
Yu Yang 41ad632341 Add NCCL Group Guard
7 years ago
Yu Yang 99fe83a020 Move nccl helper
7 years ago
Yu Yang 90f980167d Do not wait computation stream
7 years ago
Yu Yang 7ac969b88c Debug
7 years ago
Qiao Longfei 37a272e670
add executor.prepare (#9022)
7 years ago
chengduoZH eaa90d38ad add use_pinned
7 years ago
Yu Yang 599f7a87ba Refine code
7 years ago
Yu Yang 43e54079a8 Debug code
7 years ago
Yu Yang e335f01826 Add more logs
7 years ago
Yu Yang 82693e7227 Wait nccl all reduce
7 years ago
Yu Yang eb0a580e78 Add enforce
7 years ago
Yu Yang 65bc7d17d5 Add mtx to ncclAllReduce
7 years ago
Yu Yang ba227df941 Expose num_threads
7 years ago
Yu Yang 1533bf12df Use event and single thread
7 years ago
Yu Yang 95a0d7c7c1 Illegal memory access
7 years ago
Yu Yang 798e6907b4 Change mem order
7 years ago
Yu Yang 1c2b6100b0 Add
7 years ago
Yu Yang 4e43b71377 Add wait log
7 years ago
Yu Yang dbed123382 Debug
7 years ago
Yu Yang e53b6aba63 Use no thread
7 years ago
Yu Yang a8bd7b9809 Add log
7 years ago
Yu Yang 3c9cea597e Add more log
7 years ago
Yu Yang f8f1a963d9 Add debug code
7 years ago
Yu Yang fbbcedda01 Fix bug
7 years ago
Yu Yang 7643c2cbab Add flag for use event
7 years ago
Yu Yang ca4b3d2532 Use 12 threads
7 years ago
Yu Yang f251a58e85 Use base class manage events
7 years ago
Yu Yang 1dd216dc3b Wait bcast param
7 years ago
Yu Yang 4185dd48e4 Disable multi-thread
7 years ago
Yu Yang 631aa3d10a Wait all inputs ready
7 years ago
Yu Yang 9b1f4d5d62 After nccl add event
7 years ago
Yu Yang feb569f8ea Add log
7 years ago
Yu Yang 260cfe3b86 Stop Wait NCCL Stream
7 years ago
Yu Yang e025e284c6 Exchange wait op
7 years ago
Yu Yang 3238ce0672 Add wait
7 years ago
Yu Yang 8a9de67e17 Remove wait
7 years ago
Yu Yang d2cb3790e9 Wait all evernts
7 years ago
Yu Yang 4137bb4eda Add wait
7 years ago
Yu Yang 3da4159f88 Add run iter
7 years ago
Yu Yang d3c82c356e Wait multiple stream
7 years ago
Yu Yang c18c2f6ab0 Sync all computation streams at the end of run
7 years ago
chengduo 597ba3f3f2 add more times close test (#9215)
7 years ago