Paddle

Commit Graph

Author	SHA1	Message	Date
Wu Yi	29d9fb53fc	[Feature] multi process multi gpu dist training, boost v100 performance by 20% (#14661 ) * wip multi process multi gpu dist training * workable for p2p * update test=develop * change back env name test=develop * fix alloc init * fix cpu build test=devlop * fix mac tests test=develop * refine code * refine test=develop	7 years ago
zhang wenhui	abbe382e1e	Revert "Add EstiminateFlops"	7 years ago
Yu Yang	0f0e197914	Merge pull request #14666 from reyoung/feature/estiminate_flops Add EstiminateFlops	7 years ago
Wang Guibao	41e19eb431	AsyncExecutor (#14627 ) * AsyncExecutor: C++ side * Google naming conventions * Rename MultiExecutor to AsyncExecutor * pybind with async_executor * Naming convention * remove some flags and unused code * add refactored file of async_executor and data_feed * clear async executor interface and add data feed factory * split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto * Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch * refine async_executor_refactor.cc * add some files about datafeed * Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. * Interface rework * add MultiSlotDataFeed * Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side * update data_feed for add MultiSlotDataFeed * update datafeed and async_executor to run bow_net demo * fix bug that finish_set_filelist failed in multithread * delete finish_binding_memory_(flag), because it can not be marked under the current interface * Fix bug * update async_executor.py for support set_use_slots * update async_executor.py for support set_use_slots and set set_dense_slots * fix bug that when the number of files is less than the number of threads, it will fetch nan * remove redundant code, and make executor exit when set a illegal queue size * add batch_size check * add MultiSlotDesc * Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. * add some checkpoint in DataFeedDesc * add CheckFile function in MultiSlotDataFeed * update something error info * fix deaded lock bug * Fix fetch variable * Merge error * fix code style in async_executor * using one lock blocking queue replace two lock blocking queue because of some bugs * update code style * add utest for data_feed * Fix fetch var * update utest for data_feed for multithread * update SetFileList info * fix bug in utest of data_feed * Add comments for python * Add comments for python code * Fix pybind.cc with new pybind11 version * add note for DataFeedDesc's set_use_slots function * Add save_model * update data_feed_test for multi-type * add comment for executor_thread_worker * Remove unused code * update data_feed_test for generate test data file * removed unnecessary interfaces and add comments * c++ style check * update data_feed.cc * AsyncExecutor: C++ side Google naming conventions Rename MultiExecutor to AsyncExecutor pybind with async_executor Naming convention remove some flags and unused code add refactored file of async_executor and data_feed clear async executor interface and add data feed factory split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch refine async_executor_refactor.cc add some files about datafeed Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. add MultiSlotDataFeed Interface rework Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side update datafeed and async_executor to run bow_net demo update async_executor.py for support set_use_slots Fix bug update async_executor.py for support set_use_slots and set set_dense_slots fix bug that when the number of files is less than the number of threads, it will fetch nan remove redundant code, and make executor exit when set a illegal queue size add MultiSlotDesc Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. add some checkpoint in DataFeedDesc Fix fetch variable fix code style in async_executor Fix fetch var add utest for data_feed Add comments for python update utest for data_feed for multithread fix bug in utest of data_feed Add comments for python code Fix pybind.cc with new pybind11 version add note for DataFeedDesc's set_use_slots function update data_feed_test for multi-type Add save_model update data_feed_test for generate test data file removed unnecessary interfaces and add comments add comment for executor_thread_worker Remove unused code update data_feed.cc c++ style check * commit for code style * commit for code style * commit for code style * commit for code style * Comment away __init__ in async_executor.py * clang-format fix test=develop * use PADDLE_THROW instead of exit(-1); use unique_ptr to manage scope var in data_feed_test.cc * commit for update code style * commit for update code style * Add async_executor demo; Remove some methods test=develop * commit for update code style * commit for update code style * commit for update code style * update API.spec * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * Fix API.spec test=develop * Fix API.spec test=develop * Fix windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix code style test=develop * Fix code style test=develop * update datafeed * Fix code style test=develop * update data_feed_test for test Tensor test=develop * Fix code style test=develop * Fix windows build failure test=develop * Fix code style and windows build failure test=develop * Fix PYTHON3.5 build failure test=develop * AsyncExecutor API test=develop	7 years ago
Yu Yang	589b863b98	Add EstiminateFlops test=develop	7 years ago
Qiao Longfei	8ebde595c9	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refactor-prefetch test=develop	7 years ago
Qiao Longfei	145c535750	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refactor-prefetch test=develop	7 years ago
minqiyang	a02ce58f2c	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into revert_vlog test=develop	7 years ago
gongweibao	867c312bc4	Fix allreduce dependency order. (#14586 )	7 years ago
Qiao Longfei	9589babe12	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refactor-prefetch test=develop	7 years ago
minqiyang	be04d99fe4	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into revert_vlog test=develop	7 years ago
Qiao Longfei	af2f5fc824	fix some bugs	7 years ago
minqiyang	53433d7f2e	Revert the changes of VLOG test=develop	7 years ago
peizhilin	7c8c9dc9bf	fix unit test cases	7 years ago
wopeizl	d9a1f3e58e	Windows/online (#14474 ) * add recordio support * disable the openblas multi-thread on windows since no support adjust the python script * code style * code style test=develop * add create_recordio_file_reader back * fix code style test=develop * fix the gtest.cmake on windows * fix cc_test on windows * fix the win build test=develop * remove fused compile support on windows test=develop * add the jit support test=develop * add the jit support, test=develop * add the jit support, test=develop * add the jit back fix compile error on windows * rollback test=develop * test case fix * disable DSO by default on windows * exclude warpctc_op on windows * exclude the dynload_warpctc out on windows test=develop * fix the scripts error test=develop * disable avx on windows by default test=develop * re-organize the cmake file * disable mkl on windows by default * add warp_ctc back * fix the dependency * fix the dependency * fix the build issue on windows * remove unsupported flag on windows * code style * code style test=develop * fix issue * add profiler, parallel_executor back * clean up the pre-definitions on windows * fix build issue * test=develop	7 years ago
peizhilin	703b26e697	add profiler, parallel_executor back	7 years ago
Yu Yang	c8f6e70ab4	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation test=develop	7 years ago
Xin Pan	8cfda7ee0c	Merge pull request #14382 from panyx0718/fix4 Refine the pass builder and buildstrategy	7 years ago
Zeng Jinle	38d32c98b8	merge develop test=develop	7 years ago
sneaxiy	eb18d532a5	fix num_threads in fast_pe test=develop	7 years ago
Xin Pan	759ffca423	some improvements test=develop	7 years ago
Xin Pan	99dffb91d6	allow to repeatedly share and update BuildStrategy test=develop	7 years ago
sneaxiy	d231e55065	merge develop test=develop	7 years ago
Yu Yang	fdc689142c	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation test=develop	7 years ago
minqiyang	87450b9ad4	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_vlog test=develop	7 years ago
chengduo	c5b6573a5a	Fix input<tensor> (#14208 ) * fix input<tensor> test=develop * fix split_ids test=develop * ElementwiseMul should not support SelectedRows * fix scale op test=develop * change GetTensorFromVar() method to GetTensorOrSelectedRowsFromVar() * fix operator * refine MultiOutput * fix MultiOutput test=develop * disable test_dist_save_load test=develop * fix elementwise_op test=develop * add get_sparse_as_op test=develop * add info for check test=develop * rename get_sparse_as_op with extract_rows_as_op. test=develop * elementwise doesn't support selected_rows * fix regularizer * remove extract_rows_as test=develop * fix ci test=develop * add test for sum_op * fix regularizer test=develop * test=develop * fix pserver weight decay multi inputs test=develop	7 years ago
minqiyang	3db9fad764	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix_vlog test=develop	7 years ago
Xin Pan	ff6c809bfc	Merge pull request #14251 from panyx0718/fix Make OpHandle/VarHandle and ir::Node works cleaner	7 years ago
minqiyang	0c3227a523	Change the origin VLOG level to 10 times Fix code to support cpplint syntax check test=develop	7 years ago
Xin Pan	59c66532e7	add more logs and comments test=develop	7 years ago
Yu Yang	057a682ee9	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into rewrite_allocation	7 years ago
Yu Yang	c28beb8a3c	test(Pe): add dry run tests for pe (#14254 ) Dry run tests will skip `Op.Run` and just perform job scheduling. It helps to analysis dead lock in PE. test=develop	7 years ago
Xin Pan	cb2d33a851	resolve conflict test=develop	7 years ago
Xin Pan	8c11d3fed6	clean up	7 years ago
Xin Pan	0a89650507	fix more tests test=develop	7 years ago
Xin Pan	a3b27e3237	fix test=develop	7 years ago
Xin Pan	f25eb9a71d	fix some tests. test=develop	7 years ago
Xin Pan	adf5615e54	clean kGraphOp test=develop	7 years ago
Xin Pan	2e14999942	clean1 test=develop	7 years ago
Xin Pan	34b401fc6c	clean up a global graph attr.	7 years ago
Zeng Jinle	8ac2242b6e	Merge pull request #14075 from sneaxiy/remove_some_locks_in_pe Remove some locks in ParallelExecutor	7 years ago
sneaxiy	faac8a76ce	remove unnecessary codes test=develop	7 years ago
Wu Yi	306236c2c0	feature/DC asgd (#12722 ) * wip * add ref_by_trainer_id op * ready to test * fix ref inputs * refine rpc_op_handle * fix merge bug	7 years ago
sneaxiy	7ff320f8cc	merge develop	7 years ago
sneaxiy	203027ca86	test=develop	7 years ago
sneaxiy	cf1944af2a	test=develop	7 years ago
Yan Xu	d10b8efcc0	Merge pull request #14152 from Yancey1989/add_fused_broadcast_unittest add fused broadcast op unit test	7 years ago
Yu Yang	c21597cf07	fix(PE): use shared_ptr<BlockingQueue> for cross thread communication (#14136 ) It seems that the blocking queue might be destroyed early than Run method complete. It might because the Run method throw some unhandled exception. However, it should be shared_ptr when multthread access an resource. So change BlockingQueue as a shared_ptr. test=develop	7 years ago
chengduo	ed087f8232	refine op_handle (#14178 ) test=develop	7 years ago
Yancey1989	6bfa6a0a33	add fused broadcast op unit test, test=develop	7 years ago

1 2 3 4 5 ...

404 Commits (fab0ee87570f796a25cc299a50e6108594f65fce)