You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Wang Guibao 41e19eb431 AsyncExecutor (#14627) ... * AsyncExecutor: C++ side * Google naming conventions * Rename MultiExecutor to AsyncExecutor * pybind with async_executor * Naming convention * remove some flags and unused code * add refactored file of async_executor and data_feed * clear async executor interface and add data feed factory * split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto * Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch * refine async_executor_refactor.cc * add some files about datafeed * Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. * Interface rework * add MultiSlotDataFeed * Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side * update data_feed for add MultiSlotDataFeed * update datafeed and async_executor to run bow_net demo * fix bug that finish_set_filelist failed in multithread * delete finish_binding_memory_(flag), because it can not be marked under the current interface * Fix bug * update async_executor.py for support set_use_slots * update async_executor.py for support set_use_slots and set set_dense_slots * fix bug that when the number of files is less than the number of threads, it will fetch nan * remove redundant code, and make executor exit when set a illegal queue size * add batch_size check * add MultiSlotDesc * Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. * add some checkpoint in DataFeedDesc * add CheckFile function in MultiSlotDataFeed * update something error info * fix deaded lock bug * Fix fetch variable * Merge error * fix code style in async_executor * using one lock blocking queue replace two lock blocking queue because of some bugs * update code style * add utest for data_feed * Fix fetch var * update utest for data_feed for multithread * update SetFileList info * fix bug in utest of data_feed * Add comments for python * Add comments for python code * Fix pybind.cc with new pybind11 version * add note for DataFeedDesc's set_use_slots function * Add save_model * update data_feed_test for multi-type * add comment for executor_thread_worker * Remove unused code * update data_feed_test for generate test data file * removed unnecessary interfaces and add comments * c++ style check * update data_feed.cc * AsyncExecutor: C++ side Google naming conventions Rename MultiExecutor to AsyncExecutor pybind with async_executor Naming convention remove some flags and unused code add refactored file of async_executor and data_feed clear async executor interface and add data feed factory split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch refine async_executor_refactor.cc add some files about datafeed Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. add MultiSlotDataFeed Interface rework Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side update datafeed and async_executor to run bow_net demo update async_executor.py for support set_use_slots Fix bug update async_executor.py for support set_use_slots and set set_dense_slots fix bug that when the number of files is less than the number of threads, it will fetch nan remove redundant code, and make executor exit when set a illegal queue size add MultiSlotDesc Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. add some checkpoint in DataFeedDesc Fix fetch variable fix code style in async_executor Fix fetch var add utest for data_feed Add comments for python update utest for data_feed for multithread fix bug in utest of data_feed Add comments for python code Fix pybind.cc with new pybind11 version add note for DataFeedDesc's set_use_slots function update data_feed_test for multi-type Add save_model update data_feed_test for generate test data file removed unnecessary interfaces and add comments add comment for executor_thread_worker Remove unused code update data_feed.cc c++ style check * commit for code style * commit for code style * commit for code style * commit for code style * Comment away __init__ in async_executor.py * clang-format fix test=develop * use PADDLE_THROW instead of exit(-1); use unique_ptr to manage scope var in data_feed_test.cc * commit for update code style * commit for update code style * Add async_executor demo; Remove some methods test=develop * commit for update code style * commit for update code style * commit for update code style * update API.spec * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * Fix API.spec test=develop * Fix API.spec test=develop * Fix windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix code style test=develop * Fix code style test=develop * update datafeed * Fix code style test=develop * update data_feed_test for test Tensor test=develop * Fix code style test=develop * Fix windows build failure test=develop * Fix code style and windows build failure test=develop * Fix PYTHON3.5 build failure test=develop * AsyncExecutor API test=develop		6 years ago
..
CMakeLists.txt	Fix allreduce dependency order. (#14586)	6 years ago
all_reduce_deps_pass.cc	Fix allreduce dependency order. (#14586)	6 years ago
all_reduce_deps_pass.h	Fix allreduce dependency order. (#14586)	6 years ago
all_reduce_op_handle.cc	fix unit test cases	6 years ago
all_reduce_op_handle.h	fix unit test cases	6 years ago
broadcast_op_handle.cc	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into revert_vlog	6 years ago
broadcast_op_handle.h	fix unit test cases	6 years ago
broadcast_op_handle_test.cc	add fused broadcast op unit test, test=develop	6 years ago
broadcast_op_handle_test.h	fix unit test cases	6 years ago
build_strategy.cc	Fix allreduce dependency order. (#14586)	6 years ago
build_strategy.h	Fix allreduce dependency order. (#14586)	6 years ago
computation_op_handle.cc	remove unnecessary codes	6 years ago
computation_op_handle.h	remove unnecessary codes	6 years ago
container_cast.h	Clean Code	7 years ago
cow_ptr.h	Fix MixedVector	6 years ago
cow_ptr_test.cc	Revert "Revert "Merge pull request #13431 from chengduoZH/refine_lod""	6 years ago
data_balance_op_handle.cc	fix unit test cases	6 years ago
data_balance_op_handle.h	fix unit test cases	6 years ago
exception_holder.h	refactor(memory): rewrite memory allocation and make it extentable	6 years ago
execution_strategy.h	test(Pe): add dry run tests for pe (#14254)	6 years ago
fast_threaded_ssa_graph_executor.cc	merge develop	6 years ago
fast_threaded_ssa_graph_executor.h	Windows/online (#14474)	6 years ago
fetch_op_handle.cc	fix some tests.	6 years ago
fetch_op_handle.h	op compose node and update nodes.	7 years ago
fuse_vars_op_handle.cc	fix a small compile error on Mac	7 years ago
fuse_vars_op_handle.h	op compose node and update nodes.	7 years ago
fused_broadcast_op_handle.cc	[1.1] [project] train imagenet using large batch size (#13766)	6 years ago
fused_broadcast_op_handle.h	fix unit test cases	6 years ago
fused_broadcast_op_handle_test.cc	fix unit test cases	6 years ago
gather_op_handle.cc	refine op_handle (#14178)	6 years ago
gather_op_handle.h	op compose node and update nodes.	7 years ago
gather_op_handle_test.cc	fix some tests.	6 years ago
modify_op_lock_and_record_event_pass.cc	Revert the changes of VLOG	6 years ago
modify_op_lock_and_record_event_pass.h	remove_lock_in_some_ops	6 years ago
multi_devices_graph_check_pass.cc	clean up	6 years ago
multi_devices_graph_check_pass.h	code clean up and renaming	7 years ago
multi_devices_graph_pass.cc	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into refactor-prefetch	6 years ago
multi_devices_graph_pass.h	fix unit test cases	6 years ago
multi_devices_graph_print_pass.cc	clean up	6 years ago
multi_devices_graph_print_pass.h	code clean up and renaming	7 years ago
multi_devices_helper.cc	code clean up and renaming	7 years ago
multi_devices_helper.h	fix some tests.	6 years ago
op_graph_view.cc	resolve conflict	6 years ago
op_graph_view.h	resolve conflict	6 years ago
op_handle_base.cc	refine op_handle (#14178)	6 years ago
op_handle_base.h	add more logs and comments	6 years ago
op_registry.h	Add op role	7 years ago
reduce_and_gather.h	Feature/template (#13093)	7 years ago
reduce_op_handle.cc	fix unit test cases	6 years ago
reduce_op_handle.h	fix unit test cases	6 years ago
reduce_op_handle_test.cc	fix unit test cases	6 years ago
reference_count_op_handle.h	remove_lock_in_some_ops	6 years ago
reference_count_pass.cc	Revert the changes of VLOG	6 years ago
reference_count_pass.h	feature/eager_delete_tensor	7 years ago
rpc_op_handle.cc	feature/DC asgd (#12722)	6 years ago
rpc_op_handle.h	op compose node and update nodes.	7 years ago
scale_loss_grad_op_handle.cc	Revert the changes of VLOG	6 years ago
scale_loss_grad_op_handle.h	op compose node and update nodes.	7 years ago
scope_buffered_ssa_graph_executor.cc	AsyncExecutor (#14627)	6 years ago
scope_buffered_ssa_graph_executor.h	fix mac build of graph_executor	7 years ago
sequential_execution_pass.cc	Revert the changes of VLOG	6 years ago
sequential_execution_pass.h	move to pass	6 years ago
ssa_graph_executor.cc	fix	6 years ago
ssa_graph_executor.h	clean1	6 years ago
threaded_ssa_graph_executor.cc	Revert the changes of VLOG	6 years ago
threaded_ssa_graph_executor.h	Merge pull request #14251 from panyx0718/fix	6 years ago
var_handle.cc	add more logs and comments	6 years ago
var_handle.h	add more logs and comments	6 years ago
variable_visitor.cc	follow comments	7 years ago
variable_visitor.h	follow comments and clean code	7 years ago