36 Commits (61d19a8e1cf760cf805b06c6d35b85e2dd57adae)

Author	SHA1	Message	Date
Chen Weihang	8da0cd537a	Add examples for error message writing specification - NotFound, OutOfRange, AlreadyExists, PermissionDenied (#21134) ... * add examples for error msg spec, test=develop * change ENFORCE to ENFORCE_**, test=develop * add more already exists examples, test=develop	5 years ago
Chen Weihang	7e5f74b825	Add examples for error message writing specification - InvalidArgument (#21132) ... * add examples for error msg spec, test=develop * change ENFORCE to ENFORCE_**, test=develop * fix error, test=develop	5 years ago
hutuxian	969e6378b9	Pipeline Concurrency (#17402) ... Add Pipeline Concurrency Train Mode: - Cpp: pipeline_trainer & section_worker - Python: PipelineOptimizer - Add a new data_feed type: PrivateInstantDataFeed - Add a test demo of pipeline trainer and the test model is gnn - Do not support win32 now	6 years ago
dongdaxiang	98dda08a85	fix pull sparse slow problem ... test=develop	6 years ago
dongdaxiang	a0b59773af	fix code style	6 years ago
xujiaqi01	a5b1a0e12b	support multi dataset && add init model && fix bug	6 years ago
dongdaxiang	f6c9232a3d	fix dataset float32 type problem	6 years ago
xjqbest	dd67ad08a2	modify c++ and python dataset related code & fix bug	6 years ago
dongdaxiang	2486389793	add RunFromDataset in executor	6 years ago
xjqbest	824b84d185	add DataSet and InMemoryDataFeed, support load data into memory and shuffle data	6 years ago
dongdaxiang	be757096da	add pybind for fleet	6 years ago
dongdaxiang	1fe54416c9	move fs.cc and shell.cc into paddle/fluid/framework/io ... test=develop	6 years ago
dongdaxiang	afaf937010	add fs_local_open example	6 years ago
dongdaxiang	cf1360643f	add printer for fetch variable	6 years ago
dongdaxiang	39014b9f9f	fix class register problem	6 years ago
dongdaxiang	c165012031	refine device_worker and trainer code ... test=develop	6 years ago
dongdaxiang	24a8001142	make -DWITH_PSLIB=ON compilable	6 years ago
dongdaxiang	67b1d6d721	add dist_multi_trainer for distributed training, add trainer_factory and device_worker_factory so that we can easily extend new training mode, add pull dense worker which is a singleton for parameter fetching	6 years ago
dongdaxiang	855bf579d2	add dist_multi_trainer for distributed training, add trainer_factory and device_worker_factory so that we can easily extend new training mode, add pull dense worker which is a singleton for parameter fetching	6 years ago
chengduo	5a03b515ae	fix potential bug in async_executor (#15707) ... test=develop	6 years ago
dongdaxiang	2dee8f6cd5	add TrainFilesWithTimer in async_executor	6 years ago
heqiaozhi	09d669ba40	fix static_cast to const_cast	6 years ago
heqiaozhi	bd1c1724aa	add ps_instance doc	6 years ago
dongdaxiang	c9b799896e	fix tag in async_executor	6 years ago
heqiaozhi	f81957a753	refine cmake for pslib & pre_define	6 years ago
heqiaozhi	2912d5311b	fix code style bug & change pslib.cmake & change Cmakelist adapt pslib	6 years ago
dongdaxiang	c71279bc69	refine code style for async_executor.h and async_executor.cc	6 years ago
heqiaozhi	7bd16e3afa	fix some bug & add log	6 years ago
heqiaozhi	10ed9e0a6e	download & run & instance	6 years ago
heqiaozhi	016a068775	stop server	6 years ago
heqiaozhi	8e3fe2d735	add skip op	6 years ago
heqiaozhi	575ae7c6c3	refine pslib inferface & fix some bugs	6 years ago
heqiaozhi	d3ca359e44	config init & adapt to interface	6 years ago
heqiaozhi	a77fa67bbd	async_thread_trainer & libmct & pslib.cmake	6 years ago
heqiaozhi	3c239cd640	pslib	6 years ago
Wang Guibao	41e19eb431	AsyncExecutor (#14627) ... * AsyncExecutor: C++ side * Google naming conventions * Rename MultiExecutor to AsyncExecutor * pybind with async_executor * Naming convention * remove some flags and unused code * add refactored file of async_executor and data_feed * clear async executor interface and add data feed factory * split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto * Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch * refine async_executor_refactor.cc * add some files about datafeed * Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. * Interface rework * add MultiSlotDataFeed * Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side * update data_feed for add MultiSlotDataFeed * update datafeed and async_executor to run bow_net demo * fix bug that finish_set_filelist failed in multithread * delete finish_binding_memory_(flag), because it can not be marked under the current interface * Fix bug * update async_executor.py for support set_use_slots * update async_executor.py for support set_use_slots and set set_dense_slots * fix bug that when the number of files is less than the number of threads, it will fetch nan * remove redundant code, and make executor exit when set a illegal queue size * add batch_size check * add MultiSlotDesc * Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. * add some checkpoint in DataFeedDesc * add CheckFile function in MultiSlotDataFeed * update something error info * fix deaded lock bug * Fix fetch variable * Merge error * fix code style in async_executor * using one lock blocking queue replace two lock blocking queue because of some bugs * update code style * add utest for data_feed * Fix fetch var * update utest for data_feed for multithread * update SetFileList info * fix bug in utest of data_feed * Add comments for python * Add comments for python code * Fix pybind.cc with new pybind11 version * add note for DataFeedDesc's set_use_slots function * Add save_model * update data_feed_test for multi-type * add comment for executor_thread_worker * Remove unused code * update data_feed_test for generate test data file * removed unnecessary interfaces and add comments * c++ style check * update data_feed.cc * AsyncExecutor: C++ side Google naming conventions Rename MultiExecutor to AsyncExecutor pybind with async_executor Naming convention remove some flags and unused code add refactored file of async_executor and data_feed clear async executor interface and add data feed factory split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch refine async_executor_refactor.cc add some files about datafeed Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. add MultiSlotDataFeed Interface rework Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side update datafeed and async_executor to run bow_net demo update async_executor.py for support set_use_slots Fix bug update async_executor.py for support set_use_slots and set set_dense_slots fix bug that when the number of files is less than the number of threads, it will fetch nan remove redundant code, and make executor exit when set a illegal queue size add MultiSlotDesc Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. add some checkpoint in DataFeedDesc Fix fetch variable fix code style in async_executor Fix fetch var add utest for data_feed Add comments for python update utest for data_feed for multithread fix bug in utest of data_feed Add comments for python code Fix pybind.cc with new pybind11 version add note for DataFeedDesc's set_use_slots function update data_feed_test for multi-type Add save_model update data_feed_test for generate test data file removed unnecessary interfaces and add comments add comment for executor_thread_worker Remove unused code update data_feed.cc c++ style check * commit for code style * commit for code style * commit for code style * commit for code style * Comment away __init__ in async_executor.py * clang-format fix test=develop * use PADDLE_THROW instead of exit(-1); use unique_ptr to manage scope var in data_feed_test.cc * commit for update code style * commit for update code style * Add async_executor demo; Remove some methods test=develop * commit for update code style * commit for update code style * commit for update code style * update API.spec * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * Fix API.spec test=develop * Fix API.spec test=develop * Fix windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix code style test=develop * Fix code style test=develop * update datafeed * Fix code style test=develop * update data_feed_test for test Tensor test=develop * Fix code style test=develop * Fix windows build failure test=develop * Fix code style and windows build failure test=develop * Fix PYTHON3.5 build failure test=develop * AsyncExecutor API test=develop	6 years ago