158 Commits (adaec0073d02c0ea55bcabc4671ebfc8dbd3182c)

Author	SHA1	Message	Date
Chen Weihang	b916335025	Paddle error message stack shaping and optimization (#19895) ... * shape and optimize paddle error message stack, test=develop * limit exception type & add unittest, test=develop * fix multi-platform problem, test=develop * fix related unnitest failed, test=develop * add doc & fix unittest errors, test=develop * fix function name error, test=develop * update tensor test exception msg compare, test=develop * remove unittest on win32, the dir format is different, test=develop * remove useless package, test=develop * add paddle enforce handler unittest, test=develop * add exception checkout, test=develop * fix coverage failed, test=develop * fix op registry test failed, test=develop * refactor whole pr, test=develop * remove test in CMakelist, test=develop * fix coverage, test=develop	7 years ago
Huihuang Zheng	88af4ab650	Add new data layer (#19916) ... The new "fluid.data" changes old "fluid.layers.data": 1. Add shape and dtype check. 2. Remove "append_batch_size" parameter. We won't offer this in the new data layer because other deep learning platforms don't have this kind of data layer pre-processing. It may confuse users. 3. Remove "stop gradient" parameter because the data layer doesn't do back-propagation TODO： Now data layer feeded by executor is checked, will we want to check the feed data of readers in the future?	7 years ago
xujiaqi01	cedc04775c	support change shuffle and train thread num (#19841) ... * support change shuffle thread num * support change train thread num * fix receive shuffle data of each channel * data norm stop gradient * add check thread_tensor type and root_tensor type when merge metric * remove sleep in shuffle, add config * add config of pslib client to client communication * fix xbox str * add data norm op testcase * add flush in trainer finalize	7 years ago
chengduo	1f68674483	refine executor bug info (#19887) ... test=develop	7 years ago
chengduo	e340df013e	Support feed single persistable variable to PE (#19417) ... * update executor feed	7 years ago
chengduo	4278518fb0	Update CompiledProgram (#18919) ... * use PE for compiler test=develop	7 years ago
Leo Chen	8f53735437	Fix memory overwriting of tensors returned by executor (#19030) ... * fix memory overlapping of fetch var (return of executor.run), test=develop * fix wrong usage of ParallelExecutor in op_test, test=develop * remove useless parameter and simplify code * avoid tensor destruct untimely, test=develop * add testcase independent of OpTest, test=develop	7 years ago
chengduo	01c7daade7	Add checking for the fetch_list of Executor.run (#18957) ... * update exe.run	7 years ago
Zeng Jinle	8008ab4e6b	Remove legacy C++ memory optimization codes (#18834) ... * remove legacy memory optimization codes, test=develop * follow huihuang's comments,test=develop * follow luotao's comments, test=develop	7 years ago
Zeng Jinle	d3003a1620	Feature/buffer_shared_inplace (#17911) ... * feature/buffer_shared_inplace, test=develop * refine code, test=develop * fix elementwise_add op cpu inplace and sum inplace bug, test=develop * add unittest and debug log, test=develop * fix parallel_executor scope bug, polish code, test=develop * fix sum op, activation op, single_in_place_inference bug, test=develop * remove kLocalExecScopeName, test=develop * fix unittest,test=develop * fix out_var first version bug, test=develop * follow comments,test=develop	7 years ago
chengduo	55baeceddb	Enhance execution error info (#18482) ... * enhance execution error info test=develop	7 years ago
hutuxian	6e0df3102e	Refactor for Pipeline Thread Check (#18459) ... move the thread-check code from train_from_dataset to a single function add UT for the thread check function	7 years ago
xsrobin	47e2ef38e9	add "import paddle.fluid as fluid" to examples lack of it	7 years ago
hutuxian	e42057cd1a	add ut for pipeline training (#18289)	7 years ago
jiaqi	3f8031e256	dataset (#17973) ... (1) use channel instead of vector/BlockingQueue in Dataset，to keep same with existing implementation, and make code more readable and flexible (dataset single output channel or multi output channel). one previous memory out of limit problem is cause by not release memory after training. (2) add Record because MultiSlotType costs too much memory (80B)，fix memory out of limit problem. (3) add Channel, Archive in paddle/fluid/framework (4) change dataset from shared_ptr to unique_ptr in pybind (5) move create/destroy readers from trainer to dataset (6) move shuffle from datafeed to dataset. dataset holds memory, datafeed is only for load data and feed data to network. (7) fix thread num bug of Dataset when filelist size < thread num (8) support set_queue_num in InMemoryDataset	7 years ago
hutuxian	cf15c3ff1e	fix errors in python3 (#18239) ... * fix relative import error in python3 * fix debug string info	7 years ago
hutuxian	969e6378b9	Pipeline Concurrency (#17402) ... Add Pipeline Concurrency Train Mode: - Cpp: pipeline_trainer & section_worker - Python: PipelineOptimizer - Add a new data_feed type: PrivateInstantDataFeed - Add a test demo of pipeline trainer and the test model is gnn - Do not support win32 now	7 years ago
guru4elephant	d52391094d	fix prepare context redundant code problem, optimize executor by cach… (#17743) ... * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * cache sub_scope, program, var when use_program_cache=True is set * make fetch_list runable with variables, add more unittest for use_program_cache	7 years ago
guru4elephant	326bf8291a	add Run Prepared Ctx (#17616) ... add Run Prepared Ctx, fix pybind problem	7 years ago
guru4elephant	7f8bc49d00	polish_executor_and_add_ctx_cache (#17536) ... * polish_executor_and_add_ctx_cache	7 years ago
jiaqi	66d51206b1	add save/load model, shrink table, cvm, config file & fix pull dense bug (#17118) ... * add save/load model, shrink table, cvm, config file & fix pull dense bug test=develop * fix global shuffle bug, fix pull dense bug, fix release memeory bug, fix shrink error add client flush, add get data size test=develop * fix global shuffle bug test=develop * fix global shuffle bug test=develop * fix code style test=develop * fix code style & modify pslib cmake test=develop * fix error of _role_maker test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix windows compile error of fleet test=develop * fix global shuffle bug * add comment test=develop * update pslib.cmake test=develop * fix fill sparse bug test=develop * fix push sparse bug test=develop	7 years ago
qingqing01	e32c9888f5	Double backward of conv2d. (#17211) ... * Add conv2d_grad_grad_op * Extracte the cuDNN conv algo searching code in conv_cudnn_helper.h. - Now use it in conv2d_grad_grad. - Will simply the searching code in conv2d and conv2d_grad in next PR. * Enhance and fix bug in unit testing of gradient_checker. * Support to fetch empty variables，return None in Python.	7 years ago
guru4elephant	5d6a1fcf16	fix infer_from_dataset and train_from_dataset (#17243) ... * fix train_from_dataset and infer_from_dataset example * add inductive dim for data_reader, example: shape=[-1, 1], then -1 will be inducted through run-time reading of number of elements	7 years ago
lujun	e388a1fb66	Repair api example (#17221) ... Fix the following API examples: paddle.fluid.scope_guard paddle.fluid.backward.append_backward paddle.fluid.cpu_places paddle.fluid.cuda_pinned_places paddle.fluid.cuda_places paddle.fluid.in_dygraph_mode paddle.fluid.CUDAPlace paddle.fluid.CPUPlace paddle.fluid.CUDAPinnedPlace	7 years ago
chengduo	8f534696b7	Polish Executor and Compiler doc (#17262) ... * polish doc test=develop * updata parallel executor doc test=develop * update API.spec test=develop * polish code test=develop	7 years ago
Zeng Jinle	1202d3fc74	Refine model gpu memory (#16993) ... * speedup gc and inplace softmax_with_cross_entropy_grad test=develop * refine models gpu mem Merge skip vars and warning messages of mem opt remove relu mem opt test=develop * follow comments test=develop	7 years ago
dongdaxiang	05464e7c5c	add gpu training for Executor.train_from_dataset ... test=develop	7 years ago
xjqbest	5e5139283b	fix runtime error ... test=develop	7 years ago
xjqbest	271b7147cc	fix dataset bug ... test=develop	7 years ago
dongdaxiang	a53c8cd5a7	fix infer_from_dataset docs and abastract class problem ... test=develop	7 years ago
dongdaxiang	3829eac27b	fix API spec about infer_from_dataset ... test=develop	7 years ago
dongdaxiang	87027a2eef	fix API.spec problem and executor's docstring ... test=develop	7 years ago
dongdaxiang	ade9337486	fix API.spec ... test=develop	7 years ago
dongdaxiang	60b7bf6fa6	add infer_from_dataset for inference	7 years ago
dongdaxiang	8e14d8f900	add data_generator package into setup.py	7 years ago
dongdaxiang	d87ba58c14	refine document of python API, make device_worker and trainer's API private ... test=develop	7 years ago
dongdaxiang	b95b80bc76	add doc string for executor and update API.spec ... test=develop	7 years ago
dongdaxiang	6bf796df14	refine print fetch list	7 years ago
xjqbest	1ec8fab7c8	remove error line	7 years ago
xjqbest	b7940c2918	fix bug of gen_worker_desc and set_filelist, add some doc	7 years ago
dongdaxiang	68d7bf3de5	add fetch var function ... test=develop	7 years ago
dongdaxiang	9419de521f	add IO percent for multi_trainer	7 years ago
dongdaxiang	6af697adb0	add trainfileswithprofiler for downpour worker	7 years ago
dongdaxiang	b7a202aa38	add distributed optimizer factory	7 years ago
dongdaxiang	3641a78b01	add incubate for unified API	7 years ago
dongdaxiang	e657c127a8	hide opt_info in distirbuted optimizer	7 years ago
dongdaxiang	ff87698a44	refactor downpour optimization	7 years ago
dongdaxiang	b66f0074b6	fix data reading bugs in api, add VLOG(3) log for setup	7 years ago
dongdaxiang	b415ec27e8	make Dataset* as an argument	7 years ago
sneaxiy	2f54d9f995	Merge develop ... test=develop	7 years ago
Zeng Jinle	f8df9eb32e	fix api doc (#16201) ... test=develop	7 years ago
sneaxiy	ac0e0f5181	merge develop ... test=develop	7 years ago
sneaxiy	b80d76f784	merge develop	7 years ago
sneaxiy	732fa00eaf	disable gc in recurrent_op currently ... test=develop	7 years ago
sneaxiy	7c4303bc4a	Fix doc ... test=develop	7 years ago
sneaxiy	69b1ebdfa5	merge develop ... test=develop	7 years ago
Xin Pan	8b1672fe76	follow comments ... test=develop	7 years ago
Xin Pan	26e32e095a	allow compiler to use graph ... test=develop	7 years ago
sneaxiy	7160cb0f32	decoupled reader ... test=develop	7 years ago
sneaxiy	7e399b0628	rename ... test=develop	7 years ago
sneaxiy	f85245b409	test=develop	7 years ago
Xin Pan	1d31a0e10c	resolve conflicts ... test=develop	7 years ago
Xin Pan	1888337a49	tweak the executor implementation to better match origin behavior ... test=develop	7 years ago
flame	d60751fb71	add python inference api (#15248) ... add python inference api	7 years ago
Xin Pan	abdd9411b4	fix ... test=develop	7 years ago
Xin Pan	c4b09a713f	polish ... test=develop	7 years ago
Xin Pan	5f0a0286e0	add doc ... test=develop	7 years ago
Xin Pan	8ae9094e07	polish and resolve conflicts ... test=develop	7 years ago
Xin Pan	beaae61a16	polish ... test=develop	7 years ago
Xin Pan	5e928e579a	try unify Executor and ParallelExecutor ... test=develop	7 years ago
sneaxiy	3e917a934a	add scope_pool ... add module cleanup test=develop	7 years ago
Tink_Y	6d04a9cf47	fix api format and example (#14686) ... * fix api format and examples test=develop * Update executor.py test=develop * Update nn.py * Update nn.py test=develop * Update nn.py test=develop	7 years ago
Wang Guibao	41e19eb431	AsyncExecutor (#14627) ... * AsyncExecutor: C++ side * Google naming conventions * Rename MultiExecutor to AsyncExecutor * pybind with async_executor * Naming convention * remove some flags and unused code * add refactored file of async_executor and data_feed * clear async executor interface and add data feed factory * split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto * Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch * refine async_executor_refactor.cc * add some files about datafeed * Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. * Interface rework * add MultiSlotDataFeed * Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side * update data_feed for add MultiSlotDataFeed * update datafeed and async_executor to run bow_net demo * fix bug that finish_set_filelist failed in multithread * delete finish_binding_memory_(flag), because it can not be marked under the current interface * Fix bug * update async_executor.py for support set_use_slots * update async_executor.py for support set_use_slots and set set_dense_slots * fix bug that when the number of files is less than the number of threads, it will fetch nan * remove redundant code, and make executor exit when set a illegal queue size * add batch_size check * add MultiSlotDesc * Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. * add some checkpoint in DataFeedDesc * add CheckFile function in MultiSlotDataFeed * update something error info * fix deaded lock bug * Fix fetch variable * Merge error * fix code style in async_executor * using one lock blocking queue replace two lock blocking queue because of some bugs * update code style * add utest for data_feed * Fix fetch var * update utest for data_feed for multithread * update SetFileList info * fix bug in utest of data_feed * Add comments for python * Add comments for python code * Fix pybind.cc with new pybind11 version * add note for DataFeedDesc's set_use_slots function * Add save_model * update data_feed_test for multi-type * add comment for executor_thread_worker * Remove unused code * update data_feed_test for generate test data file * removed unnecessary interfaces and add comments * c++ style check * update data_feed.cc * AsyncExecutor: C++ side Google naming conventions Rename MultiExecutor to AsyncExecutor pybind with async_executor Naming convention remove some flags and unused code add refactored file of async_executor and data_feed clear async executor interface and add data feed factory split async executor into executor_thread_worker and async_executor, refactor pybind, add datafeed and corresponding proto Fix async_executor interfaces: 1) Remove all protobufs; 2) Stop after each epoch refine async_executor_refactor.cc add some files about datafeed Revert "add some files about datafeed" This reverts commit 8ee8133ab841196925a2812b76f18d2812a6701d. add MultiSlotDataFeed Interface rework Creating DataFeedDesc from .proto file, then manipulate it (add/del fields etc) from python side update datafeed and async_executor to run bow_net demo update async_executor.py for support set_use_slots Fix bug update async_executor.py for support set_use_slots and set set_dense_slots fix bug that when the number of files is less than the number of threads, it will fetch nan remove redundant code, and make executor exit when set a illegal queue size add MultiSlotDesc Revert "add MultiSlotDesc" This reverts commit 2e72ebfad364ed6b5dcc75f38ffb2a1fdec83d8e. add some checkpoint in DataFeedDesc Fix fetch variable fix code style in async_executor Fix fetch var add utest for data_feed Add comments for python update utest for data_feed for multithread fix bug in utest of data_feed Add comments for python code Fix pybind.cc with new pybind11 version add note for DataFeedDesc's set_use_slots function update data_feed_test for multi-type Add save_model update data_feed_test for generate test data file removed unnecessary interfaces and add comments add comment for executor_thread_worker Remove unused code update data_feed.cc c++ style check * commit for code style * commit for code style * commit for code style * commit for code style * Comment away __init__ in async_executor.py * clang-format fix test=develop * use PADDLE_THROW instead of exit(-1); use unique_ptr to manage scope var in data_feed_test.cc * commit for update code style * commit for update code style * Add async_executor demo; Remove some methods test=develop * commit for update code style * commit for update code style * commit for update code style * update API.spec * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * AsyncExecutor test=develop * Fix API.spec test=develop * Fix API.spec test=develop * Fix windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * FIx windows build error test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix Windows Build test=develop * Fix code style test=develop * Fix code style test=develop * update datafeed * Fix code style test=develop * update data_feed_test for test Tensor test=develop * Fix code style test=develop * Fix windows build failure test=develop * Fix code style and windows build failure test=develop * Fix PYTHON3.5 build failure test=develop * AsyncExecutor API test=develop	7 years ago
minqiyang	99d3f08920	Add print_function for all python files	8 years ago
minqiyang	1800fef142	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into port_pybind11	8 years ago
Wu Yi	8b77448d5f	hide misc APIs (#12540) ... * hide misc APIs * update * fix transformer test * update API.spec	8 years ago
minqiyang	fae5c1f514	Fix the input of executor	8 years ago
minqiyang	47561c34b0	Fix python2 CI issues	8 years ago
minqiyang	a3539845f2	Polish python code style	8 years ago
minqiyang	3ec6d60c34	Fix write bytes in dataset download	8 years ago
minqiyang	1f618c4ff9	Fix the overfix of 2to3 in xrange	8 years ago
minqiyang	7f794ea563	Replace the overfix of 2to3 with six.string_types	8 years ago
minqiyang	559d36328c	Apply 2to3 to current paddle main python code	8 years ago
Yancey1989	efd5a84986	update executor interface	8 years ago
Yancey1989	7570d8e77c	add rpc complete interface	8 years ago
Wu Yi	db67d60e31	Remove block api (#12107) ... * remove block api * remove clone_variable * hide block inner apis * update * fix tests	8 years ago
Yancey1989	1366832a41	add dist pass barrier	8 years ago
chengduo	a64844ad00	enable PE return numpy (#11704)	8 years ago
dzhwinter	15130fc8cb	"non-layer add doc for executor module" (#11602) ... * "add doc for exec" * "add more changes" * "fix based on preview" * "chagne code format"	8 years ago
Yu Yang	b94f7848c4	Merge pull request #11531 from reyoung/feature/non_layer_api_doc ... Polish Non-Layer API	8 years ago
chengduoZH	10cee7ed1f	Add doc of fetch var	8 years ago
yuyang18	ea44157b09	Add APIs	8 years ago
dzhwinter	f7c96f079b	Big data op_test benchmark, for checking output consistent in different runs. (#10646) ... * "init benchmark ops" * "untrack outputs" * "delete some usused code" * "benchmark" * "fix ci" * "fix op test" * "fix uint16 missing" * "fix ci" * "follow comments" * "fix ci" * "follow comments" * "conficts. merge develop branch" * repick * "merge develop branch"	8 years ago
dzhwinter	62c51e44d2	"add float64 tests" (#10450) ... * "add float64 tests" * "fix based comment" * "fixed based comment"	8 years ago
Yibing Liu	129b913cff	Fix the bug in executor.fetch_var	8 years ago
dzhwinter	6cfc0c1497	"polish code" (#9318) ... * "polish code" * "fix ci" * "fix ci" * "done"	8 years ago
Qiao Longfei	37a272e670	add executor.prepare (#9022) ... optimize executor.run	8 years ago
qiaolongfei	1ee77841be	add get_program_cache_key function	8 years ago
qiaolongfei	b63901f523	optimize program_cache_key, delete cache when use_program_cache is false	8 years ago
qiaolongfei	82a8e080cb	update comment of executor.run	8 years ago