137 Commits (a6beb96dd0235c236336f2db31df875b33db6635)

Author	SHA1	Message	Date
Cindy Cai	86ca31ab58	English API Docs Optimization Part 1 (#24536) ... * test=develop, test=document_fix * test=develop, test=document_fix Co-authored-by: swtkiwi <1208425345@qq.com>	6 years ago
Leo Chen	381492fca3	add try finally, test=develop (#24243)	6 years ago
guofei	2b896c1f6b	Support LoDTensorArray in fetch (#23645) ... * Support LoDTEnsorArray in fetch op test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop * Support LoDTensorArray in fetch test=develop	6 years ago
gfwm0502	d8ca66dae2	Modify documents of executor and randn and fix other errors (#23879) ... test=develop	6 years ago
Leo Chen	02b4e989b8	fix pruned_program_cache_key of Operator (#23594) ... * fix init_gflags with 'python -c', test=develop * fix pruned_program_cache_key of Operator, test=develop	6 years ago
guofei	cb1a2512dc	Make the place parameter default (#23094) ... * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop * Make the place parameter default test=develop	6 years ago
Chen Weihang	75bd350710	Implement StaticModelRunner to support dygraph fine-tune static graph pre-training model (#23171) ... * static model runner basic implement, test=develop * add run program op to execute loaded program, test=develop * refactor static model runner & run program op, test=develop * reset engine.cc to resolve conflict * adapt the change of dygraph double grad, test=develop * refactor impl to solve control flow error, test=develop * clear debug code, test=develop * fix ci str compatible error & checkout dygraph grad maker & add example, test=develop * hide api & add op test, test=develop * fix run program op test places error, test=develop * fix program by review comment, test=develop * delete change var desc name, test=develop * fix other program by review comment, test=develop * remove _static_graph_guard, test=develop * add selectedrows test, test=develop * remove desc parser, test=develop * fix detail program, test=develop * change socpe create & add test, test=develop	6 years ago
Leo Chen	a62599a888	[feature] prune program by feed and fetch_list automatically (#22474) ... * prune train program by fetch_list, test=develop * add unittest for prune, test=develop * fix pruned feed, test=develop * support ParallelExecutor and feed prune, test=develop * add comments, test=develop * update unittest, test=develop * update unittests, test=develop * remove debug code, test=develop * support cond in clone, test=develop * support cond in prune, test=develop * support multiple minimize, test=develop * support cache, test=develop * fix _copy_param_info_from, test=develop * support python2 str, test=develop * remove debug code, test=develop * fix bug of caching CompiledProgram, test=develop * fix multi_device issue, test=develop * tmp * support tuple in fetch_list and overriding use_prune, test=develop * dont use nonlocal in python2, test=develop * remove nonlocal, test=develop * code clean, test=develop * code clean, test=develop * feed list, test=develop * test adam, test=develop * follow comments, test=develop * reduce duplicate code, test=develop * update comments, test=develop	6 years ago
Leo Chen	8f2f977345	support feeding scalar when runing program , test=develop (#23214) ... * support feed_python_builtin, test=develop * add test, test=develop * support CompiledProgram, test=develop * support fluid.data, test=develop * fix ci problems, test=develop * follow comments, test=develop	6 years ago
Zeng Jinle	acfc9b8a70	Reader sequential and inference partial feed (#22699) ... * sequential reader stage 1, test=develop * fix ut, test=develop * fix iterable=False reset bug, add some logs and polish code, test=develop * inference feed partial data, test=develop * Turn on keep_order=True for test, test=develop * enhance ut to test more cases, test=develop * test commit for reverting * Revert "test commit for reverting", test=develop This reverts commit 80aef42ef52ba1ee79627d6f663a624ec4f12f58. * add ut of merged and unmerged results, test=develop * add more uts for coverages and add en doc of api, test=develop * follow comments, test=develop * change note style, test=develop	6 years ago
Zhen Wang	89cfa49156	Unmerged fetch list (#22635) ... * update ScopeBufferedSSAGraphExecutor&AsyncSSAGraphExecutor&ThreadedSSAGraphExecutor&FastThreadedSSAGraphExecutor&ParallelSSAGraphExecutor&ParallelExecutor for fetching unmerged results. * add the unit test for fetch_unmerged. * update ut for multi-card and multi-cpu. * add the error message and the user suggestion in FetchOpHandle. test=develop	6 years ago
tianshuo78520a	433cef03e5	fix typo word (#22784)	6 years ago
tianshuo78520a	d2ba91aad1	fix typo words (#22653)	6 years ago
123malin	00594c1c88	support dumping params/grads in transpiler mode (#22490)	6 years ago
tangwei12	82bc814a57	integrated HALF_ASYNC to communicator (#21869) ... * add half_async in the communicator * fix DistributedStrategy	6 years ago
Chen Weihang	8bed352278	Polish executor empty program hint message (#22167) ... * polish executor hint message, test=develop * fix punctuation error, test=develop	6 years ago
Huihuang Zheng	dd4361568e	Add ParallelExecutor Test for Cond API and Fix PE Checks Shape Bug (#22029)	6 years ago
hutuxian	c5aec2fe68	Paddlebox Related to Framework (#21586) ... * Add a single_process_multi_thread transpiler. * Add some UTs. * Fix some API description.	6 years ago
Dong Daxiang	691ced87c0	Refactor fetch handler (#21264) ... * fix fetch handler problem and refactor when a user define FetchHandler class, he or she should initialize a handler with variable dict. the key of a variable dict is a user defined name, the value of a variable dict is a Varaible generated from python API. For each fetching, a user should implement handler function in which fetched_result_dict will be available and the user can access the fetched value with user defined keys.	6 years ago
tangwei12	3b96e3d20a	fix FetchHandler (#20900) ... * bug fix, test=develop	7 years ago
Yiqun Liu	16e4d02675	Refine the cache of program, context and scope in executor. (#18483) ... * Refine the cache of program, context and scope in executor. test=develop * Refine the unittest test_executor_and_use_program_cache. * Add the test the PaddingRNN with use_program_cache=True. test=develop * Remove a check. test=develop * Refine the unittest to check whether it is correct when setting use_program_cache=True. test=develop	7 years ago
tangwei12	1d925440ca	fix fetch handler error with pslib (#20679) ... * fix fetch handler error with pslib * fix distributed lookup table op with 1 pserver	7 years ago
chengduo	f23eeeb460	update global scope doc (#20483) ... test=document_fix	7 years ago
xujiaqi01	408bd8b874	fix en doc and test=document_fix (#20441) ... * fix en doc train_from_dataset and infer_from_datatset	7 years ago
chengduo	4ba6d062f8	Fix fluid.data of executor (#20365) ... * fix fluid.data test=document_fix * fix doc batch dim test=document_fix	7 years ago
chengduo	d77e770353	Polish executor doc (#20244) ... * polish executor doc test=develop	7 years ago
Huihuang Zheng	e6d13ae5ae	Refine Error Message for New Data API (#20204) ... Refine error message for new data API	7 years ago
Youwei Song	340b2ba449	refine scope_guard, test=document_fix, test=develop (#20240)	7 years ago
tangwei12	c9139c3db3	trainer from dataset fetch targets (#19760) ... add executor.FetchHandler for train/infer from the dataset	7 years ago
Chen Weihang	b916335025	Paddle error message stack shaping and optimization (#19895) ... * shape and optimize paddle error message stack, test=develop * limit exception type & add unittest, test=develop * fix multi-platform problem, test=develop * fix related unnitest failed, test=develop * add doc & fix unittest errors, test=develop * fix function name error, test=develop * update tensor test exception msg compare, test=develop * remove unittest on win32, the dir format is different, test=develop * remove useless package, test=develop * add paddle enforce handler unittest, test=develop * add exception checkout, test=develop * fix coverage failed, test=develop * fix op registry test failed, test=develop * refactor whole pr, test=develop * remove test in CMakelist, test=develop * fix coverage, test=develop	7 years ago
Huihuang Zheng	88af4ab650	Add new data layer (#19916) ... The new "fluid.data" changes old "fluid.layers.data": 1. Add shape and dtype check. 2. Remove "append_batch_size" parameter. We won't offer this in the new data layer because other deep learning platforms don't have this kind of data layer pre-processing. It may confuse users. 3. Remove "stop gradient" parameter because the data layer doesn't do back-propagation TODO： Now data layer feeded by executor is checked, will we want to check the feed data of readers in the future?	7 years ago
xujiaqi01	cedc04775c	support change shuffle and train thread num (#19841) ... * support change shuffle thread num * support change train thread num * fix receive shuffle data of each channel * data norm stop gradient * add check thread_tensor type and root_tensor type when merge metric * remove sleep in shuffle, add config * add config of pslib client to client communication * fix xbox str * add data norm op testcase * add flush in trainer finalize	7 years ago
chengduo	1f68674483	refine executor bug info (#19887) ... test=develop	7 years ago
chengduo	e340df013e	Support feed single persistable variable to PE (#19417) ... * update executor feed	7 years ago
chengduo	4278518fb0	Update CompiledProgram (#18919) ... * use PE for compiler test=develop	7 years ago
Leo Chen	8f53735437	Fix memory overwriting of tensors returned by executor (#19030) ... * fix memory overlapping of fetch var (return of executor.run), test=develop * fix wrong usage of ParallelExecutor in op_test, test=develop * remove useless parameter and simplify code * avoid tensor destruct untimely, test=develop * add testcase independent of OpTest, test=develop	7 years ago
chengduo	01c7daade7	Add checking for the fetch_list of Executor.run (#18957) ... * update exe.run	7 years ago
Zeng Jinle	8008ab4e6b	Remove legacy C++ memory optimization codes (#18834) ... * remove legacy memory optimization codes, test=develop * follow huihuang's comments,test=develop * follow luotao's comments, test=develop	7 years ago
Zeng Jinle	d3003a1620	Feature/buffer_shared_inplace (#17911) ... * feature/buffer_shared_inplace, test=develop * refine code, test=develop * fix elementwise_add op cpu inplace and sum inplace bug, test=develop * add unittest and debug log, test=develop * fix parallel_executor scope bug, polish code, test=develop * fix sum op, activation op, single_in_place_inference bug, test=develop * remove kLocalExecScopeName, test=develop * fix unittest,test=develop * fix out_var first version bug, test=develop * follow comments,test=develop	7 years ago
chengduo	55baeceddb	Enhance execution error info (#18482) ... * enhance execution error info test=develop	7 years ago
hutuxian	6e0df3102e	Refactor for Pipeline Thread Check (#18459) ... move the thread-check code from train_from_dataset to a single function add UT for the thread check function	7 years ago
xsrobin	47e2ef38e9	add "import paddle.fluid as fluid" to examples lack of it	7 years ago
hutuxian	e42057cd1a	add ut for pipeline training (#18289)	7 years ago
jiaqi	3f8031e256	dataset (#17973) ... (1) use channel instead of vector/BlockingQueue in Dataset，to keep same with existing implementation, and make code more readable and flexible (dataset single output channel or multi output channel). one previous memory out of limit problem is cause by not release memory after training. (2) add Record because MultiSlotType costs too much memory (80B)，fix memory out of limit problem. (3) add Channel, Archive in paddle/fluid/framework (4) change dataset from shared_ptr to unique_ptr in pybind (5) move create/destroy readers from trainer to dataset (6) move shuffle from datafeed to dataset. dataset holds memory, datafeed is only for load data and feed data to network. (7) fix thread num bug of Dataset when filelist size < thread num (8) support set_queue_num in InMemoryDataset	7 years ago
hutuxian	cf15c3ff1e	fix errors in python3 (#18239) ... * fix relative import error in python3 * fix debug string info	7 years ago
hutuxian	969e6378b9	Pipeline Concurrency (#17402) ... Add Pipeline Concurrency Train Mode: - Cpp: pipeline_trainer & section_worker - Python: PipelineOptimizer - Add a new data_feed type: PrivateInstantDataFeed - Add a test demo of pipeline trainer and the test model is gnn - Do not support win32 now	7 years ago
guru4elephant	d52391094d	fix prepare context redundant code problem, optimize executor by cach… (#17743) ... * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * cache sub_scope, program, var when use_program_cache=True is set * make fetch_list runable with variables, add more unittest for use_program_cache	7 years ago
guru4elephant	326bf8291a	add Run Prepared Ctx (#17616) ... add Run Prepared Ctx, fix pybind problem	7 years ago
guru4elephant	7f8bc49d00	polish_executor_and_add_ctx_cache (#17536) ... * polish_executor_and_add_ctx_cache	7 years ago
jiaqi	66d51206b1	add save/load model, shrink table, cvm, config file & fix pull dense bug (#17118) ... * add save/load model, shrink table, cvm, config file & fix pull dense bug test=develop * fix global shuffle bug, fix pull dense bug, fix release memeory bug, fix shrink error add client flush, add get data size test=develop * fix global shuffle bug test=develop * fix global shuffle bug test=develop * fix code style test=develop * fix code style & modify pslib cmake test=develop * fix error of _role_maker test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix code style test=develop * fix windows compile error of fleet test=develop * fix global shuffle bug * add comment test=develop * update pslib.cmake test=develop * fix fill sparse bug test=develop * fix push sparse bug test=develop	7 years ago