Paddle

Commit Graph

Author	SHA1	Message	Date
Yi Liu	a873fa84ce	supports collective training with programs (#18392 ) 1. Since allreduce op has 4 reduce types, We split these four reduce types into four ops 2. We also refined the collective op code, e.g. we separated the collective op kernel into CPUKernel and CUDAKernel, and remove the device specified DeviceContext parameter in template as we already knew the target DeviceContext 3. We remove the newly added Collective op role to reduce the complexity of program and graph analysis	7 years ago
guru4elephant	357311fdb7	make fleet support mpi job submit directly (#18441 ) make fleet support mpi job submit directly.	7 years ago
chengduo	e0d8c6ac68	Add find_no_grad_vars in backward.py (#17942 ) * add not_been_used_vars to no_grad_set test=develop	7 years ago
LielinJiang	449c7a9f98	Make roi_perspective_transform op return mask and transform matrix (#18371 ) * modify roi_perspective_transform_op to output mask and transform matrix * modify comment * modify comment * modify API.spec * update API.spec * remove no use header, test=develop * resolve conflict	7 years ago
tensor-tang	a3bc804f5f	fix mac ci random fail (#18430 ) * fix mac ci random fail * use platform instead	7 years ago
xiaoting	dd3f9d19cf	replace mnist dataset url, test=develop (#18429 ) replace mnist dataset url	7 years ago
xsrobin	47e2ef38e9	add "import paddle.fluid as fluid" to examples lack of it	7 years ago
hutuxian	8a39e5c110	update api format (#18413 ) * update api format test=develop * update API.spec test=develop	7 years ago
tensor-tang	ce7a024c6d	fix py-cpuinfo mac random fail (#18383 ) * fix py-cpuinfo mac random fail * differentiate version on windows	7 years ago
Jie Fang	2b4ef509ea	init custom black white list (#18377 ) test=develop	7 years ago
guru4elephant	e83f902b98	add MultiSlotStringDataGenerator for speedup of string based user inp… (#18390 ) * add MultiSlotStringDataGenerator for speedup of string based user input data	7 years ago
Jiabin Yang	43f64a177e	Fix/program doc (#17908 ) * test=develop, add some comments for Program.clone * test=develop, add API.spec * test=develop, refine comments * refine Program doc and clone doc * test=develop, refine doc	7 years ago
chengduo	871cc15e6a	Add is_compiled_with_cuda (#18356 ) * add cuda_is_available test=develop * Fix api.spec test=develop * fix api doc test=develop	7 years ago
Wojciech Uss	8ed819d8e4	Call the test_slim_int8_* tests through absolute path (#18386 ) test=develop	7 years ago
lujun	fd6631ef2f	Fix dygraph show style (#18297 ) Fix dygraph show style for FluidDoc.	7 years ago
翟飞跃	19da59ed3f	Remove all the code, API and doc of MKL-DNN INT8v1 (#18347 )	7 years ago
chengduo	8ed33bf91f	Fix Bug-prone code of PE (#18354 ) * update pe reduce config test=develop * drop the local_exe_scopes of the previous parallel_executor test=develop	7 years ago
tangwei12	999d9a59a5	fix communicator with pyreader (#18350 ) * add is_runnning in communicator, test=develop	7 years ago
kh2se2013	27fb9cad65	add WITH_COVERAGE option, default OFF (#17872 ) * add WITH_COVERAGE option, default OFF test=develop * add coverage for python sdk test=develop * fix code style * fix COVERAGE_FILE path test=develop * remove coverage package test=develop * test = develop, run coverage as module	7 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175 ) * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	7 years ago
qingqing01	9047ac687e	Simplify multi_box_head API in detection.py and remove assign op. (#18310 ) * Simplify multi_box_head API in detection.py and remove assign op.	7 years ago
hutuxian	e42057cd1a	add ut for pipeline training (#18289 )	7 years ago
Jiabin Yang	bd61d89925	test=develop, recover ocr ut on dygraph (#18166 )	7 years ago
Yibing Liu	23941e43ec	Update lamb optimizer (#18333 ) * Update lamb optimizer test=develop, test=document_preview * Regenerate api spec test=develop, test=document_preview	7 years ago
whs	1bdfd2eb85	Fix checkpoint of Light-NAS (#18330 ) Socket can't be pickled. test=develop	7 years ago
Jiabin Yang	79bcdbbf1a	test=develop, disable basic gru related ut (#18329 )	7 years ago
Jiabin Yang	831a3e62ca	Add install check for multigpu (#18323 ) * test=develop, add_install_check_for_multigpu * test=develop, refine code to use cuda_devices	7 years ago
Zeng Jinle	f88e07a0d3	fix lod_tensor.py grammar error, test=develop (#18308 )	7 years ago
Hongyu Liu	df2eee71d8	Sequence mask support tensor (#18249 ) * sequnce mask support max length tensor input; test=develop * add rnn_impl.py; test=develop * add basic gru lstm unittest; test=develop * fix api spec; test=develop * fix sequence_mask op bug; test=develop test=document_preview * change +-x to elmentwise_op; test=develop add mkl flag; test=develop * fix rnn impl bug; test=develop * update api spec; test=develop * fix doc bug; test=develop * fix lstm bugs; test=develop	7 years ago
Jiabin Yang	9cb799be50	test=develop, Revert "Add multi gpu install check" (#18313 ) * Revert "Add multi gpu install check (#18229)" This reverts commit `61ed06b29a`. * test=develop, start ci	7 years ago
Qiao Longfei	0e08e91c18	optimize communicator merge sparse gradient test=develop (#18159 ) * optimize communicator merge sparse gradient test=develop * revert multithread selected rows merge add test=develop * follow comment test=develop	7 years ago
Jie Fang	172c2facef	init black/white lists (#17847 ) test=develop	7 years ago
chengduo	e06c69c788	Fix default value of fluid.memory_optimize (#18295 ) * fix default value of fluid.memory_optimize test=develop * fix api.spec test=develop	7 years ago
Zhaolong Xing	6978b2e48e	fix split and sampled softmax (#18280 ) test=develop	7 years ago
hutuxian	6ed73830c2	add api desc for pipeline training (#18293 )	7 years ago
liuwei1031	a736c03b10	improve doc of lstm, sequence_enumerate, softmax_with_cross_entropy, space_to_depth APIs (#18261 ) * improve doc of lstm, sequence_enumerate, softmax_with_cross_entropy, space_to_depth APIs, test=develop * update API.spec, test=develop	7 years ago
chengduo	d54e13bbec	add random seed for recurrent op test (#18274 ) test=develop	7 years ago
liuwei1031	4151d90c16	improve the hint message of memory optimize, test=develop (#18260 )	7 years ago
guru4elephant	ff399fd720	fix paddle cloud role maker bug (#18269 ) * fix paddle cloud role maker bug	7 years ago
Yibing Liu	412951d7d2	Fix ema's example & fp16 update (#18273 ) test=develop, test=document_preview	7 years ago
flame	fdf798f95a	fix double buffer example (#18169 ) test=develop test=document_preview	7 years ago
Bai Yifan	23b8b18e56	fix api doc example, test=develop (#18266 )	7 years ago
pkpk	cd9d57f5e0	fix a bug in examples of metrics.Acc	7 years ago
tensor-tang	68da8b2a01	refine core cmake warning and print more info (#18248 ) * refine core cmake warning and print more info test=develop * fix comments test=develop	7 years ago
zhaoyuchen2018	32c95f1752	Add StaticRNN.output code example (#18251 ) refine StaticRNN api doc test=develop test=document_preview	7 years ago
xiaoting	2f0d68261c	fix yolo_box example,test=develop (#18247 )	7 years ago
songhao	6b3d96254d	fix some bug when merge sparse embedding parameters, test=develop (#18223 ) 1. fix the bug that out_put_var in SaveSelectedRows would be empty string 2. use merge_sparse_lookup_table to replace sum op for load_persistables_for_inference 3. fix the bug in _clone_var_in_block_ when the var is SELECTED_ROWS.	7 years ago
jiaqi	3f8031e256	dataset (#17973 ) (1) use channel instead of vector/BlockingQueue in Dataset，to keep same with existing implementation, and make code more readable and flexible (dataset single output channel or multi output channel). one previous memory out of limit problem is cause by not release memory after training. (2) add Record because MultiSlotType costs too much memory (80B)，fix memory out of limit problem. (3) add Channel, Archive in paddle/fluid/framework (4) change dataset from shared_ptr to unique_ptr in pybind (5) move create/destroy readers from trainer to dataset (6) move shuffle from datafeed to dataset. dataset holds memory, datafeed is only for load data and feed data to network. (7) fix thread num bug of Dataset when filelist size < thread num (8) support set_queue_num in InMemoryDataset	7 years ago
liuwei1031	5d54ed4a84	improve the doc of DataFeeder and default_main_program (#18241 ) * improve the doc of DataFeeder and default_main_program * update API.spec, test=develop	7 years ago
AIFollowers	4f3acb392f	fix BilinearInitializer doc (#18242 )	7 years ago
songhao	432fda51aa	fix bug in Class MultiSlotDataGenerator's function _gen_str, test=develop (#18222 )	7 years ago
Jiabin Yang	61ed06b29a	Add multi gpu install check (#18229 ) * test=develop, add add_multi_gpu_install_check * test=develop, refine warning doc * test=develop, refine warning doc * test=develop, refine warning doc * test=develop, support multi cpu * test=develop, find right num of cuda device * test=develop, find right num of cuda device * test=develop, fix multigpu processing and fix type bug in dygraph * test=develop, fix multigpu processing and fix type bug in dygraph	7 years ago
xiaoting	b58bb80248	set src_idx > 0 for bilinear_interp_op (#18238 ) * set src_idx > 0, test=develop * add unittest and cu, test=develop	7 years ago
guru4elephant	7d76e34ec2	add more print function for timeout issue, make timeout value larger (#18219 ) * add more print function for timeout issue, make timeout value larger	7 years ago
hutuxian	cf15c3ff1e	fix errors in python3 (#18239 ) * fix relative import error in python3 * fix debug string info	7 years ago
wopeizl	222c9fe57e	fix doc for LarsMomentumOptimizer test=develop (#18208 )	7 years ago
Shuai Yuan	9a32dad811	[DOC] Fix comment code of API create_py_reader_by_data (#18193 ) * [DOC] Fix comment code of API create_py_reader_by_data. test=develop, test=document_preview * Fix code style of API comment. test=develop,test=document_preview Fix code style of API comment. test=develop,test=document_preview * update api spec of api create_py_reader_by_data * remove default config code. test=develop * remove useless code. test=develop * update create_py_reader_by_data api. test=develop	7 years ago
Zeng Jinle	ec970f1270	Fix create_lod_tensor (#18196 ) * fix_create_lod_tensor, test=develop * remove program_guard import,test=develop * fix windows numpy default int32 error, test=develop	7 years ago
Hongyu Liu	cefd0fb598	Fix slice op shape=-1 bug (#18107 ) * fix slice op bug; test=develop * fix variabel test bug; test=develop * remove slice while true; test=develop	7 years ago
Jiabin Yang	b3cbc5be76	test=develop, fix test_imperative_transformer and ocr (#18127 ) * test=develop, fix test_imperative_transformer and ocr * test=develop, remove ocr recovery part	7 years ago
lijianshe02	ff4279e3b2	fix paddle.fluid.layers.io.open_files api doc bug test=develop (#18203 ) * fix paddle.fluid.layers.io.open_files api doc bug test=develop	7 years ago
Wojciech Uss	36b60e240a	Enable MKL-DNN for slim FP32 vs. INT8 tests (#18214 ) * Enable MKL-DNN for slim FP32 vs. INT8 tests test=develop * added test for MobileNetV1 with MKL-DNN test=develop	7 years ago
bingyanghuang	976cf46010	slim QAT MKL-DNN readme (#18187 ) * QAT doc first version * follow the comments, test=develop * update the benchmark, test=develop * Change to channel-wise, test=develop	7 years ago
chengduo	5588b923f3	Add multi process reader (#18115 ) * add multi process reader test=develop	7 years ago
wangchaochaohu	a9dc534f48	fix API example (#18153 ) * API.spec test=develop * update * update test=develop * update test=develop * update * update test=develop * update test=develop * update test=develop * update test=develop * update test=develop * test=develop * update * update test=develop * update test=develop * fix test=develop	7 years ago
Qiao Longfei	778f6acf4d	disable test_async_ssa_graph_executor_mnist test=develop (#18165 )	7 years ago
翟飞跃	802ea50956	fix spelling errors (#17941 ) * fix spelling errors; test=develop * Update API.spec update md5 * Update API.spec * change the order of api;test=develop	7 years ago
zhoukunsheng	0569ff78fa	Fix doc example for greater_equal, greater_than, less_equal, not_equal, rank, reduce_all, reduce_any, sign, where, diag (#18167 ) * test=develop fix greater_than, greater_equal, less_equal, not_equal, rank, reduce_all, reduce_any, sign, where, diag doc example * test=develop fix API.spec conflict	7 years ago
Jiabin Yang	991c94f135	test=develop, add add_multi_gpu_install_check (#18157 ) * test=develop, add add_multi_gpu_install_check * test=develop, refine warning doc * test=develop, refine warning doc * test=develop, refine warning doc * test=develop, support multi cpu	7 years ago
Huihuang Zheng	bbc292920c	Fix API example code (#18176 ) The fixed APIs: 6 Methods in paddle.fluid.io.PyReader paddle.fluid.layers.Preprocessor paddle.fluid.layers.py_reader paddle.fluid.io.save_params paddle.fluid.io.save_persistables test=develop test=document_preview	7 years ago
翟飞跃	78441c5449	add mkldnn Int8v2 slim doc (#17909 )	7 years ago
lvmengsi	d658f1133b	Fix doc for transpose, conv3d and batch_norm. (#18035 ) * update some op doc, test=develop	7 years ago
翟飞跃	c2fb9b906a	change dirname mobilenet to mobilenetv1 (#18160 )	7 years ago
bingyanghuang	5cf7e7415d	update mkldnn int8v2 doc (#18177 )	7 years ago
chengduo	4978db2c10	Remove nccl dep when the number of GPU is 1 (#18158 ) * remove nccl dep when the number of GPU is 1 test=develop	7 years ago
Zeng Jinle	25ab23be28	Fix dygraph mem leak (#18082 ) * fix dygraph mem leak, test=develop * polish msg, test=develop	7 years ago
tensor-tang	1c6e560607	core replace x86cpu with py cpuinfo (#18151 ) test=develop	7 years ago
Qiao Longfei	23f8a4b1c3	assign role_maker before use (#18137 ) fix role_maker bug test=develop	7 years ago
Zeng Jinle	6eec66a1b1	Fix py_reader iterable bug (#18108 ) * fix py_reader iterable bug, test=develop * move data from buffered_reader,test=develop	7 years ago
guru4elephant	58f3e1bad7	add paddle cloud role maker for customized usage, note this is only for industrial users that have cloud environment pre-configuration (#18121 ) add paddle cloud role maker for specific cloud usage. This pr will simplifies user's configuration in distributed training.	7 years ago
qingqing01	80d2e66f9e	Update backward appending stragety to support double backward and fix some bug. (#18104 ) * Update backward.py: - If there is no input grad var in all outputs of previous ops, do not append this op into graph. - Only apply this stragety when double backward. * Update some double backward op. * Update sum_op to judge whether a tensor is empty by numel or IsInitialized().	7 years ago
Wojciech Uss	ca5642c850	unify FP32 vs. INT8 comparison tests output (#18111 ) test=develop	7 years ago
FlyingQianMM	ff83655f7e	add detection output operator for supporting retinanet (#17896 ) * test=develop add detection output for supporting retinanet * test=develop add test_layers.py * test=develop add API.spec * test=develop alter test_retinanet_detection_output.py * test=develop alter round 2 * test=develop alter retinanet_detection_output * test=develop alter paddle/fluid/API.spec * test=devlop alter detection.py * test=develop alter retinanet_detection_output * test=develop alter paddle/fluid/API.spec * test=develop alter detection.py * test=develop alter API.spec * test=develop alter retinanet_detection_output * test=develop alter paddle/fluid/API.spec * test=develop alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py * test=develop alter python/paddle/fluid/tests/unittests/test_retinanet_detection_output.py * test=develop fix grammer error * test=develop fix grammer error * test=develop fix grammer error * test=develop alter python/paddle/fluid/tests/unittests/test_layers.py * test=develop alter paddle/fluid/API.spec	7 years ago
guru4elephant	0941e3e013	add class name and timeline for test_dist_base.py (#18122 ) * add class name and timeline for test_dist_base.py	7 years ago
lujun	9089774155	fix python ver for matplotlib, test=develop (#18123 ) update config for matplotlib, because python 3.6 is required by 3.1+	7 years ago
FlyingQianMM	0aee1f0074	add sigmoid focal loss operator for supporting retinanet (#17895 ) * test=develop add sigmoid_focal_loss for supporting retinanet * test=develop add test_layers * test=develop add API.spc * test=develop alter sigmoid_focal_loss_op.cc * test=develop alter detection.py * test=develop alter API.spec * test=develop alter round 1 * test=develop alter simooid_focal_loss * test=develop alter sigmoid_focal_loss_op.cc * test=develop alter test_layers.py * test=develop alter paddle/fluid/API.spec * test=develop alter sigmoid_focal_loss_op.cu * test=develop alter paddle/fluid/operators/detection/sigmoid_focal_loss_op.cc	7 years ago
FDInSky	9e4b9d9798	Update generate_proposal_labels_op to support CascadeRCNN. (#17200 ) * Update generate_proposal_labels_op to support CascadeRCNN.	7 years ago
FlyingQianMM	9ed2f936f1	add target assign operator for supporting retinanet (#17893 ) * test=develop add target assign for retinanet * test=develop run ci * test=developp add test_layers * test=develop add APi.spec * test=develop alter round 1 * test=develop alter rpn_target_assign_op.cc * test=develop alter test_rpn_target_assign_op.py * test=develop alter rpn_target_assign_op.cc * test=develop alter API.spec * test=develop alter paddle/fluid/operators/detection/rpn_target_assign_op.cc * test=develop alter rpn_target_assign_op.cc * test=develop alter python/paddle/fluid/layers/detection.py * test=develop alter paddle/fluid/API.spec	7 years ago
Sylwester Fraczek	accb132f0f	fix slim int8 mkldnn multithreading issue (#18009 )	7 years ago
lujun	5c3cbb58e5	Update requirement for py2 and py3 (#18068 ) for py2: matplotlib<=2.2.3 ; python_version<"3.5" scipy>=0.19.0, <=1.2.1 ; python_version<"3.5" nltk>=3.2.2, <=3.4 ; python_version<"3.5" for py3: matplotlib ; python_version>="3.5" scipy ; python_version>="3.5" nltk ; python_version>="3.5" update requests version to >=2.20.0 for CEV-2018-18074	7 years ago
chengduo	24e988a471	Fix bug of scope_buffered_ssa_graph_executor (#18100 ) * fix code bug test=develop	7 years ago
Jiabin Yang	ff59866502	test=develop, fix mnist will segment fault (#18083 )	7 years ago
Hongyu Liu	d9270af931	Fix getitems slice bug (#18053 ) * fix get items slice bug; test=develop * fix unique_name bug; test=develop	7 years ago
wopeizl	26a7c1a396	add unit test to cover all parameters for print op test=develop (#18089 )	7 years ago
guru4elephant	b2cfdc3891	Refine unittest log (#18084 ) * add print log for unittest of distributed training test=develop	7 years ago
gongweibao	f5caf3443c	Fix reinitialized ncclid error! (#18025 )	7 years ago
qingqing01	e81756f1ba	Hidden paddle.fluid.layers.detection_map. (#18033 ) * Remove layers.detection_map API * Since uers can use fluid.metrics.DetectionMAP to calculate mAP of current-batch and cumulative-batch. layers.detection_map only can calculate cur-batch mAP.	7 years ago
tensor-tang	8cf25c4310	refine core warning message (#18063 ) test=develop	7 years ago
tangwei12	4c735f24ea	fix bug in fleet, test=develop (#18058 )	7 years ago
chengduo	b5a1c1463d	Update CPU_NUM config (#18059 ) * update CPU_NUM config test=develop	7 years ago

1 2 3 4 5 ...

9047 Commits (075e1cf78e4104a6c19bd64e593afac00a2f2c21)