195 Commits (0d6ea52958ffcb7ed7bda0917aecb872a96ea2ca)

Author	SHA1	Message	Date
Huihuang Zheng	12542320c5	Replace TemporaryAllocator by CUDADeviceContextAllocator (#18989) ... TemporaryAllocator is a singleton used for allocating memory for Cudnn. Since it is a singleton, we can delete it for better performance in memory. We replace TemporaryAllocator by CUDADeviceContextAllocator and CUDADeviceContextAllocation, which uses stream callback to delete the memory allocated for the stream to avoid singleton. Also added data_feed_proto to operator to fix CI in CPU compilation	6 years ago
hutuxian	1ca6ea0318	fix cmakelist deps (#19668) ... fix cmakelist deps: remove unnecessary deps and add proper op deps	6 years ago
hutuxian	c756b5d231	Paddlebox Framework (#18982) ... * Support looking up embeddings from BoxPS. * Add a _pull_box_sparse op, for now this op is not exposed to users. * Add a BoxHelper class, providing 'BeginPass', 'EndPass', 'FeedPass' functions and so on. * Add 'BoxPSDataset' in python code. * Add a compile options WITH_BOX_PS and a MACRO PADDLE_WITH_BOX_PS. * Add UT. * More concrete information pls refer to: https://github.com/PaddlePaddle/Paddle/pull/18982	6 years ago
Zeng Jinle	5b6673c44d	merge develop to solve conflict, also fix API doc, test=develop (#18823)	6 years ago
chengduo	21440b4d69	Add call stack info during compile time (#19067) ... * Add call stack info during runtime and compile time test=develop * Rename operator_call_stack test=develop * Add unit test test=develop * follow comment test=develop	6 years ago
Zeng Jinle	7ac748adb4	Open gc by default (#18836) ... * open gc by default, test=develop * fix test_train_recognize_digits and disable gc when ngraph is enabled, test=develop * fix conditional_block op eager deletion bug, test=develop * add some comments to reviewers, test=develop	6 years ago
Zeng Jinle	8008ab4e6b	Remove legacy C++ memory optimization codes (#18834) ... * remove legacy memory optimization codes, test=develop * follow huihuang's comments,test=develop * follow luotao's comments, test=develop	6 years ago
Huihuang Zheng	89bc3fd841	Support memory eager deletion on recurrent OP (#17710) ... Test PaddingRNN on V100 GPU device. Test configuration: large model, padding mode (which is the mode using recurrentOp), one GPU. GPU memory (MiB): 6414 (this PR) vs 6837 (without this PR) Speed (steps/s): 10.28 (this PR) vs 9.89 (without this PR)	6 years ago
guru4elephant	d714bf037c	remove async executor and add data_feed.proto to the deps of train demo (#18659) ... * remove async executor and add data_feed.proto to the deps of train demo	6 years ago
HaoRen	b7128bac5f	supports collective communicated training (#18175) ... * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * fix comment test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * fix prepare context redundant code problem, optimize executor by caching create_varaiables test=develop * supports collective training in executor * make fetch_list runable with variables, add more unittest for use_program_cache test=develop * use unique name for nccl_id * supports output to stream in program_to_code * insert sync_comm_stream before regularization; add skip_op_callstack capability in program_to_code * set op role in collective training * add collective op role * fix comment test=develop * remove orig file * add build optimizer by strategy * add collective strategy * refine collective strategy * add multi-process role maker * refine strategy building factory so that we can easily plugin more strategy * scale loss grad in collective sgd transpiler * add support for distributed fc * code format * revert some features for dist fc * add support for distributed fc training * test=develop add collective op unittest standard * test=develop remove the test_collective directory * test=develop remove the test_collective directory * remove slicegather test * code format for reducescatter * update attr of shard_index_op * Modify macro nccl_helper * remove test without distribute * macro collective_helper * marcro update * test=develop update support python3.5 * test=develop change gpu memory use to 0.1 when test * test=develop update ut equal func * test=develop set flags to 1.5 * test=develop fix pickle dumple py35 * test=develop fix divide in slice and add sync_comm_stream update atol and rtol to 1e-05 rm shard_index op and test modify read input from file to read from memory remove origin_program in framework and add i/o in c_sync_calc_stream * test=develop update unittest sync operator I/O	6 years ago
hutuxian	f1d458daf0	add trainer_desc proto DEPS (#18019)	6 years ago
hutuxian	969e6378b9	Pipeline Concurrency (#17402) ... Add Pipeline Concurrency Train Mode: - Cpp: pipeline_trainer & section_worker - Python: PipelineOptimizer - Add a new data_feed type: PrivateInstantDataFeed - Add a test demo of pipeline trainer and the test model is gnn - Do not support win32 now	6 years ago
Zeng Jinle	c6189637cd	Fix allocator bug (#16712) ... * Revert "Revert "Fix allocator bug"" This reverts commit 174d0d0b90. * Revert "fix travis ci" This reverts commit 5656fa9f7c. test=develop * add inlined_vector.h, test=develop * add inlined_vector_test,test=develop	6 years ago
Qiao Longfei	adf272bcec	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async-ssa-graph-executor-communicator ... test=develop	6 years ago
guru4elephant	76b49f02ee	Merge pull request #16539 from guru4elephant/train_with_pipe_reader_merge_develop ... Train with pipe reader merge develop	6 years ago
Qiao Longfei	baf02328b2	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async-ssa-graph-executor-communicator ... test=develop	6 years ago
liuwei1031	278debab71	fix comments of 16410, test=develop (#16499) ... * fix comments of 16410, test=develop * modify inplace_op_inference_test according to pass interface change, test=develop	6 years ago
dongdaxiang	720647e17f	rebase current develop and fix conflict ... test=develop	6 years ago
dongdaxiang	241d8808be	add timer to distributed executor ... test=develop	6 years ago
dongdaxiang	3c73859eec	add trainer_desc.proto to distributed executor ... test=develop	6 years ago
dongdaxiang	0030eb2a61	fix distributed building ... test=develop	6 years ago
dongdaxiang	f39b323ed7	remove trainer_library in CMakeLists ... test=develop	6 years ago
dongdaxiang	f6c9232a3d	fix dataset float32 type problem	6 years ago
xujiaqi01	ecfc7df913	add dataset factory && fix style	6 years ago
dongdaxiang	b415ec27e8	make Dataset* as an argument	6 years ago
xjqbest	dd67ad08a2	modify c++ and python dataset related code & fix bug	6 years ago
dongdaxiang	cc4def6ba5	fix some conflict for compilation	6 years ago
heqiaozhi	9bca1926c1	refactor & fix bug	6 years ago
xjqbest	2e9a836c6f	add DataSet and InMemoryDataFeed, support load data into memory and shuffle data	6 years ago
dongdaxiang	2486389793	add RunFromDataset in executor	6 years ago
xjqbest	824b84d185	add DataSet and InMemoryDataFeed, support load data into memory and shuffle data	6 years ago
dongdaxiang	1fe54416c9	move fs.cc and shell.cc into paddle/fluid/framework/io ... test=develop	6 years ago
dongdaxiang	afaf937010	add fs_local_open example	6 years ago
dongdaxiang	cf1360643f	add printer for fetch variable	6 years ago
dongdaxiang	54f047a126	fix ngraph compile option	6 years ago
dongdaxiang	dd1dc9bcf0	add common.h.in back	6 years ago
dongdaxiang	c165012031	refine device_worker and trainer code ... test=develop	6 years ago
dongdaxiang	24a8001142	make -DWITH_PSLIB=ON compilable	6 years ago
Qiao Longfei	d8974e6da0	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async-ssa-graph-executor-communicator ... test=develop	6 years ago
sneaxiy	a0f4fefb60	delete source file no_need_buffer_vars_inference.cc ... test=develop	6 years ago
Qiao Longfei	30618409db	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-async-ssa-graph-executor-communicator	6 years ago
sneaxiy	78fb3a62e0	fix env variable settting bug ... test=develop	6 years ago
sneaxiy	7000ec85d9	fix some op grad maker ... fix ctest eager deletion disable bug test=develop	6 years ago
sneaxiy	a93a9eef8f	add op registry type ... refine gc code test=develop	6 years ago
Zeng Jinle	472f16b5aa	Merge pull request #16063 from sneaxiy/enhance_gc ... Enhance gc	6 years ago
chengduo	ad80bde824	Revert "Revert "Add Event for TensorCopy"" (#16035) ... * Revert "Revert "Add Event for TensorCopy" (#16022)" This reverts commit e2da3a5b22. * use default stream test=develop	6 years ago
sneaxiy	814a759061	merge develop ... test=develop	6 years ago
sneaxiy	597dc65e76	enhance gc ... test=develop	6 years ago
chengduo	e2da3a5b22	Revert "Add Event for TensorCopy" (#16022) ... * Revert "Add Event for TensorCopy (#15953)" This reverts commit 7235fd662b. test=develop * fix CI test=develop	6 years ago
chengduo	7235fd662b	Add Event for TensorCopy (#15953) ... Add Event for TensorCopy	6 years ago