73 Commits (7c55a94de51b1de42bd444c0d8ab20c9eeabcb4e)

Author	SHA1	Message	Date
zhaoyuchen2018	72dde4abde	Refine adam op to improve performance, test=develop (#22346) ... * Refine adam op, test=develop * Fuse kernels together to reduce cpu time. * Refine paddle enforce, test=develop * Remove some comments, test=develop * Refine code,test=develop * Refine cuda kernel, test=develop * Refine code according to comments, test=develop	6 years ago
zhongpu	d0f0a2520c	test Optimizer in dygraph (#21949) ... * test Optimizer in dygraph, test=develop * add optest for Optimizer in dygraph, test=develop * fix adagrad optimizer, test=develop * fix dpsgd optimizer, test=develop * fix test_optimizer.py, test=develop * fix dpsgd optimizer, this op only support cpu, test=develop * add optest for optimizer, test=develop * add description for dpsgd, test=develop * add rmsprop to white_list in unused_var_check.cc, test=develop * polish code style, test=develop * polish code style, test=develop * delete seed attribute for DpsgdOptimizer, test=develop * change testing to debugging, test=develop	7 years ago
Aurelius84	51a86d2b6b	Optimize adam speed (#21777) ... * optimize adam speed by removing _finish_update test=develop * fix SparseAdamFunctor param list test=develop * Remove scale_op in expect_list of adam_op test=develop * fix test optimizer loss assert error test=develop * fix test optimizer loss assert error test=develop * modify PADDLE_ENFORCE usage test=develop * fix op_type in lamb_op.cc test=develop * fix errors ostream format bug test=develop * add betaPowOut in ngraph op test=develop * fix ngraph::op api for gcc8 test=develop * clean code test=develop * modify struct into class test=develop * remove code of beta1Tensor in lamb_op test=develop	7 years ago
Huihuang Zheng	1dcf6a7212	Add Much Complex Test and Fix Bugs for Control Flow cond API (#21532) ... Add tests to use dy/dx to make sure the gradient values calculated by the control flow backward is correct. Also fixed bugs detected by those tests. Fix bugs: 1. Unlike sum_op, optimizer ops don't allow uninitialized input tensor. But in conditional_block_grad_op, since the conditional_block may not run, the output gradient tensor may be uninitialized, which will cause the optimizer op error. To fix it, we should let optimizer ops support uninitialized input like sum_op or assign the uninitialized gradient to 0 when the conditional_block_grad_op doesn't run. I found there are about 10+ optimizer ops. **To be simpler, I just assign output gradient of the conditional_block_grad_op to 0 in this PR**. But it can be further explored whether we can make optimizer ops like sum_op to support uninitialized input tensor because theoretically we can speed up without the assigning in conditional_block_grad_op. 2. Infer parameter shapes during append_backward. I didn't know that all our parameters are in global block. When op_desc is inferring shapes at the sub-block, it may not know the shape of gradients of parameters whose shape information is at global block. I fixed it by inferring shapes of gradients from forward var. This PR also did some code clean up: 1. Print the var name when sgd_op catches shape error so that it is easier to debug 2. Fix a typo: dicta -> dict	7 years ago
Chen Weihang	664f958a02	Fix optimizer op infershape failed in dygraph multi-cards mode (#21374) ... * add param & grad shape check for sgd op * add _reshape_inplece interface for dygraph parallel * refine unittest based paddle/models scripts, test=develop * add unittest for parallel grad fuse, test=develop	7 years ago
hong	ac8546701d	Add dygraph execution context (#20157) ... * add_dygraph_execution_context * add dygraph infershape context and execution context; test=develop * fix imperative bug; test=develop * remove inputs outputs interface from execution context, because it have same function with inputNames; test=develop * remove tracer_test ctest; test=develop * fix split op bug; test=develop * fix unitests bug; test=develop * fix distribute test bug; test=develop * fix ngraph compile bug; test=develop * fix grad maker bug; test=develop * fix load op bugs; test=develop * fix operator.cc construct bug; test=develop * remove useless name find in operator; test=develop * add tracer_test; test=develop * fix concat, split bug; test=develop * remove tracer_test unitest; test=develop * fix attribute check bug; test=develop * add test code to fix converage; test=develop * remove useless code, change check backward input in engin; test=develop * unlock var type infer shape;test=develop * add ShareAllLoD api; test=develop * add dygraph infershape context unitest; test=develop * remove increase and decrease lod in dygraph; test=develop * addd override; test=develop * fix increase descrease lod; test=develop * fix paddle_enforce; test=develop * disable lod op dygraph check; test=develop * fix paddle enforce error; test=develop * add comment for op_registry and OperatorBase; test=develop * optimize the comment of op_registry; test=develop * fix format of comment; test=develop * fix format of comment; test=develop * optimize the format of comment; test=develop * optimize the format of the comment; test=develop * optimize comment of op_registry; test=develop	7 years ago
Kaipeng Deng	ebfb720a63	add Adam beta1/beta2 support Variable (#21234) ... * add Adam beta1/beta2 support Variable. test=develop	7 years ago
WangXi	8ac7687e36	Fix dgc accuracy by mv regularization to local (#21278)	7 years ago
hong	8c4573a3cb	GradMaker for dygraph (#19706) ... * refactor dygraph,test=develop * fix failed unittest,test=develop * polish code,test=develop * check windows ci error,test=develop try to fix windows ci error by np.allclose,test=develop * polish vlog and profiler, test=develop * try to fix preceding ops order,test=develop * test transformer in windows ci, test=develop * use python c-api to speed up tracer.trace,test=develop * test=develop, fix docker with paddle nccl problem * test=develop, add ut for debug string and gradient_accumulator * test=develop, add tests for layer/gradient_accumulator/prepared_op * test=develop, fix complie error for test_prepared_op * test=develop, add more ut for dygraph * test=develop, create API.spec for dygraph api change * optimize grad maker; test=develop * optimize grad maker * test * grad make optim; test=develop * fix unittest bugs; test=develop * add dygraph grad op maker and split_op * grad op maker refactor; test=develop * add dygraph grad maker; test=develop * fix op deformable_conv_v1_op bug; test=develop * fix deformable_conv prroi pool bugs; * fix new op grad op maker bug; test=develop * fix split by ref bug; test=develop * fix dygraph auto prune bug; test=develop * fix test_trace bug; test=develop * fix fused emb seq pool bug; test=develop * remove useless code in op_desc file; test=develop * remove useless code, StrVarBaseNode; test=develop * fix review issues; test=develop * fix rank_loss grad maker; test=develop * remove flag in VarBase; test=develop * fix distributed_notify_op compile bug ; test=develop * fix reshape op double grad; test=develop * fix expand as op; test=develop * add impertive type_defs.h for demo_train; test=develop * fix inference lib cmake; test=develop * fix inference lib; test=develop * fix infernce_lib; test=develop * fix inference cmake; test=develop * fix inference lib; test=develop * fix inference lib; test=develop * remove condition dygraph grad maker, modify local name; test=develop * fix split grad maker bug; test=develop * fix pyramid_op bug; test=develop * change travis time out limit; test=develop * restore travis; test=develop * change timeout limit; test=develop	7 years ago
Chen Weihang	26cc1fe508	Replace risky GetInputType method with secure IndicateVarDataType interface (#20668) ... * replace part of the old implementation, test=develop * restore concat op, test=develop * update all ops implemention & delete GetDataTypeOfVar func, test=develop	7 years ago
WangXi	250e72d254	Fix DGC algorithm flow to make it the same as paper (#20758)	7 years ago
jhjiangcs	766bd529d1	add optimizer:dpsgd,test=develop (#19915)	7 years ago
Chen Weihang	8cb54ede8c	Add user-friendly error message in optimizer ops to give a hint about the position sensitive problem of run(startup_program) (#19605) ... * add extra error message hint in optimizer ops * polish format & delete useless change, test=develop * extract init judue from shape compare, test=develop	7 years ago
Tao Luo	75d1571995	refine PADDLE_ENFORCE codes for unify PADDLE_ASSERT_MSG (#19603) ... test=develop	7 years ago
chengduo	7453857324	Make fuse_all_reduce_op_pass support mix_precision (#17652)	7 years ago
Yibing Liu	23941e43ec	Update lamb optimizer (#18333) ... * Update lamb optimizer test=develop, test=document_preview * Regenerate api spec test=develop, test=document_preview	7 years ago
Yibing Liu	e8990e64f6	Fix trust ratio in lamb (#17614) ... test=develop	7 years ago
Yibing Liu	f9796b1249	Add LAMB Optimizer support (#17489) ... * Add LAMB optimizer * Expose LAMB Optimizer's APIs test=develop, test=document_preview * Cleanup code & doc test=develop, test=document_preview * Update lamb optimizer's formula test=develop	7 years ago
Qiyang Min	c7f1f3ed0c	Merge pull request #16214 from velconia/imperative_infer_var_type ... Implement imperative infer var type	7 years ago
minqiyang	b40e41fbd1	Polish code style ... test=develop	7 years ago
minqiyang	ca392c7e97	Implement infer var type context	7 years ago
sneaxiy	f0d108f589	fix const_cast ... test=develop	7 years ago
tensor-tang	14a764c930	simplify the jitkernel templates and tests ... test=develop	7 years ago
tensor-tang	802f362ac4	unify the kernelfuncs cache and add unit test ... test=develop	7 years ago
tensor-tang	a0c37662b9	enable sgd jitkernel refer code and test ... test=develop	7 years ago
peizhilin	cd562f8fb7	disable the parallel mode for adam op on windows test=develop	8 years ago
Qiao Longfei	8c516a24e5	remote min_row_size_to_use_multithread in adam interface test=develop	8 years ago
Qiao Longfei	9b4fe283e1	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multithread-sparse-adam ... test=develop	8 years ago
Qiao Longfei	44b300556d	change min_row_size_to_use_multithread to parameter of adam ... test=develop	8 years ago
Qiao Longfei	87b4eb1da4	change min_param_size_to_use_multithread to min_row_size_to_use_multithread	8 years ago
Qiao Longfei	17b1b660fc	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multithread-sparse-adam ... test=develop	8 years ago
Qiao Longfei	c15270c5b2	optimize multi thread adam	8 years ago
Qiao Longfei	0e747e8d02	change the limit of thead num	8 years ago
Qiao Longfei	dfe85fb358	fix build	8 years ago
Qiao Longfei	f057bbd1d1	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix-adam-tmp-var	8 years ago
Qiao Longfei	f1c973b014	adam op should not create tmp var in compute	8 years ago
Qiao Longfei	d161215332	optimize adam multi thread	8 years ago
Qiao Longfei	7a58ad5c79	lazy mode have higher priority then multithread ... test=develop	8 years ago
Qiao Longfei	d0572bf02e	add log for lazy mode test=develop	8 years ago
Qiao Longfei	1177b0bc84	update multi thread adam	8 years ago
Qiao Longfei	3b294e2e2e	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multithread-sparse-adam	8 years ago
sneaxiy	dde3afe7b7	Merge develop ... test=develop	8 years ago
Wu Yi	856f0da0fe	Fp16 training (#14992) ... * wip * wip * wip * wip for test * add fp16 tests test=develop * fix cpu build test=develop * fix test=develop * fix py3 tests test=develop * fix lr_scheduler dtype test=develop * fix test=dvelop * test fix ci compile test=develop * fix build and merge test=develop * fallback momentumop change to general test=develop * make fp16 lr schedule simple test=develop * fix ut test=develop * fix tests test=develop * remove fp16 learning rate cast test=develop	8 years ago
minqiyang	8b6b0da062	Use adam_update ... test=develop	8 years ago
minqiyang	f4e7a47381	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into accelerate_adam ... test=develop	8 years ago
sneaxiy	c1f7e54f62	merge develop ... test=develop	8 years ago
typhoonzero	da87f7a698	Revert "[Feature] Fp16 training for resnet50 (#14850)" ... This reverts commit 3d750f9c5a.	8 years ago
Qiao Longfei	d76bda50c4	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into multithread-sparse-adam ... test=develop	8 years ago
minqiyang	6467eda534	Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into accelerate_adam ... test=develop	8 years ago
Wu Yi	3d750f9c5a	[Feature] Fp16 training for resnet50 (#14850) ... * wip * wip * wip * wip for test * add fp16 tests test=develop * fix cpu build test=develop * fix test=develop * fix py3 tests test=develop * fix lr_scheduler dtype test=develop * fix test=dvelop * test fix ci compile test=develop * fix build and merge test=develop * fallback momentumop change to general test=develop	8 years ago