* test=develop,test=document_fix, remove the out args
* Revert "test=develop,test=document_fix, remove the out args"
This reverts commit f430799bc82ee327d14a04c3bb1ea712d39d1cef.
* test=develop,test=document_fix, remove the out args
* remove all the out args
* remove all the out args
* test=develop, remove all the out args
* test=develop, remove all the out args
* test=develop, remove out args in matmul testcase
* test=develop, fix the bugs of out args testcase in logsumexp
* test=develop, fix the bugs of out args testcase in elementwise_add
* test=develop, fix the bugs of out args testcase in elementwise_div
* test=develop, fix the bugs of out args testcase in activation
* test=develop, fix the bugs of out args testcase in addcmul
* test=develop, fix the bugs of out args
* test=develop, fix the bugs of out args in API example
* test=develop, fix the bugs of out args in kron testcase
* test=develop, fix the bugs of out args in kron testcase
* test=develop, remove out args
* test=develop, remove out args
* test=develop, fix the coverage
* test=develop, polish codestyle
* test=develop, Update math.py
* test=develop, polish codestyle
* test=develop, polish codestyle
* test=develop, polish codestyle
* test=develop, polish codestyle
* test=develop, polish the test_activation_op.py
* test=develop, Update math.py
* test=develop, fix the failed CI
* test=develop, add core.ops
* test=develop, add core.ops change clamp to clip
* test=develop, add core.ops change clamp to clip
* test=develop, write testcase for clamp OP
Enhance TracedLayer Error Message
Note: this PR uses assert to check type somewhere and check_type somewhere, the reason is that the check_type skips checking when it is under dygraph mode.
* add more settings for distributed strategy
Basically, DistributedStrategy has several parts of configurations:
- BuildStrategy: the same as paddle.fluid.BuildStrategy, but the distributed arguments are moved out of BuildStrategy
- ExecutionStrategy: the same as paddle.fluid.ExecutionStrategy
- collective communication configs: nccl_comm_num, hierarchical allreduce and so on
- distributed algorithms: async_update(mainly used in PS), lars, lamb and so on
Based on the comment here b5f8784cab/paddle/fluid/framework/details/build_strategy.h (L49)
The unit test which compares Reduce and AllReduce must have diff. The PR_CI_Night runs on P40 machine and it has 8GB GPU, which is smaller than the 16GB normal CI machines. So we decrease the batch size in the past to make it runnable: https://github.com/PaddlePaddle/Paddle/pull/24651/files . Decreasing the batch size makes the difference occurs often. So this PR replace the absolute delta by relative delta.
Before this PR, the unit test failure happens with probability about 1/100. After this PR it doesn't happen.
* fix the double grad bug for the star gan. test=develop
* update the retain_graph parameter doc. test=develop
* add the unit test for the retain_graph parameter. test=develop
* Refine Model
1. Take the network (instance of Layer) as the input of Model.
2. Refine set_dict/load_dict of Layer.
3. Refine Input interface, so update code sample about Input
* New features, add sinh and cosh op, test=develop
* remove duplicate test function and remove out paramters, test=develop
* Add out paramters temporary, remove later. test=develop
* remove out args, PR 25570, test=develop
* remove TestParameter, test=developx
* add test api for static dygraph, test=develop
* add backword unittests for sinh and cosh, test=develop
* add ci check for changing op-related api without core.ops, test=develop
* generate api_source_md5 file when build, test=develop
* add failed example, test=develop
* add failed example, test=develop
* handle exception, test=develop
We found that the reason of multiple return error in SimNet is that I wrote wrong task_mode. If we set task_mode as "pairwise" correctly, which is a format of the model input data, the multiple return won't have problem in the unit test. In this PR we corrected the task_mode and enable multiple return in SimNet unit test.
This PR fixes a bug that SelectedRows cannot be supported in SimNet. The reason of this bug is that dygraph basic_engine didn't copy var's type when the var needs to be accumulated during backward. So when a var is SelectedRows and needs to be accumulated, like SimNet which calls net for two times, the var's type will be changed to default LoDTensor thus bug happens. To fix it, we just also copy the type.
Without this PR, the accumulated SelectedRows parameters in dygraph will be changed into LoDTensor. So when we fixed the bug of supporting SelectedRows in SimNet, we found `test_imperative_lod_tensor_to_selected_rows` failed and threw the error that SelectedRows was not supported for Transpose OP. To fix it, too, this PR also added support for SelectedRows for Transpose OP.
According to paddle 2.0 standard.
1, chang api to def meshgrid(*args, **kwargs) we hide name argument in **kwargs.
2, add relate ut.
3, change example code to imperative mode
Add Similarity Net as unit test. During the unit test, we found three problems:
1. The run_program_op has memory optimization error when running dy2stat net multiple times.
2. The support for SelectedRows can cause problem in dy2stat.
3. The return grammar has problem.
This PR fixes the 1. problem but modify codes for the 2. 3. problems to make PR smaller. I will fix those two problems in the next PR(s)
* fix optimizer.state_dict and LRScheduler.state_dict to save/load dygraph,test=develop
* fix optimizer.state_dict and LRScheduler.state_dict to save/load dygraph,test=develop
* Add a judgment that state_dict/set_dict is used incorrectly,test=develop
* fix some doc error,test=develop
* fix current_step_lr for _LearningRateEpochDecay,test=develop
* remove some unsed code to improve coverage,test=develop
* remove some unsed code to improve coverage,test=develop
* fix optimizer parameter is a iterator; test=develop
* fix parameter list None bug; test=develop
* use is not None; test=develop
* change list to iterable; test=develop
* fix bn & in in dy, test=develop
* update instance_norm,test=develop
* fix bugs,test=develop
* add more case in unittest,test=develop
* fix,test=develop
* fix,test=develop
* show the attr and functions of the Layer,test=develop
* add buffer for dir,test=develop
* fix __dir__,test=develop
* fix doc of Layer.__dir__, test=develop
* support tuple/list init for VarBase,test=develop
* fix doc of fluid.dygraph.to_variable,test=develop
* fix doc of fluid.dygraph.to_variable,test=develop
* fix the compatibility of PY2 and PY3 in paddle.distributed.launch
test=develop
* only pull log of local rank 0
test=develop
* log exception if UnicodeEncodeError occurs when pulling log in
paddle.distributed.launch
test=develop
Co-authored-by: SunGaofeng <peakbee@gmail.com>
* Add `matrix_nms_op`
test=develop
* Make ci happy
test=develop
* Exit early when no detection
test=develop
* Fix license year
test=develop
* Output index as well
test=develop
* Match nms2 lod behavior and add `return_index` flag
test=develop
* Make CI happy
test=develop
* Fix wording
test=develop
* add new API: MultiStepDecay, a new learing rate strategy, test=develop
* add new API: MultiStepDecay, a new learing rate strategy,test=develop
* add new API: MultiStepDecay, a new learing rate strategy,test=develop
* add base class of LearningRateEpochDecay, and MultiStepDecay, and StepDecay, test=develop
* fix doc to add coverage,test=develop
* add new api: optimizer.set_lr, test=develop
* add API doc and example code for optimizer.set_lr,test=develop
* add API doc and example code for optimizer.set_lr,test=develop
* Modified doc to :api_attr: imperative,test=develop
Support Various-Length Return Grammar in Dy2stat. This PR is a follow-up of https://github.com/PaddlePaddle/Paddle/pull/25176 .
The basic idea is putting no-value placeholder variables at `return` statement to make all `return` statement have same length, after that the static graph can have fixed fetch output (code at return_transformer.py). Then remove those no-value placeholder when we finally return dygraph result (code at partial_program.py).
However, various length return in Bert model is still not supported. The dy2stat can change the code as I wish but some ops which check shape at compile time (e.g. Reshape, MatMul) will throw error because of the no-value-placeholder may not have the required shape. Is this a matter? To me, those no-value placeholder will be replaced as really values meeting shape requirements at run time, so I think the solution should be some way to do the compile-time checking. By the way, every time when we have dynamic shape, it often causes problem in dy2stat. We should find a way to handle it in the future.
Fixing various return in Bert is my TODO thing and I will also find some other existing models for verification.
This PR added basic support for 'return' grammar in dy2stat. It supports the control flow of 'return'.
The basics idea is using a return value variable to store the early return statements and boolean state variables with if-else to skip the statements after the return statements.
**This PR is very basic support. There are some corner cases I didn't develop/test**. For example, 'return None', 'return different length of variables', 'return non-tensor and tensor together', 'no return statement'. **These corner cases will be done in my next PRs**. Target date is this week.
**Note**:
1. for the unit test, I changed test_program_translator.py because the StaticCode of `dyfunc_with_if_else` will change. To guarantee the correctness of `dyfunc_with_if_else`, I also run it in `TestRecursiveReturn` in test_return.py.
2. I commented the early return code in bert_dygraph_model.py because 'return different length of variables' is unsupported now. I also know that there are some other models used early return and we didn't enable it in the unit test. I will add support for it in next PRs and then re-enable those tests.
Zhen Wang
LielinJiang
danleifeng
wawltor
Aurelius84
wangchaochaohu
Huihuang Zheng
Dong Daxiang
WuHaobo
WangXi
Pei Yang
xujiaqi01
JZ-LIANG
gentelyang
Yi Liu
Zhang Ting
lilong12
yukavio
Bai Yifan
ShenLiang
Chen Weihang
mapingshuo
tangwei12
qingqing01
liym27
hong
cc
zhupengyang
Sylwester Fraczek
Wojciech Uss
LutaoChu
Leo Chen
cnn
Shibo Tao
Zhou Wei
Peihan
YUNSHEN XIE
yaoxuefeng
leesusu
Kaipeng Deng
Jacek Czaja
ceci3
gongweibao
Feiyu Chan
littletomatodonkey
Yang Zhang
chalsliu
Chengmo
123malin
Tao Luo
liu zhengxi
FDInSky