* Simplify code for gast.If in is_control_flow_to_transform.
* Move IsControlFlowVisitor to file utils.
* Don't use convert_call for build-in func in CallTransformer.
* Optimize api is_control_flow_to_transform.
* Polish the document of IsControlFlowVisitor.
To prepare for publishing APIs, I added tests for that we can access dy2stat through:
@fluid.dygraph.declarative
@fluid.dygraph.jit.declarative
fluid.dygraph.ProgramTranslator()
fluid.dygraph.dygraph_to_static.ProgramTranslator()
fluid.dygraph.dygraph_to_static.program_translator.ProgramTranslator()
It surprised me that we had bugs on those different usages. I have fixed them.
I also added example codes for these new APIs
This PR also pulls my current PR https://github.com/PaddlePaddle/Paddle/pull/23880, so the PR history is long. For reviewer information, you could review this PR after https://github.com/PaddlePaddle/Paddle/pull/23880 is merged
1. Rename Dygraph To Static Decorators to declarative
2. dygraph_to_static_func is still using in some training tests, I cannot delete it now.
3. Add some API docs
* add Conv/ConvTranspose layers in paddle.nn.layer, test=develop
* add example code in docstring, test=develop
* update unittests to work with test_runner, test=develop
* static model runner basic implement, test=develop
* add run program op to execute loaded program, test=develop
* refactor static model runner & run program op, test=develop
* reset engine.cc to resolve conflict
* adapt the change of dygraph double grad, test=develop
* refactor impl to solve control flow error, test=develop
* clear debug code, test=develop
* fix ci str compatible error & checkout dygraph grad maker & add example, test=develop
* hide api & add op test, test=develop
* fix run program op test places error, test=develop
* fix program by review comment, test=develop
* delete change var desc name, test=develop
* fix other program by review comment, test=develop
* remove _static_graph_guard, test=develop
* add selectedrows test, test=develop
* remove desc parser, test=develop
* fix detail program, test=develop
* change socpe create & add test, test=develop
This CR makes two changes:
1. In old loop_transformer, if a class variable, such as "self.a, foo.bar" is a loop var, the Dy2stat will fail because `def func(self.foo)` is not legal syntax. We support class variable by renaming.
2. After https://github.com/PaddlePaddle/Paddle/pull/22892 is merged, we can support `while x < 10` in dygraph. I enable those tests in corresponding Dy2stat
1. Add data_layer_not_check because it is needed in dygraph_to_static where input can be variable size
2. Remove warnings in static analysis because python cannot do exact static analysis
* Remove Read-Only Basic Type Variable in loop_vars
test=develop
* Put class support in loop for future thing
test=develop
* Modify based on reviewer's comment
test=develop
1. Add Decorator 'dygraph_to_static_program'
2. Add corresponding ProgramTranslator.get_program
3. Add ProgramTranslator.save_inference_model
4. Modified some warning information of dy2stat
5. Change program cache to contain startup_program because for users who gets program to run, they may like to initialize startup program
This PR does exact the thing as the title. The reason is that we plan to develop 4 decorators
"dygraph_to_static_code"
"dygraph_to_static_program"
"dygraph_to_static_func"
"dygraph_to_static_output"
The 4 decorators will emphasize different part when translating dygraph to static graph. Decorator name "dygraph_to_static_graph" is too big for the function it implements.
As the title, we add decorator "dygraph_to_static_code", and add related "get_code", "get_func", "get_output" for ProgramTranslator. Next step will be adding "dygraph_to_static_program"
1. Rename AutoTracer to ProgramTranslator
2. Rename cached_program to program_cache
3. Remove some functor style __call__
4. Dict key should be string but not hash code of string
1. Add support for Break and Continue in Dygraph to Static
2. Also add support for gast.Not in NodeTestTransformer
3. Also add support for logical op transformation in LoopTransformer
* add dygraph parallel env doc, test=develop
* polish details, test=develop, test=document_fix
* fix examples error in other apis, test=develop
* fix more example error in other api, test=develop
* add white list for gpu examples, test=develop, test=document_fix
1. Add basic support for `for in range` loop
2. Move `test_dygraph_to_static_*` to `dygraph_to_static` dir and rename them
3. Add test case for dict in while_loop
1. copy.deepcopy in NameVisitor should be changed to copy.copy to make hash or set work
2. read_context should be type of gast.Load()/gast.AugLoad(), not gast.Load/gast.AugLoad
Add basic support for while in translating dygraph to static
1. Analysis the variable liveness in class NameVisitor
2. Replace while key word using while_loop API
* Support fetch and run program in the process of dygraph_to_static_output. test=develop
* fix to_source(gast) and remove dygraph API such as Conv2D, Linear. test=develop
1. Considering functions, I have to change the node type from single value to a set. Because python function is allowed to return different types. The set represent all possible types
2. I added scope_name and scope_type for AstVarScope, because in python functions, variable may have different scope. For example:
```
a = 3
def foo(b):
a = 9
return a + b
```
the `a` in `foo` is different to the `a` out of `foo`. Similar to class field. The scope_name will help me to know the function name when static analysis finds a `return` sentence.
* support dygraph to static graph for simple case.
* add test for dygraph API recognition. test=develop
* support basic to_variable api. test=develop
* update dict: dygraph_class_to_static_api
* add all tests of dygraph api. test=develop
* use gast/astor instead of ast/codegen for the compatibility of PY2 and PY3. test=develop
* add arg 'num_flatten_dims' for fc ast node. test=develop
* Modify names of class by Camel-Case.
* support nested if/else
* support to derivate returns the parameter list automatically
* polish tranform function of slice
* fix modify x.numpy()[i] slice function
* support to transform ast.node into callable function
* fix get_name_ids bug and add more unittest test=develop
* fix requirements.txt test=develop
* remove useless import statement test=develop
* Fixed version compatibility issues in param of function test=develop
* use decorater to test ast_to_func test=develop
* add textwrap.dedent for source_code test=develop
* polish code comment
* fix compatibility with python2 and python3 test=develop
* fix gast version error test=develop
* fix gast repo test=develop
* polish transfer_from_node_type code test=develop
* add nested_if_else unittest test=develop
* split IfElseTransformer test=develop
* specify gast version test=develop
* fix ast_to_func root type test=develop
This PR provides very basic and simple framework for transforming Dygraph to Static Graph.
API names, final outputs are not determined yet. Feel free to modify or add class/function/type when you think the framework is not extendable for you.
* modify fc to linear in sample code, test=develop
* remove FC, test=develop
* remove warnings, test=develop
* drop fluid/imperative/README.md , test=develop
* change fc to linear, test=develop
* polish code style, test=develop
* dygraph mode support linear lr warm up; test=develop
* add unitest for linear warmup; test=develop
* add input type check; test=develop
* fix type check assert error; test=develop
* change type error; test=develop
* test=develop, fix docker with paddle nccl problem
* don't expose numerous Tensor.set(), test=develop
* fix condition, test=develop
* fix float16 bug, test=develop
* feed should be Tensor or np.array, not Variable or number, test=develop
* use forcecast to copy numpy slice to new array, test=develop
* remove float16-uint16 hacking, test=develop
* add variable method to varbase and refactor to_variable to support return varbase
* support kwargs in varbase constructor
* add VarBase constructor to support default python args
* refine varbase initial method
* reset branch
* fix ut for change VarBase error info to PaddleEnforce
* cherry is parameter change before
* overload isinstance to replace too many change of is_variable
* rm useless files
* rm useless code merged by git
* test=develop, fix some ut failed error
* test=develop, fix test_graph_wrapper
* add some tests, test=develop
* refine __getitem__, test=develop
* add tests, test=develop
* fix err_msg, test=develop
* add param & grad shape check for sgd op
* add _reshape_inplece interface for dygraph parallel
* refine unittest based paddle/models scripts, test=develop
* add unittest for parallel grad fuse, test=develop
* Add c++ global current tracer for dygraph, test=develop
* add tracer property in c++, test=develop
* support different place, test=develop
* add unittest for tracer, test=develop
* don't expose numerous Tensor.set(), test=develop
* fix condition, test=develop
* fix float16 bug, test=develop
* feed should be Tensor or np.array, not Variable or number, test=develop
* use forcecast to copy numpy slice to new array, test=develop
* remove float16-uint16 hacking, test=develop
* test=develop, fix docker with paddle nccl problem
* test=develop, Add Variable api and refine dygraph related API
* test=develop, Add Variable api and refine dygraph related API
* test=develop, refine test for new api and error info
* test=develop, refine error info and test_layers
* test=develop, add API.spec
* test=devleop, fix to_string python2 and python3 compat error and refien doc
* test=devleop, add API spec
* test=devleop, update API spec
* test=devleop, update API spec
* test=develop, invoke ci
* test=develop, fix example code
* test=develop, update API spec
* test=develop, add compat test and fix inplace campat dict error
* refactor dygraph,test=develop
* fix failed unittest,test=develop
* polish code,test=develop
* check windows ci error,test=develop
try to fix windows ci error by np.allclose,test=develop
* polish vlog and profiler, test=develop
* try to fix preceding ops order,test=develop
* test transformer in windows ci, test=develop
* use python c-api to speed up tracer.trace,test=develop
* test=develop, fix docker with paddle nccl problem
* test=develop, add ut for debug string and gradient_accumulator
* test=develop, add tests for layer/gradient_accumulator/prepared_op
* test=develop, fix complie error for test_prepared_op
* test=develop, add more ut for dygraph
* test=develop, create API.spec for dygraph api change
* test=develop, refoctor name to make it easier to understand
* test=develop, refoctor name to make it easier to understand
* test=develop, fix multi-gpu failed problem , add Tracer tests, change PADDLEENFORCE to PADDLEENFORCE_EQ
* test=develop, fix ut failed on parallel se-resnext
* test=develop, change one more PADDLE_ENFORCE
* fix con2d transpose bias by create and init it in build_onee
* fix API spec
* test=develop, invoke ci
* fix bias_attr and act has no effect error on layer norm, conv2dTranpose, billinearTensorProduct, sequece_conv. fix original_mode not used error on GRUunit. fix sample_weight not set error on NCE. Add ut for all thoese layer
* test=develop, change success standard for conv2dTranspose
* test=develop, fix test_layers to invoke some error branch
* test=develop, fix sample code
* test=develop, fix BilinearTensorProduct failed in dygraph mode
* test=develop, fix test_layers segment fault error
* test=develop, refine api
* test=develop, fix bug when error occured on save_persistable with no optimizer
* test=develop, refine waring
* test=develop, refine example code and comments
* save optimizer related vars in dygraph
* test=develop, add optimizer save and load
* test=develop, add optimizer save and load
* test=develop, merge code and add multi-optimizer save and load
* test=develop, fix test_imperative_checkpoint
* test=develop, fix include error
* test=develop, fix include error
* test=develop, renew api spec
* test=develop, refine code
* test=develop, set default value for checkpoint
* test=develop, fix ci error
* test=develop, change API.spec and make api more readable
* test=develop, refine version and time stamp
* test=develop, add example code and refine code
* test=develop, refine doc
* test=develop, change version
* for debug
* test=develop, memory optimize for dygraph using shared_ptr
* test=develop, fix travis ci showed error
* test=develop, fix bug for recurrent usage of varbase
* test=develop, init varbase when it need to be Add
Chen Weihang
qingqing01
liym27
Xing Wu
Yibing Liu
wawltor
Huihuang Zheng
songyouwei
zhongpu
Aurelius84
silingtong123
Zhou Wei
Leo Chen
Feiyu Chan
hong
Zeng Jinle
tianshuo78520a
Wojciech Uss
LielinJiang
DuYao
Jiabin Yang
chengduo
lujun
xiaoting
chengduo
Yan Xu