* Expose Net to Python
* Expose PlainNet to Python, make python can add_op, complete_add_op
* Provide a low level api to manipulate Net
* Unittest for Net::DebugString
We put forward Op's inputs, outputs and output gradients into Grad
Op's inputs, and put forward Op's input gradients into Grad Op's output.
So Grad Op's `in_out_idx`, `input_format` and 'output format' need to be
rebuilt during Op creating.
* There is a merge conflict when merge PR #2914
* Develop and PR #2914 both add `DDim::size` method, but did not
triger git merge conflict while merge.
* OperatorBase should not store OpDesc because not All op contains an
OpDesc and not all ops create from OpDesc.
* Networks do not contain OpDesc and are not created by OpDesc
* Do not register Network to OpRegistry.
* The network is directly created by the user in Python. Not from
registry.
* Correctly handle the `inputs` and `outputs` of a Network.
* Add CompleteAddOp() methods
* Remove `AddOp(OpDesc&)` in net-op. All op are added by OperatorPtr.
* Rewrite unit test for truly tested what networks do.
* optimise operator_test
All OpCreation method are generated by
`create_op_creation_methods::__bootstrap__` method, and stores in
`op_creations` object and its methods.
There are three parts to implement this feature.
1. Get all registered `OpProto` from C++ side. It is implemented in
`get_all_op_protos` method.
1. Create a function to convert `kwargs` to `OpDesc` base on each op's
`OpProto`. The `OpDescCreationMethod` class.
1. Convert `OpProto` to `docstring` by `get_docstring_from_op_proto`
method.
All three methods are unit tested. The `__bootstrap__` just combines
them together and create a method in runtime.
For details, please reference the doc string in
`create_op_creation_methods.py` and the unit test
`test_op_creation_methods.py`.
1. Add template T which indicates data type to `CopyFrom()`, `Slice()`
and `ShareData()` functions. This makes `CopyData()` code much clearer.
2. Add `set_dim()`.
3. `product(DDim)` transforms `DDim` to `vector<int>` first and then calculate
its product. That might be quite slow. For `product(dims_)` is frequently
used in Tensor, we add a mumber variable `numel_` as a cache of the
product result.
TODO: refactor `product()` to make it more efficient.
4. Unable Tensor::operator=
5. Remove the limit of POD type, because `float16` and `int8` are not POD type.
* Let OpProto support multiple and temporary
* Each input/output of Paddle's Op could be a list. Add multiple mark to
OpProto. Also add a `input_format`/`output_format` attribute if that
Op has multiple input or output. The format of that attribute please
reference the comments in `op_proto.proto`
* Add temporary mark, because some output of an Op is not used by user
but used by other op for faster computation. Explicitly mark which
output is temporary could let future memory/computation optimization.
* Add generated field to AttrProto.
* Add `AddInputs`/`AddOutputs` function
* It is more readable to invoke `AddInputs` not
`AddInput(multiple=true)`.
* Convert `op` --> `operators`
* Remove AddType in OpProtoMaker, because type is part of registry.
* Rename CPU_OR_GPU --> DEVICE_TYPE in registry macro.
1. Add `Tensor::CopyFrom`. Current version can only support CPU memory
copy. The support of GPU will be provided later by `paddle::memory`.
The current implementation of `Tensor::CopyFrom` is a little inefficient:
Every time `CopyFrom` is called, tensor will re-allocate its memory. However, if
we try to check and reuse `placeholder_`, we have to provide a template
parameter for `CopyFrom` to indicate the data type. It seems strange for
a simple copy function.
2. Add `Tensor::mutable_data(Place place)`, which directly use member
variable `dims_` as its dim parameter. This interface is required by
`Op::InferShape`.
* User can register OpKernel to its Ops. The OpKernelMap saved in
OperatorWithKernel. Each Op which inherits OperatorWithKernel will
use `OpKernel::Compute` instead of Run.
Add OperatorBase.
issue: https://github.com/PaddlePaddle/Paddle/issues/2790
Paddle design the Operator with Kernel. OperatorBase has no type and device information when create, One operator can have multiple kernels, Operator will choose a kernel to run according to context. The kernel should be bind to Operator before or during Operator running.
1. Add member variable 'DDim dims_' and a getter function 'dims()'.
'dims' is supposed to hold tensor's shape during Op::InferShape.
2. Remove 'mutable_data' which use default Place. User must specify a
explicit Place when call 'mutable_data'.
3. A PlaceHolder may be shared by more than one tensor, and some of them may be the others' slices. So we add a new member variable 'offset_' for Tensor, which is used to show the byte offset between PlaceHolder::ptr_ and where tensor's data really begins.
4. Add functions 'ShareDataFrom' and 'Slice' for Tensor.
TODO: Tensor needs a 'CopyFrom' function.
* Move static variable defined in .cc
We cannot define static variable in .h, because it will be multi-defined
errors.
Also fix some cpp syntax, like:
* Prefer to use algorithm not manually for-loop, to make code more
readable.
* Remove unused `()`.
* Enforce take a bool. It is no need `xxx==true`.
* Use range-based for-loop iterator from op_desc.attrs
* Fix a protential static variable init order error
* init op_registry.h
* dev op_registry.h
* add 'attr_checker.h', which is a draft of op attribute checker.
* rename some macro parameters
* 1. Use `Attribute` and `AttributeMap` instead of `OpDesc`. `AttributeMap` is a unordered_map of <string, Attribute>, and `Attribute` is a boost::variant object to hold multiple types of attribute value.
2. Use `PADDLE_ENFORCE` to print checkers' fail message.
3. Abstract default value operations to a new function: `DefaultChecker`.
* rename DefaultChecker to DefaultValueSetter
ZZ
* Finish op_registry
1. Complete the development of interfaces between OpRegistry and
Protobuf.
2. Add unit test for op_registry.h
* Add demo and test of custome checker
* fix merge conflict
Python should be able to manipulate Protobuf message because:
1. Python's `create_op_creation_methods` take the `OpProto` array to
generate all `op_creation_methods` in RunTime.
2. All `op_creation_methods` will create an `OpDesc` and pass it to
Paddle C++ method `CreateOp` and return the Op handle.
Here is the list of what is added in this commit:
* Add `protobuf_generate_python` if it is not defined.
* Before cmake 3.4, `protobuf_generate_python` is not defined. Just
copy the implementation of that function in `protobuf.cmake`
* Add `py_proto_compile` function in `cmake/generic.cmake`.
* It follows bazel's API interface.
* https://github.com/pubref/rules_protobuf#rules
* Add an empty package named `paddle.v2.framework`, all python code of
`paddle::framework` will be in that package.
* Generate protobuf's python module `__init__.py` by `touch` while
compiling.
* Change setup.py.in, make `paddle.v2.framework.proto` uses the
generated protobuf pythons.
* add op_desc.proto
In Operator design, we need a proto message to describe an Operator.
Third-party language such as python can build this proto message and use
AddOp(const OpDesc& op_desc) of Paddle core to construct an Op in the
Network.
It will fix#2728.
Maybe it is silly to `target_link_libraries` for static library,
because a static library do not need to link other libraries. But
it will tell cmake how to propagate dependencies.
The solution comes from
[here](http://floooh.github.io/2016/01/12/cmake-dependency-juggling.html).
* Also change op_proto_test DEPS for testing this fix works.
OpProto is a proto message that helps 3rd-party language bindings, e.g.
`Python`, to generate operator creation methods. The operator creation
method is the low-level API for 3rd-party language bindings. Op creation
methods take the user's input in that language, and convert users inputs
into `OpDesc` message, then passing that `OpDesc` message to Paddle's
C++ core and create an operator.
* A separated `attr_type.proto` is added, because that file wound
be included by `op_desc.proto` in future.
dongzhihong
fengjiayi
Yu Yang
liaogang
QI JUN
Yi Wang
Yi Wang
Qiao Longfei
Yan Chunwei
Superjom
Yi Wang