Liu Yiqun
8 years ago
commit
a310946f8f
@ -0,0 +1,73 @@
|
||||
# Design for TensorArray
|
||||
TensorArray as a new concept is borrowed from TensorFlow,
|
||||
it is meant to be used with dynamic iteration primitives such as `while_loop` and `map_fn`.
|
||||
|
||||
This concept can be used to support our new design of dynamic operations, and help to refactor some existing variant-sentence-related layers,
|
||||
such as `RecurrentGradientMachine`.
|
||||
|
||||
In [our design for dynamic RNN](https://github.com/PaddlePaddle/Paddle/pull/4401),
|
||||
`TensorArray` is used to segment inputs and store states in all time steps.
|
||||
By providing some methods similar to a C++ array,
|
||||
the definition of some state-based dynamic models such as RNN could be more natural and highly flexible.
|
||||
|
||||
## Dynamic-Related Methods
|
||||
Some basic methods should be proposed as follows:
|
||||
|
||||
### stack()
|
||||
Pack the values in a `TensorArray` into a tensor with rank one higher than each tensor in `values`.
|
||||
### unstack(axis=0)
|
||||
Unpacks the given dimension of a rank-`R` tensor into rank-`(R-1)` tensors.
|
||||
### concat()
|
||||
Return the values in the `TensorArray` as a concatenated Tensor.
|
||||
### write(index, value, data_shared=true)
|
||||
Write value into index of the TensorArray.
|
||||
### read(index)
|
||||
Read the value at location `index` in the `TensorArray`.
|
||||
### size()
|
||||
Return the number of values.
|
||||
|
||||
## LoDTensor-related Supports
|
||||
The `RecurrentGradientMachine` in Paddle serves as a flexible RNN layer; it takes variant length sequences as input,
|
||||
because each step of RNN could only take a tensor-represented batch of data as input,
|
||||
some preprocess should be taken on the inputs such as sorting the sentences by their length in descending order and cut each word and pack to new batches.
|
||||
|
||||
Such cut-like operations can be embedded into `TensorArray` as general methods called `unpack` and `pack`.
|
||||
|
||||
With these two methods, a variant-sentence-RNN can be implemented like
|
||||
|
||||
```c++
|
||||
// input is the varient-length data
|
||||
LodTensor sentence_input(xxx);
|
||||
TensorArray ta;
|
||||
Tensor indice_map;
|
||||
Tensor boot_state = xxx; // to initialize rnn's first state
|
||||
TensorArray::unpack(input, 1/*level*/, true/*sort_by_length*/, &ta, &indice_map);
|
||||
TessorArray step_outputs;
|
||||
TensorArray states;
|
||||
|
||||
for (int step = 0; step = ta.size(); step++) {
|
||||
auto state = states.read(step);
|
||||
// rnnstep is a function which acts like a step of RNN
|
||||
auto step_input = ta.read(step);
|
||||
auto step_output = rnnstep(step_input, state);
|
||||
step_outputs.write(step_output, true/*data_shared*/);
|
||||
}
|
||||
|
||||
// rnn_output is the final output of an rnn
|
||||
LoDTensor rnn_output = ta.pack(ta, indice_map);
|
||||
```
|
||||
the code above shows that by embedding the LoDTensor-related preprocess operations into `TensorArray`,
|
||||
the implementation of a RNN that supports varient-length sentences is far more concise than `RecurrentGradientMachine` because the latter mixes all the codes together, hard to read and extend.
|
||||
|
||||
|
||||
some details are as follows.
|
||||
|
||||
### unpack(level, sort_by_length)
|
||||
Split LodTensor in some `level` and generate batches, if set `sort_by_length`, will sort by length.
|
||||
|
||||
Returns:
|
||||
|
||||
- a new `TensorArray`, whose values are LodTensors and represents batches of data.
|
||||
- an int32 Tensor, which stores the map from the new batch's indices to original LoDTensor
|
||||
### pack(level, indices_map)
|
||||
Recover the original LoD-arranged LoDTensor with the values in a `TensorArray` and `level` and `indices_map`.
|
||||
@ -0,0 +1,146 @@
|
||||
## How to use Eigen in Paddle
|
||||
|
||||
Essentially, a neural network is a compute graph. T data needed for the computation is stored in `Tensor`s and its computation procedure is described by `Operator`s. An `Operator` calls the `Compute` interface in its corresponding `OpKernel` and operates on the `Tensor`.
|
||||
|
||||
|
||||
### Eigen Tensor Module
|
||||
|
||||
The Eigen Tensor module supports powerful element-wise computation. In addition, a piece of code written using it can be run on both the CPU and the GPU.
|
||||
|
||||
Note that Eigen Tensor is still being actively developed, so its tests are not completely covered and its documentation may be sparse.
|
||||
|
||||
For details on Eigen Tensor module, please see [doc 1](https://github.com/RLovelett/eigen/blob/master/unsupported/Eigen/CXX11/src/Tensor/README.md) and [doc 2](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md).
|
||||
|
||||
|
||||
### paddle::framework::Tensor
|
||||
|
||||
Paddle Tensor's is defined in the framework directory with the following interface:
|
||||
|
||||
```cpp
|
||||
class Tensor {
|
||||
public:
|
||||
/*! Return a pointer to mutable memory block. */
|
||||
template <typename T>
|
||||
inline T* data();
|
||||
|
||||
/**
|
||||
* @brief Return a pointer to mutable memory block.
|
||||
* @note If not exist, then allocation.
|
||||
*/
|
||||
template <typename T>
|
||||
inline T* mutable_data(platform::Place place);
|
||||
|
||||
/**
|
||||
* @brief Return a pointer to mutable memory block.
|
||||
*
|
||||
* @param[in] dims The dimensions of the memory block.
|
||||
* @param[in] place The place of the memory block.
|
||||
*
|
||||
* @note If not exist, then allocation.
|
||||
*/
|
||||
template <typename T>
|
||||
inline T* mutable_data(DDim dims, platform::Place place);
|
||||
|
||||
/*! Resize the dimensions of the memory block. */
|
||||
inline Tensor& Resize(const DDim& dims);
|
||||
|
||||
/*! Return the dimensions of the memory block. */
|
||||
inline const DDim& dims() const;
|
||||
|
||||
private:
|
||||
/*! holds the memory block if allocated. */
|
||||
std::shared_ptr<Placeholder> holder_;
|
||||
|
||||
/*! points to dimensions of memory block. */
|
||||
DDim dim_;
|
||||
};
|
||||
```
|
||||
|
||||
`Placeholder` is used to delay memory allocation; that is, we can first define a tensor, using `Resize` to configure its shape, and then call `mutuable_data` to allocate the actual memory.
|
||||
|
||||
```cpp
|
||||
paddle::framework::Tensor t;
|
||||
paddle::platform::CPUPlace place;
|
||||
// set size first
|
||||
t.Resize({2, 3});
|
||||
// allocate memory on CPU later
|
||||
t.mutable_data(place);
|
||||
```
|
||||
|
||||
### paddle::framework::Tensor Usage
|
||||
`AddOp` demonstrates Tensor's usage.
|
||||
|
||||
- InferShape
|
||||
|
||||
When computing a neural network's compute graph, first call every `Operator`'s `InferShape` method, and use `Resize` to configure the size of the output tensor.
|
||||
|
||||
```cpp
|
||||
void InferShape(const framework::InferShapeContext &ctx) const override {
|
||||
PADDLE_ENFORCE_EQ(ctx.Input<Tensor>("X")->dims(),
|
||||
ctx.Input<Tensor>("Y")->dims(),
|
||||
"Two input of Add Op's dimension must be same.");
|
||||
ctx.Output<Tensor>("Out")->Resize(ctx.Input<Tensor>("X")->dims());
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
- Run
|
||||
|
||||
```cpp
|
||||
void Compute(const framework::ExecutionContext& context) const override {
|
||||
auto* input0 = context.Input<Tensor>("X");
|
||||
auto* input1 = context.Input<Tensor>("Y");
|
||||
auto* output = context.Output<Tensor>("Out");
|
||||
|
||||
output->mutable_data<T>(context.GetPlace());
|
||||
|
||||
auto x = EigenVector<T>::Flatten(*input0);
|
||||
auto y = EigenVector<T>::Flatten(*input1);
|
||||
auto z = EigenVector<T>::Flatten(*output);
|
||||
|
||||
auto place = context.GetEigenDevice<Place>();
|
||||
|
||||
z.device(place) = x + y;
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
### paddle::framework::Tensor到EigenTensor的转换
|
||||
|
||||
As shown above, in actual computation, we need to transform the input and output `Tensor`s into formats Eigen supports. We show some functions in [eigen.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen.h) to implement the transformation from `paddle::framework::Tensor`to `EigenTensor/EigenMatrix/EigenVector/EigenScalar`.
|
||||
|
||||
Using EigenTensor as an example:
|
||||
|
||||
```cpp
|
||||
Tensor t;
|
||||
float* p = t.mutable_data<float>(make_ddim({1, 2, 3}), platform::CPUPlace());
|
||||
for (int i = 0; i < 1 * 2 * 3; i++) {
|
||||
p[i] = static_cast<float>(i);
|
||||
}
|
||||
|
||||
EigenTensor<float, 3>::Type et = EigenTensor<float, 3>::From(t);
|
||||
```
|
||||
|
||||
`From` is an interfacing method provided by the EigenTensor template, which implements the transformation from a `paddle::framework::Tensor` object to an EigenTensor. Since `rank` is a template parameter, it needs to be explicitly specified at the time of the transformation.
|
||||
|
||||
In Eigen, tensors with different ranks are different types, with `Vector` bring a rank-1 instance. Note that `EigenVector<T>::From` uses a transformation from an 1-dimensional Paddle tensor to a 1-dimensional Eigen tensor while `EigenVector<T>::Flatten` reshapes a paddle tensor and flattens it into a 1-dimensional Eigen tensor. Both resulting tensors are still typed EigenVector.
|
||||
|
||||
For more transformations, see the [unit tests](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/eigen_test.cc) in the `eigen_test.cc` file.
|
||||
|
||||
|
||||
|
||||
### Implementing Computation
|
||||
|
||||
While computing, the device interface is needed from the EigenTensors on the left hand side of the assignments. Note that the computation between EigenTensors only changes the data originally inthe Tensor and does not change all the shape information associated with the Tensor.
|
||||
|
||||
```cpp
|
||||
auto x = EigenVector<T>::Flatten(*input0);
|
||||
auto y = EigenVector<T>::Flatten(*input1);
|
||||
auto z = EigenVector<T>::Flatten(*output);
|
||||
auto place = context.GetEigenDevice<Place>();
|
||||
z.device(place) = x + y;
|
||||
```
|
||||
|
||||
In this code segment, input0/input1/output can be Tensors of arbitrary dimension. We are calling Flatten from EigenVector, transforming a tensor of any dimension into a 1-dimensional EigenVector. After completing computation, input0/input1/output will retain the same shape information, and they can be resized using the `Resize` interface.
|
||||
|
||||
Because the Eigen Tensor module is under-documented, please refer to `OpKernel`'s computation code in TensorFlow's [kernel module documentation](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/core/kernels).
|
||||
@ -1,5 +0,0 @@
|
||||
---
|
||||
Language: Cpp
|
||||
BasedOnStyle: Google
|
||||
Standard: Cpp11
|
||||
...
|
||||
@ -0,0 +1 @@
|
||||
../framework/.clang-format
|
||||
@ -1,5 +0,0 @@
|
||||
---
|
||||
Language: Cpp
|
||||
BasedOnStyle: Google
|
||||
Standard: Cpp11
|
||||
...
|
||||
@ -0,0 +1 @@
|
||||
../framework/.clang-format
|
||||
@ -0,0 +1 @@
|
||||
../framework/.clang-format
|
||||
@ -1,6 +1,6 @@
|
||||
if(WITH_PYTHON)
|
||||
cc_library(paddle_pybind SHARED
|
||||
SRCS pybind.cc protobuf.cc
|
||||
SRCS pybind.cc exception.cc protobuf.cc
|
||||
DEPS pybind python backward
|
||||
${GLOB_OP_LIB})
|
||||
endif(WITH_PYTHON)
|
||||
|
||||
@ -0,0 +1,34 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#include "paddle/pybind/exception.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace pybind {
|
||||
|
||||
void BindException(pybind11::module& m) {
|
||||
static pybind11::exception<platform::EnforceNotMet> exc(m, "EnforceNotMet");
|
||||
pybind11::register_exception_translator([](std::exception_ptr p) {
|
||||
try {
|
||||
if (p) std::rethrow_exception(p);
|
||||
} catch (const platform::EnforceNotMet& e) {
|
||||
exc(e.what());
|
||||
}
|
||||
});
|
||||
|
||||
m.def("__unittest_throw_exception__", [] { PADDLE_THROW("test exception"); });
|
||||
}
|
||||
|
||||
} // namespace pybind
|
||||
} // namespace paddle
|
||||
@ -0,0 +1,23 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#pragma once
|
||||
#include "paddle/platform/enforce.h"
|
||||
#include "pybind11/pybind11.h"
|
||||
namespace paddle {
|
||||
namespace pybind {
|
||||
|
||||
extern void BindException(pybind11::module& m);
|
||||
} // namespace pybind
|
||||
} // namespace paddle
|
||||
@ -0,0 +1 @@
|
||||
../framework/.clang-format
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue