Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fluid_init
wanghaoshuang
7 years ago
commit
f31bb1476c
@ -0,0 +1,175 @@
|
||||
# Varient Length supported RNN Design
|
||||
For the learning of variable length sequences, the existing mainstream frameworks such as tensorflow, pytorch, caffe2, mxnet and so on all use padding.
|
||||
|
||||
Different-length sequences in a mini-batch will be padded with zeros and transformed to same length.
|
||||
|
||||
The existing RNN implementations of the PaddlePaddle is `RecurrentLayerGroup`,
|
||||
which supports the variable length sequences without padding.
|
||||
This doc will design fluid's RNN based on this idea.
|
||||
|
||||
## Multi-layer sequence data format `LODTensor`
|
||||
At present, Paddle stores data in one mini-batch in one-dimensional array.
|
||||
|
||||
`Argument.sequenceStartPositions` is used to store information for each sentence.
|
||||
|
||||
In Paddle, `Argument.subSequenceStartPositions` is used to store 2 levels of sequence information, while higher dimensional sequences can not be supported.
|
||||
|
||||
In order to support the storage of `N-level` sequences, we define sequence information as the following data structure.
|
||||
|
||||
|
||||
```c++
|
||||
std::shared_ptr<std::vector<std::vector<int>>> lod_start_pos_;
|
||||
```
|
||||
|
||||
Or more clearly defined here
|
||||
|
||||
```c++
|
||||
typedef std::vector<int> level_t;
|
||||
std::vector<level_t> lod_start_pos;
|
||||
```
|
||||
Each `level_t` here stores a level of offset information consistent with paddle's current practice.
|
||||
|
||||
In order to transmit sequence information more transparently, we have introduced a new tensor called `LODTensor`[1].
|
||||
Its tensor-related interfaces all inherit directly from `Tensor`, but it also adds serial-related interfaces.
|
||||
Thus, when working with a `LODTensor`, ordinary `Op` is used directly as `Tensor`.
|
||||
The `Op` of the operation sequence will additionally operate the relevant interface of the `LODTensor` variable-length sequence operation.
|
||||
|
||||
The definition of `LODTensor` is as follows:
|
||||
|
||||
|
||||
```c++
|
||||
class LODTensor : public Tensor {
|
||||
public:
|
||||
size_t Levels() const { return seq_start_positions_.size(); }
|
||||
size_t Elements(int level = 0) const {
|
||||
return seq_start_positions_[level].size();
|
||||
}
|
||||
// slice of level[elem_begin: elem_end]
|
||||
// NOTE low performance in slice seq_start_positions_.
|
||||
// TODO should call Tensor's Slice.
|
||||
LODTensor LODSlice(int level, int elem_begin, int elem_end) const;
|
||||
|
||||
// slice with tensor's data shared with this.
|
||||
LODTensor LODSliceShared(int level, int elem_begin, int elem_end) const;
|
||||
|
||||
// copy other's lod_start_pos_, to share LOD info.
|
||||
// NOTE the LOD info sould not be changed.
|
||||
void ShareConstLODFrom(const LODTensor &other) {
|
||||
lod_start_pos_ = other.lod_start_pos_;
|
||||
}
|
||||
// copy other's lod_start_pos_'s content, free to mutate.
|
||||
void ShareMutableLODFrom(const LODTensor &other) {
|
||||
lod_start_pos_ = std::make_shared <
|
||||
std::vector<std::vector<int>>(other.lod_start_pos_.begin(),
|
||||
other.lod_start_pos_.end());
|
||||
}
|
||||
|
||||
private:
|
||||
std::shared_ptr<std::vector<std::vector<int>>> lod_start_pos_;
|
||||
};
|
||||
```
|
||||
Among them, `lod_start_pos_` uses `shared_ptr` to reduce the cost of storage and replication.
|
||||
`LODTensor` can be thought as an extension of `Tensor`, which is almost completely compatible with the original `Tensor`.
|
||||
|
||||
## How to support the framework
|
||||
### Replace `Tensor` with `LoDTensor`
|
||||
To implement the passing of `LODTensor`, most `Tensor` in the framework need to be replaced with `LODTensor`.
|
||||
Simple implementation, directly **replace all previous `Tensor` with `LODTensor`** , where you can directly modify the `Tensor` interface created in `pybind.cc`.
|
||||
|
||||
In addition, the user may need to perceive the existence of a sequence (such as the sequence of the visualization needs to parse the output sequence in the model), so some of the serial operation APIs also need to be exposed to the python layer.
|
||||
|
||||
### Transmit `lod_start_pos` along with the Op call chain
|
||||
`lod_start_pos` is passed along with the Op call chain
|
||||
The framework needs to support the following features to implement the transmit of `lod_start_pos`:
|
||||
|
||||
1. Implement the transfer as `shared_ptr`
|
||||
- Do not modify the contents of `lod_start_pos` as a consumer
|
||||
- Modify producer of `lod_start_pos` as producer
|
||||
- Conventions consumer only needs to copy `shared_ptr` passed over
|
||||
- producer needs to create its own independent memory to store its own independent modifications and expose `shared_ptr` to subsequent consumer
|
||||
- Since the transfer process is implemented by copying `shared_ptr`, the framework only needs to pass `lod_start_pos` once.
|
||||
|
||||
2. Op is transparent enough not to sense `lod_start_pos`
|
||||
3. Producer Op that needs to modify `lod_start_pos` can update its `lod_start_pos` data when `Run`
|
||||
|
||||
## sorted by length
|
||||
After sorting by length, the batch size from the forward time step will naturally decrement, and you can directly plug it into Net to do the batch calculation.
|
||||
|
||||
For example, the original input:
|
||||
|
||||
```
|
||||
origin:
|
||||
xxxx
|
||||
xx
|
||||
xxx
|
||||
|
||||
-> sorted:
|
||||
xxxx
|
||||
xxx
|
||||
xx
|
||||
```
|
||||
|
||||
After `SegmentInputs`, there will be 4 time steps, the input of each time step is as follows (vertical arrangement)
|
||||
|
||||
```
|
||||
0 1 2 3
|
||||
x x x x
|
||||
x x x
|
||||
x x
|
||||
```
|
||||
|
||||
In order to track the changes before and after sorting, use here
|
||||
|
||||
```c++
|
||||
struct SortedSeqItem {
|
||||
void *start{nullptr};
|
||||
void *end{nullptr};
|
||||
};
|
||||
|
||||
std::vector<SortedSeqItem> sorted_seqs;
|
||||
```
|
||||
To track the position of the sequence after sorting, and add a new interface
|
||||
|
||||
```c++
|
||||
std::vector<SortedSeqItem> SortBySeqLen(const LODTensor& tensor);
|
||||
```
|
||||
Due to the sequence of input sequences, the following existing interfaces need to be modified:
|
||||
|
||||
- InitMemories, memory needs to be rearranged according to `sorted_seqs`
|
||||
- SetmentInputs
|
||||
- ConcatOutputs
|
||||
|
||||
In addition, because `sorted_seqs` needs to be multiplexed with `RecurrentGradientOp`, it will become a new output of `RecurrentOp`.
|
||||
It is passed in as an input to `RecurrentGradientOp`.
|
||||
|
||||
## InitMemories
|
||||
Due to the sequence change, the order of the elements on the `boot_memories` batch also needs to be rearranged accordingly.
|
||||
|
||||
## SegmentInputs
|
||||
|
||||
`SegmentInputs` relies on the information of `sorted_seqs` to cut the original sequence from the horizontal to the input of each step in the sorted sequence order.
|
||||
|
||||
the transition is as follows:
|
||||
```
|
||||
origin:
|
||||
xxxx
|
||||
xx
|
||||
xxx
|
||||
|
||||
|
|
||||
|
|
||||
\ /
|
||||
!
|
||||
0 1 2 3
|
||||
x x x x
|
||||
x x x
|
||||
x x
|
||||
```
|
||||
## ConcatOutputs
|
||||
`ConcatOutputs` needs
|
||||
|
||||
- Restore the output of each time step back to the original input sequence order (to prevent the order of Infer phase from being upset)
|
||||
- Concat each sequence as a regular mini-batch representation
|
||||
|
||||
## references
|
||||
1. [Level of details](https://en.wikipedia.org/wiki/Level_of_detail)
|
||||
@ -1 +0,0 @@
|
||||
../../v2/getstarted/quickstart_cn.rst
|
||||
@ -1 +0,0 @@
|
||||
../../v2/getstarted/quickstart_en.rst
|
||||
@ -0,0 +1,49 @@
|
||||
Quick Start
|
||||
============
|
||||
|
||||
Quick Install
|
||||
-------------
|
||||
|
||||
You can use pip to install PaddlePaddle with a single command, supports
|
||||
CentOS 6 above, Ubuntu 14.04 above or MacOS 10.12, with Python 2.7 installed.
|
||||
Simply run the following command to install, the version is cpu_avx_openblas:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle
|
||||
|
||||
If you need to install GPU version (cuda7.5_cudnn5_avx_openblas), run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle-gpu
|
||||
|
||||
For more details about installation and build: :ref:`install_steps` .
|
||||
|
||||
Quick Use
|
||||
---------
|
||||
|
||||
Create a new file called housing.py, and paste this Python
|
||||
code:
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import paddle.dataset.uci_housing as uci_housing
|
||||
import paddle.fluid as fluid
|
||||
|
||||
with fluid.scope_guard(fluid.core.Scope()):
|
||||
# initialize executor with cpu
|
||||
exe = fluid.Executor(place=fluid.CPUPlace())
|
||||
# load inference model
|
||||
[inference_program, feed_target_names,fetch_targets] = \
|
||||
fluid.io.load_inference_model(uci_housing.fluid_model(), exe)
|
||||
# run inference
|
||||
result = exe.run(inference_program,
|
||||
feed={feed_target_names[0]: uci_housing.predict_reader()},
|
||||
fetch_list=fetch_targets)
|
||||
# print predicted price is $12,273.97
|
||||
print 'Predicted price: ${:,.2f}'.format(result[0][0][0] * 1000)
|
||||
|
||||
Run :code:`python housing.py` and voila! It should print out a list of predictions
|
||||
for the test housing data.
|
||||
@ -0,0 +1,197 @@
|
||||
/* Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#include "mkldnn.hpp"
|
||||
#include "paddle/fluid/framework/tensor.h"
|
||||
#include "paddle/fluid/operators/mul_op.h"
|
||||
#include "paddle/fluid/platform/device_context.h"
|
||||
#include "paddle/fluid/platform/mkldnn_helper.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace operators {
|
||||
|
||||
using paddle::framework::Tensor;
|
||||
using paddle::platform::MKLDNNDeviceContext;
|
||||
|
||||
template <typename Format = mkldnn::memory::format>
|
||||
mkldnn::memory::desc type(const std::vector<int>& dims, Format&& f) {
|
||||
return platform::MKLDNNMemDesc(dims, mkldnn::memory::data_type::f32, f);
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
class MulMKLDNNOpKernel : public paddle::framework::OpKernel<T> {
|
||||
void Compute(const paddle::framework::ExecutionContext& ctx) const override {
|
||||
PADDLE_ENFORCE(paddle::platform::is_cpu_place(ctx.GetPlace()),
|
||||
"It must use CPUPlace.");
|
||||
|
||||
auto& dev_ctx = ctx.template device_context<MKLDNNDeviceContext>();
|
||||
auto mkldnn_engine = dev_ctx.GetEngine();
|
||||
|
||||
auto input = ctx.Input<Tensor>("X");
|
||||
auto weight = ctx.Input<Tensor>("Y");
|
||||
|
||||
PADDLE_ENFORCE(input->dims().size() & (2 | 4),
|
||||
"Input must be with 2 or 4 dimensions, i.e. NC or NCHW");
|
||||
PADDLE_ENFORCE(weight->dims().size() & (2 | 4),
|
||||
"Weights must be with 2 or 4 dimensions, i.e. OI or OIHW");
|
||||
|
||||
std::vector<int> w_tz = paddle::framework::vectorize2int(weight->dims());
|
||||
std::vector<int> src_tz = paddle::framework::vectorize2int(input->dims());
|
||||
|
||||
auto src_md =
|
||||
src_tz.size() != 2
|
||||
? type(src_tz, mkldnn::memory::format::nchw)
|
||||
: type({src_tz[0], src_tz[1]}, mkldnn::memory::format::nc);
|
||||
|
||||
auto dst_md = type({src_tz[0], w_tz[1]}, mkldnn::memory::format::nc);
|
||||
|
||||
auto weights_md =
|
||||
src_tz.size() != 2
|
||||
? type({w_tz[1], src_tz[1], src_tz[2], src_tz[3]},
|
||||
mkldnn::memory::format::oihw)
|
||||
: type({w_tz[1], src_tz[1]}, mkldnn::memory::format::oi);
|
||||
|
||||
auto output = ctx.Output<Tensor>("Out");
|
||||
T* output_data = output->mutable_data<T>(ctx.GetPlace());
|
||||
|
||||
const std::string key = ctx.op().Output("Out");
|
||||
const std::string key_fc_pd = key + "@mul_pd";
|
||||
|
||||
const T* input_data = input->data<T>();
|
||||
const T* w_data = weight->data<T>();
|
||||
|
||||
auto dst_memory = mkldnn::memory({dst_md, mkldnn_engine}, output_data);
|
||||
|
||||
auto src_memory = mkldnn::memory({src_md, mkldnn_engine},
|
||||
platform::to_void_cast(input_data));
|
||||
|
||||
auto weights_memory = mkldnn::memory({weights_md, mkldnn_engine},
|
||||
platform::to_void_cast(w_data));
|
||||
|
||||
auto pd = platform::MKLDNNFwdPrimitiveDesc<mkldnn::inner_product_forward>(
|
||||
mkldnn_engine, src_md, weights_md, dst_md);
|
||||
|
||||
dev_ctx.SetBlob(key_fc_pd, pd);
|
||||
|
||||
auto forward = mkldnn::inner_product_forward(*pd, src_memory,
|
||||
weights_memory, dst_memory);
|
||||
|
||||
std::vector<mkldnn::primitive> pipeline = {forward};
|
||||
mkldnn::stream(mkldnn::stream::kind::eager).submit(pipeline).wait();
|
||||
}
|
||||
};
|
||||
|
||||
template <typename T>
|
||||
class MulMKLDNNGradOpKernel : public paddle::framework::OpKernel<T> {
|
||||
public:
|
||||
void Compute(const paddle::framework::ExecutionContext& ctx) const override {
|
||||
PADDLE_ENFORCE(paddle::platform::is_cpu_place(ctx.GetPlace()),
|
||||
"It must use CPUPlace.");
|
||||
|
||||
auto& dev_ctx = ctx.template device_context<MKLDNNDeviceContext>();
|
||||
auto mkldnn_engine = dev_ctx.GetEngine();
|
||||
|
||||
const Tensor* input = ctx.Input<Tensor>("X");
|
||||
const Tensor* w = ctx.Input<Tensor>("Y");
|
||||
|
||||
const Tensor* out_grad = ctx.Input<Tensor>(framework::GradVarName("Out"));
|
||||
Tensor* input_grad = ctx.Output<Tensor>(framework::GradVarName("X"));
|
||||
Tensor* w_grad = ctx.Output<Tensor>(framework::GradVarName("Y"));
|
||||
|
||||
const std::string key = ctx.op().Input("Out");
|
||||
const std::string key_fc_pd = key + "@mul_pd";
|
||||
|
||||
const T* input_data = input->data<T>();
|
||||
const T* w_data = w->data<T>();
|
||||
const T* out_grad_data = out_grad->data<T>();
|
||||
T* input_grad_data = nullptr;
|
||||
T* w_grad_data = nullptr;
|
||||
|
||||
if (input_grad) {
|
||||
input_grad_data = input_grad->mutable_data<T>(ctx.GetPlace());
|
||||
}
|
||||
if (w_grad) {
|
||||
w_grad_data = w_grad->mutable_data<T>(ctx.GetPlace());
|
||||
}
|
||||
|
||||
std::vector<int> src_tz = paddle::framework::vectorize2int(input->dims());
|
||||
std::vector<int> w_tz = paddle::framework::vectorize2int(w->dims());
|
||||
|
||||
auto src_md =
|
||||
src_tz.size() != 2
|
||||
? type(src_tz, mkldnn::memory::format::nchw)
|
||||
: type({src_tz[0], src_tz[1]}, mkldnn::memory::format::nc);
|
||||
|
||||
auto dst_md = type({src_tz[0], w_tz[1]}, mkldnn::memory::format::nc);
|
||||
|
||||
auto weights_md =
|
||||
src_tz.size() != 2
|
||||
? type({w_tz[1], src_tz[1], src_tz[2], src_tz[3]},
|
||||
mkldnn::memory::format::oihw)
|
||||
: type({w_tz[1], src_tz[1]}, mkldnn::memory::format::oi);
|
||||
|
||||
auto src_memory = mkldnn::memory({src_md, mkldnn_engine},
|
||||
platform::to_void_cast(input_data));
|
||||
|
||||
auto dst_memory = mkldnn::memory({dst_md, mkldnn_engine},
|
||||
platform::to_void_cast(out_grad_data));
|
||||
|
||||
auto weight_memory = mkldnn::memory({weights_md, mkldnn_engine},
|
||||
platform::to_void_cast(w_data));
|
||||
|
||||
auto pd =
|
||||
std::static_pointer_cast<mkldnn::inner_product_forward::primitive_desc>(
|
||||
dev_ctx.GetBlob(key_fc_pd));
|
||||
|
||||
PADDLE_ENFORCE(pd != nullptr, "Fail to find pd in device context");
|
||||
|
||||
if (w_grad) {
|
||||
auto weights_grad_memory = mkldnn::memory(
|
||||
{weights_md, mkldnn_engine}, platform::to_void_cast(w_grad_data));
|
||||
|
||||
auto bwd_weight_pd = platform::MKLDNNBwdPrimitiveDesc<
|
||||
mkldnn::inner_product_backward_weights>(mkldnn_engine, *pd, src_md,
|
||||
weights_md, dst_md);
|
||||
|
||||
auto bwd_weights_prim = mkldnn::inner_product_backward_weights(
|
||||
bwd_weight_pd, src_memory, dst_memory, weights_grad_memory);
|
||||
|
||||
std::vector<mkldnn::primitive> pipeline{bwd_weights_prim};
|
||||
mkldnn::stream(mkldnn::stream::kind::eager).submit(pipeline).wait();
|
||||
}
|
||||
|
||||
if (input_grad) {
|
||||
auto src_grad_memory = mkldnn::memory(
|
||||
{src_md, mkldnn_engine}, platform::to_void_cast(input_grad_data));
|
||||
|
||||
auto bwd_data_pd =
|
||||
platform::MKLDNNBwdPrimitiveDesc<mkldnn::inner_product_backward_data>(
|
||||
mkldnn_engine, *pd, src_md, weights_md, dst_md);
|
||||
|
||||
auto bwd_data_prim = mkldnn::inner_product_backward_data(
|
||||
bwd_data_pd, dst_memory, weight_memory, src_grad_memory);
|
||||
|
||||
std::vector<mkldnn::primitive> pipeline{bwd_data_prim};
|
||||
mkldnn::stream(mkldnn::stream::kind::eager).submit(pipeline).wait();
|
||||
}
|
||||
}
|
||||
};
|
||||
} // namespace operators
|
||||
} // namespace paddle
|
||||
|
||||
REGISTER_OP_KERNEL(mul, MKLDNN, ::paddle::platform::CPUPlace,
|
||||
paddle::operators::MulMKLDNNOpKernel<float>);
|
||||
|
||||
REGISTER_OP_KERNEL(mul_grad, MKLDNN, ::paddle::platform::CPUPlace,
|
||||
paddle::operators::MulMKLDNNGradOpKernel<float>);
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue