wangmeng28
9 years ago
commit
b3cd67967e
@ -0,0 +1,23 @@
|
||||
# Executor Design Doc
|
||||
|
||||
## Motivation
|
||||
|
||||
We use executor to do the runtime evaluation of a `ProgramDesc`.
|
||||
|
||||
## Overview
|
||||
|
||||
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs.
|
||||
|
||||
### What does executor do?
|
||||
|
||||
It evaluates all the operators in the `block_id`th block of a `ProgramDesc`.
|
||||
|
||||
### What does executor NOT do?
|
||||
|
||||
It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run.
|
||||
|
||||
It does not do graph partitioning, meaning dividing the `ProgramDesc` into several small pieces and executing them on different devices.
|
||||
|
||||
## Implementation
|
||||
|
||||
`Executor` evaluates a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [[code]](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/executor.cc)
|
||||
@ -0,0 +1,78 @@
|
||||
# Design Doc: InferVarType
|
||||
|
||||
## The Problem Posed
|
||||
|
||||
The variable in our design can hold variant types. Such as `LoDTensor` and `SelectedRows`. An operator should be able to inference the variable types of its output.
|
||||
|
||||
For example, a `lookup table` operator takes two `LoDTensor`; one is a float tensor as the embedding table, the other is an int tensor as word ID. The gradient operator of `lookup table` will generate a `SelectedRows` as its output. A `sum` operator can take both `LoDTensor` and `SelectedRows` as its inputs and will generate a `LoDTensor` if any of its inputs is `LoDTensor`, otherwise, the `sum` operator will generate `SelectedRows` as its output.
|
||||
|
||||
The variable type will be constant at runtime. Every variable's type can either be set by the user (input data and parameter) or be inferred by the operator in compile time.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
The `InferVarType` is a compile-time function which is registered to each operator. The inferface of that function is:
|
||||
|
||||
|
||||
```c++
|
||||
using InferVarTypeFN = std::function<
|
||||
void (const OpDescBind& /*op_desc*/, BlockDescBind* /*block*/)>;
|
||||
```
|
||||
|
||||
It takes an operator description as its input and will write the output variable type and store them in block description.
|
||||
|
||||
The `InferVarTypeFN` will be registered in `OpInfo`, to replace `infer_var_type_` field. The `OpInfo` should be
|
||||
|
||||
```cpp
|
||||
struct OpInfo {
|
||||
InferVarTypeFN infer_var_type_;
|
||||
...
|
||||
};
|
||||
```
|
||||
|
||||
The default `InferVarType` will set output type as `LoDTensor`. It can be done by `GetInferVarType()`.
|
||||
|
||||
```cpp
|
||||
void DefaultInferVarType(const OpDescBind& op_desc, BlockDescBind* block) {
|
||||
// set the output type of variable as `LoDTensor`.
|
||||
// ...
|
||||
}
|
||||
|
||||
struct OpInfo {
|
||||
InferVarTypeFN infer_var_type_;
|
||||
InferVarTypeFN GetInferVarType() const {
|
||||
if (infer_var_type_) {
|
||||
return infer_var_type_;
|
||||
} else {
|
||||
return DefaultInferVarType;
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
## Register InferVarType
|
||||
|
||||
We provide a thin base class for registering an `InferVarTypeFN`. To use a base class will ease the implementation of registry since we can detect the registry entry is an `InferVarTypeFN` or not.
|
||||
|
||||
```cpp
|
||||
class VarTypeInferer {
|
||||
public:
|
||||
virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const = 0;
|
||||
}
|
||||
```
|
||||
|
||||
Operator developers can write the specialize `VarTypeInferer` as follow.
|
||||
|
||||
```cpp
|
||||
class SpecialVarTypeInferer : public VarTypeInferer {
|
||||
public:
|
||||
virtual void operator()(const OpDescBind& op_desc, BlockDescBind* block) const {
|
||||
// .. own logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then user can register the `InferVarType` just like `GradOpDescMaker` and `OpInfoMaker`.
|
||||
|
||||
```
|
||||
REGISTER_OPERATOR(some_op, OpType, SpecialVarTypeInferer, ...);
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,50 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#pragma once
|
||||
#include "paddle/framework/scope.h"
|
||||
#include "paddle/framework/variable.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace framework {
|
||||
|
||||
template <typename T>
|
||||
void SetFeedVariable(const LoDTensor& input, const std::string& var_name,
|
||||
size_t index) {
|
||||
// If var_name Variable is not found in GlobalScope, a new variable will
|
||||
// be created.
|
||||
Variable* g_feed_value = GetGlobalScope().Var(var_name);
|
||||
auto& feed_inputs =
|
||||
*(g_feed_value->GetMutable<std::vector<paddle::framework::LoDTensor>>());
|
||||
if (index >= feed_inputs.size()) {
|
||||
feed_inputs.resize(index + 1);
|
||||
}
|
||||
// shared data with input tensor
|
||||
feed_inputs[index].ShareDataWith<T>(input);
|
||||
// set lod
|
||||
feed_inputs[index].set_lod(input.lod());
|
||||
}
|
||||
|
||||
LoDTensor& GetFetchVariable(const std::string& var_name, size_t index) {
|
||||
// Since we want to fetch LodTensor from a variable, the variable must
|
||||
// be created alreadly.
|
||||
Variable* g_fetch_value = GetGlobalScope().FindVar(var_name);
|
||||
auto& fetch_outputs =
|
||||
*(g_fetch_value->GetMutable<std::vector<paddle::framework::LoDTensor>>());
|
||||
PADDLE_ENFORCE_LT(index, fetch_outputs.size());
|
||||
return fetch_outputs[index];
|
||||
}
|
||||
|
||||
} // namespace framework
|
||||
} // namespace paddle
|
||||
@ -0,0 +1,24 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#pragma once
|
||||
#include <vector>
|
||||
#include "paddle/framework/lod_tensor.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace framework {
|
||||
using FeedFetchType = LoDTensor;
|
||||
using FeedFetchList = std::vector<FeedFetchType>;
|
||||
} // namespace framework
|
||||
} // namespace paddle
|
||||
@ -0,0 +1,29 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#pragma once
|
||||
#include "paddle/framework/type_defs.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace framework {
|
||||
|
||||
class VarTypeInference {
|
||||
public:
|
||||
virtual ~VarTypeInference() {}
|
||||
virtual void operator()(const OpDescBind& op_desc,
|
||||
BlockDescBind* block) const = 0;
|
||||
};
|
||||
|
||||
} // namespace framework
|
||||
} // namespace paddle
|
||||
@ -0,0 +1,104 @@
|
||||
/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License. */
|
||||
|
||||
#include "paddle/framework/var_type_inference.h"
|
||||
#include "gtest/gtest.h"
|
||||
#include "paddle/framework/op_registry.h"
|
||||
#include "paddle/framework/operator.h"
|
||||
#include "paddle/framework/program_desc.h"
|
||||
|
||||
namespace paddle {
|
||||
namespace framework {
|
||||
|
||||
class SumOpMaker : public OpProtoAndCheckerMaker {
|
||||
public:
|
||||
SumOpMaker(OpProto *proto, OpAttrChecker *op_checker)
|
||||
: OpProtoAndCheckerMaker(proto, op_checker) {
|
||||
AddInput("X", "").AsDuplicable();
|
||||
AddOutput("Out", "");
|
||||
AddComment("");
|
||||
}
|
||||
};
|
||||
|
||||
class SumOpVarTypeInference : public VarTypeInference {
|
||||
public:
|
||||
void operator()(const OpDescBind &op_desc,
|
||||
BlockDescBind *block) const override {
|
||||
auto &inputs = op_desc.Input("X");
|
||||
auto default_var_type = VarDesc::SELECTED_ROWS;
|
||||
|
||||
bool any_input_is_lod_tensor = std::any_of(
|
||||
inputs.begin(), inputs.end(), [block](const std::string &name) {
|
||||
return block->Var(name)->GetType() == VarDesc::LOD_TENSOR;
|
||||
});
|
||||
if (any_input_is_lod_tensor) {
|
||||
default_var_type = VarDesc::LOD_TENSOR;
|
||||
}
|
||||
|
||||
auto out_var_name = op_desc.Output("Out").front();
|
||||
block->Var(out_var_name)->SetType(default_var_type);
|
||||
}
|
||||
};
|
||||
} // namespace framework
|
||||
} // namespace paddle
|
||||
|
||||
REGISTER_OPERATOR(sum, paddle::framework::NOP, paddle::framework::SumOpMaker,
|
||||
paddle::framework::SumOpVarTypeInference);
|
||||
REGISTER_OPERATOR(sum_without_infer_var_type, paddle::framework::NOP,
|
||||
paddle::framework::SumOpMaker);
|
||||
|
||||
namespace paddle {
|
||||
namespace framework {
|
||||
|
||||
TEST(InferVarType, sum_op) {
|
||||
auto &prog = ProgramDescBind::Instance(&GetProgramDesc());
|
||||
auto *op = prog.Block(0)->AppendOp();
|
||||
op->SetType("sum");
|
||||
op->SetInput("X", {"test_a", "test_b", "test_c"});
|
||||
op->SetOutput("Out", {"test_out"});
|
||||
|
||||
prog.Block(0)->Var("test_a")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test_b")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test_c")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test_out");
|
||||
|
||||
op->InferVarType(prog.Block(0));
|
||||
|
||||
ASSERT_EQ(VarDesc::SELECTED_ROWS, prog.Block(0)->Var("test_out")->GetType());
|
||||
|
||||
prog.Block(0)->Var("test_b")->SetType(VarDesc::LOD_TENSOR);
|
||||
op->InferVarType(prog.Block(0));
|
||||
ASSERT_EQ(VarDesc::LOD_TENSOR, prog.Block(0)->Var("test_out")->GetType());
|
||||
}
|
||||
|
||||
TEST(InferVarType, sum_op_without_infer_var_type) {
|
||||
auto &prog = ProgramDescBind::Instance(&GetProgramDesc());
|
||||
auto *op = prog.Block(0)->AppendOp();
|
||||
op->SetType("sum_without_infer_var_type");
|
||||
op->SetInput("X", {"test2_a", "test2_b", "test2_c"});
|
||||
op->SetOutput("Out", {"test2_out"});
|
||||
|
||||
prog.Block(0)->Var("test2_a")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test2_b")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test2_c")->SetType(VarDesc::SELECTED_ROWS);
|
||||
prog.Block(0)->Var("test2_out");
|
||||
|
||||
op->InferVarType(prog.Block(0));
|
||||
|
||||
ASSERT_EQ(VarDesc_VarType_LOD_TENSOR,
|
||||
prog.Block(0)->Var("test2_out")->GetType());
|
||||
}
|
||||
|
||||
} // namespace framework
|
||||
} // namespace paddle
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue