+
+
+
+| 接口 |
+说明 |
+
+
+
+
+ 
+ |
+输入
1. `ProgramDesc`
2. `Scope`
3.`block_id`
解释执行步骤
1. 创建所有 Variables
2. 逐一创建 Operator 并运行
+ |
+
+
+
+
+---
+### Operator/OpWithKernel/Kernel
+
+
+
+ 
+
+
+- operator 无状态，Operator的核心是==Run==方法
+- 一个operator可以注册多个kernel
+- operator 可以无 kernel：while_op 、ifelse op
+
+
+
+---
+#### Fluid Operator vs. PaddlePaddle layers
+
+
+
+
+| Layer |
+Operator |
+
+
+
+
+ 
+ |
+
+ 
+ |
+
+
+
+1. 内部维护状态
2. 包含forward和backward方法 |
+1. 内部无状态
2. 只有Run方法 |
+
+
+
+
+
+
+---
+
+### ==4.== 内存管理
+
+---
+### 目标
+
+- 为异构设备提供统一的内存分配、回收接口
+- 最小化管理内存所需的时间，最小化管理开销
+- 减少内存碎片
+- 将内存管理与计算（Operators/Kernels）完全剥离
+- 统一内存管理是内存优化的基础
+
+---
+
+
+
+### Memory 接口
+
+- 内存管理模块向上层应用逻辑提供三个基础接口：
+ ```cpp
+ template
+ void* Alloc(Place place, size_t size);
+
+ template
+ void Free(Place place, void* ptr);
+
+ template
+ size_t Used(Place place);
+
+ struct Usage : public boost::static_visitor {
+ size_t operator()(const platform::CPUPlace& cpu) const;
+ size_t operator()(const platform::CUDAPlace& gpu) const;
+ };
+ ```
+- 模板参数 `Place` 指示内存分配发生的设备
+- 实现时，需特化支持的 `Place`， 提供以上三个接口的实现
+
+
+
+---
+### 代码结构
+
+
+
+内存管理模块可以理解为由以下两部分构成：
+
+1. SystemAllocator：实际从物理设备上分配、释放的内存的接口
+1. BuddyAllocator：内存管理算法
+
+
+
+---
+### System Allocator
+
+
+
+- SystemAllocator 是实现物理内存分配、回收的基类
+ - 不同设备上的内存分配和回收终将转化为标准接口调用
+ - 为不同设备实现MemoryAllocator，继承自SystemAllocator
+
+ ```cpp
+ class SystemAllocator {
+ public:
+ virtual ~SystemAllocator() {}
+ virtual void* Alloc(size_t& index, size_t size) = 0;
+ virtual void Free(void* p, size_t size, size_t index) = 0;
+ virtual bool UseGpu() const = 0;
+ };
+ ```
+
+
+---
+
+### CPU/GPU Allocator
+
+
+
+```cpp
+class CPUAllocator : public SystemAllocator {
+ public:
+ virtual void* Alloc(size_t& index, size_t size);
+ virtual void Free(void* p, size_t size, size_t index);
+ virtual bool UseGpu() const;
+};
+
+#ifdef PADDLE_WITH_CUDA
+class GPUAllocator : public SystemAllocator {
+ public:
+ virtual void* Alloc(size_t& index, size_t size);
+ virtual void Free(void* p, size_t size, size_t index);
+ virtual bool UseGpu() const;
+ private:
+ size_t gpu_alloc_size_ = 0;
+ size_t fallback_alloc_size_ = 0;
+};
+#endif
+```
+- CPUAllocator和GPUAllocator分别继承自SystemAllocator，分别调用相应的标准库函数实现物理内存的分配和释放。
+- 一旦大块、连续的物理内存分配之后，将通过内存管理算法实现内存的按块分配、回收、重用等。
+
+
+
+---
+### CPU Allocator
+
+
+
+- CPU 内存的分配提供两种选项：
+ 1. non-pinned memory：可分页内存
+ 2. pinned memory：页锁定内存
+ - 分配过大的页锁定内存有可能因为系统可使用的分页内存减少，影响系统性能，默认CPU下分配的是可分页内存
+
+- 通过gflags进行设置一次性分配内存的大小以及是否使用页锁定内存。
+
+ ```cpp
+ DEFINE_bool(use_pinned_memory, true, "If set, allocate cpu pinned memory.");
+ DEFINE_double(fraction_of_cpu_memory_to_use, 1,
+ "Default use 100% of CPU memory for PaddlePaddle,"
+ "reserve the rest for page tables, etc");
+ ```
+
+
+
+---
+### GPU Allocator
+
+
+
+- 通过 cudaMalloc 分配GPU显存
+- GPUAllocator::Alloc 首先会计算指定GPU device上的可用显存
+ - 如果可用显存小于请求分配大小，调用cudaMalloc进行分配
+ - 如果可用显存不足，目前会报错退出。
+- 通过gflags控制GPU下一次性分配显存的大小：
+
+ ```cpp
+ DEFINE_double(fraction_of_gpu_memory_to_use, 0.92,
+ "Default use 92% of GPU memory for PaddlePaddle,"
+ "reserve the rest for page tables, etc");
+ ```
+
+
+
+---
+#### 内存管理算法: [Buddy Memory Allocation](https://en.wikipedia.org/wiki/Buddy_memory_allocation)
+
+
+
+- Memory Arena：一次性分配大块连续内存，之后会基于这块内存进行内存管理：动态分配、释放、重用内存块。
+- 伙伴内存分配：
+ - 将内存划分为 2 的幂次方个分区，使用 best-fit 方法来分配内存请求。
+ - 当释放内存时，检查 buddy 块，查看相邻的内存块是否也已被释放。如果是，将内存块合并，以最小化内存碎片。
+ - 分配的内存在物理内存的自然边界对齐，提高内存访问效率。
+ - 算法的时间效率高，单使用 best-fit 方法的缘故，会产生一定的内存浪费
+
+
+
+---
+
+### Buddy Allocator
+
+
+
+- BuddyAllocator 是一个单例，每个设备（如： GPU/CPU(0)/GPU(1)） 拥有一个BuddyAllocator
+- BuddyAllocator 内部拥有一个私有成员变量 SystemAllocator
+- 当请求的内存超过BuddyAllocator管理的空余内存时，将会调用SystemAllocator去指定的设备上分配物理内存
+
+
+
+---
+### 实例：CPU 下内存管理接口的实现
+
+
+
+- 对上层应用，统一通过BuddyAllocator来实现内存的分配、释放以及用量查询
+ ```cpp
+ template <>
+ void* Alloc(platform::CPUPlace place, size_t size) {
+ VLOG(10) << "Allocate " << size << " bytes on " << platform::Place(place);
+ void* p = GetCPUBuddyAllocator()->Alloc(size);
+ VLOG(10) << " pointer=" << p;
+ return p;
+ }
+
+ template <>
+ void Free(platform::CPUPlace place, void* p) {
+ VLOG(10) << "Free pointer=" << p << " on " << platform::Place(place);
+ GetCPUBuddyAllocator()->Free(p);
+ }
+
+ template <>
+ size_t Used(platform::CPUPlace place) {
+ return GetCPUBuddyAllocator()->Used();
+ }
+ ```
+
+
+---
+### ==5.== 多设备支持
+
+---
+### 多设备支持（一）
+
+
+
+- step 1：添加Place类型，由用户实现添加到框架
+ - 可以将Place类型理解为一个整数加上一个枚举型，包括：设备号 + 设备类型
+
+
+ 
+
+- DeviceContext
+ - 不同的Place会对应一个相应的DeviceContext，用于组织管理与设备相关的信息
+ - 例如，GpuDeviceContext中会管理Cuda stream
+ - 目前实现中一些特殊的库也会对应有自己的DeviceContext：例如：
+ ```cpp
+ class MKLDNNDeviceContext : public CPUDeviceContext {……}
+ ```
+ - 每种设备对应的DeviceContext需要管理的内容不尽相同，视具体需求来实现
+
+
+
+---
+
+### 多设备支持（二）
+
+
+
+- step 2: 增加KernelType，为相应的KernelType注册Kernel对象，由用户实现注册给框架 可以按照：
+ 1. Place 执行设备
+ 1. DataType 执行数据类型 FP32/FP64/INT32/INT64
+ 1. Memory layout： 运行时 Tensor 在内存中的排布格式 NCHW、 NHWC
+ 1. 使用的库
+
+ 来区分Kernel，为同一个operator注册多个 Kernel。
+
+ ```cpp
+ struct OpKernelType {
+ proto::DataType data_type_;
+ DataLayout data_layout_;
+ platform::Place place_;
+ LibraryType library_type_;
+ }
+ ```
+
+
+
+---
+
+### 多设备支持（三）
+
+
+
+step 3: 运行时的 KernelType 推断和Kernel切换，按需要修改Kernel推断和Kernel切换规则
+- Expected Kernel：期待调用的Kernel：由（1）`Place`和计算精度决定；或（2）用户在配置中显示指定使用的计算库，如`cudnn`、`mkldnn`等。
+- Actual Kernel：运行时从`Operator`的输入（`Variable`）可以推断出实际需要的`KernelType`
+- 当Expected Kernel和Actual Kernel不一致的时候，框架会插入`data_transformer`或者`data_layerout_transform`等，保证Expected Kernel可以执行，包括：
+ - CPUPlace -> GPUPlace ：跨设备内存复制
+ - NCHW -> nChw8c ：Layout转换
+ - FP32 -> FP16 ：精度转换 _**尚未支持**_
+ - ……
+- 以上过程实现在OperatorWithKernel类的Run方法中 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/operator.cc#L497)
+
+
+
+---
+## ==6.== while_op
+
+---
+### while_op
+
+
+
+- 循环执行一段`Program`，直到条件operator判断循环条件不满足时终止循环
+- while_op 的特殊之处：
+ 1. while_op 没有 kernel
+ 1. while_op 拥有自己的`Block`，会形成一段嵌套的`Block`
+ 1. ==while_op 内部创建了一个 Executor，来循环执行`Block`==
+
+- while_op 输入输出 ： LoDTensorArray
+ ```cpp
+ namespace paddle {
+ namespace framework {
+ using LoDTensorArray = std::vector;
+ }
+ }
+ ```
+ - 每一次循环，从原始输入中“切出”一个片段
+ - LoDTensorArray 在Python端暴露，是Fluid支持的基础数据结构之一，用户可以直接创建并使用
+
+
+
+---
+### while_op [Run](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/while_op.cc#L42) 方法概览
+
+
+
+```cpp
+
+void Run(const framework::Scope &scope,
+ const platform::Place &dev_place) const override {
+ PADDLE_ENFORCE_NOT_NULL(scope.FindVar(Input(kCondition)));
+ auto &cond = scope.FindVar(Input(kCondition))->Get();
+ PADDLE_ENFORCE_EQ(cond.dims(), paddle::framework::make_ddim({1}));
+
+ framework::Executor executor(dev_place);
+ auto *block = Attr(kStepBlock);
+
+ auto *program = block->Program();
+ auto step_scopes =
+ scope.FindVar(Output(kStepScopes))->GetMutable();
+
+ while (cond.data()[0]) {
+ auto ¤t_scope = scope.NewScope();
+ step_scopes->push_back(¤t_scope);
+ executor.Run(*program, ¤t_scope, block->ID(),
+ false /*create_local_scope*/);
+ }
+}
+
+```
+
+
+
+---
+### while_op 的重要应用：Dynamic RNN
+
+---
+
+### 什么是 `dynamicRNN` ?
+
+
+
+
+1. 用户可以自定义在一个时间步之内的计算, 框架接受序列输入数据，在其上循环调用用户定义的单步计算
+1. 可学习参数在多个时间步之间共享
+1. `dynamicRNN` 由 `while_op` 实现
+1. 如果`dynamicRNN`中定义了`memory`，将会构成一个循环神经网络，否则其行为就等于在输入序列上循环调用预定义的单步计算
+
+
+
+---
+
+#### `dynamic RNN` 用户接口
+
+
+
+
+
+
+- `dynamicRNN` 中的重要元素
+ 1. **step input**: `dynamicRNN` 每个时间步的输入
+ 1. **step function**: 用户定义的单步计算
+ 1. **memory**: 用于形成循环连接
+ 1. **external/static memory**：单步计算的每一步都可以全部读取到的外部输入
+
+
+
+---
+
+#### dynamicRNN 中的 Memory
+
+
+
+`dynamicRNN`中`memory`的行为非常类似于 C++ 中的引用变量
+ - `memory` “指向” 一个operator的输出变量，记作： A
+ - `memory` 可以被 LoDTensor 初始化（当LoD信息为空时，为非序列，否则为序列）,默认`memory`被初始化为零
+ - `memory` 在 operator A 前向计算之后，进行前向计算
+ - 当 `memory` 的前向计算会 "指向" A 的输出 LoDTensor
+ - `memory` 的输出可以是另一个 operator 的输入，于是形成了“循环”连接
+
+
+
+---
+
+### DynamicRNN 实现细节
+
+
+
+- `while_op` 无法独立构成dynamicRNN，必须和一组相关的 operator 及数据结构配合
+ - 依赖的 operators (这里仅列出最重要的，并非全部):
+ - `lod_rank_table` operator
+ - `lod_tensor_to_array` operator
+ - `array_to_lod_tensor` operator
+ - `shrink_memory` operator
+ - 依赖的数据结构
+ - `TensorArray`
+ - `LoDRankTable`
+
+- 在Fluid中，RNN接受变长序列输入，无需填充，以上数据结构和相关的operator配合工作，实现了对变长输入以batch计算
+
+
+
+---
+
+### `dynamicRNN` 如何实现 batch 计算 ?
+
+
+
+- 问题：
+ - RNN 可以看作是一个展开的前向网络，前向网络的深度是最长序列的长度
+ - 如果不对变长序列进行填充，将它们填充到一样长度，每个mini-batch输入将会不等长，每个样本展开长度不一致，导致前向和反向计算实现困难
+
+
+
+----
+##### 实例 ：RNN encoder-decoder with attention
+
+
+
+- 以机器翻译的RNN encoder-decoder 模型（涉及了`dynamicRNN`的所有设计要素）为例，下图是 RNN encoder-decoder 的原始输入：
+
+ 
Figure. RNN encoder-decoder 原始batch 输入数据
+
+
+- source word sequences 是encoder RNN的输出，是一个LoDTensor
+- target word sequences 是look_uptable的输入，是一个LoDTensor
+- 上图中一个矩形方块是CPU/GPU内存中一片连续的内存空间，表示一个dense vector
+
+
+
+---
+
+### `dynamicRNN` 如何实现 batch 计算 ?
+
+
+
+1. 对一个mini batch中不等长样本进行排序，最长样本变成batch中的第一个，最短样本是batch中最后一个
+ - `LoDTensor` -> `LoDRankTable` :heavy_plus_sign: `lod_rank_table operaator`
+ - 可以将`LoDRankTable`理解为对LoDTensor中的多个序列按照长度排序LoDRankTable 存储了排序之后的index
+
+2. 构建每个时间步的batch输入：随着时间步增加，每个时间步的batch输入可能会逐渐缩小
+ - `TensorArray` :heavy_plus_sign: `lod_tensor_to_array` -> `LoDTensor` (without LoD)
+3. 每个时间步输出写入一个输出 `LoDTensorArray`
+3. `dynamicRNN`循环结束后, 按照`LoDRankTable`中记录的信息对输出`LoDTensorArray`重排序，还原会原始输入顺序
+ - `TensorArray` :heavy_plus_sign: `array_to_lod_tensor` -> `LoDTensor`
+
+
+
+---
+
+### 运行实例
+
+
+
+
+
+---
+### 运行实例
+
+
+
+
+
+
+
+- 执行到第5~7个batch时，batch size将会缩小
+
+
+
+---
+### 运行实例
+
+
+
+
+
+
+
+- 第5 ~ 7个batch时RNN的`memory`会发生什么？
+ - `memory` 指向某个operator的输出Tensor，在该operator前向计算之后，“取回”其计算结果
+ - 5 ~ 7时，遇到了序列的结束，==下一个时间步计算不再需要在已经结束的序列上展开==
+ - 在`dynamicRNN`中`shrink_memory` operator 用来缩小`memory`的batch输入
+
+
+
+---
+### 运行实例：batch 1 ~ 2
+
+
+
Figure. 第1、2个batch输入dynamicRNN的batch输入
+
+
+---
+### 运行实例：batch 3 ~ 4
+
+
+
Figure. 第3、4个batch输入dynamicRNN的batch输入
+
+
+---
+
+### 运行实例：batch 5 ~ 7
+
+
+
Figure. 第5、6、7个batch输入dynamicRNN的batch输入
+
+
+---
+### ==7.== Fluid 代码结构
+
+---
+### Fluid 代码结构
+
+
+
+
+| 代码结构 |
+模块结构 |
+
+
+
+
+|
+
+
+
+ |
+
+
+
+
+ |
+
+
+
+
+
+---
+
+### ==8.== 文档总结
+
+---
+
+
+- 设计概览
+ - 重构概览 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/refactorization.md)
+ - fluid [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/fluid.md)
+ - fluid_compiler [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/motivation/fluid_compiler.md)
+- 核心概念
+ - variable 描述 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/var_desc.md)
+ - Tensor [->](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/tensor.md)
+ - LoDTensor [->](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md)
+ - TensorArray [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/tensor_array.md)
+ - Program [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md)
+ - Block [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md)
+ - Scope [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md)
+
+---
+
+- 重要功能模块
+ - backward [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/backward.md)
+ - 内存优化 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/memory_optimization.md)
+ - evaluator [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/executor.md)
+ - python API [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/python_api.md)
+ - regularization [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/regularization.md)
+
+- 开发指南
+ - 支持新设硬件设备库 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/support_new_device.md)
+ - 添加新的Operator [->](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/new_op_cn.md)
+ - 添加新的Kernel [->](
+https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/new_op_kernel_en.md)
+
+
+
+---
+
+### ==9.== 开发指南
+
+---
+
+#### 建议开发环境：使用 Docker 编译和测试
+
+
+
+Docker编译PaddlePaddle源码: [->](http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/build_and_install/docker_install_cn.html)
+
+PaddlePaddle 在 Dockerhub 地址：[->](
+ https://hub.docker.com/r/paddlepaddle/paddle/tags/)
+
+1. 获取PaddlePaddle的Docker镜像
+ ```bash
+ docker pull paddlepaddle/paddle:latest-dev
+ ```
+
+1. 启动 docker container
+
+ ```bash
+ docker run -it -v $PWD/Paddle:/paddle paddlepaddle/paddle:latest-dev /bin/bash
+ ```
+
+1. 进入docker container后，从源码编译，请参考文档 [->]( http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/build_and_install/build_from_source_cn.html)
+
+
+
+---
+
+### 一些说明
+
+
+
+1. PaddlePaddle的Docker镜像为了减小体积，默认没有安装vim，可以在容器中执行`apt-get install -y vim`来安装vim。
+1. 开发推荐使用tag为`latest-dev`的镜像，其中打包了所有编译依赖。`latest`及`lastest-gpu`是production镜像，主要用于运行PaddlePaddle程序。
+2. 在Docker中运行GPU程序，推荐使用nvidia-docker，[否则需要将CUDA库和设备挂载到Docker容器内](http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/build_and_install/docker_install_cn.html)。
+
+
+ ```bash
+ nvidia-docker run -it -v $PWD/Paddle:/paddle paddlepaddle/paddle:latest-dev /bin/bash
+ ```
+
+
+
+
+
+---
+
+### [如何贡献](http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/dev/contribute_to_paddle_cn.html)
+
+
+
+- ==提交PullRequest前请务必阅读==： [->](http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/dev/contribute_to_paddle_cn.html)
+- 代码要求
+ 1. 代码注释遵守 Doxygen 的样式
+ 1. 确保编译器选项 WITH_STYLE_CHECK 已打开，并且编译能通过代码样式检查
+ 1. 所有代码必须具有单元测试，且能够通过所有单元测试
+- 使用 `pre-commit` 钩子提交Pull Request
+ 1. 帮助格式化源代码（C++，Python）
+ 1. 在提交前自动检查一些基本事宜：如每个文件只有一个 EOL，Git 中不要添加大文件等
+ 1. 安装pre-commit，并在PaddlePaddle根目录运行：
+ ```bash
+ ➜ pip install pre-commit
+ ➜ pre-commit install
+ ```
+
+
+---
+
+### 如何贡献
+
+
+
+1. 开始开发之前请先建立issue。
+ - 让其它同学知道某项工作已经有人在进行，以避免多人开发同一功能的情况。
+1. 提交PR必须关联相关的issue。做法请参考：[->](https://help.github.com/articles/closing-issues-using-keywords/)
+ - 目的：为了在提交的版本中留有记录描述这个PR是为了开发什么样的功能，为了解决什么样的问题。
+ - 当PR被merge后，关联的issue会被自动关闭。
+1. PR review 中，reviewer的每条comment都必须回复。
+ - 如修改完可直接回复：Done。
+ - 目的：review comment 中可能会有（1）询问类型的问题；（2）可以在下一个PR修改的问题；（3）comment意见不合理等。需要明确回复，以便reviewer和其他人有历史可查，便于区分是否已经进行修改，或者准备下一个PR修改，或者意见不合理可以不用进行修改。
+
+
+
+---
+
+### ==10.== 添加新的 Operator
+
+---
+
+### 概念简介
+
+
+
+添加一个新的operator，会涉及实现以下C++类的派生类：
+
+1. `framework::OperatorBase`: Operator(简写，Op)基类。
+1. `framework::OpKernel`: Op计算函数的基类，称作Kernel。
+1. `framework::OperatorWithKernel`：继承自OperatorBase，Op有计算函数，称作有Kernel。
+1. `class OpProtoAndCheckerMaker`：描述该Op的输入、输出、属性、注释,主要用于Python API接口生成
+
+依据是否包含kernel，可以将Op分为两种：
+1. 包含Kernel的Op：继承自OperatorWithKernel，==绝大多数operator都属于这一类==
+1. 不包含kernel的Op，继承自OperatorBase，只有少量Op属于这一类，例如while_op，ifelse_op
+
+这里主要介绍带Kernel的Op如何编写。
+
+
+
+---
+
+#### 添加新的Operator需要修改/添加哪些文件？
+
+
+
+
+
+
+| 内容 |
+定义位置 |
+
+
+
+
+|
+OpProtoMake定义
+ |
+
+`.cc`文件，Backward Op不需要OpProtoMaker
+ |
+
+
+|
+Op定义
+ |
+
+`.cc`文件
+ |
+
+
+|
+Kernel实现
+ |
+
+CPU、CUDA共享Kernel实现在`.h`文件中，否则，CPU 实现在`.cc`文件中，CUDA 实现在`.cu`文件中。
+ |
+
+
+
+|
+注册Op
+ |
+
+Op注册实现在`.cc`文件；Kernel注册CPU实现在`.cc`文件中，CUDA实现在`.cu`文件中
+ |
+
+
+
+
+
+- 添加 Operator 之前请阅读：[Operator 命名规范](https://github.com/PaddlePaddle/Paddle/blob/63cca04cfd488a4dab6d6273fd04a8017ef45932/doc/fluid/dev/name_convention.md)及[Operator Markdown注释规范](https://github.com/PaddlePaddle/Paddle/blob/63cca04cfd488a4dab6d6273fd04a8017ef45932/doc/fluid/dev/op_markdown_format.md)。
+- 实现新的op都添加至目录[paddle/operators](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/fluid/operators)下，文件命名以`*_op.h`（如有） 、 `*_op.cc` 、`*_op.cu`（如有）结尾。
+- 根据文件名自动构建op和Python端绑定，请务必遵守以上命名，否则需要进一步修改PyBind相关文件及CMakeLists.txt。
+
+
+---
+
+###### 实现带Kernel的Operator step1: 定义ProtoMaker类
+
+
+
+下面均以[clip_op](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/clip_op.h)为例进行介绍
+
+- clip_op计算公式：$Out = \min(\max(X, min), max)$
+- 首先定义`ProtoMaker`来描述该Op的输入、输出，并添加注释（*下面代码段的中注释进行了简化，实现时需按照规范添加注释*）：
+
+ ```cpp
+ template
+ class ClipOpMaker : public framework::OpProtoAndCheckerMaker {
+ public:
+ ClipOpMaker(OpProto* proto, OpAttrChecker* op_checker)
+ : OpProtoAndCheckerMaker(proto, op_checker) {
+ AddInput("X","(Tensor)The input of clip op.");
+ AddOutput("Out", "(Tensor),The output of clip op.");
+ AddAttr(
+ "min", "(float),Minimum value.");
+ AddAttr(
+ "max", "(float),Maximum value.");
+ AddComment(R"DOC(
+ ……
+ )DOC");
+ }
+ };
+ ```
+
+
+
+---
+
+###### 实现带Kernel的Operator step2: 定义Operator类
+
+
+
+下面的代码段实现了`clip_op`的定义：
+
+```cpp
+class ClipOp : public framework::OperatorWithKernel {
+ public:
+ using framework::OperatorWithKernel::OperatorWithKernel;
+
+ void InferShape(framework::InferShapeContext* ctx) const override {
+ PADDLE_ENFORCE(ctx->HasInput("X"),
+ "Input(X) of ClipOp should not be null.");
+ PADDLE_ENFORCE(ctx->HasOutput("Out"),
+ "Output(Out) of ClipOp should not be null.");
+ auto x_dims = ctx->GetInputDim("X");
+ auto max = ctx->Attrs().Get("max");
+ auto min = ctx->Attrs().Get("min");
+ PADDLE_ENFORCE_LT(min, max, "max should be greater than min.");
+ ctx->SetOutputDim("Out", x_dims);
+ ctx->ShareLoD("X", /*->*/ "Out");
+ }
+};
+```
+
+
+---
+
+### Operator 类中需要完成的工作
+
+
+
+1. clip_op 继承自`OperatorWithKernel`，
+
+ ```cpp
+ using framework::OperatorWithKernel::OperatorWithKernel;
+ ```
+ 表示使用基类`OperatorWithKernel`的构造函数。
+
+1. 重写`InferShape`接口。
+ - `InferShape` 为const函数，不能修改Op的成员变
+ - `InferShape` 的参数为 `const framework::InferShapeContext &ctx`，从中可获取到输入输出以及属性
+ - `InferShape` 会被调用两次，一次是编译时（创建op），一次是运行时（调用op的`Run`方法时），需要完成以下功能：
+ 1. 做检查， 尽早报错：检查输入数据维度、类型等是否合法
+ 2. 设置输出Tensor的形状
+
+通常`OpProtoMaker`和`Op`类的定义写在`.cc`文件中。
+
+
+
+---
+
+### 补充说明
+
+
+
+1. `InferShape`目前支持两种实现方式，二者最后都会生成一个functor注册给OpInfo结构体。
+ 1. 继承framework::InferShapeBase，实现为一个functor（参考 [mul_op](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/mul_op.cc#L22)）
+ 2. override InferShape函数（参考 [clip_op](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/clip_op.cc#L24)）
+
+1. 什么是`functor` ?
+
+ - 类或结构体仅重载了`()`，一般是可被多个kernel复用的计算函数。
+
+
+
+ ```cpp
+ template
+ class CrossEntropyFunctor {
+ public:
+ void operator()(const platform::CPUDeviceContext& ctx,
+ framework::Tensor* out,
+ const framework::Tensor* prob,
+ const framework::Tensor* labels, const bool softLabel) {
+ ……
+ }
+ };
+ ```
+
+
+ - 在 clip_op 内也会看到将一段计算函数抽象为functor的使用法： [->](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/clip_op.h#L27)。
+
+
+
+---
+
+###### 实现带Kernel的Operator step3: 定义OpKernel类
+
+
+
+- `ClipKernel`继承自`framework::OpKernel`，带有下面两个模板参数:
+ 1. `typename DeviceContext`: 表示设备类型，不同设备共享同一个Kernel时，需添加该模板参数。不共享时，需要提供针对不同设备的特化实现。
+ 1. `typename T` : 表示支持的数据类型，如`float`, `double`等
+
+- 在`ClipKernel`类中重写`Compute`方法
+ 1. `Compute`接受输入参数：`const framework::ExecutionContext& context`
+ - `ExecutionContext` 是从 `Scope`中将运行时Op的输入、输出`Variable`组织在一起，使得Op在调用`Compute`方法时，能够简单地通过名字拿到需要的输入输出`Variable`
+ - 与`InferShapeContext`相比，`ExecutionContext` 中增加了设备类型
+ 1. 在`Compute`函数里实现`OpKernel`的具体计算逻辑
+
+
+
+---
+#### ClipKernel 代码概览
+
+
+
+```cpp
+template
+class ClipKernel : public framework::OpKernel {
+ public:
+ void Compute(const framework::ExecutionContext& context) const override {
+ auto max = context.Attr("max");
+ auto min = context.Attr("min");
+ auto* x = context.Input("X");
+ auto* out = context.Output("Out");
+ T* out_data = out->mutable_data(context.GetPlace());
+ const T* x_data = x->data();
+ int64_t numel = x->numel();
+ Transform trans;
+ trans(context.template device_context(), x_data,
+ x_data + numel, out_data, ClipFunctor(min, max));
+ }
+};
+```
+
+- 为了使`OpKernel`的计算过程书写更加简单，并且CPU、CUDA的代码可以复用， Fluid 使用 Eigen 作为基础的矩阵运算库
+- Fluid对Eigen unsupported Tensor提供了一些基本的封装，可以在`Compute`接口中直接调用
+ - 关于在PaddlePaddle中如何使用Eigen库，请参考[使用文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/dev/use_eigen_cn.md)。
+
+
+
+---
+###### 实现带Kernel的Operator step4: 实现反向Op
+
+
+
+- ==**反向Op没有`ProtoMaker`**==，除此之外定义与实现方式前向Op完全一致，不再赘述
+- 这里仅对反向Op的输入输出进行说明：
+ 1. 反向Op的输入
+ - 前向Op的输出
+ - 反向传播过程中传递给当前Op的梯度
+ - 需要注意，Fluid中，不区分Cost Op和中间层Op，所有Op都必须正确处理接收到的梯度
+ 2. 反向Op的输出
+ - 对可学习参数的求导结果
+ - 对所有输入的求导结果
+
+
+
+
+---
+
+###### 实现带Kernel的Operator step5: 注册Op及Kernel
+
+
+
+至此Op和Op kernel都已经实现完毕，接下来，需要在`.cc`和`cu`文件中注册op和kernel
+
+1. 在`.cc`文件中注册前向、反向Op类，注册CPU Kernel。
+
+
+
+ ```cpp
+ namespace ops = paddle::operators;
+ REGISTER_OP(clip, ops::ClipOp, ops::ClipOpMaker, clip_grad,
+ ops::ClipOpGrad);
+ REGISTER_OP_CPU_KERNEL(
+ clip, ops::ClipKernel);
+ REGISTER_OP_CPU_KERNEL(
+ clip_grad, ops::ClipGradKernel);
+ ```
+
+ - 在上面的代码片段中：
+
+ 1. `REGISTER_OP` ： 注册`ops::ClipOp`类，类型名为`clip`，该类的`ProtoMaker`为`ops::ClipOpMaker`，注册`ops::ClipOpGrad`，类型名为`clip_grad`
+ 1. `REGISTER_OP_WITHOUT_GRADIENT` ： 用于注册没有反向的Op，例如：优化算法相关的Op
+ 1. `REGISTER_OP_CPU_KERNEL` ：注册`ops::ClipKernel`类，并特化模板参数为`paddle::platform::CPUPlace`和`float`类型，同理，注册`ops::ClipGradKernel`类
+
+
+1. 按照同样方法，在`.cu`文件中注册GPU Kernel
+ - 如果CUDA Kernel的实现基于Eigen，需在 `.cu`的开始加上宏定义 `#define EIGEN_USE_GPU`
+
+
+
+---
+
+##### 编译和Python端绑定
+
+
+
+- 运行下面命令可以仅编译新添加的Op：
+
+ ```
+ make mul_op
+ ```
+ - 需注意，运行单元测试需要编译整个工程
+
+- 如果遵循前文的文件命名规则，构建过程中，会自动为新增的op添加Python端绑定，并链接到生成的lib库中
+
+
+
+---
+
+###### 实现带Kernel的Operator step6: 添加前向单测及梯度检测
+
+
+
+- 新增Op的单元测试统一添加至：[python/paddle/v2/fluid/tests/unittests](https://github.com/PaddlePaddle/Paddle/tree/develop/python/paddle/fluid/tests/unittests)目录
+- 前向Operator单测
+
+ 1. Op单元测试继承自`OpTest`，各项具体的单元测试在`TestClipOp`里完成，所有单测case都以`TestXX`命名
+ 1. 单元测试Operator，需要：
+ 1. 在`setUp`函数定义输入、输出，以及相关的属性参数
+ 1. 生成随机的输入数据
+ 1. 在Python脚本中实现与前向operator相同的计算逻辑，得到输出值，与operator前向计算的输出进行对比
+ 1. 反向梯度检测流程测试框架已经实现，直接调用相应接口`check_grad`即可
+
+- `clip_op` 单测代码请参考 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/unittests/test_clip_op.py)，这里不再展开
+
+
+
+---
+#### 编译执行单测
+
+
+
+- `python/paddle/v2/framework/tests` 目录下新增的 `test_*.py` 单元测试会被自动加入工程进行编译
+
+ - 运行单元测试测时需要编译整个工程，并且编译时需要打开`WITH_TESTING`, 即`cmake paddle_dir -DWITH_TESTING=ON`
+- 编译成功后，执行下面的命令来运行单元测试：
+
+ ```bash
+ make test ARGS="-R test_mul_op -V"
+ ```
+
+ 或者:
+
+ ```
+ ctest -R test_mul_op
+ ```
+
+
+---
+
+### 添加Op的一些注意事项
+
+
+
+- 为每个Op创建单独的`*_op.h`（如有）、`*_op.cc`和`*_op.cu`（如有）。不允许一个文件中包含多个Op，将会导致编译出错。
+- 注册Op时的类型名，需要和该Op的名字一样。不允许在`A_op.cc`里面，注册`REGISTER_OP(B, ...)`，会导致单元测试出错。
+- 如果Op没有实现CUDA Kernel，不要创建空的`*_op.cu`，会导致单元测试出错。
+- 如果多个Op依赖一些共用的函数，可以创建非`*_op.*`格式的文件来存放，如`gather.h`文件。
+
+
+
+---
+
+### ==10.== 使用相关问题
+
+---
+
+### 定义前向计算
+
+
+
+- 当在python端执行时：
+ ```python
+ import paddle.v2.fluid as fluid
+ ```
+ [`framework.py`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/framework.py#L1040)定义了两个全局`Program`:
+ ```python
+ # program is a global instance.
+ _main_program_ = Program()
+ _startup_program_ = Program()
+ ```
+
+- 前向定义的过程就是不断往`mian_program`中添加Op和Variable
+- 如果需要执行一个新的`mian_program`时，可以调用调用：
+ ```python
+ def switch_main_program(program):
+ """
+ Switch the main program to a new program.
+ This funtion returns the previous main program.
+ """
+ ……
+ ```
+
+
+---
+
+### 自定义参数的初始化
+
+
+
+- 调用`fluid.ParamAttr(……)`接口，自定义参数的初始化
+
+ ```python
+ w_param_attrs = ParamAttr(name=None,
+ initializer=UniformInitializer(low=-1.0, high=1.0, seed=0),
+ learning_rate=1.0,
+ regularizer=L1Decay(1.0),
+ trainable=True,
+ clip=GradientClipByValue(-1.0, 1.0),
+ )
+ y_predict = fluid.layers.fc(input=x, size=1, param_attr=w_param_attrs)
+ ```
+
+- 补充问题：如何创建 `Variable`
+ ```python
+ cur_program = Program()
+ cur_block = cur_program.current_block()
+ new_var = cur_block.create_var(name="X", shape=[-1, 16, 16], dtype="float32")
+ ```
+
+
+
+---
+
+### 添加反向Op
+
+
+
+- 调用`fluid.backward.append_backward(X)`（`X`是一个Variable），来为一段前向`ProgramDesc`添加反Op
+
+ ```python
+ data = fluid.layers.data(name="data", shape=(2,3,4))
+ out = fluid.layers.fc(input=data,size=128,act=None)
+ loss = fluid.layers.reduce_sum(out)
+ fluid.backward.append_backward(loss=loss)
+ ```
+
+- 添加优化相关的Op
+ ```python
+ sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.001)
+ sgd_optimizer.minimize(loss)
+ ```
+
+- 可以随时调用`print(fluid.default_main_program())`来输出当前的`main_program`
+
+- 当构建完成整个`Program`后，调用下面的接口执行内存优化：
+ ```python
+ fluid.memory_optimize(fluid.default_main_program())
+ ```
+ - _注：内存优化目前仍在持续开发中，有可能不够稳定。_
+
+
+
+---
+
+### 总结：编译时执行流程
+
+
+
+- 用户定义前向计算
+- 添加反向Op到`default_main_program`
+- 添加 gradient clipping Op 到
+- 添加 regularization Op 到`default_main_program`
+- 为指定的优化算法，添加相关的状态 variable of optimizer 到`default_startup_program`
+ - 状态相关 variable是指如学习率, 历史 momentum, 二阶momentum等
+- 添加初始化 variable 的Op 到 `default_startup_program`
+- 为整个网络最后一个op，添加设置其接受到的梯度的Op到`default_main_program`
+- 进行内存优化规划
+
+
+
+---
+
+### Feed 数据 (一)：通过 feed 字典
+
+
+
+- 执行executor的run方法时，指定feed字典，feed op 会将指定的数据放到`x`和`y`两个Variable中
+ ```python
+ y_data = np.random.randint(0, 8, [1]).astype("int32")
+ y_tensor = core.Tensor()
+ y_tensor.set(y_data, place)
+
+ x_data = np.random.uniform(0.1, 1, [11, 8]).astype("float32")
+ x_tensor = core.Tensor()
+ x_tensor.set(x_data, place)
+ ……
+ cost = exe.run(
+ fluid.default_main_program(),
+ feed={'x': x_tensor,
+ 'y': y_tensor},
+ fetchlist=[avg_cost])
+ ```
+
+- 这种方法较为底层，一般用于单测中
+
+
+
+---
+
+### Feed 数据 (二)：使用 DataFeeder接口
+
+
+
+- 编写一个data_reader函数，data_reader是一个Python generator
+
+ ```python
+ def demo_reader():
+ def random_generator():
+ yield np.random.uniform(0.1, 1, [4]), np.random.randint(0, 1, [1])
+ return random_generator
+ ```
+- 在训练任务中使用 DataFeeder 接口
+ ```python
+ cost = exe.run(
+ fluid.default_main_program(),
+ feed={'x': x_tensor,
+ 'y': y_tensor},
+ fetchlist=[avg_cost])
+
+ train_reader = paddle.batch(
+ paddle.reader.shuffle(demo_reader(), buf_size=500), batch_size=4)
+ feeder = fluid.DataFeeder(place=place, feed_list=[x, y])
+ for data in train_reader():
+ cost = exe.run(
+ fluid.default_main_program(),
+ feed=feeder.feed(data),
+ fetch_list=[cost])
+ ```
+
+
+
+---
+
+### 常见问题
+
+
+
+- 如何使用 evaluator ? [->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_label_semantic_roles.py#L168)
+
+ ```python
+ accuracy = fluid.evaluator.Accuracy(input=predict, label=label)
+ for pass_id in range(PASS_NUM):
+ accuracy.reset()
+ for data in train_reader():
+ loss, acc = exe.run(fluid.default_main_program(),
+ feed=feeder.feed(data),
+ fetch_list=[avg_cost] + accuracy.metrics)
+ pass_acc = accuracy.eval(exe)
+ # acc 当前一个batch 的 accuracy
+ # pass_acc 当前batch 的 accuracy
+ pass_total_acc = accuracy.eval(exe) # 整个pass的accuracy
+ ```
+
+- 如何在训练中测试？[->](https://github.com/dzhwinter/benchmark/blob/master/fluid/vgg16.py#L144)
+- 如何保存训练好的模型？[->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_recognize_digits.py#L143)
+- 如何加载训练好的模型进行预测？[->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_recognize_digits.py#L154)
+- 如何在同一个训练任务中定义多个Program，并交替运行？ [->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/demo/fc_gan.py)
+- 如何profile？Fluid 实现了profile 工具，可以直接调用。请参考示例 [->](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/unittests/test_profiler.py)
+
+
+
+
+---
diff --git a/doc/fluid/getstarted/index_cn.rst b/doc/fluid/getstarted/index_cn.rst
index 75af7354be..3daea71d09 100644
--- a/doc/fluid/getstarted/index_cn.rst
+++ b/doc/fluid/getstarted/index_cn.rst
@@ -17,3 +17,4 @@
:maxdepth: 1
concepts/use_concepts_cn.rst
+ developer's_guide_to_paddle_fluid.md
diff --git a/doc/fluid/getstarted/index_en.rst b/doc/fluid/getstarted/index_en.rst
index 75a43f4af8..fb20bb4f24 100644
--- a/doc/fluid/getstarted/index_en.rst
+++ b/doc/fluid/getstarted/index_en.rst
@@ -16,3 +16,4 @@ Here is an example of linear regression. It introduces workflow of PaddlePaddle,
:maxdepth: 1
concepts/index_en.rst
+ developer's_guide_to_paddle_fluid.md
diff --git a/doc/fluid/getstarted/quickstart_cn.rst b/doc/fluid/getstarted/quickstart_cn.rst
index 135beb75d0..6a964d4f85 100644
--- a/doc/fluid/getstarted/quickstart_cn.rst
+++ b/doc/fluid/getstarted/quickstart_cn.rst
@@ -11,7 +11,7 @@ PaddlePaddle支持使用pip快速安装，目前支持CentOS 6以上, Ubuntu 14.
pip install paddlepaddle
-如果需要安装支持GPU的版本（cuda7.5_cudnn5_avx_openblas），需要执行：
+如果需要安装支持GPU的版本（cuda8.0_cudnn5_avx_openblas），需要执行：
.. code-block:: bash
@@ -28,18 +28,18 @@ PaddlePaddle支持使用pip快速安装，目前支持CentOS 6以上, Ubuntu 14.
import paddle.dataset.uci_housing as uci_housing
import paddle.fluid as fluid
-
+
with fluid.scope_guard(fluid.core.Scope()):
# initialize executor with cpu
exe = fluid.Executor(place=fluid.CPUPlace())
- # load inference model
+ # load inference model
[inference_program, feed_target_names,fetch_targets] = \
fluid.io.load_inference_model(uci_housing.fluid_model(), exe)
# run inference
- result = exe.run(inference_program,
- feed={feed_target_names[0]: uci_housing.predict_reader()},
+ result = exe.run(inference_program,
+ feed={feed_target_names[0]: uci_housing.predict_reader()},
fetch_list=fetch_targets)
- # print predicted price is $12,273.97
+ # print predicted price is $12,273.97
print 'Predicted price: ${:,.2f}'.format(result[0][0][0] * 1000)
执行 :code:`python housing.py` 瞧！ 它应该打印出预测住房数据的清单。
diff --git a/doc/fluid/getstarted/quickstart_en.rst b/doc/fluid/getstarted/quickstart_en.rst
index df6619cfd0..680122f258 100644
--- a/doc/fluid/getstarted/quickstart_en.rst
+++ b/doc/fluid/getstarted/quickstart_en.rst
@@ -12,7 +12,7 @@ Simply run the following command to install, the version is cpu_avx_openblas:
pip install paddlepaddle
-If you need to install GPU version (cuda7.5_cudnn5_avx_openblas), run:
+If you need to install GPU version (cuda8.0_cudnn5_avx_openblas), run:
.. code-block:: bash
@@ -31,18 +31,18 @@ code:
import paddle.dataset.uci_housing as uci_housing
import paddle.fluid as fluid
-
+
with fluid.scope_guard(fluid.core.Scope()):
# initialize executor with cpu
exe = fluid.Executor(place=fluid.CPUPlace())
- # load inference model
+ # load inference model
[inference_program, feed_target_names,fetch_targets] = \
fluid.io.load_inference_model(uci_housing.fluid_model(), exe)
# run inference
- result = exe.run(inference_program,
- feed={feed_target_names[0]: uci_housing.predict_reader()},
+ result = exe.run(inference_program,
+ feed={feed_target_names[0]: uci_housing.predict_reader()},
fetch_list=fetch_targets)
- # print predicted price is $12,273.97
+ # print predicted price is $12,273.97
print 'Predicted price: ${:,.2f}'.format(result[0][0][0] * 1000)
Run :code:`python housing.py` and voila! It should print out a list of predictions
diff --git a/doc/fluid/howto/cluster/fluid_cluster_train_cn.md b/doc/fluid/howto/cluster/fluid_cluster_train_cn.md
index b99b90056b..55326940ce 100644
--- a/doc/fluid/howto/cluster/fluid_cluster_train_cn.md
+++ b/doc/fluid/howto/cluster/fluid_cluster_train_cn.md
@@ -168,13 +168,13 @@ cd /paddle/python/paddle/fluid/tests/book
第二步，启动Parameter Server：
```bash
-PADDLE_INIT_PORT=6174 PADDLE_INIT_PSERVERS=192.168.1.2 TRAINERS=2 POD_IP=192.168.1.2 PADDLE_INIT_TRAINER_ID=1 TRAINING_ROLE=PSERVER python test_fit_a_line.py
+PADDLE_PSERVER_PORT=6174 PADDLE_PSERVER_IPS=192.168.1.2 PADDLE_TRAINERS=2 PADDLE_CURRENT_IP=192.168.1.2 PADDLE_TRAINER_ID=1 PADDLE_TRAINING_ROLE=PSERVER python test_fit_a_line.py
```
执行命令后请等待出现提示： ```Server listening on 192.168.1.2:6174 ```, 表示Paramter Server已经正常启动。
第三步，启动Trainer：
```bash
-PADDLE_INIT_PORT=6174 PADDLE_INIT_PSERVERS=192.168.1.3 TRAINERS=2 POD_IP=192.168.1.3 PADDLE_INIT_TRAINER_ID=1 TRAINING_ROLE=TRAINER python test_fit_a_line.py
+PADDLE_PSERVER_PORT=6174 PADDLE_PSERVER_IPS=192.168.1.3 PADDLE_TRAINERS=2 PADDLE_CURRENT_IPP=192.168.1.3 PADDLE_TRAINER_ID=1 PADDLE_TRAINING_ROLE=TRAINER python test_fit_a_line.py
```
由于我们定义的Trainer的数量是2个，因此需要在另外一个计算节点上再启动一个Trainer。
diff --git a/doc/fluid/howto/cluster/fluid_recordio.md b/doc/fluid/howto/cluster/fluid_recordio.md
new file mode 100644
index 0000000000..92859e8f62
--- /dev/null
+++ b/doc/fluid/howto/cluster/fluid_recordio.md
@@ -0,0 +1,127 @@
+# How to use RecordIO in Fluid
+
+If you want to use RecordIO as your training data format, you need to convert to your training data
+to RecordIO files and reading them in the process of training, PaddlePaddle Fluid provides some
+interface to deal with the RecordIO files.
+
+## Generate RecordIO File
+
+Before start training with RecordIO files, you need to convert your training data
+to RecordIO format by `fluid.recordio_writer.convert_reader_to_recordio_file`, the sample codes
+as follows:
+
+```python
+ reader = paddle.batch(mnist.train(), batch_size=1)
+ feeder = fluid.DataFeeder(
+ feed_list=[ # order is image and label
+ fluid.layers.data(
+ name='image', shape=[784]),
+ fluid.layers.data(
+ name='label', shape=[1], dtype='int64'),
+ ],
+ place=fluid.CPUPlace())
+ fluid.recordio_writer.convert_reader_to_recordio_file('./mnist.recordio', reader, feeder)
+```
+
+The above code snippet would generate a RecordIO `./mnist.recordio` on your host.
+
+**NOTE**: we recommend users to set `batch_size=1` when generating the recordio files so that users can
+adjust it flexibly while reading it.
+
+## Use the RecordIO file in a Local Training Job
+
+PaddlePaddle Fluid provides an interface `fluid.layers.io.open_recordio_file` to load your RecordIO file
+and then you can use them as a Layer in your network configuration, the sample codes as follows:
+
+```python
+ data_file = fluid.layers.io.open_recordio_file(
+ filename="./mnist.recordio",
+ shapes=[(-1, 784),(-1, 1)],
+ lod_levels=[0, 0],
+ dtypes=["float32", "int32"])
+ data_file = fluid.layers.io.batch(data_file, batch_size=4)
+
+ img, label = fluid.layers.io.read_file(data_file)
+ hidden = fluid.layers.fc(input=img, size=100, act='tanh')
+ prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
+ loss = fluid.layers.cross_entropy(input=prediction, label=label)
+ avg_loss = fluid.layers.mean(loss)
+
+ fluid.optimizer.Adam(learning_rate=1e-3).minimize(avg_loss)
+
+ place = fluid.CPUPlace()
+
+ exe = fluid.Executor(place)
+ exe.run(fluid.default_startup_program())
+ avg_loss_np = []
+
+ # train a pass
+ batch_id = 0
+ while True:
+ tmp, = exe.run(fetch_list=[avg_loss])
+
+ avg_loss_np.append(tmp)
+ print(batch_id)
+ batch_id += 1
+```
+
+## Use the RecordIO files in Distributed Training
+
+1. generate multiple RecordIO files
+
+For a distributed training job, you may have multiple trainer nodes,
+and one or more RecordIO files for one trainer node, you can use the interface
+`fluid.recordio_writer.convert_reader_to_recordio_files` to convert your training data
+into multiple RecordIO files, the sample codes as follows:
+
+```python
+ reader = paddle.batch(mnist.train(), batch_size=1)
+ feeder = fluid.DataFeeder(
+ feed_list=[ # order is image and label
+ fluid.layers.data(
+ name='image', shape=[784]),
+ fluid.layers.data(
+ name='label', shape=[1], dtype='int64'),
+ ],
+ place=fluid.CPUPlace())
+ fluid.recordio_writer.convert_reader_to_recordio_files(
+ filename_suffix='./mnist.recordio', batch_per_file=100, reader, feeder)
+```
+
+The above codes would generate multiple RecordIO files on your host like:
+
+```bash
+.
+ \_mnist-00000.recordio
+ |-mnist-00001.recordio
+ |-mnist-00002.recordio
+ |-mnist-00003.recordio
+ |-mnist-00004.recordio
+```
+
+2. open multiple RecordIO files by `fluid.layers.io.open_files`
+
+For a distributed training job, the distributed operator system will schedule trainer process on multiple nodes,
+each trainer process reads parts of the whole training data, we usually take the following approach to make the training
+data allocated by each trainer process as uniform as possiable:
+
+```python
+def gen_train_list(file_pattern, trainers, trainer_id):
+ file_list = glob.glob(file_pattern)
+ ret_list = []
+ for idx, f in enumerate(file_list):
+ if (idx + trainers) % trainers == trainer_id:
+ ret_list.append(f)
+ return ret_list
+
+trainers = int(os.getenv("PADDLE_TRAINERS"))
+trainer_id = int(os.getenv("PADDLE_TRAINER_ID"))
+data_file = fluid.layers.io.open_files(
+ filenames=gen_train_list("./mnist-[0-9]*.recordio", 2, 0),
+ thread_num=1,
+ shapes=[(-1, 784),(-1, 1)],
+ lod_levels=[0, 0],
+ dtypes=["float32", "int32"])
+img, label = fluid.layers.io.read_file(data_files)
+...
+```
diff --git a/doc/fluid/howto/cluster/nccl2_rdma_training.md b/doc/fluid/howto/cluster/nccl2_rdma_training.md
new file mode 100644
index 0000000000..cecd5c3a7a
--- /dev/null
+++ b/doc/fluid/howto/cluster/nccl2_rdma_training.md
@@ -0,0 +1,110 @@
+# Distributed Training with NCCL2 and RDMA
+
+When doing distributed multi-GPU training, network bandwith often becomes the
+bottle neck. We introduce a way to use NCCL2 to do such training job to
+achieve best performace.
+
+## Prepare Hardwares with RDMA and Multiple GPUs
+
+I'm using two Linux servers each of them is installed with 8 GPUs and
+one 100Gb RDMA card.
+Base environment is:
+
+* OS: CentOS 7.4
+* RDMA device: "Mellanox Technologies MT27700 Family [ConnectX-4]"
+* Kernel version: `4.4.88-1.el7.elrepo.x86_64`
+* Docker version: `1.12.6`
+* Docker storage driver: `overlay2`
+* IP addresses: 192.168.16.30,192.168.16.34
+
+In general, the steps including:
+
+1. Install GPU drivers
+1. Install RDMA drivers
+1. Install "InfiniBand Support"
+1. Use docker to run tests and make sure GPUs and RDMA can work inside
+ the container.
+
+I'll ommit section "Install GPU drivers" because we can find it easily
+somewhere else.
+
+### Install RDMA drivers
+
+For my case, I've got two machines with device
+"Mellanox Technologies MT27700 Family [ConnectX-4]" installed. The OS was
+"CentOS 7.4" and I updated the kernel to version 4.4 so that docker can
+work with latest overlay2 filesystem.
+
+***NOTE: before you start, make sure you have a way to get a console
+of the server other than ssh because we may need to re-configure the
+network device.***
+
+1. Go to http://www.mellanox.com/page/products_dyn?product_family=26,
+ download `MLNX_OFED` software in the bottom of the page, and upload it
+ onto the server.
+1. Run `./mlnxofedinstall --add-kernel-support` in the software package.
+1. Run `/etc/init.d/openibd restart` to make everything work, note that
+ this operation may cause the network goes down if you are using this
+ RDMA device as default network device and use ssh to login the server.
+1. Re-configure the network interface, for example:
+ `ifconfig eth2 192.168.16.30/20 up`, then add routes if needed:
+ `ip route add default via 192.168.16.1 dev eth2`.
+1. Do the same thing on the other node.
+1. Use `ping` to test if the two nodes have typical ICMP connection.
+1. Use either `udaddy` or `ib_write_bw` to test the network connection is
+ ready and have the desired bandwith.
+
+### Prepare Docker Image to Run RDMA Programs
+
+1. Build a docker image using cuda base image like: `nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04` and install paddlepaddle whl
+ package in it.
+1. Start a docker container and mount GPU driver libs into it (you can
+ skip this step if you are using nvidia-docker).
+1. Mount RDMA dirvers and libs into the docker image (see below section),
+ also `udaddy` and `ib_write_bw` if needed.
+1. Mount GPU devices and RDMA devices into the container using `--device`
+ or just use privileged mode `--privileged`.
+1. Start the container using host network mode: `--net=host`
+
+### RDMA Library Files Needed
+
+Usually, `MLNX_OFED` install latest supported libs under
+`/usr/lib64/mlnx_ofed/valgrind`. Other libs also needed to run RDMA programs
+is listed below. These libs must be mounted into the docker container.
+
+* Libs under `/usr/lib64/mlnx_ofed/valgrind`
+ * libibcm.so
+ * libibverbs.so
+ * libmlx4.so
+ * libmlx5.so
+ * libmlx5-rdmav2.so
+ * librdmacm.so
+* Other libs:
+ * libnl-3.so.200
+ * libnl-route-3.so.200
+ * libnuma.so.1
+
+## Start to Run the Training Job
+
+Setting NCCL environment variables to turn NCCL switches on and off:
+
+
+| Env Name | Description |
+| --- | --- |
+| NCCL_SOCKET_IFNAME | The RDMA device, e.g. eth2 |
+| NCCL_P2P_DISABLE | Set to 1 to disable P2P transfer between GPUs |
+| NCCL_IB_DISABLE | Set to 1 to disable using RDMA |
+| NCCL_IB_CUDA_SUPPORT | Set to 1 to enable GPU Direct if supported |
+| NCCL_DEBUG | Set debug level: VERSION, WARN, INFO |
+
+My two servers are: `192.168.16.30,192.168.16.34`, On node 1, Run :
+
+```bash
+PADDLE_TRAINER_ID=0 PADDLE_PORT=48372 PADDLE_WORKERS=192.168.16.30,192.168.16.34 POD_IP=192.168.16.30 stdbuf -oL python vgg16.py
+```
+
+On node 2, Run:
+
+```bash
+PADDLE_TRAINER_ID=1 PADDLE_PORT=48372 PADDLE_WORKERS=192.168.16.30,192.168.16.34 POD_IP=192.168.16.34 stdbuf -oL python vgg16.py
+```
diff --git a/doc/fluid/howto/index_cn.rst b/doc/fluid/howto/index_cn.rst
index 97aeaf167d..b57af64f44 100644
--- a/doc/fluid/howto/index_cn.rst
+++ b/doc/fluid/howto/index_cn.rst
@@ -3,5 +3,6 @@
.. toctree::
:maxdepth: 1
-
+
+ inference/index_cn.rst
optimization/index_cn.rst
diff --git a/doc/fluid/howto/inference/build_and_install_lib_cn.rst b/doc/fluid/howto/inference/build_and_install_lib_cn.rst
new file mode 100644
index 0000000000..91357dd8c8
--- /dev/null
+++ b/doc/fluid/howto/inference/build_and_install_lib_cn.rst
@@ -0,0 +1,97 @@
+安装与编译C++预测库
+===========================
+
+直接下载安装
+-------------
+
+====================== ========================================
+版本说明 C++预测库
+====================== ========================================
+cpu_avx_mkl `fluid.tgz `_
+cpu_avx_openblas `fluid.tgz `_
+cpu_noavx_openblas `fluid.tgz `_
+cuda7.5_cudnn5_avx_mkl `fluid.tgz `_
+cuda8.0_cudnn5_avx_mkl `fluid.tgz `_
+cuda8.0_cudnn7_avx_mkl `fluid.tgz `_
+cuda9.0_cudnn7_avx_mkl `fluid.tgz `_
+====================== ========================================
+
+从源码编译
+----------
+用户也可以从 PaddlePaddle 核心代码编译C++预测库，只需在编译时配制下面这些编译选项：
+
+================= =========
+选项 值
+================= =========
+CMAKE_BUILD_TYPE Release
+FLUID_INSTALL_DIR 安装路径
+WITH_FLUID_ONLY ON（推荐）
+WITH_SWIG_PY OFF（推荐
+WITH_PYTHON OFF（推荐）
+WITH_GPU ON/OFF
+WITH_MKL ON/OFF
+================= =========
+
+建议按照推荐值设置，以避免链接不必要的库。其它可选编译选项按需进行设定。
+
+下面的代码片段从github拉取最新代码，配制编译选项（需要将PADDLE_ROOT替换为PaddlePaddle预测库的安装路径）：
+
+ .. code-block:: bash
+
+ pip install paddlepaddle-gpu
+ PADDLE_ROOT=/path/of/capi
+ git clone https://github.com/PaddlePaddle/Paddle.git
+ cd Paddle
+ mkdir build
+ cd build
+ cmake -DFLUID_INSTALL_DIR=$PADDLE_ROOT \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DWITH_FLUID_ONLY=ON \
+ -DWITH_SWIG_PY=OFF \
+ -DWITH_PYTHON=OFF \
+ -DWITH_MKL=OFF \
+ -DWITH_GPU=OFF \
+ ..
+ make
+ make inference_lib_dist
+
+成功编译后，使用C++预测库所需的依赖（包括：（1）编译出的PaddlePaddle预测库和头文件；（2）第三方链接库和头文件；（3）版本信息与编译选项信息）
+均会存放于PADDLE_ROOT目录中。目录结构如下：
+
+ .. code-block:: text
+
+ PaddleRoot/
+ ├── CMakeCache.txt
+ ├── paddle
+ │ └── fluid
+ │ ├── framework
+ │ ├── inference
+ │ ├── memory
+ │ ├── platform
+ │ ├── pybind
+ │ └── string
+ ├── third_party
+ │ ├── boost
+ │ │ └── boost
+ │ ├── eigen3
+ │ │ ├── Eigen
+ │ │ └── unsupported
+ │ └── install
+ │ ├── gflags
+ │ ├── glog
+ │ ├── mklml
+ │ ├── protobuf
+ │ ├── snappy
+ │ ├── snappystream
+ │ └── zlib
+ └── version.txt
+
+version.txt 中记录了该预测库的版本信息，包括Git Commit ID、使用OpenBlas或MKL数学库、CUDA/CUDNN版本号，如：
+
+ .. code-block:: text
+
+ GIT COMMIT ID: c95cd4742f02bb009e651a00b07b21c979637dc8
+ WITH_MKL: ON
+ WITH_GPU: ON
+ CUDA version: 8.0
+ CUDNN version: v5
diff --git a/doc/fluid/howto/inference/index_cn.rst b/doc/fluid/howto/inference/index_cn.rst
new file mode 100644
index 0000000000..a903423548
--- /dev/null
+++ b/doc/fluid/howto/inference/index_cn.rst
@@ -0,0 +1,8 @@
+预测库
+------------
+
+.. toctree::
+ :maxdepth: 1
+
+ build_and_install_lib_cn.rst
+ inference_support_in_fluid_cn.md
diff --git a/doc/fluid/howto/inference/inference_support_in_fluid_cn.md b/doc/fluid/howto/inference/inference_support_in_fluid_cn.md
new file mode 100644
index 0000000000..309b17fccd
--- /dev/null
+++ b/doc/fluid/howto/inference/inference_support_in_fluid_cn.md
@@ -0,0 +1,304 @@
+# 使用指南
+
+## 目录：
+
+- Python Inference API
+- Inference C++ API
+- Inference实例
+- Inference计算优化
+
+## Python Inference API **[改进中]**
+- 保存Inference模型 ([链接](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/io.py#L295))
+
+ ```python
+ def save_inference_model(dirname,
+ feeded_var_names,
+ target_vars,
+ executor,
+ main_program=None,
+ model_filename=None,
+ params_filename=None):
+ ```
+ Inference模型和参数将会保存到`dirname`目录下：
+ - 序列化的模型
+ - `model_filename`为`None`，保存到`dirname/__model__`
+ - `model_filename`非`None`，保存到`dirname/model_filename`
+ - 参数
+ - `params_filename`为`None`，单独保存到各个独立的文件，各文件以参数变量的名字命名
+ - `params_filename`非`None`，保存到`dirname/params_filename`
+
+- 两种存储格式
+ - 参数保存到各个独立的文件
+ - 如，设置`model_filename`为`None`、`params_filename`为`None`
+
+ ```bash
+ $ cd recognize_digits_conv.inference.model
+ $ ls
+ $ __model__ batch_norm_1.w_0 batch_norm_1.w_2 conv2d_2.w_0 conv2d_3.w_0 fc_1.w_0 batch_norm_1.b_0 batch_norm_1.w_1 conv2d_2.b_0 conv2d_3.b_0 fc_1.b_0
+ ```
+ - 参数保存到同一个文件
+ - 如，设置`model_filename`为`None`、`params_filename`为`__params__`
+
+ ```bash
+ $ cd recognize_digits_conv.inference.model
+ $ ls
+ $ __model__ __params__
+ ```
+- 加载Inference模型([链接](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/io.py#L380))
+ ```python
+ def load_inference_model(dirname,
+ executor,
+ model_filename=None,
+ params_filename=None):
+ ...
+ return [program, feed_target_names, fetch_targets]
+ ```
+
+## 链接Fluid Inference库
+- 示例项目([链接](https://github.com/luotao1/fluid_inference_example.git))
+
+ - GCC配置
+ ```bash
+ $ g++ -o a.out -std=c++11 main.cc \
+ -I${PADDLE_ROOT}/ \
+ -I${PADDLE_ROOT}/third_party/install/gflags/include \
+ -I${PADDLE_ROOT}/third_party/install/glog/include \
+ -I${PADDLE_ROOT}/third_party/install/protobuf/include \
+ -I${PADDLE_ROOT}/third_party/eigen3 \
+ -L${PADDLE_ROOT}/paddle/fluid/inference -lpaddle_fluid \
+ -lrt -ldl -lpthread
+ ```
+
+ - CMake配置
+ ```cmake
+ include_directories(${PADDLE_ROOT}/)
+ include_directories(${PADDLE_ROOT}/third_party/install/gflags/include)
+ include_directories(${PADDLE_ROOT}/third_party/install/glog/include)
+ include_directories(${PADDLE_ROOT}/third_party/install/protobuf/include)
+ include_directories(${PADDLE_ROOT}/third_party/eigen3)
+ target_link_libraries(${TARGET_NAME}
+ ${PADDLE_ROOT}/paddle/fluid/inference/libpaddle_fluid.so
+ -lrt -ldl -lpthread)
+ ```
+
+ - 设置环境变量：
+ `export LD_LIBRARY_PATH=${PADDLE_ROOT}/paddle/fluid/inference:$LD_LIBRARY_PATH`
+
+
+
+## C++ Inference API
+
+- 推断流程([链接](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/test_helper.h#L91))
+
+ - 1、 初始化设备
+ ```cpp
+ #include "paddle/fluid/framework/init.h"
+ paddle::framework::InitDevices(false);
+ ```
+
+ - 2、 定义place，executor，scope
+ ```cpp
+ auto place = paddle::platform::CPUPlace();
+ auto executor = paddle::framework::Executor(place);
+ auto* scope = new paddle::framework::Scope();
+ ```
+
+ - 3、 加载模型
+ ```cpp
+ #include "paddle/fluid/inference/io.h"
+ auto inference_program = paddle::inference::Load(executor, *scope, dirname);
+ // or
+ auto inference_program = paddle::inference::Load(executor,
+ *scope,
+ dirname + "/" + model_filename,
+ dirname + "/" + params_filename);
+ ```
+
+ - 4、 获取`feed_target_names`和`fetch_target_names`
+ ```cpp
+ const std::vector& feed_target_names = inference_program->GetFeedTargetNames();
+ const std::vector& fetch_target_names = inference_program->GetFetchTargetNames();
+ ```
+
+ - 5、 准备`feed`数据
+ ```cpp
+ #include "paddle/fluid/framework/lod_tensor.h"
+ std::vector cpu_feeds;
+ ...
+ std::map feed_targets;
+ for (size_t i = 0; i < feed_target_names.size(); ++i) {
+ // Please make sure that cpu_feeds[i] is right for feed_target_names[i]
+ feed_targets[feed_target_names[i]] = cpu_feeds[i];
+ }
+ ```
+
+ - 6、 定义`Tensor`来`fetch`结果
+ ```cpp
+ std::vector cpu_fetchs;
+ std::map fetch_targets;
+ for (size_t i = 0; i < fetch_target_names.size(); ++i) {
+ fetch_targets[fetch_target_names[i]] = cpu_fetchs[i];
+ }
+ ```
+
+ - 7、 执行`inference_program`
+ ```cpp
+ executor.Run(*inference_program, scope, feed_targets, fetch_targets);
+ ```
+
+ - 8、 使用`fetch`数据
+ ```cpp
+ for (size_t i = 0; i < cpu_fetchs.size(); ++i) {
+ std::cout << "lod_i: " << cpu_fetchs[i]->lod();
+ std::cout << "dims_i: " << cpu_fetchs[i]->dims();
+ std::cout << "result:";
+ float* output_ptr = cpu_fetchs[i]->data();
+ for (int j = 0; j < cpu_fetchs[i]->numel(); ++j) {
+ std::cout << " " << output_ptr[j];
+ }
+ std::cout << std::endl;
+ }
+ ```
+ 针对不同的数据，4. - 8.可执行多次。
+
+ - 9、 释放内存
+ ```cpp
+ delete scope;
+ ```
+
+
+- 接口说明
+
+ ```cpp
+ void Run(const ProgramDesc& program, Scope* scope,
+ std::map& feed_targets,
+ std::map& fetch_targets,
+ bool create_vars = true,
+ const std::string& feed_holder_name = "feed",
+ const std::string& fetch_holder_name = "fetch");
+ ```
+ - 使用Python API `save_inference_model`保存的`program`里面包含了`feed_op`和`fetch_op`，用户提供的`feed_targets`、`fetch_targets`必须和`inference_program`中的`feed_op`、`fetch_op`保持一致。
+ - 用户提供的`feed_holder_name`和`fetch_holder_name`也必须和`inference_program`中`feed_op`、`fetch_op`保持一致，可使用`SetFeedHolderName`和`SetFetchHolderName`接口重新设置`inferece_program`
+ - 默认情况下，除了`persistable`属性设置为`True`的`Variable`之外，每次执行`executor.Run`会创建一个局部`Scope`，并且在这个局部`Scope`中创建和销毁所有的`Variable`，以最小化空闲时的内存占用。
+ - `persistable`属性为`True`的`Variable`有：
+ - Operators的参数`w`、`b`等
+ - `feed_op`的输入变量
+ - `fetch_op`的输出变量
+
+
+- **不在每次执行时创建和销毁变量
+ ([PR](https://github.com/PaddlePaddle/Paddle/pull/9301))**
+ - 执行`inference_program`
+ ```cpp
+ // Call once
+ executor.CreateVariables(*inference_program, scope, 0);
+ // Call as many times as you like
+ executor.Run(
+ *inference_program, scope, feed_targets, fetch_targets, false);
+ ```
+ - **优点**
+ - 节省了频繁创建、销毁变量的时间（约占每次`Run`总时间的1% ~ 12%）
+ - 执行结束后可获取所有Operators的计算结果
+ - **缺点**
+ - 空闲时也会占用大量的内存
+ - 在同一个`Scope`中，相同的变量名是公用同一块内存的，容易引起意想不到的错误
+
+
+- **不在每次执行时创建Op([PR](https://github.com/PaddlePaddle/Paddle/pull/9630))**
+ - 执行`inference_program`
+ ```cpp
+ // Call once
+ auto ctx = executor.Prepare(*inference_program, 0);
+ // Call as many times as you like if you have no need to change the inference_program
+ executor.RunPreparedContext(ctx.get(), scope, feed_targets, fetch_targets);
+ ```
+ - **优点**
+ - 节省了频繁创建、销毁Op的时间
+ - **缺点**
+ - 一旦修改了`inference_program`，则需要重新创建`ctx`
+
+
+- **多线程共享Parameters([链接](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/test_multi_thread_helper.h))**
+ - 主线程
+ - 1、 初始化设备
+ - 2、 定义`place`，`executor`，`scope`
+ - 3、 加载模型，得到`inference_program`
+ - 从线程
+ - **复制`inference_program`得到`copy_program`，修改`copy_program`的`feed_holder_name`和`fetch_holder_name`**
+ ```cpp
+ auto copy_program = std::unique_ptr(
+ new paddle::framework::ProgramDesc(*inference_program));
+ std::string feed_holder_name = "feed_" + paddle::string::to_string(thread_id);
+ std::string fetch_holder_name = "fetch_" + paddle::string::to_string(thread_id);
+ copy_program->SetFeedHolderName(feed_holder_name);
+ copy_program->SetFetchHolderName(fetch_holder_name);
+ ```
+ - 4、 获取`copy_program`的`feed_target_names`和`fetch_target_names`
+ - 5、 准备feed数据，定义Tensor来fetch结果
+ - 6、 执行`copy_program`
+ ```cpp
+ executor->Run(*copy_program, scope, feed_targets, fetch_targets, true, feed_holder_name, fetch_holder_name);
+ ```
+ - 7、 使用fetch数据
+ - 主线程
+ - 8、 释放资源
+
+
+- 基本概念
+ - 数据相关：
+ - [Tensor](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/tensor.md)，一个N维数组，数据可以是任意类型（int，float，double等）
+ - [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/lod_tensor.md)，带LoD(Level-of-Detail)即序列信息的Tensor
+ - [Scope](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md)，记录了变量Variable
+ - 执行相关：
+ - [Executor](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/executor.md)，无状态执行器，只跟设备相关
+ - Place
+ - CPUPlace，CPU设备
+ - CUDAPlace，CUDA GPU设备
+ - 神经网络表示：
+ - [Program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/program.md).
+
+ 详细介绍请参考[**Paddle Fluid开发者指南**](https://github.com/lcy-seso/learning_notes/blob/master/Fluid/developer's_guid_for_Fluid/Developer's_Guide_to_Paddle_Fluid.md)
+
+
+
+## Inference实例
+
+ 1. fit a line: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_fit_a_line.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_fit_a_line.cc)
+ 1. image classification: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_image_classification.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_image_classification.cc)
+ 1. label semantic roles: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_label_semantic_roles.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_label_semantic_roles.cc)
+ 1. recognize digits: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_recognize_digits.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_recognize_digits.cc)
+ 1. recommender system: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_recommender_system.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_recommender_system.cc)
+ 1. understand sentiment: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_understand_sentiment.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_understand_sentiment.cc)
+ 1. word2vec: [Python](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_word2vec.py), [C++](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/inference/tests/book/test_inference_word2vec.cc)
+
+
+## Inference计算优化
+- 使用Python推理优化工具([inference_transpiler](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/inference_transpiler.py))
+ ```python
+ class InferenceTranspiler:
+ def transpile(self, program, place, scope=None):
+ ...
+ if scope is None:
+ scope = global_scope()
+ ...
+ ```
+ - 使用`InferenceTranspiler`将会直接修改`program`。
+ - 使用`InferenceTranspiler`会修改参数的值，请确保`program`的参数在`scope`内。
+- 支持的优化
+ - 融合batch_norm op的计算
+- 使用示例([链接](https://github.com/Xreki/Xreki.github.io/blob/master/fluid/inference/inference_transpiler.py))
+ ```python
+ import paddle.fluid as fluid
+ # NOTE: Applying the inference transpiler will change the inference_program.
+ t = fluid.InferenceTranspiler()
+ t.transpile(inference_program, place, inference_scope)
+ ```
+
+
+
+
+## 内存使用优化
+- 使用Python内存优化工具([memory_optimization_transipiler](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/memory_optimization_transpiler.py))
+ ```python
+ fluid.memory_optimize(inference_program)
+ ```
diff --git a/doc/fluid/howto/optimization/benchmark/README.md b/doc/fluid/howto/optimization/benchmark/README.md
deleted file mode 120000
index db30af7f53..0000000000
--- a/doc/fluid/howto/optimization/benchmark/README.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../../../benchmark/cluster/README.md
\ No newline at end of file
diff --git a/doc/fluid/howto/optimization/benchmark/vgg16/README.md b/doc/fluid/howto/optimization/benchmark/vgg16/README.md
deleted file mode 120000
index ca963ef5f0..0000000000
--- a/doc/fluid/howto/optimization/benchmark/vgg16/README.md
+++ /dev/null
@@ -1 +0,0 @@
-../../../../../../benchmark/cluster/vgg16/README.md
\ No newline at end of file
diff --git a/doc/fluid/howto/optimization/cpu_profiling_cn.md b/doc/fluid/howto/optimization/cpu_profiling_cn.md
index 8266dec3c6..198a05a79e 100644
--- a/doc/fluid/howto/optimization/cpu_profiling_cn.md
+++ b/doc/fluid/howto/optimization/cpu_profiling_cn.md
@@ -1,3 +1,5 @@
+# CPU性能调优
+
此教程会介绍如何使用Python的cProfile包、Python库yep、Google perftools来进行性能分析 (profiling) 与调优（performance tuning）。
Profling 指发现性能瓶颈。系统中的瓶颈可能和程序员开发过程中想象的瓶颈相去甚远。Tuning 指消除瓶颈。性能优化的过程通常是不断重复地 profiling 和 tuning。
@@ -8,7 +10,7 @@ PaddlePaddle 用户一般通过调用 Python API 编写深度学习程序。大
* Python 与 C++ 混合代码的性能分析
-# Python代码的性能分析
+## Python代码的性能分析
### 生成性能分析文件
diff --git a/doc/fluid/howto/optimization/cpu_profiling_en.md b/doc/fluid/howto/optimization/cpu_profiling_en.md
index e95556dd60..216694965b 100644
--- a/doc/fluid/howto/optimization/cpu_profiling_en.md
+++ b/doc/fluid/howto/optimization/cpu_profiling_en.md
@@ -1,3 +1,5 @@
+# Tune CPU performance
+
This tutorial introduces techniques we use to profile and tune the
CPU performance of PaddlePaddle. We will use Python packages
`cProfile` and `yep`, and Google's `perftools`.
@@ -14,7 +16,7 @@ the profiling and tuning of
1. the Python code and
1. the mixture of Python and C++ code.
-# Profiling the Python Code
+## Profiling the Python Code
### Generate the Performance Profiling File
diff --git a/doc/fluid/howto/optimization/host_memory_profiling_cn.md b/doc/fluid/howto/optimization/host_memory_profiling_cn.md
new file mode 100644
index 0000000000..7fb0883dd9
--- /dev/null
+++ b/doc/fluid/howto/optimization/host_memory_profiling_cn.md
@@ -0,0 +1,89 @@
+# 堆内存分析和优化
+
+计算机程序都可能有内存泄漏的风险。**内存泄漏**一般是由于程序在堆(heap)上分配了内存而没有释放，随着程序的运行占用的内存越来越大，一方面会影响程序的稳定性，可能让运行速度越来越慢，或者造成oom，甚至会影响运行程序的机器的稳定性，造成宕机。
+
+
+目前有很多内存泄漏分析工具，比较经典的有[valgrind](http://valgrind.org/docs/manual/quick-start.html#quick-start.intro), [gperftools](https://gperftools.github.io/gperftools/)。
+
+因为Fluid是用Python驱动C++ core来运行，valgrind直接分析非常困难，需要自己编译debug版本的、带valgrind支持的专用Python版本，而且输出的信息中大部分是Python自己的符号和调用信息，分析起来很困难，另外使用valgrind会让程序运行速度变得非常慢，所以不建议使用。
+
+本教程主要介绍[gperftools](https://gperftools.github.io/gperftools/)的使用。
+
+gperftool主要支持以下四个功能：
+
+- thread-caching malloc
+- heap-checking using tcmalloc
+- heap-profiling using tcmalloc
+- CPU profiler
+
+Paddle也提供了基于gperftool的[CPU性能分析教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/howto/optimization/cpu_profiling_cn.md)。
+
+对于堆内存的分析，主要用到thread-caching malloc和heap-profiling using tcmalloc。
+
+## 环境
+
+本教程基于paddle提供的Docker开发环境paddlepaddle/paddle:latest-dev，基于Ubuntu 16.04.4 LTS环境。
+
+## 使用流程
+
+- 安装google-perftools
+
+```
+apt-get install libunwind-dev
+apt-get install google-perftools
+```
+
+- 安装pprof
+
+```
+go get -u github.com/google/pprof
+```
+
+- 设置运行环境
+
+```
+export PPROF_PATH=/root/gopath/bin/pprof
+export PPROF_BINARY_PATH=/root/gopath/bin/pprof
+export LD_PRELOAD=/usr/lib/libtcmalloc.so.4
+```
+
+- 使用heap profile来运行python程序。本质上是周期性的对堆的分配情况做一次快照。
+
+```
+# HEAPPROFILE 设置生成的堆分析文件的目录和文件前缀
+# HEAP_PROFILE_ALLOCATION_INTERVAL 设置每分配多少存储dump一次dump，默认1GB
+env HEAPPROFILE="./perf_log/test.log" HEAP_PROFILE_ALLOCATION_INTERVAL=209715200 python trainer.py
+```
+
+随着程序的运行，会在perf_log这个文件夹下生成很多文件，如下：
+
+```
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0001.heap
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0002.heap
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0003.heap
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0004.heap
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0005.heap
+-rw-r--r-- 1 root root 1.0M Jun 1 15:00 test.log.0006.heap
+```
+
+- 使用pprof对heap文件进行分析。分析有两种模式：
+ - 完整模式。会对当前heap做一个分析，显示目前分配内存一些调用路径。
+
+ ```
+ pprof --pdf python test.log.0012.heap
+ ```
+ 上述命令会生成一个profile00x.pdf的文件，可以直接打开，例如：[memory_cpu_allocator](https://github.com/jacquesqiao/Paddle/blob/bd2ea0e1f84bb6522a66d44a072598153634cade/doc/fluid/howto/optimization/memory_cpu_allocator.pdf)。从下图可以看出，在CPU版本fluid的运行过程中，分配存储最多的模块式CPUAllocator. 而别的模块相对而言分配内存较少，所以被忽略了，这对于分配内存泄漏是很不方便的，因为泄漏是一个缓慢的过程，在这种图中是无法看到的。
+
+ 
+
+ - Diff模式。可以对两个时刻的heap做diff，把一些内存分配没有发生变化的模块去掉，而把增量部分显示出来。
+ ```
+ pprof --pdf --base test.log.0010.heap python test.log.1045.heap
+ ```
+ 生成的结果为：[`memory_leak_protobuf`](https://github.com/jacquesqiao/Paddle/blob/bd2ea0e1f84bb6522a66d44a072598153634cade/doc/fluid/howto/optimization/memory_leak_protobuf.pdf)
+
+ 从图中可以看出：ProgramDesc这个结构，在两个版本之间增长了200MB+，所以这里有很大的内存泄漏的可能性，最终结果也确实证明是这里造成了泄漏。
+
+ 
+ 
+
diff --git a/doc/fluid/howto/optimization/timeline.md b/doc/fluid/howto/optimization/timeline.md
deleted file mode 100644
index 96481ae2a6..0000000000
--- a/doc/fluid/howto/optimization/timeline.md
+++ /dev/null
@@ -1,27 +0,0 @@
-# how to use timeline tool to do profile
-
-1. Add `with profiler.profiler(...)` to the main training loop. After run, the code will generate a profile record file `/tmp/profile`. **Warning**: Please do not run too many batches when use profiler to record timeline information, for the profile record will grow with the batch number.
-
- ```python
- with profiler.profiler('All', 'total', '/tmp/profile') as prof:
- for pass_id in range(pass_num):
- for batch_id, data in enumerate(train_reader()):
- exe.run(fluid.default_main_program(),
- feed=feeder.feed(data),
- fetch_list=[],
- use_program_cache=True)
- ...
- ```
-
-1. Run `python paddle/tools/timeline.py` to process `/tmp/profile`, it will generate another
-file `/tmp/timeline` by default. You can change the path by cmd parameter, please take a look at
-[timeline.py](https://github.com/PaddlePaddle/Paddle/blob/develop/tools/timeline.py) for details.
-
-1. Open chrome and visit , use `load` button to load the generated `timeline` file.
-
- 
-
-1. The resulting timeline should be like:
-
-
- 
diff --git a/doc/fluid/howto/optimization/timeline_cn.md b/doc/fluid/howto/optimization/timeline_cn.md
new file mode 100644
index 0000000000..faf39f276d
--- /dev/null
+++ b/doc/fluid/howto/optimization/timeline_cn.md
@@ -0,0 +1,32 @@
+# 如何使用timeline工具做性能分析
+
+1. 在训练的主循环外加上`profiler.start_profiler(...)`和`profiler.stop_profiler(...)`。运行之后，代码会在`/tmp/profile`目录下生成一个profile的记录文件。
+
+ **提示：**
+ 请不要在timeline记录信息时运行太多次迭代，因为timeline中的记录数量和迭代次数是成正比的。
+
+ ```python
+ for pass_id in range(pass_num):
+ for batch_id, data in enumerate(train_reader()):
+ if pass_id == 0 and batch_id == 5:
+ profiler.start_profiler("All")
+ elif pass_id == 0 and batch_id == 10:
+ profiler.stop_profiler("total", "/tmp/profile")
+ exe.run(fluid.default_main_program(),
+ feed=feeder.feed(data),
+ fetch_list=[])
+ ...
+ ```
+
+1. 运行`python paddle/tools/timeline.py`来处理`/tmp/profile`，这个程序默认会生成一个`/tmp/timeline`文件，你也可以用命令行参数来修改这个路径，请参考[timeline.py](https://github.com/PaddlePaddle/Paddle/blob/develop/tools/timeline.py)。
+```python
+python Paddle/tools/timeline.py --profile_path=/tmp/profile --timeline_path=timeline
+```
+
+1. 打开chrome浏览器，访问，用`load`按钮来加载生成的`timeline`文件。
+
+ 
+
+1. 结果如下图所示，可以放到来查看timetime的细节信息。
+
+ 
diff --git a/doc/fluid/howto/optimization/timeline_en.md b/doc/fluid/howto/optimization/timeline_en.md
new file mode 100644
index 0000000000..6f963c6b4d
--- /dev/null
+++ b/doc/fluid/howto/optimization/timeline_en.md
@@ -0,0 +1,33 @@
+# how to use timeline tool to do profile
+
+1. Add `profiler.start_profiler(...)`和`profiler.stop_profiler(...)` to the main training loop. After run, the code will generate a profile record file `/tmp/profile`. **Warning**: Please do not run too many batches when use profiler to record timeline information, for the profile record will grow with the batch number.
+
+ ```python
+ for pass_id in range(pass_num):
+ for batch_id, data in enumerate(train_reader()):
+ if pass_id == 0 and batch_id == 5:
+ profiler.start_profiler("All")
+ elif pass_id == 0 and batch_id == 10:
+ profiler.stop_profiler("total", "/tmp/profile")
+ exe.run(fluid.default_main_program(),
+ feed=feeder.feed(data),
+ fetch_list=[])
+ ...
+ ```
+
+1. Run `python paddle/tools/timeline.py` to process `/tmp/profile`, it will generate another
+file `/tmp/timeline` by default. You can change the path by cmd parameter, please take a look at
+[timeline.py](https://github.com/PaddlePaddle/Paddle/blob/develop/tools/timeline.py) for details.
+
+```python
+python Paddle/tools/timeline.py --profile_path=/tmp/profile --timeline_path=timeline
+```
+
+1. Open chrome and visit , use `load` button to load the generated `timeline` file.
+
+ 
+
+1. The resulting timeline should be like:
+
+
+ 
diff --git a/doc/fluid/howto/performance/error_clip.md b/doc/fluid/howto/performance/error_clip.md
index 58aa73b8cd..749cf7693c 100644
--- a/doc/fluid/howto/performance/error_clip.md
+++ b/doc/fluid/howto/performance/error_clip.md
@@ -78,7 +78,7 @@ def error_clip_callback(block, context):
op_desc = block.desc.op(block.desc.op_size() - 1)
for grad_n in filter(lambda n: grad_to_var.has_key(n),
op_desc.output_arg_names()):
- fwd_var = block.var_recursive(grad_to_var[grad_n])
+ fwd_var = block.__var_recursive(grad_to_var[grad_n])
error_clip = getattr(fwd_var, "error_clip", None)
if not (error_clip is None or isinstance(error_clip,
BaseErrorClipAttr)):
diff --git a/doc/fluid/images/1.png b/doc/fluid/images/1.png
new file mode 100644
index 0000000000..67daf566f9
Binary files /dev/null and b/doc/fluid/images/1.png differ
diff --git a/doc/fluid/images/2.png b/doc/fluid/images/2.png
new file mode 100644
index 0000000000..43367777f4
Binary files /dev/null and b/doc/fluid/images/2.png differ
diff --git a/doc/fluid/images/3.png b/doc/fluid/images/3.png
new file mode 100644
index 0000000000..481021ef30
Binary files /dev/null and b/doc/fluid/images/3.png differ
diff --git a/doc/fluid/images/4.png b/doc/fluid/images/4.png
new file mode 100644
index 0000000000..4279f41e06
Binary files /dev/null and b/doc/fluid/images/4.png differ
diff --git a/doc/fluid/images/LoDTensor.png b/doc/fluid/images/LoDTensor.png
new file mode 100644
index 0000000000..75369f5378
Binary files /dev/null and b/doc/fluid/images/LoDTensor.png differ
diff --git a/doc/fluid/images/compile_run_time.png b/doc/fluid/images/compile_run_time.png
new file mode 100644
index 0000000000..0bc9b2fd0e
Binary files /dev/null and b/doc/fluid/images/compile_run_time.png differ
diff --git a/doc/fluid/images/executor.png b/doc/fluid/images/executor.png
new file mode 100644
index 0000000000..b29c0d779e
Binary files /dev/null and b/doc/fluid/images/executor.png differ
diff --git a/doc/fluid/images/fluid_examples.png b/doc/fluid/images/fluid_examples.png
new file mode 100644
index 0000000000..aa99472c0f
Binary files /dev/null and b/doc/fluid/images/fluid_examples.png differ
diff --git a/doc/fluid/images/fluid_module_1.png b/doc/fluid/images/fluid_module_1.png
new file mode 100644
index 0000000000..554782ba54
Binary files /dev/null and b/doc/fluid/images/fluid_module_1.png differ
diff --git a/doc/fluid/images/fluid_module_2.png b/doc/fluid/images/fluid_module_2.png
new file mode 100644
index 0000000000..4219efccbb
Binary files /dev/null and b/doc/fluid/images/fluid_module_2.png differ
diff --git a/doc/fluid/images/layer.png b/doc/fluid/images/layer.png
new file mode 100644
index 0000000000..e46db4c9c6
Binary files /dev/null and b/doc/fluid/images/layer.png differ
diff --git a/doc/fluid/images/operator1.png b/doc/fluid/images/operator1.png
new file mode 100644
index 0000000000..3975b06f61
Binary files /dev/null and b/doc/fluid/images/operator1.png differ
diff --git a/doc/fluid/images/operator2.png b/doc/fluid/images/operator2.png
new file mode 100644
index 0000000000..b7bb1fae20
Binary files /dev/null and b/doc/fluid/images/operator2.png differ
diff --git a/doc/fluid/images/place.png b/doc/fluid/images/place.png
new file mode 100644
index 0000000000..14e77511d6
Binary files /dev/null and b/doc/fluid/images/place.png differ
diff --git a/doc/fluid/images/print_fluid_program.png b/doc/fluid/images/print_fluid_program.png
new file mode 100644
index 0000000000..e8e459e1b3
Binary files /dev/null and b/doc/fluid/images/print_fluid_program.png differ
diff --git a/doc/fluid/images/program_desc1.png b/doc/fluid/images/program_desc1.png
new file mode 100644
index 0000000000..0656336914
Binary files /dev/null and b/doc/fluid/images/program_desc1.png differ
diff --git a/doc/fluid/images/program_desc2.png b/doc/fluid/images/program_desc2.png
new file mode 100644
index 0000000000..db5bfa1231
Binary files /dev/null and b/doc/fluid/images/program_desc2.png differ
diff --git a/doc/fluid/images/raw_input.png b/doc/fluid/images/raw_input.png
new file mode 100644
index 0000000000..0725f92d2b
Binary files /dev/null and b/doc/fluid/images/raw_input.png differ
diff --git a/doc/fluid/images/scope_variable_tensor.png b/doc/fluid/images/scope_variable_tensor.png
new file mode 100644
index 0000000000..59b0de6fb3
Binary files /dev/null and b/doc/fluid/images/scope_variable_tensor.png differ
diff --git a/doc/fluid/images/sorted_input.png b/doc/fluid/images/sorted_input.png
new file mode 100644
index 0000000000..ff60112836
Binary files /dev/null and b/doc/fluid/images/sorted_input.png differ
diff --git a/doc/fluid/images/transpiler.png b/doc/fluid/images/transpiler.png
new file mode 100644
index 0000000000..422973c0dc
Binary files /dev/null and b/doc/fluid/images/transpiler.png differ
diff --git a/doc/fluid/images/user_interface.png b/doc/fluid/images/user_interface.png
new file mode 100644
index 0000000000..ffc94e3d89
Binary files /dev/null and b/doc/fluid/images/user_interface.png differ
diff --git a/doc/fluid/index_cn.rst b/doc/fluid/index_cn.rst
index d878d192ca..6b1ef3ceed 100644
--- a/doc/fluid/index_cn.rst
+++ b/doc/fluid/index_cn.rst
@@ -1,12 +1,16 @@
- PaddlePaddle Fluid
-==========================
+.. PaddlePaddle Fluid documentation master file, created by
+ sphinx-quickstart on Thu Jun 7 17:04:53 2018.
+ You can adapt this file completely to your liking, but it should at least
+ contain the root `toctree` directive.
+
+##############
+欢迎使用 Fluid
+##############
.. toctree::
- :maxdepth: 1
+ :maxdepth: 1
- getstarted/index_cn.rst
- build_and_install/index_cn.rst
- design/index_cn.rst
- howto/index_cn.rst
- dev/index_cn.rst
- faq/index_cn.rst
+ new_docs/beginners_guide/index.rst
+ new_docs/user_guides/index.rst
+ new_docs/advanced_usage/index.rst
+ new_docs/faq/index_cn.rst
diff --git a/doc/fluid/new_docs/advanced_usage/benchmark.rst b/doc/fluid/new_docs/advanced_usage/benchmark.rst
new file mode 100644
index 0000000000..7854263bf8
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/benchmark.rst
@@ -0,0 +1,120 @@
+#################
+如何进行基准测试
+#################
+
+本文介绍如何给深度学习框架做基准测试。基准测试主要包含验证模型的精度和性能两方面，下文包含搭建测试环境，选择基准测试模型，验证测试结果等几方面内容。
+
+验证深度学习框架，可分为训练和测试两个阶段， 验证指标略有不同，本文只介绍训练阶段的指标验证。训练阶段关注的是模型训练集上的精度，训练集是完备的，因此关注大batch\_size下的训练速度,关注吞吐量，例如图像模型常用的batch\_size=128, 多卡情况下会加大；预测阶段关注的是在测试集上的精度，线上服务测试数据不能提前收集，因此关注小batch\_size下的预测速度，关注延迟，例如预测服务常用的batch\_size=1, 4等。
+
+`Fluid `__ 是PaddlePaddle从0.11.0版本开始引入的设计，本文的基准测试在该版本上完成。
+
+
+环境搭建
+""""""""""""
+
+基准测试中模型精度和硬件、框架无关，由模型结构和数据共同决定；性能方面由测试硬件和框架性能决定。框架基准测试为了对比框架之间的差异，控制硬件环境，系统库等版本一致。下文中的对比实验都在相同的硬件条件和系统环境条件下进行.
+
+
+不同架构的GPU卡性能差异巨大，在验证模型在GPU上训练性能时，可使用NVIDIA提供的工具:code `nvidia-smi` 检验当前使用的GPU型号，如果测试多卡训练性能，需确认硬件连接是 `nvlink `__ 或 `PCIe `__ 。 同样地，CPU型号会极大影响模型在CPU上的训练性能。可读取`/proc/cpuinfo`中的参数，确认当前正在使用的CPU型号。
+
+下载GPU对应的Cuda Tool Kit和 Cudnn，或者使用NVIDIA官方发布的nvidia-docker镜像 `nvidia-docker `__, 镜像内包含了Cuda和Cudnn，本文采用这种方式。 Cuda Tool Kit包含了GPU代码使用到的基础库，影响在此基础上编译出的Fluid二进制运行性能。
+
+准备好Cuda环境后，从github上的下载Paddle并源码编译，会生成对应的最适合当前GPU的sm\_arch二进制\ `sm\_arch `__\ 。另外，cudnn对卷积类任务影响巨大，在基准测试中需要小版本一致，例如Cudnn7.0.2与Cudnn7.1.4在Resnet上有5%以上差异。
+
+
+选择基准模型
+""""""""""""
+
+对框架做基准测试，需要覆盖不同训练任务和不同大小的模型，本文中选取了图像和NLP的最为常用的5个模型。
+
+============ ============ ================= ============
+任务种类 模型名称 网络结构 数据集
+============ ============ ================= ============
+图像分类 mnist Lenet mnist
+图像分类 VGG VGG-16 Flowers102
+图像分类 Resnet Resnet-50 Flowers102
+文本分类 Stacked-LSTM Stacked-LSTM IMDB
+机器翻译 seq-seq Stacked-LSTM wmt14
+============ ============ ================= ============
+
+其中mnist, VGG, Resnet属于CNN模型, stacked-lstm, seq2seq代表RNN模型。
+`benchmark `__
+基准模型测试脚本中，均跳过了前几个batch的训练过程，原因是加载数据和分配显存受系统当前运行情况影响，会导致统计性能不准确。运行完若干个轮次后，统计对应指标。
+
+
+基准模型的数据的选择方面，数据量大且验证效果多的公开数据集为首选。图像模型VGG和resnet, 本文选择了 `flowers102 `__ ，图像大小预处理为和Imagenet相同大小，因此性能可直接对比
+NLP模型的公开且影响力大数据集较少，seq2seq模型选择了wmt14数据，stacked-lstm模型中选择了 `imdb `__ 数据。
+
+
+注意，图像模型每条样本大小相同，图像经过变换后大小一致，因此经过的计算路径基本相同，计算速度和显存占用波动较小，可以从若干个batch的数据中采样得到当前的训练性能数据。而NLP模型由于样本长度不定，计算路径和显存占用也不相同，因此只能完整运行若干个轮次后，统计速度和显存消耗。
+显存分配是特别耗时的操作，因此Fluid默认会占用所有可用显存空间形成显存池，用以加速计算过程中的显存分配。如果需要统计模型真实显存消耗，可设置环境变量`FLAGS_fraction_of_gpu_memory_to_use=0.0`，观察最大显存开销。
+
+
+测试过程
+""""""""""""
+
+- CPU 单机单线程测试
+
+测试CPU上单线程的性能，先设置CUDA的环境变量为空，``CUDA_VISIBLE_DEVICES=``，并通过环境变量关闭OpenMP和MKL的多线程 ``OMP_NUM_THREADS=1``， ``MKL_NUM_THREADS=1;``。
+然后代码中设置为使用CPUPlace，如果使用Paddle代码库中的脚本，只需要命令行参数传入 use_gpu=False即可。
+
+.. code-block:: python
+
+ >>> import paddle.fluid as fluid
+ >>> place = fluid.CPUPlace()
+
+.. code:: bash
+
+ docker run -it --name CASE_NAME --security-opt seccomp=unconfined -v $PWD/benchmark:/benchmark paddlepaddle/paddle:latest-dev /bin/bash
+
+
+- GPU 单机单卡测试
+
+本教程使用了Cuda8, Cudnn7.0.1。来源为:code `nvidia/cuda:8.0-cudnn7-devel-ubuntu16.04`
+
+.. code:: bash
+
+ nvidia-docker run -it --name CASE_NAME --security-opt seccomp=unconfined -v $PWD/benchmark:/benchmark -v /usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu paddlepaddle/paddle:latest-dev /bin/bash
+在单卡上测试，设置CUDA的环境变量使用一块GPU，``CUDA_VISIBLE_DEVICES=0``
+然后代码中设置为使用CUDAPlace，如果使用Paddle代码库中的脚本，只需要命令行参数传入 use_gpu=True即可。
+
+.. code-block:: python
+
+ >>> import paddle.fluid as fluid
+ >>> place = fluid.CUDAPlace(0) // 0 指第0块GPU
+
+
+测试结果
+""""""""""""
+
+本教程对比相同环境下的Fluid0.12.0和TensorFlow1.4.0的性能表现。
+硬件环境为 CPU: Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz, GPU: TITAN X(Pascal) 12G x 1, Nvidia-Driver 384.90。
+系统环境为Ubuntu 16.04.3 LTS, 本文中采用了docker环境，系统版本为nvidia-docker17.05.0-ce。
+测试的Fluid版本为\ `v.0.12.0 `__ 。
+TensorFlow版本为\ `v.1.4.0-rc1 `__ 。
+使用的脚本和配置见\ `benchmark `__ 。
+图表中统计单位为samples/秒。
+
+- CPU 单机单线程测试结果
+
+ ================ ==================== ===================
+ Speed Fluid CPU TensorFlow CPU
+ ================ ==================== ===================
+ mnist 1298.75 samples/s 637.57 samples/s
+ VGG-16 0.4147 images/s 0.1229 images/s
+ Resnet-50 1.6935 images/s 0.3657 images/s
+ Stacked-LSTM 472.3225 words/s 48.2293words/s
+ Seq2Seq 217.1655 words/s 28.6164 words/s
+ ================ ==================== ===================
+
+- GPU 单机单卡测试结果
+
+ =============== ===================== =================
+ Speed Fluid GPU TensorFlow GPU
+ =============== ===================== =================
+ mnist 19710.90 samples/s 15576.3 samples/s
+ VGG-16 59.83327 images/s 40.9967 images/s
+ Resnet-50 105.84412 97.8923 images/s
+ Stacked-LSTM 1319.99315 1608.2526 words/s
+ Seq2Seq 7147.89081 6845.1161 words/s
+ =============== ===================== =================
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/anakin_arm_benchmark.md b/doc/fluid/new_docs/advanced_usage/deploy/anakin_arm_benchmark.md
new file mode 100644
index 0000000000..08ea379f81
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/anakin_arm_benchmark.md
@@ -0,0 +1,56 @@
+# Anakin ARM 性能测试
+
+## 测试环境和参数:
++ 测试模型Mobilenetv1, mobilenetv2, mobilenet-ssd
++ 采用android ndk交叉编译，gcc 4.9，enable neon， ABI： armveabi-v7a with neon -mfloat-abi=softfp
++ 测试平台
+ - 荣耀v9(root): 处理器:麒麟960, 4 big cores in 2.36GHz, 4 little cores in 1.8GHz
+ - nubia z17:处理器:高通835, 4 big cores in 2.36GHz, 4 little cores in 1.9GHz
+ - 360 N5:处理器:高通653, 4 big cores in 1.8GHz, 4 little cores in 1.4GHz
++ 多线程：openmp
++ 时间：warmup10次，运行10次取均值
++ ncnn版本：来源于github的master branch中commits ID：307a77f04be29875f40d337cfff6df747df09de6（msg:convert LogisticRegressionOutput)版本
++ TFlite版本：来源于github的master branch中commits ID：65c05bc2ac19f51f7027e66350bc71652662125c（msg:Removed unneeded file copy that was causing failure in Pi builds)版本
+
+在BenchMark中本文将使用**`ncnn`**、**`TFlite`**和**`Anakin`**进行性能对比分析
+
+## BenchMark model
+
+> 注意在性能测试之前，请先将测试model通过[External Converter](#10003)转换为Anakin model
+> 对这些model，本文在ARM上进行多线程的单batch size测试。
+
+- [Mobilenet v1](#11) *caffe model 可以在[这儿](https://github.com/shicai/MobileNet-Caffe)下载*
+- [Mobilenet v2](#22) *caffe model 可以在[这儿](https://github.com/shicai/MobileNet-Caffe)下载*
+- [mobilenet-ssd](#33) *caffe model 可以在[这儿](https://github.com/chuanqi305/MobileNet-SSD)下载*
+
+### mobilenetv1
+
+ |platform | Anakin (1) | Anakin (2) | Anakin (4) | ncnn (1) | ncnn (2) | ncnn (4) | TFlite (1) | TFlite (2) | TFlite (4)|
+ |:---: | :---: | :---: | :---:| :---:| :---:| :---:| :---:| :---:| :---:|
+ |麒麟960|107.7ms|61.1ms|38.2ms|152.8ms|85.2ms|51.9ms|152.6ms|nan|nan|
+ |高通835|105.7ms|63.1ms|~~46.8ms~~|152.7ms|87.0ms|~~92.7ms~~|146.9ms|nan|nan|
+ |高通653|120.3ms|64.2ms|46.6ms|202.5ms|117.6ms|84.8ms|158.6ms|nan|nan|
+
+### mobilenetv2
+
+ |platform | Anakin (1) | Anakin (2) | Anakin (4) | ncnn (1) | ncnn (2) | ncnn (4) | TFlite (1) | TFlite (2) | TFlite (4)|
+ |:---: | :---: | :---: | :---:| :---:| :---:| :---:| :---:| :---:| :---:|
+ |麒麟960|93.1ms|53.9ms|34.8ms|144.4ms|84.3ms|55.3ms|100.6ms|nan|nan|
+ |高通835|93.0ms|55.6ms|41.1ms|139.1ms|88.4ms|58.1ms|95.2ms|nan|nan|
+ |高通653|106.6ms|64.2ms|48.0ms|199.9ms|125.1ms|98.9ms|108.5ms|nan|nan|
+
+### mobilenet-ssd
+
+ |platform | Anakin (1) | Anakin (2) | Anakin (4) | ncnn (1) | ncnn (2) | ncnn (4) | TFlite (1) | TFlite (2) | TFlite (4)|
+ |:---: | :---: | :---: | :---:| :---:| :---:| :---:| :---:| :---:| :---:|
+ |麒麟960|213.9ms|120.5ms|74.5ms|307.9ms|166.5ms|104.2ms|nan|nan|nan|
+ |高通835|213.0ms|125.7ms|~~98.4ms~~|292.9ms|177.9ms|~~167.8ms~~|nan|nan|nan|
+ |高通653|236.0ms|129.6ms|96.0ms|377.7ms|228.9ms|165.0ms|nan|nan|nan
+
+## How to run those Benchmark models?
+
+1. 首先, 使用[External Converter](../docs/Manual/Converter_en.md)对caffe model 进行转换
+2. 然后将转换后的Anakin model和编译好的benchmark_arm 二进制文件通过'adb push'命令上传至测试机
+3. 接着在测试机含有Anakin model的目录中运行'./benchmark_arm ./ anakin_model.anakin.bin 1 10 10 1' 命令
+4. 最后，终端显示器上将会打印该模型的运行时间
+5. 其中运行命令的参数个数和含义可以通过运行'./benchmark_arm'看到
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/anakin_example.md b/doc/fluid/new_docs/advanced_usage/deploy/anakin_example.md
new file mode 100644
index 0000000000..e6b9e18fe2
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/anakin_example.md
@@ -0,0 +1,28 @@
+# Example
+Anakin目前只支持NCHW的格式
+示例文件在test/framework/net下
+
+## 在NV的GPU上运行CNN模型
+示例文件为打开example_nv_cnn_net.cpp，整体流程如下：
+- 将模型的的path设置为anakin模型的路径，初始化NV平台的图对象。 anakin模型可以通过转换 器转化caffe或fluid的模型得到
+- 根据模型设置网络图的输入尺寸，进行图优化
+- 根据优化后的网络图初始化网络执行器
+- 取出网络的输入tensor，将数据拷贝到输入tensor
+- 运行推导
+- 取出网络的输出tensor
+
+以NV平台为例演示Anakin框架的使用方法，注意编译时需要打开GPU编译开关
+
+## 在X86上运行RNN模型
+示例文件为example_x86_rnn_net.cpp
+整体流程与在NV的GPU上运行CNN模型相似，不同之处如下：
+- 使用X86标识初始化图对象和网络执行器对象
+- rnn模型的输入尺寸是可变的，初始化图时的输入维度是维度的最大值，输入维度N代表总的词的个数。还需要设置输入tensor的seq_offset来标示这些词是如何划分为句子的,如{0,5,12}表示共有12个词，其中第0到第4个词是第一句话，第5到第11个词是第二句话
+
+以X86平台为例演示Anakin框架的使用方法，注意编译时需要打开X86编译开关
+
+## 在NV的GPU上使用Anakin的线程池运行CNN模型
+示例文件为example_nv_cnn_net_multi_thread.cpp ，示例使用worker的同步预测接口
+整体流程与在NV的GPU上运行CNN模型相似，不同之处如下：
+- 用模型地址和线程池大小初始化worker对象
+- 将输入tensor注入任务队列,获得输出tensor
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/anakin_gpu_benchmark.md b/doc/fluid/new_docs/advanced_usage/deploy/anakin_gpu_benchmark.md
new file mode 100644
index 0000000000..667f9396f1
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/anakin_gpu_benchmark.md
@@ -0,0 +1,170 @@
+# Anakin GPU Benchmark
+
+## Machine:
+
+> CPU: `12-core Intel(R) Xeon(R) CPU E5-2620 v2 @2.10GHz`
+> GPU: `Tesla P4`
+> cuDNN: `v7`
+
+
+## Counterpart of anakin :
+
+The counterpart of **`Anakin`** is the acknowledged high performance inference engine **`NVIDIA TensorRT 3`** , The models which TensorRT 3 doesn't support we use the custom plugins to support.
+
+## Benchmark Model
+
+The following convolutional neural networks are tested with both `Anakin` and `TenorRT3`.
+ You can use pretrained caffe model or the model trained by youself.
+
+> Please note that you should transform caffe model or others into anakin model with the help of [`external converter ->`](../docs/Manual/Converter_en.md)
+
+
+- [Vgg16](#1) *caffe model can be found [here->](https://gist.github.com/jimmie33/27c1c0a7736ba66c2395)*
+- [Yolo](#2) *caffe model can be found [here->](https://github.com/hojel/caffe-yolo-model)*
+- [Resnet50](#3) *caffe model can be found [here->](https://github.com/KaimingHe/deep-residual-networks#models)*
+- [Resnet101](#4) *caffe model can be found [here->](https://github.com/KaimingHe/deep-residual-networks#models)*
+- [Mobilenet v1](#5) *caffe model can be found [here->](https://github.com/shicai/MobileNet-Caffe)*
+- [Mobilenet v2](#6) *caffe model can be found [here->](https://github.com/shicai/MobileNet-Caffe)*
+- [RNN](#7) *not support yet*
+
+We tested them on single-GPU with single-thread.
+
+### VGG16
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 8.8690 | 8.2815 |
+| 2 | 15.5344 | 13.9116 |
+| 4 | 26.6000 | 21.8747 |
+| 8 | 49.8279 | 40.4076 |
+| 32 | 188.6270 | 163.7660 |
+
+- GPU Memory Used (`MB`)
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 963 | 997 |
+| 2 | 965 | 1039 |
+| 4 | 991 | 1115 |
+| 8 | 1067 | 1269 |
+| 32 | 1715 | 2193 |
+
+
+### Yolo
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 16.4596| 15.2124 |
+| 2 | 26.6347| 25.0442 |
+| 4 | 43.3695| 43.5017 |
+| 8 | 80.9139 | 80.9880 |
+| 32 | 293.8080| 310.8810 |
+
+- GPU Memory Used (`MB`)
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 1569 | 1775 |
+| 2 | 1649 | 1815 |
+| 4 | 1709 | 1887 |
+| 8 | 1731 | 2031 |
+| 32 | 2253 | 2907 |
+
+### Resnet50
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 4.2459 | 4.1061 |
+| 2 | 6.2627 | 6.5159 |
+| 4 | 10.1277 | 11.3327 |
+| 8 | 17.8209 | 20.6680 |
+| 32 | 65.8582 | 77.8858 |
+
+- GPU Memory Used (`MB`)
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 531 | 503 |
+| 2 | 543 | 517 |
+| 4 | 583 | 541 |
+| 8 | 611 | 589 |
+| 32 | 809 | 879 |
+
+### Resnet101
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 7.5562 | 7.0837 |
+| 2 | 11.6023 | 11.4079 |
+| 4 | 18.3650 | 20.0493 |
+| 8 | 32.7632 | 36.0648 |
+| 32 | 123.2550 | 135.4880 |
+
+- GPU Memory Used (`MB)`
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 701 | 683 |
+| 2 | 713 | 697 |
+| 4 | 793 | 721 |
+| 8 | 819 | 769 |
+| 32 | 1043 | 1059 |
+
+### MobileNet V1
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 45.5156 | 1.3947 |
+| 2 | 46.5585 | 2.5483 |
+| 4 | 48.4242 | 4.3404 |
+| 8 | 52.7957 | 8.1513 |
+| 32 | 83.2519 | 31.3178 |
+
+- GPU Memory Used (`MB`)
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 329 | 283 |
+| 2 | 345 | 289 |
+| 4 | 371 | 299 |
+| 8 | 393 | 319 |
+| 32 | 531 | 433 |
+
+### MobileNet V2
+
+- Latency (`ms`) of different batch
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 65.6861 | 2.9842 |
+| 2 | 66.6814 | 4.7472 |
+| 4 | 69.7114 | 7.4163 |
+| 8 | 76.1092 | 12.8779 |
+| 32 | 124.9810 | 47.2142 |
+
+- GPU Memory Used (`MB`)
+
+| BatchSize | TensorRT | Anakin |
+| --- | --- | --- |
+| 1 | 341 | 293 |
+| 2 | 353 | 301 |
+| 4 | 385 | 319 |
+| 8 | 421 | 351 |
+| 32 | 637 | 551 |
+
+## How to run those Benchmark models?
+
+> 1. At first, you should parse the caffe model with [`external converter`](https://github.com/PaddlePaddle/Anakin/blob/b95f31e19993a192e7428b4fcf852b9fe9860e5f/docs/Manual/Converter_en.md).
+> 2. Switch to *source_root/benchmark/CNN* directory. Use 'mkdir ./models' to create ./models and put anakin models into this file.
+> 3. Use command 'sh run.sh', we will create files in logs to save model log with different batch size. Finally, model latency summary will be displayed on the screen.
+> 4. If you want to get more detailed information with op time, you can modify CMakeLists.txt with setting `ENABLE_OP_TIMER` to `YES`, then recompile and run. You will find detailed information in model log file.
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/anakin_tutorial.md b/doc/fluid/new_docs/advanced_usage/deploy/anakin_tutorial.md
new file mode 100644
index 0000000000..5efbc89abd
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/anakin_tutorial.md
@@ -0,0 +1,639 @@
+# Anakin 使用教程 ##
+
+本教程将会简略的介绍Anakin的工作原理，一些基本的Anakin API，以及如何调用这些API。
+
+## 内容 ###
+
+- [Anakin的工作原理](#principle)
+- [Anakin APIs](#api)
+- [示例代码](#example)
+
+## Anakin的工作原理 ###
+
+
+
+用Anakin来进行前向计算主要分为三个步骤：
+
+- 将外部模型通过[Anakin Parser](Converter_ch.md)解析为Anakin模型
+ 在使用Anakin之前，用户必须将所有其他模型转换成Anakin模型，我们提供了转换脚本，用户可通过[Anakin Parser](Converter_ch.md)进行模型转换。
+- 生成Anakin计算图
+ 加载Anakin模型生成原始计算图，然后需要对原始计算图进行优化。你只需要调用相应的API优化即可。
+- 执行计算图
+ Anakin会选择不同硬件平台执行计算图。
+
+
+## Anakin APIs ###
+### Tensor ####
+
+`Tensor`提供基础的数据操作和管理，为ops提供统一的数据接口。`Tensor`包含以下几个属性：
+
+- Buffer
+ 数据存储区
+- Shape
+ 数据的维度信息
+- Event
+ 用于异步计算的同步
+
+ `Tensor` 类包含三个`Shape`对象， 分别是`_shape`, `_valid_shape`和 `offset`。 `_shape`为`tensor`真正空间信息，`_valid_shape`表示当前`tensor`使用的空间信息， `_offset`表示当前`tensor`数据指针相对于真正数据空间的信息。 `Tensor`不同维度与分别与数学中的向量、矩阵等相对应如下表所示。
+
+
+Dimentions | Math entity |
+ :----: | :----:
+1 | vector
+2 | matrix
+3 | 3-tensor
+n | n-tensor
+
+#### 声明tensor对象
+
+`Tensor`接受三个模板参数:
+
+
+```c++
+ template
+ class Tensor .../* Inherit other class */{
+ //some implements
+ ...
+ };
+```
+
+TargetType是平台类型，如X86，GPU等等，在Anakin内部有相应的标识与之对应；datatype是普通的数据类型，在Anakin内部也有相应的标志与之对应；[LayOutType](#layout)是数据分布类型，如batch x channel x height x width [NxCxHxW], 在Anakin内部用一个struct来标识。 Anakin中数据类型与基本数据类型的对应如下:
+
+1. TargetType
+
+ Anakin TargetType | platform
+ :----: | :----:|
+ NV | NVIDIA GPU
+ ARM | ARM
+ AMD | AMD GPU
+ X86 | X86
+ NVHX86 | NVIDIA GPU with Pinned Memory
+
+2. DataType
+
+Anakin DataType | C++ | Description
+:---: | :---: | :---: |
+AK_HALF | short | fp16
+AK_FLOAT | float | fp32
+AK_DOUBLE | double | fp64
+AK_INT8 | char | int8
+AK_INT16 | short | int16
+AK_INT32 | int | int32
+AK_INT64 | long | int64
+AK_UINT8 | unsigned char | uint8
+AK_UINT16 | unsigned short | uint8
+AK_UINT32 | unsigned int | uint32
+AK_STRING | std::string | /
+AK_BOOL | bool | /
+AK_SHAPE | / | Anakin Shape
+AK_TENSOR | / | Anakin Tensor
+
+
+3. LayOutType
+
+Anakin LayOutType ( Tensor LayOut ) | Tensor Dimention | Tensor Support | Op Support
+:---: | :---: | :---: | :---: |
+W | 1-D | YES | NO
+HW | 2-D | YES | NO
+WH | 2-D | YES | NO
+NW | 2-D | YES | YES
+NHW | 3-D | YES |YES
+NCHW ( default ) | 4-D | YES | YES
+NHWC | 4-D | YES | NO
+NCHW_C4 | 5-D | YES | YES
+
+
+理论上，Anakin支持申明1维以上的tensor，但是对于Anakin中的Op来说，只支持NW、NHW、NCHW、NCHW_C4这四种LayOut，其中NCHW是默认的LayOutType，NCHW_C4是专门针对于int8这种数据类型的。
+
+
+例子
+
+> 下面的代码将展示如何使用tensor， 我们建议先看看这些示例。
+
+> 要想获得更多关于tensor的信息， 请参考 *soure_path/core/tensor.h*
+
+> 1. 使用shape对象初始化tensor
+``` c++
+ //create a null tensor. A null tensor holds for nothing.
+ //tensor's buffer is resident at CPU and its datatype is AK_FLOAT.
+ //tensor's Layout is NCHW(default)
+ Tensor mytensor;
+
+ //1. using shape object to create a tensor.
+ Shape shape1(NUM); //1-D shape. NUM is the number of dimention.
+ Tensor mytensor1(shape1); //1-D tensor.
+
+ // A 4-D shape
+ Shape shape2(N, C, H, W); // batch x channel x height x width
+```
+
+>`注意：Shape的维度必须和tensor的`[LayoutType](#layout)`相同，比如Shape(N,C,H,W), 那么Tensor的 LayoutType必须是NCHW，否则会出错。如下列代码所示`
+
+
+```c++
+ // A 4-D tensor.
+ Tensor mytensor2(shape2); //right
+
+ //A 4-D tensor which is resident at GPU and its datatype is AK_INT8
+ Tensor mytensor3(shape2); //right
+
+ Tensor mytensor4(shape2); //wrong!! shape's dimetion must be equal to tensor's Layout.
+ Tensor mytensor5(shape2); //wrong!!!!
+
+```
+
+> 2. 使用现有的数据和shape初始化tensor
+
+```c++
+
+ /**
+ * A construtor of Tensor.
+ * data_ptr is a pointer to any data type of data
+ * TargetType is type of a platform [Anakin TargetType]
+ * id : device id
+ * shape: a Anakin shape
+ */
+ Tensor(Dtype* data_ptr, TargetType_t target, int id, Shape shape);
+
+ //using existing data feed to a tensor
+ Tensor mytensor(data_ptr, TargetType, device_id, shape); //shape must has dimention (N, C, H, W).
+
+```
+
+> 3. 使用tensor初始化tensor
+
+```c++
+ Tensor tensor(exist_tensor);
+```
+
+
+> 提示： 你可以用` typedef Tensor Tensor4d_X86 `方便定义tensor
+
+
+#### 填充tensor数据区
+
+
+填充数据区得看你申明tensor的方式， 下面展示了如何填充tensor的数据区。
+
+```c++
+首先来看看tensor的四种声明方式：
+
+1. Tensor mytensor;
+2. Tensor mytensor1(shape1);
+3. Tensor mytensor(data_ptr, TargetType, device_id, shape);
+4. Tensor tensor(exist_tensor);
+
+
+相关的声明方式的数据填充方法如下：
+
+1：声明一个空的tensor，此时没有为其分配内存，所以，我们需要手动的为其分配内存。
+
+ //parama shape
+ mytensor.re_alloc(Shape shape);
+
+ //Get writable pointer to mytensor.
+ //parama index (int): where you start to write.
+ //Dtype is your data type such int, float or double.
+ Dtype *p = mytensor.mutable_data(index/*=0*/);
+ //write data to mytensor
+ for(int i = 0; i < mytensor.size(); i++){
+ p[i] = 1.0f;
+ }
+ //do something ...
+
+2: 这种声明方式会自动分配内存
+
+ //Get writable pointer to mytensor.
+ //parama index (int): where you start to write.
+ //Dtype is your data type such int, float or double.
+ Dtype *p = mytensor1.mutable_data(index/*=0*/);
+ //write data to mytensor
+ for(int i = 0; i < mytensor.size(); i++){
+ p[i] = 1.0f;
+ }
+ //do something ...
+
+
+3：在该种声明方式中，我们仍不需要手动为其分配内存。但在构造函数内部是否为其分配内存，得依情况而定。如果data_ptr和申明的
+tensor都在都一个目标平台上，那么该tensor就会与data_ptr共享内存空间，相反，如果他们不在同一个平台上（如data_ptr在X86上，而
+tensor在GPU上），那么此时tensor就会开辟一个新的内存空间，并将data_ptr所指向的数据拷贝到tensor的buffer中。
+
+ //Get writable pointer to mytensor.
+ //parama index (int): where you start to write.
+ //Dtype is your data type such int, float or double.
+ Dtype *p = mytensor.mutable_data(index/*=0*/);
+ //write data to mytensor
+ for(int i = 0; i < mytensor.size(); i++){
+ p[i] = 1.0f;
+ }
+ //do something ...
+
+4：该种方式仍不需要手动分配内存
+
+ //Get writable pointer to mytensor.
+ //parama index (int): where you start to write.
+ //Dtype is your data type such int, float or double.
+ Dtype *p = mytensor.mutable_data(index/*=0*/);
+ //write data to mytensor
+ for(int i = 0; i < mytensor.size(); i++){
+ p[i] = 1.0f;
+ }
+ //do something ...
+
+
+另外，你还可以获取一个tensor的可读指针，示例如下：
+ //Get read-only pointer to mytensor.
+ //parama index (int): where you start to read.
+ //Dtype is your data type such int, float or double.
+ Dtype *p = mytensor.data(index/*=0*/);
+ //do something ...
+```
+
+如果想更详细的了解tensor，请查阅*soure_path/saber/core/tensor.h*
+
+#### 获取tensor的shape
+
+```c++
+//some declarations
+// ...
+Shape shape = mytensor.shape();
+
+//Get a first dimetion size of tesor, if it has.
+int d1 = shape[0];
+
+//Get a second dimention size of tensor, if it has.
+int d2 = shape[1];
+
+...
+
+//Get a n-th dimention size of tensor, if it has.
+int dn = shape[n-1];
+
+
+//Get a tensor's dimention
+int dims = mytensor.dims();
+
+//Get the size of tensor.
+//size = d1 x d2 x ... x dn.
+int size = mytensor.size();
+
+//Get the size of tensor at interval [Di, Dj)
+// form i-th dimention to j-th dimention, but not including the j-th dimention.
+// which means di x (di+1) x ... x (dj -1)
+int size = mytensor.count(start, end);
+```
+
+#### 设置tensor的shape
+
+我们可以用tensor的成员函数set_shape来设置tensor的shape。 下面是set_shape的定义
+
+
+```c++
+/**
+ * \brief set a tensor's shape
+ * \param valid_shape [a Shape object]
+ * \param shape [a Shape object]
+ * \param offset [a Shape object]
+ * \return the status of this operation, that means whether it success * or not.
+ */
+SaberStatus set_shape(Shape valid_shape, Shape shape = Shape::zero(TensorAPI::layout_dims::value), Shape offset = Shape::minusone(TensorAPI::layout_dims::value));
+```
+
+这个成员函数只设置tensor的shape。这些shape对象(valid_shape, shape, offset)的[LayOutType](#layout)必须和当前的tensor的相应三个shape对象的LayOutType相同，如果不同就会出错，返回SaberInvalidValue。 如果相同，那么将成功设置tensor的shape。
+
+```c++
+
+// some declarations
+// ...
+//valid_shape, shape , offset are Shape object;
+//All these Shape object's LayOutType must be equal to mytensor's.
+mytensor.set_shape(valid_shape, shape, offset);
+
+```
+
+#### 重置 tensor的shape
+
+```c++
+//some declarations
+Shape shape, valid_shape, offset;
+
+//do some initializations
+...
+mytensor.reshape(valid_shape, shape, offset);
+```
+
+注意： Reshape操作仍然需要shape的[LayOutType](#layout) 与tensor的相同
+
+
+### Graph ###
+
+`Graph`类负责加载Anakin模型生成计算图、对图进行优化、存储模型等操作。
+
+#### 图的声明
+
+与`Tensor`一样，graph也接受三个模板参数。
+
+```c++
+
+template
+class Graph ... /* inherit other class*/{
+
+ //some implements
+ ...
+
+};
+```
+
+前面已经介绍过[TargetType](#target)和[DataType](#datatype)是Anakin内部自定义数据类型。[TargetType](#target)表示平台类型 (如NV、X86), [DataType](#datatype)是Anakin基本数据类型与C++/C中的基本数据类型相对应。 [Precision](#precision)为op所支持的精度类型, 稍后我们在介绍它。
+
+
+```c++
+
+//Create a empty graph object.
+Graph graph = Graph tmp();
+
+//Create a pointer to a empty graph.
+Graph *graph = new Graph();
+
+//Create a pointer to a empty graph.
+auto graph = new Graph();
+
+```
+
+#### 加载 Anakin 模型
+
+```c++
+//some declarations
+...
+auto graph = new Graph();
+std::string model_path = "the/path/to/where/your/models/are";
+const char *model_path1 = "the/path/to/where/your/models/are";
+
+//Loading Anakin model to generate a compute graph.
+auto status = graph->load(model_path);
+
+//Or this way.
+auto status = graph->load(model_path1);
+//Check whether load operation success.
+if(!status){
+ std::cout << "error" << endl;
+ //do something...
+}
+
+```
+
+#### 优化计算图
+
+```c++
+//some declarations
+...
+//Load graph.
+...
+//According to the ops of loaded graph, optimize compute graph.
+graph->Optimize();
+
+```
+
+> 注意： 第一次加载原始图，必须要优化。
+
+#### 保存模型
+
+你可以在任何时候保存模型， 特别的， 你可以保存一个优化的模型，这样，下次再加载模型时，就不必进行优化操作。
+
+
+```c++
+//some declarations
+...
+//Load graph.
+...
+// save a model
+//save_model_path: the path to where your model is.
+auto status = graph->save(save_model_path);
+
+//Checking
+if(!status){
+ cout << "error" << endl;
+ //do somethin...
+}
+```
+
+#### 重新设置计算图里的tensor的shape
+
+```c++
+//some declarations
+...
+//Load graph.
+...
+vector shape{10, 256, 256, 10};
+//input_name : std::string.
+//Reshape a tensor named input_name.
+graph->Reshape(input_name, shape);//Note: shape is a vector, not a Shape object.
+```
+
+#### 设置 batch size
+
+`Graph` 支持重新设置batch size的大小。
+
+```c++
+//some declarations
+...
+//Load graph.
+...
+//input_name : std::string.
+//Reset a tensor named input_name.
+int new_batch_size = 4;
+graph->ResetBatchSize(input_name, new_batch_size);
+```
+
+### Net ###
+
+
+`Net` 是计算图的执行器。你可以通过Net对象获得输入和输出
+#### Creating a graph executor
+
+`Net`接受四个模板参数。
+
+
+```c++
+template
+class Net{
+ //some implements
+ ...
+
+};
+```
+由于有些Op可能支持多种精度，我们可以通过Precision来指定。OpRunType表示同步或异步类型，异步是默认类型。OpRunType::SYNC表示同步，在GPU上只有单个流；OpRunType::ASYNC表示异步，在GPU上有多个流并以异步方式执行。实际上，Precision和OpRunType都是enum class, 详细设计请参考*source_root/framework/core/types.h*.
+
+
+1. Precision
+
+Precision | Op support
+:---: | :---:
+Precision::INT4 | NO
+Precision::INT8 | NO
+Precision::FP16 | NO
+Precision::FP32 | YES
+Precision::FP64 | NO
+
+现在Op的精度只支持FP32， 但在将来我们会支持剩下的Precision.
+
+
+
+2. OpRunType
+
+OpRunType | Sync/Aync |Description
+:---: | :---: | :---:
+OpRunType::SYNC | Synchronization | single-stream on GPU
+OpRunType::ASYNC | Asynchronization | multi-stream on GPU
+
+用graph对象创建一个执行器。
+```c++
+//some declarations
+...
+//Create a pointer to a graph.
+auto graph = new Graph();
+//do something...
+...
+
+//create a executor
+Net executor(*graph);
+
+```
+
+#### 获取输入输出tensor
+
+
+获取输入输出tensor，并填充输入tensor的buffer。如果想要获取输入和输出tensor，那么必须指定输入的名字，如"input_0", "input_1", "input_2", ..., 必须传入如上字符串才能够获得输入tensor。另外，如果想知道input_i对应哪个输入，你需要去dash board查看，如何使用dash board请看[Anakin Parser](Converter_ch.md)。请看如下示例代码
+
+```c++
+//some declaratinos
+...
+
+//create a executor
+//TargetType is NV [NVIDIA GPU]
+Net executor(*graph);
+
+//Get the first input tensor.
+//The following tensors(tensor_in0, tensor_in2 ...) are resident at GPU.
+//Note: Member function get_in returns an pointer to tensor.
+Tensor* tensor_in0 = executor.get_in("input_0");
+
+//If you have multiple input tensors
+//You just type this code below.
+Tensor* tensor_in1 = executor.get_in("input_1");
+...
+auto tensor_inn = executor.get_in("input_n");
+```
+
+当得到输入tensor之后，就可以填充它的数据区了。
+
+```c++
+//This tensor is resident at GPU.
+auto tensor_d_in = executor.get_in("input_0");
+
+//If we want to feed above tensor, we must feed the tensor which is resident at host. And then copy the host tensor to the device's one.
+
+//using Tensor4d = Tensor;
+Tensor4d tensor_h_in; //host tensor;
+//Tensor tensor_h_in;
+
+//Allocate memory for host tensor.
+tensor_h_in.re_alloc(tensor_d_in->valid_shape());
+//Get a writable pointer to tensor.
+float *h_data = tensor_h_in.mutable_data();
+
+//Feed your tensor.
+/** example
+for(int i = 0; i < tensor_h_in.size(); i++){
+ h_data[i] = 1.0f;
+}
+*/
+//Copy host tensor's data to device tensor.
+tensor_d_in->copy_from(tensor_h_in);
+
+// And then
+```
+
+
+类似的，我们可以利用成员函数get_out来获得输出tensor。但与获得输入tensor不同的是， 我们需要指定输入tensor结点的名字，这个可以从dash board中看到，请从[Anakin Parser](Converter_ch.md)中查看dash board的使用方法。假如有个输出结点叫pred_out, 那么我们可以通过如下代码获得相应的输出tensor：
+```c++
+//Note: this tensor are resident at GPU.
+Tensor* tensor_out_d = executor.get_out("pred_out");
+
+```
+
+
+#### Executing graph
+
+
+当一切准备就绪后，我们就可以执行真正的计算了！
+```c++
+executor.prediction();
+```
+
+## 示例代码 ##
+
+下面的例子展示了如何调用Anakin。
+
+在这儿之前， 请确保你已经有了Anakin模型。如果还没有，那么请使用[Anakin Parser](Converter_ch.md)转换你的模型。
+
+### Single-thread
+
+单线程例子在 *source_root/test/framework/net/net_exec_test.cpp`*
+
+```c++
+
+std::string model_path = "your_Anakin_models/xxxxx.anakin.bin";
+// Create an empty graph object.
+auto graph = new Graph();
+// Load Anakin model.
+auto status = graph->load(model_path);
+if(!status ) {
+ LOG(FATAL) << " [ERROR] " << status.info();
+}
+// Reshape
+graph->Reshape("input_0", {10, 384, 960, 10});
+// You must optimize graph for the first time.
+graph->Optimize();
+// Create a executer.
+Net net_executer(*graph);
+
+//Get your input tensors through some specific string such as "input_0", "input_1", and
+//so on.
+//And then, feed the input tensor.
+//If you don't know Which input do these specific string ("input_0", "input_1") correspond with, you can launch dash board to find out.
+auto d_tensor_in_p = net_executer.get_in("input_0");
+Tensor4d h_tensor_in;
+auto valid_shape_in = d_tensor_in_p->valid_shape();
+for (int i=0; icopy_from(h_tensor_in);
+
+//Do inference.
+net_executer.prediction();
+
+//Get result tensor through the name of output node.
+//And also, you need to see the dash board again to find out how many output nodes are and remember their name.
+
+//For example, you've got a output node named obj_pre_out
+//Then, you can get an output tensor.
+auto d_tensor_out_0_p = net_executer.get_out("obj_pred_out"); //get_out returns a pointer to output tensor.
+auto d_tensor_out_1_p = net_executer.get_out("lc_pred_out"); //get_out returns a pointer to output tensor.
+//......
+// do something else ...
+//...
+//save model.
+//You might not optimize the graph when you load the saved model again.
+std::string save_model_path = model_path + std::string(".saved");
+auto status = graph->save(save_model_path);
+if (!status ) {
+ LOG(FATAL) << " [ERROR] " << status.info();
+}
+
+```
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/build_and_install_lib_cn.rst b/doc/fluid/new_docs/advanced_usage/deploy/build_and_install_lib_cn.rst
new file mode 100644
index 0000000000..3884284ea0
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/build_and_install_lib_cn.rst
@@ -0,0 +1,99 @@
+.. _install_or_build_cpp_inference_lib:
+
+安装与编译C++预测库
+===========================
+
+直接下载安装
+-------------
+
+====================== ========================================
+版本说明 C++预测库
+====================== ========================================
+cpu_avx_mkl `fluid.tgz `_
+cpu_avx_openblas `fluid.tgz `_
+cpu_noavx_openblas `fluid.tgz `_
+cuda7.5_cudnn5_avx_mkl `fluid.tgz `_
+cuda8.0_cudnn5_avx_mkl `fluid.tgz `_
+cuda8.0_cudnn7_avx_mkl `fluid.tgz `_
+cuda9.0_cudnn7_avx_mkl `fluid.tgz `_
+====================== ========================================
+
+从源码编译
+----------
+用户也可以从 PaddlePaddle 核心代码编译C++预测库，只需在编译时配制下面这些编译选项：
+
+================= =========
+选项 值
+================= =========
+CMAKE_BUILD_TYPE Release
+FLUID_INSTALL_DIR 安装路径
+WITH_FLUID_ONLY ON（推荐）
+WITH_SWIG_PY OFF（推荐
+WITH_PYTHON OFF（推荐）
+WITH_GPU ON/OFF
+WITH_MKL ON/OFF
+================= =========
+
+建议按照推荐值设置，以避免链接不必要的库。其它可选编译选项按需进行设定。
+
+下面的代码片段从github拉取最新代码，配制编译选项（需要将PADDLE_ROOT替换为PaddlePaddle预测库的安装路径）：
+
+ .. code-block:: bash
+
+ pip install paddlepaddle-gpu
+ PADDLE_ROOT=/path/of/capi
+ git clone https://github.com/PaddlePaddle/Paddle.git
+ cd Paddle
+ mkdir build
+ cd build
+ cmake -DFLUID_INSTALL_DIR=$PADDLE_ROOT \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DWITH_FLUID_ONLY=ON \
+ -DWITH_SWIG_PY=OFF \
+ -DWITH_PYTHON=OFF \
+ -DWITH_MKL=OFF \
+ -DWITH_GPU=OFF \
+ ..
+ make
+ make inference_lib_dist
+
+成功编译后，使用C++预测库所需的依赖（包括：（1）编译出的PaddlePaddle预测库和头文件；（2）第三方链接库和头文件；（3）版本信息与编译选项信息）
+均会存放于PADDLE_ROOT目录中。目录结构如下：
+
+ .. code-block:: text
+
+ PaddleRoot/
+ ├── CMakeCache.txt
+ ├── paddle
+ │ └── fluid
+ │ ├── framework
+ │ ├── inference
+ │ ├── memory
+ │ ├── platform
+ │ ├── pybind
+ │ └── string
+ ├── third_party
+ │ ├── boost
+ │ │ └── boost
+ │ ├── eigen3
+ │ │ ├── Eigen
+ │ │ └── unsupported
+ │ └── install
+ │ ├── gflags
+ │ ├── glog
+ │ ├── mklml
+ │ ├── protobuf
+ │ ├── snappy
+ │ ├── snappystream
+ │ └── zlib
+ └── version.txt
+
+version.txt 中记录了该预测库的版本信息，包括Git Commit ID、使用OpenBlas或MKL数学库、CUDA/CUDNN版本号，如：
+
+ .. code-block:: text
+
+ GIT COMMIT ID: c95cd4742f02bb009e651a00b07b21c979637dc8
+ WITH_MKL: ON
+ WITH_GPU: ON
+ CUDA version: 8.0
+ CUDNN version: v5
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/convert_paddle_to_anakin.md b/doc/fluid/new_docs/advanced_usage/deploy/convert_paddle_to_anakin.md
new file mode 100644
index 0000000000..56ca582b2b
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/convert_paddle_to_anakin.md
@@ -0,0 +1,73 @@
+# 模型转换指南
+
+Anakin 支持不同框架的模型预测。但由于格式的差别，Anakin 需要您预先转换模型。本文档介绍如何转换模型。
+
+## 简介
+
+Anakin 模型转换器输入支持 Caffe 和 Fluid 两种格式的预测模型，模型包含网络结构（model 或 prototxt）和权重参数（param 或 caffemodel）。
+
+模型转换的输出是一个 bin 文件，它作为 Anakin 框架的 graph 参数导入。
+
+您还可以使用模型转换器的 launch board 功能生成网络结构的 HTML 预览。
+
+
+## 系统要求
+
+- python 2.7+
+- pyyaml
+- flask
+- protobuf 3.5+
+
+
+## 用法
+
+### 1、环境
+转换器所需的依赖标注于 *系统要求* 一节。
+
+### 2、配置
+您需要对 *config.yaml* 文件进行修改以告知您的需求。工程中给出了 *config.yaml* 示例，下面作进一步说明。
+
+#### config.yaml
+```bash
+OPTIONS:
+ Framework: CAFFE # 依框架类型填写 CAFFE 或 FLUID
+ SavePath: ./output # 转换结束后模型的保存位置
+ ResultName: googlenet # 输出模型的名字
+ Config:
+ LaunchBoard: ON # 是否生成网络结构预览页面
+ Server:
+ ip: 0.0.0.0
+ port: 8888 # 从一个可用端口访问预览页面
+ OptimizedGraph: # 当您使用了 Anakin 框架的 Optimized 功能时，才应该打开此项
+ enable: OFF
+ path: /path/to/anakin_optimized_anakin_model/googlenet.anakin.bin.saved
+ LOGGER:
+ LogToPath: ./log/ # 生成日志的路径
+ WithColor: ON
+
+TARGET:
+ CAFFE:
+ # 当 Framework 为 CAFFE 时需填写
+ ProtoPaths:
+ - /path/to/caffe/src/caffe/proto/caffe.proto
+ PrototxtPath: /path/to/your/googlenet.prototxt
+ ModelPath: /path/to/your/googlenet.caffemodel
+
+ FLUID:
+ # 当 Framework 为 FLUID 时需填写
+ Debug: NULL
+ ProtoPaths:
+ - /
+ PrototxtPath: /path/to/fluid/inference_model
+ ModelPath: /path/to/fluid/inference_model
+ # ...
+```
+
+### 3、转换
+在完成配置文件的修改后，您只需执行 ```python converter.py``` 就可以进行模型转换了。
+
+
+### 4、预览
+最后一步，就是在浏览器中查看令人振奋的转换结果！网址是在 *config.yaml* 中配置的，例如 http://0.0.0.0:8888 。
+
+> 注意：若您使用了默认的 IP 地址 0.0.0.0，请在预览时使用真实的服务器地址 real_ip:port 替代它。
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/how_to_add_anakin_op.md b/doc/fluid/new_docs/advanced_usage/deploy/how_to_add_anakin_op.md
new file mode 100644
index 0000000000..f2783eb9f5
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/how_to_add_anakin_op.md
@@ -0,0 +1,405 @@
+# 如何增加新的Operator
+
+## 基本概念
+
+简单介绍下几个同Operator相关的基本概念，详情请参考设计文档。
+
+```framework```: 上层的逻辑代码，负责从parser中获取参数及weights，添加op时主要修改framework/operator目录下的内容。
+
+```saber```: 底层的实现代码，Anakin通过saber封装了不同的backends，不同的实现(impl)分别特化出自己的实现，外层framework通过不同的template进入各自的impl完成调用。各个op的parameter放在saber/saber_funcs_param.h文件中，增加op主要修改saber/funcs下的内容。
+
+saber的文件结构：
+* saber/funcs下的是各个funcs的外部接口，这一层的op与具体的设备实现无关，只与各op完成的功能有关。由于跟实现(impl)无关，本层文件明均不带impl。
+* saber/funcs/impl下是各个op的impl声明，特定设备需要完成该层声明的特化版本，如saber/funcs/impl/x86实现了上一层impl声明的x86特化版本，saber/funcs/impl/cuda实现了上一层impl声明的NV特化版本。当增加新的backends时需要特化出新的实现。本层代码同实现相关，均带有```impl_```前缀。
+* saber/funcs/impl/cuda/base/cuda_c内有cuda```.cu```扩展名的文件，添加cuda的kernel需要在该文件目录下添加。
+* saber/funcs/impl/cuda/base/sass 内有不同架构的汇编代码编译的静态库。
+
+### 涉及到的基类及各个类之前的关系
+
+简单介绍相关的基类
+
+* ```anakin::Operator```: framework的operator基类，位于framework/core/operator/operator.h
+
+* ```anakin::saber::BaseFunc```: saber对外的op接口基类，提供统一的对外接口，位于saber/funcs/base.h。BaseFunc的```compute_output_shape```接口只根据input的shape和param的参数计算输出的shape，并通过```tensor```的```set_shape```接口(只设置shape，不分配空间)设置到output中。```operator()```接口为各个op的计算接口。
+
+* ```ankain::saber::ImplBase```: saber设备实现的op的接口，所有设备相关实现的基类。位于saber/funcs/impl/impl_base.h。实现版本中这里分为两类，一类以```vender_```为前缀，带有```vender_```代码意为使用第三方库来实现该op，如cudnn的conv，或mkl的conv等等，这类op的性能我们难以调优，因此单独列为一类。另一类是带有源码的saber实现，这些实现都带有```saber_```为前缀，此类实现带有源码，能够通过后续优化不断提升性能，实现起名时需要注意这一点。
+
+## 添加operator
+
+添加一个新的op需要以下几步：
+
+1. 添加saber的param
+2. 定义saber的Operator类
+3. 定义新的impl声明
+3. 完成新的impl实现
+4. 增加framework的实现或特化
+
+接下来就针对这几步，以一个简单例子为例介绍实现。
+
+例如我们要添加新的Mul op。给出计算公式如下：$$Out = alpha \dot X * Y$$
+
+### 为operator增加param
+
+涉及到的文件：```saber/saber_funcs_param.h```。如果之前已经存在需要添加的op的param，这一步可以跳过。
+这里```XXXParam```是一个```struct```。包含一个无参数的构造函数，含参数的构造函数，复制构造函数，```operator=()```及```operator==()```。
+```
+template // 能够获得target, datatype, layout
+struct MulParam{
+ MulParam()
+ : alpha(0)
+ {}
+ MulParam(float alpha_in)
+ : alpha(alpha_in)
+ {}
+ MulParam(const MulParam& right)
+ : alpha(right.alpha)
+ {}
+ MulParam &operator=(const MulParam &right) {
+ alpha = right.alpha;
+ }
+ bool operator==(const MulParam &right) {
+ return alpha == right.alpha;
+ }
+ float alpha;
+};
+```
+
+### 定义Operator类
+涉及到的文件:```saber/funcs/mul.h```。如果之前定义过该op的类，这里需要修改输入的impl定义头文件。
+下面给出一个相对完整的定义结构供参考。
+```
+//不同的设备需要包含对应的operator实现.[详见](#impl)
+#ifdef NVIDIA_GPU
+#include "saber/funcs/impl/cuda/saber_mul.h"
+#include "saber/funcs/impl/cuda/vender_mul.h"
+#endif
+//如果一个设备现在还没有对应的operator实现，需要包含声明。[详见](#declare)
+#ifdef USE_X86_PLACE
+#include "saber/funcs/impl/impl_mul.h"
+#endif
+namespace anakin {
+namespace saber {
+template
+class Mul : public BaseFunc<
+ Tensor,
+ Tensor,
+ Tensor,
+ ImplBase, MulParam> {
+public:
+ using BaseFunc<
+ Tensor,
+ Tensor,
+ Tensor,
+ ImplBase, MulParam>::BaseFunc;
+ Mul() = default;
+ typedef Tensor InDataTensor;
+ typedef Tensor OutDataTensor;
+ typedef Tensor OpTensor;
+ typedef MulParam Param_t;
+ typedef std::vector Input_v;
+ typedef std::vector Output_v;
+ typedef std::vector Shape_v;
+
+ virtual SaberStatus compute_output_shape(const Input_v &input,
+ Output_v &output, Param_t ¶m) override {
+ //计算输出的shape，
+ Shape output_shape = (input[0]->valid_shape());
+ /* code */
+ return output[0]->set_shape(output_shape);
+ }
+ virtual SaberStatus init_impl(ImplEnum implenum) override {
+ // 不同设备均使用此init_impl, 此接口创建对应impl的实现。
+ switch (implenum) {
+ case VENDER_IMPL:
+ this->_impl.push_back(new VenderMul );
+ return SaberSuccess;
+ case SABER_IMPL:
+ this->_impl.push_back(new SaberMul );
+ return SaberSuccess;
+ default:
+ return SaberUnImplError;
+ }
+ }
+private:
+ virtual void pick_best_static() override {
+ if (true) // some condition?
+ this->_best_impl = this->_impl[0];
+ }
+ virtual void pick_best_specify(ImplEnum implenum) override {
+ this->_best_impl = this->_impl[0];
+ }
+};
+} // namespace saber
+} // namespace anakin
+```
+
+### 为operator增加新的impl声明
+
+涉及的文件:```saber/funcs/impl/impl_mul.h```。不同的设备都特化同一个声明，特化版本放在对应的文件夹下，这里的声明就是给出所有设备的统一声明。下面给出一个参考。
+```
+#include "saber/funcs/impl/impl_macro.h"
+namespace anakin{
+namespace saber{
+DEFINE_OP_CLASS(Mul, MulParam); // 第一个参数是op的名字，第二个是对应param的名字
+}
+}
+```
+
+### 完成新的operator特定后端实现
+
+涉及的文件:```saber/funcs/impl/xxx/vender_mul.h```或```saber/funcs/impl/xxx/saber_mul.h```
+这里```xxx```指代特定的一种设备。```vender```是指的使用第三方库实现的op，```saber```指的源码实现的op。这里以cuda的vender实现为例，简单介绍一下特化出的函数的几个基本接口。
+
+```
+// include 对应的声明
+#include "saber/funcs/impl/impl_mul.h"
+
+namespace anakin{
+namespace saber{
+template
+class VenderMul :
+ public ImplBase<
+ Tensor,
+ Tensor,
+ Tensor,
+ MulParam > >
+{
+public:
+ typedef Tensor DataTensor_in;
+ typedef Tensor DataTensor_out;
+ typedef Tensor OpTensor;
+ typedef typename DataTensor_in::Dtype InDataType;
+ typedef typename DataTensor_out::Dtype OutDataType;
+ typedef typename OpTensor::Dtype OpDataType;
+ VenderMul(){}
+ ~VenderMul() {}
+
+ virtual SaberStatus init(const std::vector& inputs,
+ std::vector& outputs,
+ MulParam& param, Context& ctx) {
+ this->_ctx = ctx;
+ create(inputs, outputs, param, ctx);
+ }
+
+ virtual SaberStatus create(const std::vector& inputs,
+ std::vector& outputs,
+ MulParam& param, Context& ctx) {
+ // set内部参数
+ }
+
+ virtual SaberStatus dispatch(const std::vector& inputs,
+ std::vector& outputs,
+ MulParam& param) {
+ // dispatch kernel.
+ }
+
+private:
+};
+}
+}
+```
+```init```和```create```的区别：```init```接口是第一次初始化op的时候进入的接口，此函数只在第一次初始化op时调用，这个接口一般放一些只需要执行一次的代码，如malloc或者create之类的函数。```create```函数除了第一次init执行外，在输入发生变化或者param发生变化时会再次触发，create一般放置set函数，设置内部变量，当input发生变化时这里执行一些同input或weights直接相关的代码。但create因为触发位置在网络内，如果```create```函数执行了一些严重耗时的操作，这里会拖慢整个op的执行时间，需要慎重选择操作放置的位置。
+### 添加framework的特化
+
+涉及的文件:```framework/operators/mul.h```和```framework/operators/mul.cpp```。
+这里简单介绍下如果添加或修改framework内的operator
+
+```
+#include "framework/core/base.h"
+#include "framework/core/data_types.h"
+#include "framework/core/operator/operator.h"
+#include "utils/logger/logger.h"
+#include "saber/funcs/mul.h" // 需要包对应的saber头文件
+namespace anakin {
+namespace ops {
+template
+class MulHelper;
+
+template
+class Mul : public Operator {
+public:
+ Mul() {}
+ /// forward impl
+ virtual void operator() (OpContext &ctx,
+ const std::vector >& ins,
+ std::vector >& outs) {
+ LOG(ERROR) << "Not Impl Yet Operator power::type>().type_info()<<">";
+ }
+ friend class MulHelper;
+};
+template
+class MulHelper : public OperatorHelper {
+public:
+ MulHelper() = default;
+ ~MulHelper();
+ Status InitParam() override;
+
+ Status Init(OpContext &ctx,
+ const std::vector >& ins,
+ std::vector >& outs) override;
+ Status InferShape(const std::vector >& ins,
+ std::vector >& outs) override;
+
+public:
+ saber::MulParam> _param_mul;
+ saber::Mul _funcs_mul;
+};
+}
+} /* namespace anakin */
+```
+对应的```.cpp```文件如下：
+```
+#include "framework/operators/mul.h"
+
+namespace anakin {
+namespace ops {
+
+#ifdef USE_CUDA
+template<>
+void Mul::operator()(
+ OpContext& ctx,
+ const std::vector >& ins,
+ std::vector >& outs) {
+ auto* impl =
+ static_cast*>(this->_helper);
+ auto& param =
+ static_cast*>(this->_helper)->_param_mul;
+ impl->_funcs_mul(ins, outs, param, ctx);
+}
+#endif
+
+template
+Status MulHelper::InitParam() {
+ auto alpha = GET_PARAMETER(float, alpha);
+ MulParam> param_mul(alpha);
+ _param_mul = param_mul;
+ return Status::OK();
+}
+
+template
+Status MulHelper::Init(OpContext& ctx,
+ const std::vector >& ins,
+ std::vector >& outs) {
+
+ SABER_CHECK(_funcs_mul.init(ins, outs, _param_mul, SPECIFY, VENDER_IMPL, ctx));
+ return Status::OK();
+}
+
+template
+Status MulHelper::InferShape(const
+ std::vector >& ins,
+ std::vector >& outs) {
+ SABER_CHECK(_funcs_mul.compute_output_shape(ins, outs, _param_mul));
+ return Status::OK();
+}
+
+#ifdef USE_CUDA
+template class MulHelper;
+#endif
+#ifdef USE_ARM_PLACE
+template class MulHelper;
+#endif
+// register helper
+#ifdef USE_CUDA
+ANAKIN_REGISTER_OP_HELPER(Mul, MulHelper, NV, AK_FLOAT, Precision::FP32);
+#endif
+#ifdef USE_ARM_PLACE
+ANAKIN_REGISTER_OP_HELPER(Mul, MulHelper, ARM, AK_FLOAT, Precision::FP32);
+#endif
+//! register op
+ANAKIN_REGISTER_OP(Mul)
+.Doc("Mul operator")
+#ifdef USE_CUDA
+.__alias__("mul")
+#endif
+#ifdef USE_ARM_PLACE
+.__alias__("mul")
+#endif
+.num_in(1)
+.num_out(1)
+.Args("alpha", " alpha of Mul "); //注册
+
+} /* namespace ops */
+
+} /* namespace anakin */
+```
+
+## 实现单元测试
+涉及的文件:```test/saber/xxx/test_saber_funcs_mul_xxx.cpp```
+在对应的test下需要添加新的单元测试
+
+```
+TEST(TestSaberFuncNV, test_depthwise_conv) {
+
+ // init tensors and some param.
+
+ // start Reshape & doInfer
+ Context ctx1(0, 1, 1);
+
+ // create param
+ MulParam > param(alpha);
+
+ std::vector*> input;
+ std::vector*> output;
+
+ // create saber op
+ Mul mul;
+
+ // compute output shape
+ mul.compute_output_shape(input, output, param);
+
+ // re_alloc output tensors memory based on output shape
+ output[0]->re_alloc(output[0]->shape());
+
+ // init saber op(calling init and create)
+ mul.init(input, output, param, SPECIFY, VENDER_IMPL, ctx1);
+
+ // call operator()
+ mul(input, output, param, ctx1);
+
+ // cuda specified, record events
+ cudaStream_t cuda_stream = ctx1.get_compute_stream();
+ output[0]->record_event(cuda_stream);
+ output_dev.sync();
+
+ // param changed
+ param.alpha = 2.0;
+ // auto calling saber op(create and dispatch)
+ mul(input, output, param, ctx1);
+
+ cudaDeviceSynchronize();
+ CUDA_CHECK(cudaPeekAtLastError());
+}
+
+int main(int argc, const char** argv){
+ anakin::saber::Env::env_init();
+
+ // initial logger
+ //logger::init(argv[0]);
+ InitTest();
+ RUN_ALL_TESTS(argv[0]);
+ return 0;
+}
+
+```
+## 调试及注意事项
+
+一个op需要有对外的op接口和内部实现，由于存在saber/funcs/impl的非特化版本声明，当有op在某种设备下没有对应实现时，也能够编译，但此时是没有任何实现的空实现，
diff --git a/doc/fluid/new_docs/advanced_usage/deploy/how_to_support_new_device_in_anakin.md b/doc/fluid/new_docs/advanced_usage/deploy/how_to_support_new_device_in_anakin.md
new file mode 100644
index 0000000000..a1f75f5e95
--- /dev/null
+++ b/doc/fluid/new_docs/advanced_usage/deploy/how_to_support_new_device_in_anakin.md
@@ -0,0 +1,459 @@
+# 如何支持一个新的设备
+
+## 概览
+
+添加一个新的设备需要以下3个步骤：
+
+* [在`CMakeList`中添加设备的支持](#0001)
+* [在`saber`中添加设备的实现](#0002)
+* [在`framework`中添加设备的具体化或实例化](#0003)
+
+假设新设备的名称为`TNEW`, 以下将以这个设备名称进行演示。
+
+## 在`CMakeList`中添加设备的支持 ##
+
+* 修改根目录`CMakeList.txt`
+```cmake
+#select the plantform to build
+anakin_option(USE_GPU_PLACE "Select the build mode for GPU place." NO)
+anakin_option(USE_X86_PLACE "Select the build mode for X86 place." NO)
+anakin_option(USE_ARM_PLACE "Select the build mode for ARM place." NO)
+anakin_option(USE_TNEW_PLACE "Select the build mode for ARM place." YES)
+```
+
+* 修改`saber/CMakeList.txt`
+
+根据新增设备的目录完善`saber`目录下的`CMakeList.txt`。
+```cmake
+if(USE_TNEW_PLACE)
+ anakin_fetch_files_with_suffix(${ANAKIN_SABER}/core/impl/tnew "cpp" ANAKIN_SABER_BASE_SRC)
+ anakin_fetch_files_with_suffix(${ANAKIN_SABER}/funcs/impl/tnew "cpp" ANAKIN_SABER_BASE_SRC)
+endif()
+```
+
+* 修改`test/CMakeList.txt`
+
+新增设备的单测文件放在`test/saber/tnew`目录下，修改`test`目录下的`CMakeList.txt`。
+```cmake
+if(USE_TNEW_PLACE)
+ anakin_fetch_files_with_suffix(${ANAKIN_UNIT_TEST}/saber/tnew "cpp" ANAKIN_TEST_CASE_SRC)
+endif()
+```
+
+* 修改`cmake/anakin_config.h.in`
+```c++
+// plantform to use
+#cmakedefine USE_GPU_PLACE
+
+#cmakedefine USE_X86_PLACE
+
+#cmakedefine USE_ARM_PLACE
+
+#cmakedefine USE_TNEW_PLACE
+```
+
+* 其他依赖和编译选项
+修改`cmake`目录下的`compiler_options.cmake`和`find_modules.cmake`
+
+
+## 在`saber`中添加设备的实现 ##
+`saber`是`Anakin`的基础计算库，对外提供设备无关的统一的API，设备相关的实现都会封装到`TargetWrapper`中。
+
+### 在`saber/saber_types.h`中添加设备
+
+```c++
+enum TargetTypeEnum {
+ eINVALID = -1,
+ eNV = 1,
+ eAMD = 2,
+ eARM = 3,
+ eX86 = 4,
+ eNVHX86 = 5,
+ eTNEW = 6
+};
+
+typedef TargetType NV;
+typedef TargetType ARM;
+typedef TargetType AMD;
+typedef TargetType X86;
+typedef TargetType TNEW;
+
+```
+
+### 在`saber/core`中添加设备的实现
+
+1. 在`target_traits.h`中添加新设备
+
+* 增加设备类型
+```c++
+struct __cuda_device{};
+struct __arm_device{};
+struct __amd_device{};
+struct __x86_device{};
+struct __tnew_device{};
+```
+
+* `TargetTypeTraits`模板具体化
+```c++
+template <>
+struct TargetTypeTraits {
+ typedef __xxx_target target_category;//根据实际设备是host端还是device端进行选择
+ typedef __tnew_device target_type;
+};
+```
+
+2. 在`data_traits.h`中特化`DataTrait`模板类
+
+如果设备需要特殊的数据类型，则特化出设备的`DataTrait`类的实现，例如opencl数据类型的实现如下：
+```c++
+#ifdef USE_OPENCL
+struct ClMem{
+ ClMem(){
+ dmem = nullptr;
+ offset = 0;
+ }
+
+ ClMem(cl_mem* mem_in, int offset_in = 0) {
+ dmem = mem_in;
+ offset = offset_in;
+ }
+
+ ClMem(ClMem& right) {
+ dmem = right.dmem;
+ offset = right.offset;
+ }
+
+ ClMem& operator=(ClMem& right) {
+ this->dmem = right.dmem;
+ this->offset = right.offset;
+ return *this;
+ }
+
+ ClMem& operator+(int offset_in) {
+ this->offset += offset_in;
+ return *this;
+ }
+
+ int offset{0};
+ cl_mem* dmem;
+};
+
+template <>
+struct DataTrait {
+ typedef ClMem Dtype;
+ typedef float dtype;
+};
+
+template <>
+struct DataTrait {
+ typedef ClMem Dtype;
+ typedef double dtype;
+};
+
+template <>
+struct DataTrait {
+ typedef ClMem Dtype;
+ typedef char dtype;
+};
+#endif //use_opencl
+```
+
+3. 在`target_wrapper.h`中特化`TargetWrapper`模板类
+
+特化`TargetWrapper`模板类，在`target_wrapper.h`中声明函数，具体如下：
+```c++
+template <>
+struct TargetWrapper { //根据TNEW的具体类型修改__xxx_target，__host_target或者__device_target
+
+ typedef xxx_event event_t; //根据设备实现xxx_event
+ typedef xxx_stream stream_t; //根据设备实现xxx_stream
+
+ static void get_device_count(int& count);
+
+ static void set_device(int id);
+
+ //We should add strategy to avoid malloc directly
+ static void mem_alloc(void** ptr, size_t n);
+
+ static void mem_free(void* ptr);
+
+ static void mem_set(void* ptr, int value, size_t n);
+
+ static void create_event(event_t& event, bool flag = false);
+
+ static void create_stream(stream_t& stream);
+
+ static void create_stream_with_flag(stream_t& stream, unsigned int flag);
+
+ static void create_stream_with_priority(stream_t& stream, unsigned int flag, int priority);
+
+ static void destroy_stream(stream_t& stream);
+
+ static void destroy_event(event_t& event);
+
+ static void record_event(event_t& event, stream_t stream);
+
+ static void query_event(event_t& event);
+
+ static void sync_event(event_t& event);
+
+ static void sync_stream(event_t& event, stream_t& stream);
+
+ static void sync_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, __DtoD);
+
+ static void async_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, stream_t& stream, __DtoD);
+
+ static void sync_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, __HtoD);
+
+ static void async_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, stream_t& stream, __HtoD);
+
+ static void sync_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, __DtoH);
+
+ static void async_memcpy(void* dst, int dst_id, const void* src, int src_id, \
+ size_t count, stream_t& stream, __DtoH);
+
+ static void sync_memcpy_p2p(void* dst, int dst_dev, const void* src, \
+ int src_dev, size_t count);
+
+ static void async_memcpy_p2p(void* dst, int dst_dev, const void* src, \
+ int src_dev, size_t count, stream_t& stream);
+
+ static int get_device_id();
+};
+
+```
+
+4. 在`impl/`目录下添加设备目录和实现
+
+在`saber/core/impl`目录下添加设备目录`tnew`。
+* 实现`TargetWrapper`结构体中各函数的定义。
+如果`TargetWrapper`的实现与默认的模板类一致，则不用特化出该类。
+
+```c++
+typedef TargetWrapper TNEW_API;
+void TNEW_API::get_device_count(int &count) {
+ // add implementation
+}
+
+void TNEW_API::set_device(int id){
+ // add implementation
+}
+
+void TNEW_API::mem_alloc(void** ptr, size_t n){
+ // add implementation
+}
+
+void TNEW_API::mem_free(void* ptr){
+ if(ptr != nullptr){
+ // add implementation
+ }
+}
+...
+
+```
+
+* 特化实现`device.h`中的`Device`
+
+```c++
+template <>
+void Device::create_stream() {
+ // add implementation
+}
+
+template <>
+void Device::get_info() {
+
+ // add implementation
+}
+
+```
+
+### 在`saber/funcs`中实现设备相关的op
+
+参考[如何增加新的Operator](addCustomOp.md)
+
+
+## 在`framework`中添加设备的具体化或实例化 ##
+
+### `framework/core`
+
+* `net.cpp`中添加实例化
+
+```c++
+#ifdef USE_TNEW_PLACE
+template class Net;
+template class Net;
+#endif
+```
+
+* `operator_func.cpp`中添加实例化
+
+```c++
+#ifdef USE_TNEW_PLACE
+template class OperatorFunc;
+#endif
+```
+
+* `worker.cpp`中添加实例化
+
+```c++
+#ifdef USE_TNEW_PLACE
+template class Worker;
+template class Worker;
+#endif
+```
+
+* `operator_attr.cpp`中添加实例化
+
+```c++
+template
+OpAttrWarpper& OpAttrWarpper::__alias__(const std::string& op_name);
+template
+OpAttrWarpper& OpAttrWarpper::__alias__(const std::string& op_name);
+template
+OpAttrWarpper& OpAttrWarpper::__alias__(const std::string& op_name);
+```
+
+* `parameter.h`中添加设备的实现
+
+```c++
+#ifdef USE_TNEW_PLACE
+template
+class PBlock {
+public:
+ typedef Tensor4d::type> type;
+
+ PBlock() {
+ _inner_tensor = std::make_shared();
+ }
+ ...
+}
+#endif //TNEW
+```
+
+* `type_traits_extend.h`中添加设备的实现
+
+```c++
+template<>
+struct target_host {
+ typedef saber::X86 type; //根据TNEW选择正确的host type
+};
+```
+
+### `framework/graph`
+
+* `graph.cpp`中添加实例化
+
+```c++
+ #ifdef USE_TNEW_PLACE
+ template class Graph;
+ template class Graph;
+ template class Graph;
+ #endif
+```
+
+### `framework/model_parser`
+
+* `parser.cpp`中添加实例化
+
+```c++
+ #ifdef USE_TNEW_PLACE
+ template
+ Status load(graph::Graph* graph,
+ const char* model_path);
+ template
+ Status load(graph::Graph* graph,
+ const char* model_path);
+ template
+ Status load(graph::Graph* graph,
+ const char* model_path);
+
+ template
+ Status save(graph::Graph* graph,
+ std::string& model_path);
+ template
+ Status save(graph::Graph* graph,
+ std::string& model_path);
+ template
+ Status save(graph::Graph* graph,
+ std::string& model_path);
+
+ template
+ Status load(graph::Graph* graph,
+ std::string& model_path);
+ template
+ Status load(graph::Graph* graph,
+ std::string& model_path);
+ template
+ Status load(graph::Graph* graph,
+ std::string& model_path);
+
+ template
+ Status save(graph::Graph* graph,
+ const char* model_path);
+ template
+ Status save(graph::Graph* graph,
+ const char* model_path);
+ template
+ Status save(graph::Graph* graph,
+ const char* model_path);
+ #endif
+```
+
+* `model_io.cpp`中添加实例化
+
+```c++
+#ifdef USE_TNEW_PLACE
+template class NodeIO;
+template class NodeIO
-As the figure above, we describe a global view of asynchronously update process and use
+As the figure above, we describe a global view of the asynchronous update process and use
the parameter `w1` as an example to introduce the steps:
1. For each gradient variables, they may distribute on different GPU card and aggregate
them while they are all calculated.
-1. Split the gradient variable into multiple blocks according to the number of PServer
+1. Split the gradient variable into multiple blocks according to the number of PS
instances and then send them.
-1. PServer would run an `Optimize Block` using a specified optimize algorithm to update
+1. PS would run an `Optimize Block` using a specified optimize algorithm to update
the specified parameter.
-1. The trainer will fetch latest parameter from PServer before running forward Op which depends
+1. The trainer will fetch the latest parameter from PS before running forward Op which depends
on the specified parameter.
1. Broadcast the received variable into multiple GPU cards and continue to run the next
mini-batch.
@@ -40,8 +43,8 @@ mini-batch.
- For the multiple devices distributed training, we need to aggregate the gradient
variables which placed on different devices firstly and then schedule a `SendVars` Operator to
-send the gradient variables to the multiple PServer instances.
-- Schedule `FetchVars` operator to fetch the latest parameter from PServer before running
+send the gradient variables to the multiple PS instances.
+- Schedule `FetchVars` operator to fetch the latest parameter from PS before running
the forward ops.
- There could be a large number of gradient variables to be sent, so we need to use another
thread pool(IO Threadpool) whose a number of the schedulable threads is larger than the
diff --git a/doc/fluid/design/dist_train/dist_train_nccl2.md b/doc/fluid/design/dist_train/dist_train_nccl2.md
new file mode 100644
index 0000000000..aa7455ec5d
--- /dev/null
+++ b/doc/fluid/design/dist_train/dist_train_nccl2.md
@@ -0,0 +1,35 @@
+# Distributed Training with NCCL2
+
+We design a pattern that can enable training with `ParallelExecutor` and
+using [NCCL2](https://developer.nvidia.com/nccl) as it's collective
+communication library.
+
+In `ParallelExecutor` we can use `AllReduce` or `Reduce` and `Broadcast`
+to do multi GPU training. And if we initialize NCCL2 communicators as
+ranks in a distributed environment, we can simply run the `ParallelExecutor`
+as a distributed program! The only thing that may be different than in
+the single node version is that we need to broadcast the NCCL unique ID
+to all the nodes, and initialize communicators using that ID, so NCCL2
+will know each other as ranks.
+
+To achieve this feature, we introduce a new operator: `gen_nccl_id` op,
+so we are ***not*** "bind to" running NCCL2 with MPI, we can run it in
+what ever platform you like.
+
+It have two running modes:
+
+1. Generate and broadcast mode, which should be used on trainer 0;
+1. Listen and fetch mode, which should be used on trainers other than 0.
+
+In both two modes, this op can save the NCCL ID into current scope as a
+persistable variable, Then we can insert this op at the end of
+"startup program" of fluid, so that all workers can get the same ID to
+initialize NCCL communicator objects.
+
+
+
+The above figure indicates the general process when training with NCCL2
+distributed. Each trainer have the number of communicators equal to the
+number of GPUs, but the ranks should match the global ranks number: here
+we have total 8 GPUs, so `nranks==8`, for each trainer, the ranks should
+be from 0 ~ 3 on trainer 0 and 4 ~ 7 on trainer 1.
diff --git a/doc/fluid/design/dist_train/distributed_lookup_table_design.md b/doc/fluid/design/dist_train/distributed_lookup_table_design.md
index 9887291389..e284e1ec5c 100644
--- a/doc/fluid/design/dist_train/distributed_lookup_table_design.md
+++ b/doc/fluid/design/dist_train/distributed_lookup_table_design.md
@@ -1,6 +1,6 @@
# Design Doc: Distributed Lookup Table Operator
-A lookup table operator in PaddlePaddle where the table could be out
+A distribute lookup table operator in PaddlePaddle where the table could be out
of the memory of a computer.
## Background
@@ -24,14 +24,14 @@ memory, so we'd need a distributed storage service, which supports the
lookup of rows.
The following figure illustrates the multiplication of x with two
-non-zero elements, or say, two symbols, and a lookup table W:
+non-zero elements, or say two symbols, and a lookup table W:
### The Backward Algorithm
The backward algorithm computes W'(x) using W(x). W'(x) has the same
-scale of size as W(x) and is much smaller than W.
+the scale of size as W(x) and is much smaller than W.
To optimize W given W', we can do simple SGD update:
@@ -44,85 +44,46 @@ $$W = f(W, W')$$
The following figure illustrates the backward pass of the lookup
operator: 
-## Distributed Storage Service
-
-The forward algorithm requires a distributed storage service for W.
-The backward algorithm prefers that the storage system can apply the
-optimization algorithm on W. The following two sections describe two
-solutions -- the former doesn't require that the storage service can
-do optimization, the latter does.
-
-### Storage Service Doesn't Optimize
-
-In this design, we use highly-optimized distributed storage, e.g.,
-memcached, as the storage service, and we run the optimization
-algorithm on parameter servers of PaddlePaddle. The following figure
-illustrates the training process.
-
-
-
-
diff --git a/doc/fluid/design/multi_devices/kernel_selection.md b/doc/fluid/design/multi_devices/kernel_selection.md
index 967317d5d2..4d2aab87b8 100644
--- a/doc/fluid/design/multi_devices/kernel_selection.md
+++ b/doc/fluid/design/multi_devices/kernel_selection.md
@@ -74,10 +74,10 @@ void OperatorWithKernel::Run(
auto kernel_type_for_var = this->GetKernelTypeForVar(...);
if (kernel_type_for_var.place_ != expected_kernel_key.place_) {
auto* trans_var = new_scope.Var(var_name);
- auto* out = DataTransform(expected_kernel_key,
+ auto* out = TransformData(expected_kernel_key,
kernel_type_for_var,
*tensor_in);
- CopyVariableWithTensor(...);
+ SetTensorToVariable(...);
}
}
diff --git a/doc/fluid/design/multi_devices/operator_kernel_type.md b/doc/fluid/design/multi_devices/operator_kernel_type.md
index 8c1bc8f76a..5e391bd62b 100644
--- a/doc/fluid/design/multi_devices/operator_kernel_type.md
+++ b/doc/fluid/design/multi_devices/operator_kernel_type.md
@@ -75,7 +75,7 @@ Different layout leads to different implementation of the operator kernel. There
- The inference of Layout is at run-time, not at compile-time.
-- Every operator has to implement different kernels for different layouts. Let's take MKLDNN as an example. If we want to implement an MKLDNN convolution operator, we have to implement all the kernels for different layouts, which are listed [here](http://01org.github.io/mkl-dnn/structmkldnn_1_1memory.html). And we will have a special macro to register kernels for MKLDNN operators.
+- Every operator has to implement different kernels for different layouts. Let's take MKLDNN as an example. If we want to implement an MKLDNN convolution operator, we have to implement all the kernels for different layouts, which are listed [here](http://intel.github.io/mkl-dnn/structmkldnn_1_1memory.html). And we will have a special macro to register kernels for MKLDNN operators.
`Layout` is also defined as a enum variable:
diff --git a/doc/fluid/design/quantization/fixed_point_quantization.md b/doc/fluid/design/quantization/fixed_point_quantization.md
new file mode 100644
index 0000000000..085352fc56
--- /dev/null
+++ b/doc/fluid/design/quantization/fixed_point_quantization.md
@@ -0,0 +1,110 @@
+Fixed-point quantization uses lower bits, for example, 2-bit, 3-bit or 8-bit fixed point to represent weights and activations, which usually are in singe-precision float-point with 32 bits. The fixed-point representation has advantages in reducing memory bandwidth, lowering power consumption and computational resources as well as the model storage requirements. It is especially important for the inference in embedded-device deployment.
+
+According to some experiments, the apporach to quantize the model trained in float point directly works effectively on the large models, like the VGG model having many parameters. But the accuracy drops a lot for the small model. In order to improve the tradeoff between accuracy and latency, many quantized training apporaches are proposed.
+
+This document is to design a quantized training framework on Fluid. The first part will introduce how to quantize, The second part will describe the quantized training framework. The last part will illustrate how to calculate the quantization scale.
+
+
+### How to quantize
+
+There are many ways to quantize the float value to fixed-point value. For example:
+
+$$ r = min(max(x, a), b)$$
+$$ s = \frac{b - a}{n - 1} $$
+$$ q = \left \lfloor \frac{r - a}{s} \right \rceil $$
+
+where, $x$ is the float value to be quantized, $[a, b]$ is the quantization range, $a$ is the minimum value and $b$ is the maximal value. $\left \lfloor \right \rceil$ denotes rounding to the nearest integer. If the quantization level is $k$, $n$ is $2^k$, for example, $k$ is 8 and $n$ is 256. $q$ is the quantized integer.
+
+
+The quantization we applied is parameterized by the number of quantization levels and maximum absolute value:
+
+$$ M = max(abs(x)) $$
+$$ q = \left \lfloor \frac{x}{M} * (n - 1) \right \rceil $$
+
+where, $x$ is the float value to be quantized, $M$ is maximum absolute value. $\left \lfloor \right \rceil$ denotes rounding to the nearest integer. For 8 bit quantization, $n=2^{8}=256$. $q$ is the quantized integer.
+
+
+Wether the *min-max* quantization or *max-abs* quantization, they also can be represent:
+
+$q = scale * r + b$
+
+We call *min-max*, *max-abs* as the quantization arguments, also call them quantization scale or quantization range.
+
+
+How to calculate the quantization scale (or maximum absolute value) for inference will be described in the last part.
+
+
+### Training Framework
+
+#### Forward pass
+
+The forward pass is simulated quantization, see Figure 1.
+
+The training framework is as following figure.
+
+
+
+
+
+
+
+