Paddle/paddle/fluid/inference/analysis

Inference Analysis

The inference/analysis module is used to analyze and optimize the inference program, it references some philosophy from LLVM/analysis, and make the various optimization features be pluggable and co-exist in a pipeline.

We borrowed some concepts from LLVM, such as

• Passes to implement optimization that traverse the inference program,
• DataFlowGraph to represent the data flow graph built from a program,
• PassManager to manage a sequence of Passes over a graph.

There are some other basic concepts here

• Node, the node in a DataFlowGraph,
  • Function, the Operator in Fluid,
  • Value, the Variable in Fluid;
• Argument, the argument that treat as the input and output of all Passes in the pipeline,

How it works

The inference/analysis module make all the passes in a pipeline, and works in such way:

1. Build a DataFlowGraph from a Fluid inference ProgramDesc,
2. Call the middle passes one by one, the same DataFlowGraph is passed across all the passes,
3. Transform a new ProgramDesc from the modified DataFlowGraph.

The new optimization features can be added as an independent Pass and controlled by gflags, each pass will generate unified debug information or visualization for better debugging.

Supported Passes

FluidToDataFlowGraphPass

Transform the fluid ProgramDesc to a DataFlowGraph to give an abstract representation for all the middle passes, this should be the first pass of the pipeline.

DataFlowGraphToFluidPass

Generate a final ProgramDesc from a data flow graph, this should be the last pass of the pipeline.

TensorRTSubgraphNodeMarkPass

Mark the Node that are supported by TensorRT, this pass will generate a visualization file which can be used for debugging.

TensorRTSubGraphPass

Split the sub-graph that are can be accelerated by TensorRT.

DFG_GraphvizDrawPass

This pass is just for debug, it will visualize the DataFlowGraph using the graphviz tool.

It can be used as a helper class that draws the modified graph after each pass.

Utilities

There is some helper function/class for analysis.

• dot.h give a easy to use interface for generating DOT codes,
• graph_traits.h contains the graph traversal algorithms, it uses iterator to make the algorithms easy to share across different passes.