m53297601
/
Paddle
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '24ced1d0b9'
${ noResults }
Paddle/paddle/contrib/inference
History
Luo Tao 24ced1d0b9
add independent demo for test static fluid library 		7 years ago
..
demo add independent inference demo on teamcity 7 years ago
demo_ci add independent demo for test static fluid library 7 years ago
CMakeLists.txt Merge pull request #12052 from tensor-tang/refine/infer/api/static 7 years ago
README.md mv contrib to paddle/ for unified compile (#10815) 7 years ago
check_symbol.sh add some symbol check 7 years ago
high_level_api.md inference doc fix grammer (#11718) 7 years ago
high_level_api_cn.md inference API init cn (#11731) 7 years ago
paddle_inference_api.cc update 7 years ago
paddle_inference_api.h feature/analysis to support sub-graph for TRT engine (#11538) 7 years ago
paddle_inference_api.map inference api symbol hidden 7 years ago
paddle_inference_api.sym inference api static lib symbol hidden 7 years ago
paddle_inference_api_anakin_engine.cc inference/unify output buffer management (#11569) 7 years ago
paddle_inference_api_anakin_engine.h feature/anakin ci (#11330) 7 years ago
paddle_inference_api_anakin_engine_tester.cc inference/unify output buffer management (#11569) 7 years ago
paddle_inference_api_impl.cc feature/analysis to support sub-graph for TRT engine (#11538) 7 years ago
paddle_inference_api_impl.h Move fluid::framework::InitDevices into fluid::platform (#11757) 7 years ago
paddle_inference_api_tensorrt_subgraph_engine.cc feature/analysis to support sub-graph for TRT engine (#11538) 7 years ago
test_paddle_inference_api.cc fix develop build issue (#10978) 7 years ago
test_paddle_inference_api_impl.cc fix compile warning in inference related codes 7 years ago
test_paddle_inference_api_tensorrt_subgraph_engine.cc feature/analysis to support sub-graph for TRT engine (#11538) 7 years ago

README.md

Embed Paddle Inference in Your Application

Paddle inference offers the APIs in C and C++ languages.

One can easily deploy a model trained by Paddle following the steps as below:

  1. Optimize the native model;
  2. Write some codes for deployment.

Let's explain the steps in detail.

Optimize the native Fluid Model

The native model that get from the training phase needs to be optimized for that.

  • Clean the noise such as the cost operators that do not need inference;
  • Prune unnecessary computation fork that has nothing to do with the output;
  • Remove extraneous variables;
  • Memory reuse for native Fluid executor;
  • Translate the model storage format to some third-party engine's, so that the inference API can utilize the engine for acceleration;

We have an official tool to do the optimization, call paddle_inference_optimize --help for more information.

Write some codes

Read paddle_inference_api.h for more information.