242 Commits (2c0a4a347015d2201fde5acdd2c0bb411a43f8f0)

Author	SHA1	Message	Date
Pei Yang	2480bdef6c	change hard_swish from plugin to layer (#29177) ... * change hard_swish from plugin to layer * add ut when threshold != scale	5 years ago
Pei Yang	f860de4af7	support clip op trt converter (#29411)	5 years ago
Shang Zhizhou	b9e76a0103	detect tensorRT plugin fp16 in runtime (#27933) ... * remove -DSUPPORTS_CUDA_FP16 in cuda.cmake * comile with cuda9 * add some unittest * notest;test=coverage * add unittest for trt plugin swish && split * update ernie unittest * fix some error message * remove repeated judgement of CUDA version in mbEltwiseLayerNormOpConverter * fix comile errror when CUDA_ARCH_NAME < Pascal" * fix comile error * update unittest timeout * compile with cuda9 * update error msg * fix code style * add some comments * add define IF_CUDA_ARCH_SUPPORT_FP16 * rename IF_CUDA_ARCH_SUPPORT_FP16 to CUDA_ARCH_FP16_SUPPORTED	5 years ago
Pei Yang	994673bf4f	change avg pooling and global pooling to trt layer in dynamic shape mode (#28702) ... * change avg pooling and global pooling to trt layer * add support for static shape global pooling * modify trt errmsg	5 years ago
Shang Zhizhou	8699f38d08	裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (#28517) ... * skip_layernorm_op done * add unittest * slice op convertor support trt < 6 * skip_layernorm only work in ernie	5 years ago
Shang Zhizhou	ea851796e5	TensorRT中ernie模型推理性能优化，支持变长输入 (#28367) ... * fp16 result ok * change -DWITH_NVINFER_PLUGIN toconfig.EnableTensorRtOSS * auto detect special slice op converter for ernie with trt oss * ernie oss only support fp16 * fix special_slice_plugin serialize bug * matmul in tensorrt ok * ernie unittest ok * add matmul tensorrt unittest * remove demo code	5 years ago
Pei Yang	602d2ce5c9	change avg pooling from trt plugin to trt layer (#28032)	5 years ago
Shang Zhizhou	bbc837ee72	add info log for trt input dynamic shape check (#27796) ... * add info log for trt input dynamic shape check * fix error msg error	5 years ago
Pei Yang	ae6e40a7fd	Add unittests and OP version registry for tensorrt_subgraph_pass (#27544) ... * add unittests and op version register for tensorrt_subgraph_pass * rename to test_trt_subgraph_pass.py * fix softmax converter diff when padding dim=1	5 years ago
wanghuancoder	df43905f12	use iwyu clean include (#27267) ... * use iwyu clean include, test=develop, test=win * compilation error, test=develop * fix compilation error2, test=develop * fix compilation error3, test=develop * fix compilation error4, test=develop * fix compilation error5, test=develop * fix compilation error6, test=develop * fix compilation error7, test=develop * fix compilation error8, test=develop * fix compilation error8, test=develop * fix compilation error10, test=develop * fix compilation error11, test=develop	5 years ago
Chen Weihang	765064476b	Polish some lost invalid error message (#27445) ... * polish some lost error msg * add some math file to white list * polish detail based reviewer commnet	5 years ago
Pei Yang	fda54c0212	errmsg refine of trt plugin (#27309)	5 years ago
Pei Yang	a5ef246cac	Optimize emb_eltwise_layernorm_plugin and support fp16 (#27128)	5 years ago
Shang Zhizhou	47fdc60ecc	Optimize slice trt plugin (#26970) ... * optimize slice TRT plugin This patch removes unnecessary barrier for data transfer of needed offset, so data transfer can be overlap with GPU kernel execution. This patch also fixes incorrect name of slice plugin. That is, replaces "layernorm" with "slice" test=develop * add serialize/deserialize to slice plugin * add static shape slice trt plugin * fix slice trt op convertor dynamic shape bug * fix format by clang-format * fix pylint format error * fix problems commented by peiyang Co-authored-by: Ryan Jeng <rjeng@nvidia.com>	5 years ago
Shang Zhizhou	e6e2e53782	Optimize error report (#27254) ... * optimize errror report * add test case for pad op converter * fix some spelling mistake commented by peiyang	5 years ago
Pei Yang	aae41c6fca	refine error message related to paddle-TRT (#27256)	5 years ago
Zhaolong Xing	932bbe955b	fix pool trt plugin bug (#26463) ... test=develop	5 years ago
zlsh80826	ad6e3dd69c	[Paddle-TRT] Stack op plugin (#25605) ... * add stack_op to CMakeLists * add dim=3 support for scale op * add trt stack op, test=develop * remove debug message * add stack plugin serialize * remove slice, scale op, will add later * enhence error message * revise trt ernie test to conver the stack op CI testi, test=develop * add stack op serialization * fix test shape after adding stack op * remove slice op, will add after implementing serialization * roll back to min_graph=5 to avoid using slice op * fix scale op output layer * implement stack op createPlugin * use workspace and move the defination to .cu * move stack plugin creator definition to .cu, test=develop	5 years ago
Pei Yang	78a530c219	[Paddle-TRT] TRT dynamic shape support PaddleSlim quant models (#26536) ... * support trt dynamic shape int8 * add unittest * add support for sigmoid; adapt to trt6+ api	5 years ago
zlsh80826	ac63c7cdef	fix a skip_layernorm bug, test=develop (#26800)	5 years ago
Pei Yang	e3f8e5cf5c	trt int8 support conv2d_transpose (#26636)	5 years ago
Pei Yang	379222c3f1	add output scale and trt op teller support for hard_swish and hard_sigmoid (#26499)	5 years ago
Zhaolong Xing	b7a86e92a8	fix dy shape bug in trt7.1 (#26273) ... test=develop	5 years ago
Pei Yang	beb0ca5fab	Fix TRT plugin registry without TRT lib (#25982) ... * fix trt plugin registry without trt lib * support trt4 * refine code style	5 years ago
Pei Yang	b717895f64	Fix registering trt plugin (#25744) ... * develop dynamic shape serilization * add test param for gelu * fix bugs * delete redundant comments * debug * fix conflict. test=develop * fix bug. test=develop * add trt dynamic shape serialized support * fix ernie serialized bug test=develop * fix codestyle test=develop * fix bug test=develop * fix bug.test=develop * modify cmakelist test=develop * fix bug test=develop * fix error message. test=develop * fix trt register plugin based on pr#25003 * add trt dynload * fix deserialization bug of not finding plugin registration * refine code style * recover engine key in tensorrt_subgraph_pass * for ci coverage * add unittest for deserialization Co-authored-by: haozech <chenhaoze94@gmail.com>	5 years ago
Pei Yang	9e9a569dae	add trt int8 support for elementwise_mul and scale (#25676)	5 years ago
Pei Yang	eef98b7f86	add macro check for using TRT api dynamicRangeIsSet() (#25694)	5 years ago
Pei Yang	f82baed866	fix trt instance norm plugin on gcc8. test=develop (#25730)	5 years ago
Jeng Bai-Cheng	fc93266b0a	Improve qkv transpose performance (#23919) ... Use vector instruction (LDG.128) to improve qkv transpose. It provides 1.4X speedup at same GPU base frequency. test=develop	5 years ago
Zhaolong Xing	7b7e605189	[Fix BUGs]: fix multhead matmul pass's instable bug (#25123) ... * fix multhead matmul's instable test=develop * fix multihead matmul bug test=develop * fix converage problem test=develop	5 years ago
Pei Yang	b2f5a149e7	[Paddle-TRT] Better Paddle-TensorRT support for PaddleSlim quant models (#25097) ... * Paddle-TensorRT support slim QAT. test=develop * add comments. test=develop * use RenameInput instead of ResetInputs. test=develop	5 years ago
Zhaolong Xing	843581154f	fix emb eltwise layernorm (#24873) ... test=develop	5 years ago
Leo Chen	fa657b3dbb	fix bug of prelu when rank not equal 4, test=develop (#25067) ... * fix bug of prelu when rank not equal 4, test=develop * fix prelu inference, test=develop * fix api, test=develop * fix shape when mode is chennel, test=develop * remove debug code, test=develop * add unittest, test=develop	5 years ago
Jeng Bai-Cheng	bef4afa6de	bugfix for unique_ptr of IOptimizationProfile (#23917) ... This commit fixs the compiling bug regarding unique_ptr of IOptimizationProfile. IOptimizationProfile has protected dtor and is controlled by TensorRT internally. Application shouldn't delete the pointer of IOptimizationProfile. See TensorRT document: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/classnvinfer1_1_1_i_builder.html#a9ac47e100454151d8206ac91d543299a test=develop	5 years ago
zlsh80826	49e4ee27e1	[Paddle-TRT] slice kernel optimization (#24783) ... * parallel move shared data test=develop * test=develop	5 years ago
Chen Weihang	d1062d5278	Replace all errors thrown by LOG(FATAL) with PADDLE_THROW (#24759) ... * remove REPLACE_ENFORCE_GLOG compile option & add ci rule prohibit LOG(FATAL) using, test=develop * remove ci test case, test=develop * replace all LOG(FATAL) & polish message, test=develop * fix typo, test=develop * polish error info detail, test=develop	5 years ago
Pei Yang	181b1f5a30	adjust minimum trt version for hard_sigmoid converter to 5130. test=develop (#24746)	5 years ago
zlsh80826	fdbe114b12	[Paddle-TRT] use float constant instead of double test=develop (#24544)	5 years ago
Zhaolong Xing	f68d4fb3f1	fix bert bug using trt6 when compile with CUDA_ARCH_NAME=All (#24517) ... test=develop	5 years ago
Chen Weihang	aa0f254fbe	Add macro BOOST_GET to enrich the error information of boost :: get (#24175) ... * add new macro BOOST_GET_SAFELY & unittests, test=develop * add different macro type, test=develop * fix get macro type in executor, test=develop * four macro part change backup * using one macro for all case, test=develop * revert attribute change, test=develop * change to three func to solve gcc4.8 bug, test=develop * polish some details, test=develop	5 years ago
Zhaolong Xing	35148d17f7	[BUG]: Head number can only be > 1 on multihead op (#23974) ... * support the head number == 1 test=develop * fix slice op error. test=develop	5 years ago
Zhou Wei	7817003795	Optimize the error messages of paddle CUDA API (#23816) ... * Optimize the error messages of paddle CUDA API, test=develop * fix the error messages of paddle CUDA API, test=develop * Refactoring PADDLE_ENFORCE_CUDA_SUCCESS, and apply to curand/cudnn/cublas/NCCL,test=develop * remove build_ex_string,test=develop * merge conflict,test=develop	5 years ago
Zhaolong Xing	133f1fc123	[Eernie TRT]: add slice op and add emb eltwise layernorm fp16 support (#23723) ... * refine ernie trt dynamic shape support 1. add slice op converter 2. add emb eltwise layernorm fp16 support test=develop * fix dynamic shape test ut test=develop * fix comments. test=develop * fix comments test=develop	5 years ago
chenhaoze	b7d185d6ca	OP clip, merge_lod_tensor, convert/elementwise error message enhancement (#23742) ... * OP clip, merge_lod_tensor, convert/elementwise error message enhancement. test=develop	5 years ago
Pei Yang	c528f1d4f3	[Paddle-TRT] Add hard_sigmoid and hard_swish support(support MobilenetV3) (#23672) ... * add hard_sigmoid trt op converter * add hard_swish op converter and plugin. test=develop * add macro to adapt lower trt version. test=develop	5 years ago
Pei Yang	015acdbfb1	Refine error message of leaky_relu, tensorrt_engine, split, prelu op converter (#23661)	5 years ago
Zhaolong Xing	3acb047a20	[Paddle-TRT]: add eltwise,pool2d, prelu, scale, concat, gelu dynamic shape support (#23396) ... * add elementwise pool2d, prelu, shuffle channel test=develop * add scale and refine concat eltwise conveter test=develop * refine elementwise converter test=develop * refine ut test and enforce error. test=develop * modify const cast test=develop	5 years ago
Zhaolong Xing	ed5766ffbc	refine act conv2d pool2d trt converter log (#23605) ... * refine act conv2d pool2d fc, trt converter log test=develop * fix comments test=develop	5 years ago
Pei Yang	28f04c6a5e	refine shuffle channel errmsg, test=develop (#23520)	5 years ago
Pei Yang	3d5d217030	Revert "[Paddle-TRT] Add hard_sigmoid and hard_swish support(support MobilenetV3) (#23536)", test=develop (#23642) ... This reverts commit cdc6d4e292.	5 years ago