Paddle

Commit Graph

Author	SHA1	Message	Date
yaoxuefeng	c67c391682	refine fleet dataset class api (#27133 )	4 years ago
Chen Weihang	8d53172794	move DataLoader._worker_loop to top level (#27247 ) * move worker loop to top level * move reader process loop to top level * fix failed unittests	4 years ago
Chen Weihang	21ea2976dd	fix static dataloader default pin memory set (#26390 ) * fix static dataloader default pin memory set * fix related unittests	5 years ago
Dong Daxiang	50a5bcfc9d	【paddle.fleet】paddle.fleet -> paddle.distributed.fleet. (#26186 ) * move paddle.fleet to paddle.distributed.fleet	5 years ago
Kaipeng Deng	dbc88bb900	Add iterable dataset support for multiprocess DataLoader (#25558 ) * add IterableDataset support in multiprocess DataLoader. test=develop	5 years ago
Chen Weihang	3c8daa9b89	Add pin memory control for BufferedReader (#26026 ) * add pin memory control * fix buffered reader init problem * fix unittest error * add unittest for coverage	5 years ago
xujiaqi01	20c8432ae3	move dataset from paddfle.fluid to paddle.fleet (#25887 ) * move dataset to fleet test=develop * fix test=develop * fix test=develop * fix test=develop * test=develop * test=develop * test=develop * test=develop * test=develop * test=develop * test=develop	5 years ago
Kaipeng Deng	20ee36bdea	make default_collate_fn visible. test=develop (#25244 ) * make default_collate_fn visible. test=develop	5 years ago
WangXi	1c7215ace4	Speedup reader check_input_array when item is array (#25395 )	5 years ago
Leo Chen	c7a63908b0	replace some logging.warn() with warings.warn(), test=develop (#25025 )	5 years ago
Chen Weihang	2e7e69d0cb	remove imperative data loader place limit, test=develop (#24641 )	5 years ago
Kaipeng Deng	80cf3c3c4d	Refine DataLoader support multi-processing (#23107 ) * add DataLoader, Dataset, BatchSampler	5 years ago
Zeng Jinle	bba740710d	add cuda resource pool for BufferedReader, test=develop (#23152 )	5 years ago
Zeng Jinle	acfc9b8a70	Reader sequential and inference partial feed (#22699 ) * sequential reader stage 1, test=develop * fix ut, test=develop * fix iterable=False reset bug, add some logs and polish code, test=develop * inference feed partial data, test=develop * Turn on keep_order=True for test, test=develop * enhance ut to test more cases, test=develop * test commit for reverting * Revert "test commit for reverting", test=develop This reverts commit 80aef42ef52ba1ee79627d6f663a624ec4f12f58. * add ut of merged and unmerged results, test=develop * add more uts for coverages and add en doc of api, test=develop * follow comments, test=develop * change note style, test=develop	5 years ago
Chen Weihang	1644926a6c	Polish detail implement of dygraph data loader (#22878 ) * polish detail implement of data loader, test=develop * solve coverage ci problem, test=develop	5 years ago
Chen Weihang	2e9e8c056a	add platform limit for cleanup function, test=develop (#22846 )	5 years ago
Chen Weihang	7d8d573453	Speed up dygraph DataLoader based on shared memory and LoDTensor serialization (#22541 ) * add lodtensor share memory & serialization, test=develop * fix windows compile error, test=develop * deal vartype pickle & fix unittest matching error message, test=develop * update timeout variable name, test=develop * refactor memory map implement, test=develop * clear mmap file discripter when exit unexpectedly, test=develop * remove the child process fd in advance, test=develop * remove mmap fds after Queue.put in child process, test=develop * add hard unittests for register exit func, test=develop * fix python2 compatibility problem in unittest, test=develop * fix exception unittest error, test=develop * polish code based review comment, test=develop	5 years ago
tianshuo78520a	433cef03e5	fix typo word (#22784 )	5 years ago
Chen Weihang	192110725a	Add Queue.get delay for multiprocess data loader (#22604 ) * add get delay for multiprocess data loader, test=develop * add unittest for coverage ci, test=develop * add timeout unittest, test=develop * increase the delay time, test=develop	5 years ago
Zeng Jinle	26da700ab2	Fix static graph dataloader early reset bug (#22365 ) * fix dataloader early reset bug, test=develop * change implementation, test=develop * fix ut, test=develop	5 years ago
Chen Weihang	35efbe6d95	Speeding up dygraph DataLoader with multiprocessing (#21762 ) * add multiprocess for dygraph data loader, test=develop * polish code & add safe gurad, test=develop * refactor dygraph dataloader & add signal handler, test=develop * fix member initializer compile error on ci, test=develop * fix member initializer compile error one more, test=develop * remove useless config, test=develop * skip windows incompatible problem, test=develop * add unittest for coverage, test=coverage * add more exception unittest case, test=develop * deal with signal handler coverage, test=develop * polish code & add signal handler tests, test=develop * deal with coverage ci problem, test=develop * split data loader test & coverage ci fix, test=develop * remove test_imperative_data_loader_with_exception, test=develop * remove singal process except test case, test=develop * add exception tests again & remove sample list test, test=develop * split normal and exception unittests to diff class, test=develop * polish doc for use_multiprocess effect in static mode, test=develop	5 years ago
Chen Weihang	68999b6c7a	simplify dygraph data loader code, test=develop (#21722 )	5 years ago
Chen Weihang	d96acc3363	Refine dygraph DataLoader implementation (#21634 ) * refine dygraph dataloader & polish related code, test=develop * refine code based review comment, test=develop	5 years ago
Huihuang Zheng	95ba4bd2ab	Add shape and type check at read_op (#20754 )	5 years ago
Zeng Jinle	40effc61af	Refine py_reader exit (#20331 ) * refine py_reader exit, test=develop * fix multiprocess_reader exception unittest, test=develop * increase code coverage for legacy fluid.layers.py_reader, test=develop	5 years ago
guofei	9b85f40140	Modify English documents (#20452 )	5 years ago
Zeng Jinle	7fb958b2e1	replace data layers, test=develop, test=document_fix (#20368 )	5 years ago
liuwei1031	bda7eab77d	improve the error message when handling ndarray with unsupported dtype (#19949 ) * impove error message when passing ndarray with object dtype * imporve message format * change assert to raise TypeError * remind user how to locate the irregular data instead of printing * add unittest for input array type check	5 years ago
Zeng Jinle	0436efd6a3	Unify DataLoader APIs (#19305 ) * unify DataLoader APIs, test=develop * integrate iterable CPU Dataset, test=develop add GPU dataset supporting, test=develop * add unittests for dataset, test=develop * add more docs to dataloader apis, test=develop, test=document_preview * refine doc, test=develop * refine doc again, test=develop * increase coverage, test=develop	5 years ago
Zeng Jinle	d07ad4c605	add more traceback to py_reader error msg, test=develop (#18722 )	6 years ago
Zeng Jinle	41ab76e55b	add friendly error msg to py_reader (#18316 )	6 years ago
Huihuang Zheng	bbc292920c	Fix API example code (#18176 ) The fixed APIs: 6 Methods in paddle.fluid.io.PyReader paddle.fluid.layers.Preprocessor paddle.fluid.layers.py_reader paddle.fluid.io.save_params paddle.fluid.io.save_persistables test=develop test=document_preview	6 years ago
wopeizl	841553e13f	use pyreader to read data in dygraph mode (#17314 ) * use pyreader to read data * add return_list to PyReader to support return value represented as list	6 years ago
Huihuang Zheng	648320bb6c	Fix some data and reader related API code (#17202 ) * Fix data and reader related api doc * Fix data and reader related api doc Review and fix the example code in some reader related API doc. These APIs are: Fix existing API example codes: paddle.fluid.io.PyReader paddle.fluid.layers.batch paddle.fluid.layers.data paddle.fluid.layers.Preprocessor paddle.fluid.layers.py_reader paddle.fluid.program_guard Add new example codes: paddle.fluid.io.PyReader.decorate_batch_generator paddle.fluid.io.PyReader.decorate_sample_generator paddle.fluid.io.PyReader.decorate_sample_list_generator paddle.fluid.io.PyReader.reset paddle.fluid.io.PyReader.start test=develop * Add changes to API.spec after changing doc. test=develop * Add blanks after python example code test=develop * Add blank line at py_reader example code test=develop * Merge API.spec test=develop * Modify reader.py based on reviewer's comment test=develop * Modify API.spec after changing doc test=develop * Change reader.py based on reviewer's comment * Modify example code of decorate_sample_generator test=develop * Fix example code of PyReader based on reviewer test=develop	6 years ago
sneaxiy	bb166a1e10	fix API.spec test=develop	6 years ago
sneaxiy	3a09693f5c	change API name test=develop	6 years ago
sneaxiy	aa2335c218	add py_reader doc	6 years ago
sneaxiy	3334c279d0	add sample_generator test=develop	6 years ago
sneaxiy	7b5a9d75d9	add cache reader test=develop	6 years ago
sneaxiy	69b1ebdfa5	merge develop test=develop	6 years ago
sneaxiy	2e67f8ae88	add doc test=develop	6 years ago
sneaxiy	c545f1ed8f	unify API test=develop	6 years ago
sneaxiy	b17541a9c1	fix hang bug	6 years ago
sneaxiy	27e1a9ee5a	fix hang bug test=develop	6 years ago
sneaxiy	7160cb0f32	decoupled reader test=develop	6 years ago

45 Commits (34091533c95291dd96459af65dacf04a9cce9fa2)