d4d236bcce
fix: use MindDataset by column_names get data error in some situation
5 years ago
b520ca9087
fix pk sampler in mindrecord
5 years ago
772e6c1461
Cleanup dataset UT: test_batch, save_and_check support
5 years ago
36fffb7706
Added example md5 generation
...
Comparison example
Added md5 and comparison example for py_transforms
Added md5 check for images
5 years ago
83c68ca2ef
Skip pyfunc test case
5 years ago
26cb3e8a5f
Added test function to show that seed doesn't work.
...
Added testcase to show that c image aug don't use seed properly
Added passing test cases
Added working testcases for using seed
Added additional test cases to show seed use
Added test case for seed
5 years ago
c56fe3aa2d
modify take op with an operator
5 years ago
8af10eb51e
!875 Reject python OP in operations argument for C++ uniform augmentation OP
...
Merge pull request !875 from AdelShafiei/ua_py
5 years ago
d15bd04bfe
added input validation to reject python op in C++ uniform augmentation operations list
5 years ago
a606c2e4da
!872 [Dataset] Add schema support for GeneratorDataset
...
Merge pull request !872 from JunhanHu/generator_schema
5 years ago
2303453753
!869 Random data op
...
Merge pull request !869 from JesseKLee/random_data_op
5 years ago
c5a8ffe4f4
Add schema support for GeneratorDataset
5 years ago
5236d0c3c0
Replace print with logger.info
5 years ago
8d3695f666
!672 Added UT for uniform augmentation C++ OP
...
Merge pull request !672 from AdelShafiei/ua_ut
5 years ago
270bf831a9
Random Data Op
5 years ago
34bfa2f7c9
fix skip
5 years ago
3322e65da9
added ut for uniform augment C++ op
5 years ago
b37db1edf5
!603 [MD] update pk sampler in minddataset
...
Merge pull request !603 from liyong126/update_pk_sampler
5 years ago
f82e63fecc
!671 Added testcase for sync_wait
...
Merge pull request !671 from EricZ/master
5 years ago
0e3054d527
!466 Deepcopy problem when pyfunc cannot be pickled
...
Merge pull request !466 from h.farahat/deepcopy
5 years ago
bfba630aa2
update pK_sampler
5 years ago
8c3931cf1d
fix first epoch always shuffle with default seed in random sampler
...
fix ut
5 years ago
2d115cd04e
Added example for multiple iterator
...
Added new testcase for multi iterator
Addressing review
Fixed typo
5 years ago
a9e9266149
Deepcopy problem when pyfunc cannot be pickled
5 years ago
aad5771a62
!524 Added support for UA augmentation ops with tests
...
Merge pull request !524 from alashkari/ua-ops
5 years ago
56e7a7deb5
Added UniformAugment + Python Augmentation Ops
5 years ago
dc0491caf9
!508 [Dataset] Adding sync_wait operator for dataset
...
Merge pull request !508 from EricZ/master
5 years ago
cd94518769
X# This is a combination of 2 commits.
...
Initial commit for dataset op python
Added signature to barrier
Adde compiling barrier code
Rebasing, fixed new compile errors
Final fix for make_unique
Added pybind API for barrier
Fixed pyfunc invocation
python interface - sync_wait
!1 sync_wait python interface
* python interface - sync_wait
fix test
update test
update test
Added new test case
add test case
test for shuffle + batch
Added two-sync test case
Restrited that no shuffle after sync
Added sync to pipeline info
block first databuffer as well
Intelligently get batch size
Fix default case
Lock Pair shares among all iterators
Added fix for empty character
Fixed up test case formatting
Fix end of epoch in sync_wait
Fixing CI
5 years ago
78001ac9e6
Add multiprocessing support for Mindspore.Dataset.GeneratorDataset
5 years ago
fb18671b28
!506 [Dataset] Multiprocessing support for Pyfunc
...
Merge pull request !506 from JunhanHu/multiprocess_pyfunc
5 years ago
b13e7bc31a
Add python multiprocessing support for Mindspore.dataset
5 years ago
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
5 years ago
d9e4dcc33b
!483 Optimize skip dataset op
...
Merge pull request !483 from jiangzhiwen/dataset/skip_opt
5 years ago
f1542a90a3
add pk sampler
5 years ago
e1b109e8b8
optimize skip dataset op
5 years ago
60df369100
Fixup py Normalize doc: takes input CHW
5 years ago
6369cf27bd
!406 added first row crc check for when reading tfrecord files
...
Merge pull request !406 from Peilin/first-row-crc-check
5 years ago
98fbd30a5b
!460 [Data]Add filter operation
...
Merge pull request !460 from xulei/filter_master
5 years ago
822a3160e4
!404 [Dataset] Add Python Sampler support for CPP dataset
...
Merge pull request !404 from JunhanHu/python_sampler_for_cpp_dataset
5 years ago
c705ea5e5b
add filterOp code
5 years ago
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
...
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
5 years ago
2795e492ff
TextFileDataset
5 years ago
43a2e99833
Add python sampler support for CPP dataset
5 years ago
f0c07c3fa6
Realize take op and add ut
5 years ago
80333e9f55
!435 Fix dataset serialize and deserialize for MindDataset
...
Merge pull request !435 from anthonyaje/fix_serdes_dataset
5 years ago
40f0a4a4f4
!333 Add skip op to Dataset
...
Merge pull request !333 from jiangzhiwen/dataset/skip
5 years ago
9e1b5efd1d
!434 Bug in cleaning dataset iterators
...
Merge pull request !434 from h.farahat/multi_itr_bug
5 years ago
ea297c0889
Fix dataset serdes for MindDataset
5 years ago
3c02c82771
Bug in weak reference.
...
Add new test cases
5 years ago
3f7054dccb
add skip dataset op
5 years ago
cf026096a6
!183 Mindspore.dataset CPP sampler for GeneratorDataset
...
Merge pull request !183 from JunhanHu/cpp_sampler
5 years ago
9739d3b048
Add CPP sampler support for GeneratorDataset
5 years ago
30de261c3c
!243 Support nested repeat
...
Merge pull request !243 from h.farahat/nested_repeat
5 years ago
0fc23eee0f
Support nested repeat
5 years ago
1a1cbc6814
implemention of new api: apply
5 years ago
0ce83e39e1
fix TestShardSampleWrongNumber
...
adjust internal api
5 years ago
11403492ae
add mindrecord subset random sampler
5 years ago
59a714c654
Correct shuffle UT buffer_size > #dataset-row as valid
5 years ago
c688265671
fix: when use MindDataset block_reade=True hung
5 years ago
bc4602b58e
fix and remove useless import of example, st, ut
5 years ago
5c22c088bb
!69 Enable skipped dataset zip operator python unit tests
...
Merge pull request !69 from cathwong/ckw_dataset_py_ut1
5 years ago
6a1b865c91
check num_samples
5 years ago
2e881276ab
Enable skipped dataset zip python unit tests
...
Signed-off-by: Cathy Wong <cathy.wong@huawei.com>
5 years ago
8c88b39da1
Optimize the execution time of test case test_rgb_hsv.py
5 years ago
34e42bd6f9
1. add more log info for dataset & mindrecord, 2. add two new testcase for MindDataset
5 years ago
930a1fb0a8
initial version
...
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
5 years ago
jonyguo
liyong
Cathy Wong
eric
Junhan Hu
ms_yan
mindspore-ci-bot
Adel Shafiei
Jesse Lee
jiangzhiwen
Zirui Wu
hesham
Amir Lashkari
qianlong
xulei2020
Peilin Wang
yanghaitao
anthonyaje
jzw
xiefangqi
anzhengqi
zhunaipan