Tinazhang
7322839b04
add UTs for LinearTransformation, ToPIL, ToType
5 years ago
Peilin Wang
0cbcc7200b
made shuffle determinisitc for subsequent epochs
5 years ago
mindspore-ci-bot
458436186c
!1365 Clean up work for text python sub-package
...
Merge pull request !1365 from h.farahat/text_namespace
5 years ago
mindspore-ci-bot
6f733ec113
!1308 Stage 2 of adding support for string Tensor
...
Merge pull request !1308 from h.farahat/string_tensor2
5 years ago
hesham
6c21e556c4
Clean up work for text python package
5 years ago
heleiwang
599a449e0b
Support processing GNN data
5 years ago
hesham
df361d1d26
Change mem layout of string tensor
...
add support for MindRecord and TFRecord
----
optimize tensorshape
optimize tensorshape and FlatIndex
TFRecord and MindRecord support for string tensor
Modify mem layout
Add new constructor
Add method Allocate
Change some GetMutableBuffer usages to AllocateBuffer
5 years ago
mindspore-ci-bot
58e6d7d950
!1341 Added lookup and vocab to mindspore.dataset.text
...
Merge pull request !1341 from ZiruiWu/vocab_and_lookup
5 years ago
jonwe
bb51bb88d7
add compress in mindrecord
5 years ago
Zirui Wu
25ab2ef303
Implemented lookup and vocab
5 years ago
qianlong
451c20a6f5
Add UnicodeCharTokenizer for nlp
5 years ago
xulei2020
163b6b7ea7
add jieba c++ code
5 years ago
Tinazhang
17cecf2cf5
Added TCs to RandomCrop and RandomCropAndResize and modified visalize() calling
5 years ago
jinyaohui
bcfaff97f9
clean pylint
5 years ago
jiangzhiwen
cb2814b498
flat_map first commit
5 years ago
mindspore-ci-bot
ab031ee9ea
!1126 VOCDataset support object detection function
...
Merge pull request !1126 from xiefangqi/voc_support_detection
5 years ago
xiefangqi
c937bad53f
minddata support voc
5 years ago
Cathy Wong
913074e656
Cleanup dataset UT: resolve skipped test units
5 years ago
Cathy Wong
49ef53f164
Cleanup dataset UT: util.py internals
5 years ago
mindspore-ci-bot
2860fd9338
!984 Add unit test case for HWC2CHW.
...
Merge pull request !984 from Tinazhang/hwc2chw
5 years ago
Tinazhang
c8b5586c7f
add unit test for HWC2CHWC
5 years ago
Cathy Wong
58226addd6
Cleanup dataset UT: use md5 npz in test_zip for images
5 years ago
Cathy Wong
772e6c1461
Cleanup dataset UT: test_batch, save_and_check support
5 years ago
eric
36fffb7706
Added example md5 generation
...
Comparison example
Added md5 and comparison example for py_transforms
Added md5 check for images
5 years ago
Jesse Lee
270bf831a9
Random Data Op
5 years ago
yanghaitao
1cb3bead32
skip black line for TextFileDataset
5 years ago
qianlong
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
5 years ago
liyong
f1542a90a3
add pk sampler
5 years ago
mindspore-ci-bot
6369cf27bd
!406 added first row crc check for when reading tfrecord files
...
Merge pull request !406 from Peilin/first-row-crc-check
5 years ago
xulei2020
c705ea5e5b
add filterOp code
5 years ago
Peilin Wang
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
...
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
5 years ago
yanghaitao
2795e492ff
TextFileDataset
5 years ago
Cathy Wong
59a714c654
Correct shuffle UT buffer_size > #dataset-row as valid
5 years ago
yanzhenxiang2020
5a4f17bfb6
fix mindrecord ut long time
5 years ago
Cathy Wong
2e881276ab
Enable skipped dataset zip python unit tests
...
Signed-off-by: Cathy Wong <cathy.wong@huawei.com>
5 years ago
zhunaipan
930a1fb0a8
initial version
...
Signed-off-by: leonwanghui <leon.wanghui@huawei.com>
5 years ago