18b519ae0f
add sentence piece
5 years ago
fbd9e3d402
vocdataset output multi-columns
5 years ago
1a518e1cff
!3144 [MD] Fix ResizeOp behaviour
...
Merge pull request !3144 from nhussain/dataset_check
5 years ago
c984b0fc88
fix size tuple
5 years ago
5647889c0d
Added AutoContrast Op
5 years ago
37e8439c0a
!3077 supporting cpp invert operation
...
Merge pull request !3077 from ava/invert
5 years ago
35c3a63701
support cpp invert operation
5 years ago
04eb4f89c7
Cleanup dataset UT: Replace save_and_check
5 years ago
ba0143402c
!3067 Cleanup dataset UT: Remove unneeded tf data files and tests
...
Merge pull request !3067 from cathwong/ckw_dataset_ut_cleanup6
5 years ago
ae1ed327ba
Cleanup dataset UT: Remove unneeded data files and tests
5 years ago
340d98a4d1
added test case to cifar_op
...
update cifar10 dataset
fixing missing error handling code in validator
5 years ago
48654be14d
!3040 BoundingBoxAugment op fix ratio
...
Merge pull request !3040 from islam_amin/bounding_box_augment_bugs
5 years ago
56da3b0ae1
Fixing ratio bug with BoundingBoxAugment
5 years ago
7c1bc5192b
Initial Drop of CacheOp Phase I
5 years ago
7a046a1d70
support get_edge_feature
5 years ago
2c3c204b14
!2982 supporting cpp unit tests for randomresizewithbbox and resizewithbbox
...
Merge pull request !2982 from ava/CPP_test3
5 years ago
b2bfb0342b
supporting cpp unit tests for random_resize_with_bbox_op and resize_with_bbox_op.
5 years ago
506f8f134e
fix, cpp ut's
...
fix cmakefile
fixed log statments in cpp UT tests
5 years ago
3cc9217f06
!2926 [Dataset C++ UT] Implement C++ UTs for RandomHorizontalFlipWithBBox and BoundingBoxAugment
...
Merge pull request !2926 from islam_amin/object_ops_cc_ut_floats
5 years ago
284cc910d5
C++ UTs for RandomHorizontalFlipWithBBox and Bounding Box Augment
5 years ago
421b80bf8f
updating python unit tests based on floating point bounding boxes + adding coco dataset in unit tests
5 years ago
9ba937b1af
!2920 Updating 3 BBox AugOps for Float + Common Files for BBox Float Support - (VertFlip/RandomCrop/RandomResizeCrop)
...
Merge pull request !2920 from danishnxt/AugOps-M1-Python
5 years ago
78c370f72a
First Commit - New Infrastructure - Python UT tests + Common Aug Files
...
fix accidental change overwrite
fix 2
updated inits from 0 to 0.0 for float
improvd python ut comments
updated macro
5 years ago
dfc3409f67
Update RandomHorizontalFlipWithBBox and BoundingBouxAugment C++ Ops to use floats
5 years ago
dc03019200
unskip md5 testcase for RandomPerspective
5 years ago
3197f9fb35
!2733 Schema clean up
...
Merge pull request !2733 from EricZ/schema_fix
5 years ago
88bb65768e
Clean up part 1
...
Removed extra fields in schema
Fixed test cases
Fixing CI
5 years ago
5cd3136355
remove local defined mse and add missing mse/md5 validation
5 years ago
32a2d5d21c
!2629 get monitor sampling interval from json file
...
Merge pull request !2629 from yanghaitao/yht_sampling_interval
5 years ago
cae77c0c22
BasicTokenizer not case fold on preserverd words
5 years ago
fd701b8504
set monitor sampling interval through json file
5 years ago
a1b17bd2e2
!2593 python unit tests for randomResizeWithBBox and ResizeWithBBox
...
Merge pull request !2593 from ava/python_ut_tests
5 years ago
1e869146e9
applying comments removing VOC
5 years ago
277aba5326
dataset: Fixup docs; remove pylint disabled messages in UT
5 years ago
e11c953225
!2587 Implementing Python UTs for BoundingBoxAugment and RandomHorizontalFlipWithBBox
...
Merge pull request !2587 from imaaamin/object_ops_ut
5 years ago
0f58f0338e
updating ut for RandomHorizontalFlipWithBBox and BBoxAugment
5 years ago
3d1ecaaeb5
updated UT test for Python (3) AugOps with BBox - MD5 checks + imrpv comments
5 years ago
a8f392ca42
updated test util file + new BoundingBoxCheck + fixed VOCDataset annotations
...
Style Error fix
fixed PyLint problem
reverting testVOC2012 due to CI break for existing test
reverting testVOC2012 due to CI break for existing test-2
updated old error messages to confirm with global standard
addressing PR 2355 Comments - 1
addressing PR 2355 Comments - 2
addressing PR 2355 Comments - 3
5 years ago
cf82aa9035
init remove storage op in c++
...
init remove storage op test case c++
remove source c++ files
5 years ago
90bb9320aa
!2314 Implemented RandomHorizontalFlipWithBBox and BoundingBoxAugment C++ Ops
...
Merge pull request !2314 from imaaamin/object_ops_pr
5 years ago
0af156d169
added two ops for object detection, random horizontal flip with bbox and bouding box augment. along with their unit tests
5 years ago
b6e9504b31
phase I of Vocab rework
...
phase II vocab rework
added more test cases
fix api doc string
address review cmts and fix CI
address ci complains
fix review cmts
ci
5 years ago
2ecd5bdf25
!2239 [MD] convert csv to mindrecord
...
Merge pull request !2239 from liyong126/csv_to_mindrecord
5 years ago
8d1dae46ac
Throw error when load config failed
5 years ago
7369950a93
convert csv to mindrecord
5 years ago
4f16f036be
Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp
...
add CaseFold, NormalizeUTF8
add RegexReplace
add RegexTokenizer
add BasicTokenizer
add WordpieceTokenizer
add BertTokenizer
5 years ago
ca94916ef9
!2184 CocoDataset add category_id check
...
Merge pull request !2184 from xiefangqi/xfq_fix_coco_issue
5 years ago
625f2421b5
!1977 random walk in gnn node2vec
...
Merge pull request !1977 from JonathanY/randomwalk
5 years ago
aab190fcdb
CocoDataset add category_id validation
5 years ago
3353a20d8b
add tool: tfrecord to mindrecord
5 years ago
dceb3c2be4
!2026 Change C++ behavior in RandomCropAndResize
...
Merge pull request !2026 from MahdiRahmaniHanzaki/I1K3FL-random-resized-crop
5 years ago
add29f95f5
resolved issues in the comments
5 years ago
87d2c27c7f
random walk v1
5 years ago
e0e167a000
add CLUE dataset
5 years ago
bc22c172b8
add TensorDataset and its ut
5 years ago
b3da41bd7a
!1913 CocoDataset implementation
...
Merge pull request !1913 from xiefangqi/xfq_support_coco
5 years ago
097cbcc0cb
CocoDataset implementation
5 years ago
fe0c3e9a8f
!1908 Add UT to six data vision transforms ops
...
Merge pull request !1908 from Tinazhang/ut-normalize
5 years ago
3ece8dd090
1. support get_all_edges, get_nodes_from_edge, get_sampled_neighbors, get_neg_sampled_neighbors and graph_info API
...
2. mod cora and citeseer conversion script
5 years ago
c3de5c7a54
adding new TCs to six ops and skip md5 case in RandomPerspective
5 years ago
976b4793d1
!1544 Python Unit Test for TenCrop
...
Merge pull request !1544 from MahdiRahmaniHanzaki/I1IHNG-ten-crop
5 years ago
f4a61fdbb2
Adding new UT cases and missing TCs in PY Ops.
5 years ago
dfc097019b
Added TenCrop test
...
Added unit tests for both testing the functinality of the TenCrop and
its error messages. Due to the similarity of this method to FiveCrop the
test cases are similar to FiveCrop test cases.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
added error_msg function call in the main method
refactored the test and added visual representation of the results
Separated the two error cases into two different functions and used the
visualize function in util.py to plot the result of TenCrop.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
Added new test cases
Added new test cases including test case for checking the error message
when the size variable is not a positive integer, test case for
rectangle crop, test case for vertical flip setting, and testing for
similarity of the result of TenCrop for the same input data in different runs.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
changed visualize in test_five_crop
Changed the visualize function in test_five_crop to use the already
existing function in util.py
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
made generate_golden variable global
5 years ago
abca62f407
10:00 26/5 clean pylint
5 years ago
aeb4c52f2d
!1379 Creating new UTs to three vision transformation ops
...
Merge pull request !1379 from Tinazhang/linearTrans
5 years ago
7322839b04
add UTs for LinearTransformation, ToPIL, ToType
5 years ago
0cbcc7200b
made shuffle determinisitc for subsequent epochs
5 years ago
458436186c
!1365 Clean up work for text python sub-package
...
Merge pull request !1365 from h.farahat/text_namespace
5 years ago
6f733ec113
!1308 Stage 2 of adding support for string Tensor
...
Merge pull request !1308 from h.farahat/string_tensor2
5 years ago
6c21e556c4
Clean up work for text python package
5 years ago
599a449e0b
Support processing GNN data
5 years ago
df361d1d26
Change mem layout of string tensor
...
add support for MindRecord and TFRecord
----
optimize tensorshape
optimize tensorshape and FlatIndex
TFRecord and MindRecord support for string tensor
Modify mem layout
Add new constructor
Add method Allocate
Change some GetMutableBuffer usages to AllocateBuffer
5 years ago
58e6d7d950
!1341 Added lookup and vocab to mindspore.dataset.text
...
Merge pull request !1341 from ZiruiWu/vocab_and_lookup
5 years ago
bb51bb88d7
add compress in mindrecord
5 years ago
25ab2ef303
Implemented lookup and vocab
5 years ago
451c20a6f5
Add UnicodeCharTokenizer for nlp
5 years ago
163b6b7ea7
add jieba c++ code
5 years ago
17cecf2cf5
Added TCs to RandomCrop and RandomCropAndResize and modified visalize() calling
5 years ago
bcfaff97f9
clean pylint
5 years ago
cb2814b498
flat_map first commit
5 years ago
ab031ee9ea
!1126 VOCDataset support object detection function
...
Merge pull request !1126 from xiefangqi/voc_support_detection
5 years ago
c937bad53f
minddata support voc
5 years ago
913074e656
Cleanup dataset UT: resolve skipped test units
5 years ago
49ef53f164
Cleanup dataset UT: util.py internals
5 years ago
2860fd9338
!984 Add unit test case for HWC2CHW.
...
Merge pull request !984 from Tinazhang/hwc2chw
5 years ago
c8b5586c7f
add unit test for HWC2CHWC
5 years ago
58226addd6
Cleanup dataset UT: use md5 npz in test_zip for images
5 years ago
772e6c1461
Cleanup dataset UT: test_batch, save_and_check support
5 years ago
36fffb7706
Added example md5 generation
...
Comparison example
Added md5 and comparison example for py_transforms
Added md5 check for images
5 years ago
270bf831a9
Random Data Op
5 years ago
1cb3bead32
skip black line for TextFileDataset
5 years ago
db80f4ff92
The num_samples and numRows in schema for TFRecordDataset are conflict
5 years ago
f1542a90a3
add pk sampler
5 years ago
6369cf27bd
!406 added first row crc check for when reading tfrecord files
...
Merge pull request !406 from Peilin/first-row-crc-check
5 years ago
c705ea5e5b
add filterOp code
5 years ago
9bc2134cb7
added checking of first row crc to find invalid tfrecord files
...
addressed code review comments. added check in python layer to exclude directories and to raise an error if a pattern does not match any file
fixed clang format
fixed cppcheck
fixed cppcheck (used std::accumulate and std::copy_if). regenerated tfrecord file to contain correct header, it was a dummy header before
fixed cppcheck: added const reference for string parameter for lambdas, fixed clang format: whitespace adjustments
more clang whitespace fixes...
changed print to logger.info
5 years ago
2795e492ff
TextFileDataset
5 years ago
59a714c654
Correct shuffle UT buffer_size > #dataset-row as valid
5 years ago
5a4f17bfb6
fix mindrecord ut long time
5 years ago
2e881276ab
Enable skipped dataset zip python unit tests
...
Signed-off-by: Cathy Wong <cathy.wong@huawei.com>
5 years ago
xulei2020
xiefangqi
mindspore-ci-bot
nhussain
islam_amin
avakh
Cathy Wong
tinazhang
Jesse Lee
heleiwang
Danish Farid
Eric
qianlong
yanghaitao
ms_yan
Zirui Wu
liyong
jonyguo
Mahdi
Jonathan Yan
jiangzhiwen
mindspore-ci-bot
Yang
Peilin Wang
hesham
jinyaohui
mindspore-ci-bot
yanzhenxiang2020