5515016dba
remove graphengine changes
...
remove graphengine changes
concat op
Truncate Pair
concat_op
remove graph engine changes
6 years ago
ffc8a3c362
!2274 add PadEndOp
...
Merge pull request !2274 from xunxue/padend
6 years ago
e4322baece
!2316 [Dataset] Move python tree transformation into cpp pass
...
Merge pull request !2316 from JunhanHu/alter_node_cpp
6 years ago
0af156d169
added two ops for object detection, random horizontal flip with bbox and bouding box augment. along with their unit tests
6 years ago
87aa9c8f7a
PadEndOp
6 years ago
57f3732ac3
Move python tree transformation into cpp pass.
6 years ago
b6e9504b31
phase I of Vocab rework
...
phase II vocab rework
added more test cases
fix api doc string
address review cmts and fix CI
address ci complains
fix review cmts
ci
6 years ago
8f2674850b
address API doc style and content
...
addr review cmts
revert
ci
private class
ci
fix ci
address ci
address review cmts
fix style err
6 years ago
5850b99161
!2211 Add TruncatePair Op to dataset
...
Merge pull request !2211 from h.farahat/pair_truncate
6 years ago
d0c5071cab
!1993 [Dataset] Fix codedex.
...
Merge pull request !1993 from luoyang/pylint
6 years ago
8d1dae46ac
Throw error when load config failed
6 years ago
b9495a9ccc
Truncate Pair
6 years ago
ea829e89c3
!2198 Add MaskOp to dataset
...
Merge pull request !2198 from h.farahat/mask_op
6 years ago
808380a9f9
!2142 Cleanup dataset UT: refactor visualize()
...
Merge pull request !2142 from Tinazhang/reorg-visualize
6 years ago
fdafc690f8
repair problem in NumpySlicesDataset
6 years ago
169eaf73c7
reorganize util function visualize()
6 years ago
dee8471d23
!1993 [Dataset] Fix codedex.
6 years ago
4f16f036be
Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp
...
add CaseFold, NormalizeUTF8
add RegexReplace
add RegexTokenizer
add BasicTokenizer
add WordpieceTokenizer
add BertTokenizer
6 years ago
f2462bb00d
Mask Op
6 years ago
ca94916ef9
!2184 CocoDataset add category_id check
...
Merge pull request !2184 from xiefangqi/xfq_fix_coco_issue
6 years ago
625f2421b5
!1977 random walk in gnn node2vec
...
Merge pull request !1977 from JonathanY/randomwalk
6 years ago
3adbde56b2
!2162 SliceOp
...
Merge pull request !2162 from h.farahat/slice_op
6 years ago
898b2fdecd
!2187 Fix comment display issues in BuildVocabDataset
...
Merge pull request !2187 from ZiruiWu/vocab_rework
6 years ago
2794883644
fix selected minor issues
...
fix review comments
6 years ago
aab190fcdb
CocoDataset add category_id validation
6 years ago
d4f8f57c7e
add split in minddataset
6 years ago
e2012a1de9
Slice Op
6 years ago
dceb3c2be4
!2026 Change C++ behavior in RandomCropAndResize
...
Merge pull request !2026 from MahdiRahmaniHanzaki/I1K3FL-random-resized-crop
6 years ago
add29f95f5
resolved issues in the comments
6 years ago
2cab44a637
!2028 FillOp for BERT
...
Merge pull request !2028 from nhussain/fillOp_learning
6 years ago
dd9bf09f0a
added FillOp for #119 - special Ops
6 years ago
848e07d022
initial commit, start of BucketBatchByLengthOp
...
c implementation done, just need to call batch/pad
added python api and validator
added pybind/de_pipeline stuff, fixed some compile errors, figure out how null py::function works
added tiny bit of doc
integrated with static batch methods
fixed some bugs
some more bug fixes and cleanup
ci fix
fix ci
ci fix
fix ci
added test_cases and debugged
addressed code review comments
addressed code review comments
ci fix
ci fix
addressed code review comments
addressed code review comments
6 years ago
87d2c27c7f
random walk v1
6 years ago
a8cf83acf2
!1932 Add CLUE dataset
...
Merge pull request !1932 from jiangzhiwen/dataset/clue
6 years ago
e0e167a000
add CLUE dataset
6 years ago
a22763b612
Bug in CIFAR after removing GetMutabble buffer
...
Fixes # I1KIPC
6 years ago
bb2082daec
fix: MindDataset padded log error
6 years ago
3085e51e45
!1922 check parameter num_samples of sampler
...
Merge pull request !1922 from yanghaitao/yht_check_num_samples_sampler
6 years ago
1b5d9a1194
!1981 Implemented from_dataset, a way to build a vocab from existing dataset
...
Merge pull request !1981 from ZiruiWu/from_dataset_dev
6 years ago
880ce5ea26
implemented from_dataset
...
fix complie error
more tests
address CI complains
fix ci
adress review comments
address review cmts
6 years ago
4e333a2f22
MindData profiler infrastructure
6 years ago
0a95223f25
!1910 RandomColorAdjust error for grayscale images
...
Merge pull request !1910 from MahdiRahmaniHanzaki/I1J9SQ-random-color-adjust-bug
6 years ago
8f16cff12e
add para check for sampler
6 years ago
1de7271afc
add floatxx test case
6 years ago
bc22c172b8
add TensorDataset and its ut
6 years ago
251a6667a5
!1989 fix MindDataset distribute shuffle error
...
Merge pull request !1989 from guozhijian/fix_distribute_shuffle
6 years ago
b3da41bd7a
!1913 CocoDataset implementation
...
Merge pull request !1913 from xiefangqi/xfq_support_coco
6 years ago
097cbcc0cb
CocoDataset implementation
6 years ago
96859f44b6
fix: MindDataset distribute shuffle bug
6 years ago
ba2e2dced4
fixed some warnings and typos
6 years ago
2812cd57db
fix pylint error
6 years ago
fe0c3e9a8f
!1908 Add UT to six data vision transforms ops
...
Merge pull request !1908 from Tinazhang/ut-normalize
6 years ago
3ece8dd090
1. support get_all_edges, get_nodes_from_edge, get_sampled_neighbors, get_neg_sampled_neighbors and graph_info API
...
2. mod cora and citeseer conversion script
6 years ago
5eb95599f6
!1874 Updates to string tensor
...
Merge pull request !1874 from h.farahat/string_tensor_updates
6 years ago
c3de5c7a54
adding new TCs to six ops and skip md5 case in RandomPerspective
6 years ago
f837ddc956
- Bug when empty strings sent to Python
...
- Support accepting Numpy of str as input
- Support batching strings
- Core logic of batch&pad is static
- Make Pad a utility function
6 years ago
feff8899ac
support padding samples
6 years ago
5c21616293
!1807 Implemented Ngram TensorOp for dataset
...
Merge pull request !1807 from ZiruiWu/ngram_dev
6 years ago
dbf9936ec4
Implemented n-gram for dataset TensorOp
6 years ago
51bc0c0460
consistent design for num_samples
...
updates
more update
more work
more fixin
post rebase updates
clang formatting
code review recovery
ci fixes
updates
update
update
update
6 years ago
33fe66fd1e
fix error handling for bigger crop size case
6 years ago
5522edc4b5
!1713 [Dataset] Clean pylint.
6 years ago
7b911886ec
Fixing some tiny faults about Pylint in my code(ops)
6 years ago
7878743400
!1638 fixed rounding edge case in split
...
Merge pull request !1638 from Peilin/splitOp-after-testing
6 years ago
0e3dd8149a
!1703 Adding sync_wait input check
...
Merge pull request !1703 from EricZ/master
6 years ago
c366b3fb18
!1704 fix num rows bug in sampler
...
Merge pull request !1704 from Peilin/fix-random-sampler-bug
6 years ago
8e1b2107ce
Added example for sync_wait bad batch_num
...
Added broken test case for cv
Fixed error handling in sync_wait
6 years ago
976b4793d1
!1544 Python Unit Test for TenCrop
...
Merge pull request !1544 from MahdiRahmaniHanzaki/I1IHNG-ten-crop
6 years ago
3ef3d1433d
fixed num_samples bug for RandomSampler
6 years ago
f4a61fdbb2
Adding new UT cases and missing TCs in PY Ops.
6 years ago
5469be2a97
fixed bug for split, RandomSampler and some other cleanup
...
add another test case
typo
merge conflict
another PR changed testing behavior, updated test cases in this commit
added input check for use_sampler
addressed code review comments
fixed pylint, not related to my changes
fixed edge case of rounding in getting split sizes
fix pylint
6 years ago
dfc097019b
Added TenCrop test
...
Added unit tests for both testing the functinality of the TenCrop and
its error messages. Due to the similarity of this method to FiveCrop the
test cases are similar to FiveCrop test cases.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
added error_msg function call in the main method
refactored the test and added visual representation of the results
Separated the two error cases into two different functions and used the
visualize function in util.py to plot the result of TenCrop.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
Added new test cases
Added new test cases including test case for checking the error message
when the size variable is not a positive integer, test case for
rectangle crop, test case for vertical flip setting, and testing for
similarity of the result of TenCrop for the same input data in different runs.
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
changed visualize in test_five_crop
Changed the visualize function in test_five_crop to use the already
existing function in util.py
Signed-off-by: Mahdi <mahdi.rahmani.hanzaki@huawei.com>
made generate_golden variable global
6 years ago
21da86b393
!1591 add get_dataset_size for CelebADataset
...
Merge pull request !1591 from yanghaitao/yht_celeba_get_dataset_size
6 years ago
cc6c7a3f60
add get_dataset_size to celebadataset
6 years ago
0f4dc13e8e
!1478 [Dataset] Fix CI warnings, clean pylint.
...
Merge pull request !1478 from luoyang/pylint
6 years ago
9b2a778d94
!1478 [Dataset] clean pylint.
6 years ago
5e4728c50f
add voc support split
6 years ago
d4c93575e7
fixed bug for split, RandomSampler and some other cleanup
...
add another test case
typo
merge conflict
another PR changed testing behavior, updated test cases in this commit
added input check for use_sampler
addressed code review comments
fixed pylint, not related to my changes
6 years ago
abca62f407
10:00 26/5 clean pylint
6 years ago
0a8ef2fe72
!1455 Cleanup dataset UT: restore config support
...
Merge pull request !1455 from cathwong/ckw_dataset_config_util
6 years ago
f891e1755c
Cleanup dataset UT: restore config support
6 years ago
aeb4c52f2d
!1379 Creating new UTs to three vision transformation ops
...
Merge pull request !1379 from Tinazhang/linearTrans
6 years ago
718a96430f
!1389 Security issues in recent string tensor PR
...
Merge pull request !1389 from h.farahat/fix_security_issues
6 years ago
a6b8451a33
!1380 make ShuffleOp have deterministic behavior for subsequent epochs
...
Merge pull request !1380 from Peilin/shuffle-subsequent-epoch-deterministic
6 years ago
f28f883cac
fix comments error and modify parameter check
6 years ago
c0f0e154f5
Clean up work for text python package
6 years ago
41456ac824
!1369 dataset: delete StorageDataset
...
Merge pull request !1369 from ms_yan/del_storage
6 years ago
7322839b04
add UTs for LinearTransformation, ToPIL, ToType
6 years ago
0cbcc7200b
made shuffle determinisitc for subsequent epochs
6 years ago
61639be1e0
!1375 Cleanup dataset UT: remove num_parallel_workers=1 in test_exception
...
Merge pull request !1375 from cathwong/ckw_dataset_ut_cleanup5
6 years ago
a528797253
!1377 Code Fix for Uniform Augmentation
...
Merge pull request !1377 from Tinazhang/cc
6 years ago
458436186c
!1365 Clean up work for text python sub-package
...
Merge pull request !1365 from h.farahat/text_namespace
6 years ago
b390883c6a
Bug fix
6 years ago
702005d403
Cleanup dataset UT: remove num_parallel_workers=1 in test_exception
6 years ago
d5e896b51c
delete storageDataset Op API and its test case
6 years ago
6f733ec113
!1308 Stage 2 of adding support for string Tensor
...
Merge pull request !1308 from h.farahat/string_tensor2
6 years ago
34236ce1f1
fix pylint
6 years ago
3363d4e834
!1249 Add GNN dataset processing API
...
Merge pull request !1249 from heleiwang/hlw_gnn_data
6 years ago
6c21e556c4
Clean up work for text python package
6 years ago
599a449e0b
Support processing GNN data
6 years ago
nhussain
mindspore-ci-bot
islam_amin
xunxue
Junhan Hu
Zirui Wu
ms_yan
hesham
Tinazhang
Yang
qianlong
xiefangqi
liyong
Mahdi
Peilin Wang
Jonathan Yan
jiangzhiwen
jonyguo
Alexey Shevlyakov
yanghaitao
heleiwang
Jamie Nisbet
liuwenhao4
mindspore-ci-bot
Eric
Cathy Wong