m53297601
/
mindspore
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'bd777e0710'
${ noResults }
mindspore/tests/ut/data/dataset/testTokenizerData
History
Xiao Tianci 0659473535
add TruncateSequencePair, ToNumber C++ API and enable three test cases 		5 years ago
..
1.txt Add UnicodeCharTokenizer for nlp 6 years ago
basic_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
bert_tokenizer.txt BasicTokenizer not case fold on preserverd words 6 years ago
normalize.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
regex_replace.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
regex_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
sentencepiece_tokenizer.txt add sentence piece 6 years ago
to_number.txt add TruncateSequencePair, ToNumber C++ API and enable three test cases 5 years ago
wordpiece_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago