m53297601
/
mindspore
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'b8fc889368'
${ noResults }
mindspore/tests/ut/data/dataset/testTokenizerData
History
xulei2020 18b519ae0f
add sentence piece 		6 years ago
..
1.txt Add UnicodeCharTokenizer for nlp 6 years ago
basic_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
bert_tokenizer.txt BasicTokenizer not case fold on preserverd words 6 years ago
normalize.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
regex_replace.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
regex_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago
sentencepiece_tokenizer.txt add sentence piece 6 years ago
wordpiece_tokenizer.txt Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp 6 years ago