m53297601
/
mindspore
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '408159e301'
${ noResults }
mindspore/tests/ut/data/dataset/testTokenizerData/regex_tokenizer.txt

3 lines
70 B
Raw Normal View History Unescape

Add WhitespaceTokenizer and UnicodeScriptTokenizer for nlp add CaseFold, NormalizeUTF8 add RegexReplace add RegexTokenizer add BasicTokenizer add WordpieceTokenizer add BertTokenizer
5 years ago
Welcome to Shenzhen!
北京欢迎您!Welcome to Beijing!
12￥+36￥=?