1aa16fb431
!13999 modify BERT、TinyBERT README and modify TinyBERT network script
...
From: @wang_hua_2019
Reviewed-by: @c_34
Signed-off-by:
4 years ago
e0d85aecea
modify tinybert for MindSpore BERT
4 years ago
da60f433f1
removed the useless link of apply form
4 years ago
669a32355c
!13769 remove control_depend from py file
...
From: @huangbingjian
Reviewed-by: @hwhewei,@zh_qh
Signed-off-by: @zh_qh
4 years ago
eaa7ec5cd2
1. fixed for markdownlint errors.
...
2. fixed spell errors.
4 years ago
72ae1799f3
remove control_depend from py file
4 years ago
c1a619ccfe
add AdamWeightDecayOp
4 years ago
15d37e5db9
imporve convergence of loss in bert
4 years ago
7188a14215
modify api detect_overflow name in TrainOneStepWithLossScaleCell
4 years ago
4d69e21933
optimizer class TrainOneStepWithLossScaleCell
4 years ago
7d9a783993
[auto-monad] Support side-effects by auto-monad
...
The basic idea is: exploits data dependency to control the execution order
of side-effect operations, and keep the semantics of ANF unchanged.
The ControlDepend primitive is removed and there are two primitives added:
1. UpdateState:
```
a = Assign(para, value)
```
became:
```
a = Assign(para, value, u)
u = UpdateState(u, a)
```
2. Load:
```
x = Add(para, value)
```
became:
```
p = Load(para, u)
x = Add(p, value)
u = UpdateState(u, p)
```
4 years ago
3bf5681ebc
!12093 adapt api for optimizer import
...
From: @sl_wang
Reviewed-by: @wang_zi_dong
Signed-off-by:
4 years ago
8f8eee4b5e
bert thor supports lr configuration in config.py
4 years ago
51e7b6481a
!12034 modify instructions of preprocessing msra dataset
...
From: @shibeiji
Reviewed-by: @c_34,@guoqi1024
Signed-off-by: @c_34
4 years ago
1c12b84757
!11992 [Bert][Gpu]Sync modify of bert script from r1.1 to master
...
From: @hanhuifeng2020
Reviewed-by: @gaoxiong1,@anyrenwei
Signed-off-by: @anyrenwei
4 years ago
95ccebb811
fix bugs of MSRA dataset preprocessing
4 years ago
ed4207f698
!12022 Add a new optimizer THOR option to BERT pretrain script.
...
From: @i_yangmh
Reviewed-by: @wang_zi_dong,@kisnwang
Signed-off-by: @kisnwang
4 years ago
0b312c2385
thor generalization code submit
4 years ago
67a4c62b4b
Add new optimizer THOR option to BERT pretrain script.
4 years ago
53d4510ea6
[Bert][Gpu]Sync modify of bert script from r1.1 to master
4 years ago
e897eb4c41
!11915 Change TensorAdd to Add, merge from r1.1 to master
...
From: @liangzhibo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
4 years ago
df265b6d6b
!11912 update submoudle akg, close graph kernel ascend ci testcases
...
From: @looop5
Reviewed-by:
Signed-off-by:
4 years ago
9ec100d069
Change TensorAdd to Add, from r1.1 to master
4 years ago
0161209e40
update submoudle akg, close graph kernel ascend ci testcases
4 years ago
cc18b206c9
bert ner for adaption of MSRA dataset
4 years ago
9fa0499fa0
Change GatherV2 to Gather r1.1 to master
4 years ago
8b1a8a6bc1
bert ner for msra dataset
4 years ago
3ec3f038ad
extract bert embedding tables in construct
4 years ago
4899a0b4c1
fix code spell
4 years ago
8eae3262f3
fix some description about bert and yolov3
4 years ago
52953f16fc
!11027 Use logic id instead of physic id in get_distribute_pretrain_cmd.py
...
From: @c_34
Reviewed-by: @ljl0711,@liangchenghui
Signed-off-by: @liangchenghui
4 years ago
30560be800
!11066 GPU add restrict for bert script
...
From: @VectorSL
Reviewed-by: @gaoxiong1,@dylangeng,@anyrenwei
Signed-off-by: @gaoxiong1
4 years ago
e2b740e812
add physic_logic_ids in get_distribute_pretrain_cmd.py
...
change ways to get physic_logic_ids
4 years ago
c13cd24e38
add restrict for gpu only
4 years ago
84303c539f
!11003 upload chinese readme
...
From: @yoonlee666
Reviewed-by:
Signed-off-by:
4 years ago
2d1e8beec7
add chinese readme
4 years ago
33277fe64b
fix bert init
4 years ago
f0b08e8bff
all reduce after each step in gradients accumulation mode for bert
4 years ago
811a785555
!10886 GPU update bert scripts
...
From: @VectorSL
Reviewed-by: @gaoxiong1,@c_34,@gaoxiong1
Signed-off-by: @c_34
4 years ago
0c97835662
update control flow int adamweightdecay for bert
4 years ago
02c107b0e9
!10874 修改GPU设置device_id的问题
...
From: @yuzhenhua666
Reviewed-by: @c_34,@linqingke
Signed-off-by: @c_34
4 years ago
6b858480c8
fix GPU device_id bug
4 years ago
812b4b0eab
extract embedding table from unified interface
4 years ago
65bfefd92b
bert on gpu for pre training script supports loss scale
4 years ago
31fed1a2f6
change code to import APIs from mindspore.dataset rather than mindspore.dataset.engine
4 years ago
4128fb1155
modify export to support mindir
4 years ago
3988376b67
Performance optimization of Bert on GPU by the graph_kernel
4 years ago
62652cc29d
bert readme update
4 years ago
59ca2ac708
!9693 add multi machine instruction for bert
...
From: @yoonlee666
Reviewed-by:
Signed-off-by:
4 years ago
fe9443bfc6
add multi-machine
4 years ago
mindspore-ci-bot
wang_hua_2019
caojiewen
huangbingjian
VectorSL
chenhaozhe
wangnan39@huawei.com
He Wei
mwang
shibeiji
MingHan-Y
hanhuifeng2020
l00591931
looop5
yoonlee666
yuzhenhua
Xiao Tianci
wilfChen