!24 Synchronization code to ms-incubator
Merge pull request !24 from changzherui/sy-codepull/915/head
mindspore-ci-bot
5 years ago
committed by
Gitee
commit
22cc03a54a
@ -0,0 +1,26 @@
|
|||||||
|
<!-- Thanks for sending a pull request! Here are some tips for you:
|
||||||
|
|
||||||
|
If this is your first time, please read our contributor guidelines: https://gitee.com/mindspore/mindspore/blob/master/CONTRIBUTING.md
|
||||||
|
-->
|
||||||
|
|
||||||
|
**What type of PR is this?**
|
||||||
|
> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
|
||||||
|
>
|
||||||
|
> /kind bug
|
||||||
|
> /kind task
|
||||||
|
> /kind feature
|
||||||
|
|
||||||
|
|
||||||
|
**What does this PR do / why do we need it**:
|
||||||
|
|
||||||
|
|
||||||
|
**Which issue(s) this PR fixes**:
|
||||||
|
<!--
|
||||||
|
*Automatically closes linked issue when PR is merged.
|
||||||
|
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
|
||||||
|
-->
|
||||||
|
Fixes #
|
||||||
|
|
||||||
|
**Special notes for your reviewers**:
|
||||||
|
|
||||||
|
|
||||||
@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: RFC
|
||||||
|
about: Use this template for the new feature or enhancement
|
||||||
|
labels: kind/feature or kind/enhancement
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Background
|
||||||
|
- Describe the status of the problem you wish to solve
|
||||||
|
- Attach the relevant issue if have
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
- Describe the general solution, design and/or pseudo-code
|
||||||
|
|
||||||
|
## Trail
|
||||||
|
| No. | Task Description | Related Issue(URL) |
|
||||||
|
| --- | ---------------- | ------------------ |
|
||||||
|
| 1 | | |
|
||||||
|
| 2 | | |
|
||||||
@ -0,0 +1,43 @@
|
|||||||
|
---
|
||||||
|
name: Bug Report
|
||||||
|
about: Use this template for reporting a bug
|
||||||
|
labels: kind/bug
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
<!-- Thanks for sending an issue! Here are some tips for you:
|
||||||
|
|
||||||
|
If this is your first time, please read our contributor guidelines: https://github.com/mindspore-ai/mindspore/blob/master/CONTRIBUTING.md
|
||||||
|
-->
|
||||||
|
|
||||||
|
## Environment
|
||||||
|
### Hardware Environment(`Ascend`/`GPU`/`CPU`):
|
||||||
|
> Uncomment only one ` /device <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
|
||||||
|
>
|
||||||
|
> `/device ascend`</br>
|
||||||
|
> `/device gpu`</br>
|
||||||
|
> `/device cpu`</br>
|
||||||
|
|
||||||
|
### Software Environment:
|
||||||
|
- **MindSpore version (source or binary)**:
|
||||||
|
- **Python version (e.g., Python 3.7.5)**:
|
||||||
|
- **OS platform and distribution (e.g., Linux Ubuntu 16.04)**:
|
||||||
|
- **GCC/Compiler version (if compiled from source)**:
|
||||||
|
|
||||||
|
## Describe the current behavior
|
||||||
|
|
||||||
|
|
||||||
|
## Describe the expected behavior
|
||||||
|
|
||||||
|
|
||||||
|
## Steps to reproduce the issue
|
||||||
|
1.
|
||||||
|
2.
|
||||||
|
3.
|
||||||
|
|
||||||
|
## Related log / screenshot
|
||||||
|
|
||||||
|
|
||||||
|
## Special notes for this issue
|
||||||
|
|
||||||
|
|
||||||
@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: Task
|
||||||
|
about: Use this template for task tracking
|
||||||
|
labels: kind/task
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task Description
|
||||||
|
|
||||||
|
|
||||||
|
## Task Goal
|
||||||
|
|
||||||
|
|
||||||
|
## Sub Task
|
||||||
|
| No. | Task Description | Issue ID |
|
||||||
|
| --- | ---------------- | -------- |
|
||||||
|
| 1 | | |
|
||||||
|
| 2 | | |
|
||||||
|
|
||||||
@ -0,0 +1,24 @@
|
|||||||
|
<!-- Thanks for sending a pull request! Here are some tips for you:
|
||||||
|
|
||||||
|
If this is your first time, please read our contributor guidelines: https://github.com/mindspore-ai/mindspore/blob/master/CONTRIBUTING.md
|
||||||
|
-->
|
||||||
|
|
||||||
|
**What type of PR is this?**
|
||||||
|
> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
|
||||||
|
>
|
||||||
|
> `/kind bug`</br>
|
||||||
|
> `/kind task`</br>
|
||||||
|
> `/kind feature`</br>
|
||||||
|
|
||||||
|
**What does this PR do / why do we need it**:
|
||||||
|
|
||||||
|
|
||||||
|
**Which issue(s) this PR fixes**:
|
||||||
|
<!--
|
||||||
|
*Automatically closes linked issue when PR is merged.
|
||||||
|
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
|
||||||
|
-->
|
||||||
|
Fixes #
|
||||||
|
|
||||||
|
**Special notes for your reviewers**:
|
||||||
|
|
||||||
|
Before Width: | Height: | Size: 121 KiB After Width: | Height: | Size: 35 KiB |
@ -1,55 +0,0 @@
|
|||||||
# Copyright 2020 Huawei Technologies Co., Ltd
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
# ============================================================================
|
|
||||||
|
|
||||||
"""
|
|
||||||
network config setting, will be used in main.py
|
|
||||||
"""
|
|
||||||
|
|
||||||
from easydict import EasyDict as edict
|
|
||||||
import mindspore.common.dtype as mstype
|
|
||||||
from mindspore.model_zoo.Bert_NEZHA import BertConfig
|
|
||||||
bert_cfg = edict({
|
|
||||||
'epoch_size': 10,
|
|
||||||
'num_warmup_steps': 0,
|
|
||||||
'start_learning_rate': 1e-4,
|
|
||||||
'end_learning_rate': 1,
|
|
||||||
'decay_steps': 1000,
|
|
||||||
'power': 10.0,
|
|
||||||
'save_checkpoint_steps': 2000,
|
|
||||||
'keep_checkpoint_max': 10,
|
|
||||||
'checkpoint_prefix': "checkpoint_bert",
|
|
||||||
'DATA_DIR' = "/your/path/examples.tfrecord"
|
|
||||||
'SCHEMA_DIR' = "/your/path/datasetSchema.json"
|
|
||||||
'bert_config': BertConfig(
|
|
||||||
batch_size=16,
|
|
||||||
seq_length=128,
|
|
||||||
vocab_size=21136,
|
|
||||||
hidden_size=1024,
|
|
||||||
num_hidden_layers=24,
|
|
||||||
num_attention_heads=16,
|
|
||||||
intermediate_size=4096,
|
|
||||||
hidden_act="gelu",
|
|
||||||
hidden_dropout_prob=0.0,
|
|
||||||
attention_probs_dropout_prob=0.0,
|
|
||||||
max_position_embeddings=512,
|
|
||||||
type_vocab_size=2,
|
|
||||||
initializer_range=0.02,
|
|
||||||
use_relative_positions=True,
|
|
||||||
input_mask_from_dataset=True,
|
|
||||||
token_type_ids_from_dataset=True,
|
|
||||||
dtype=mstype.float32,
|
|
||||||
compute_type=mstype.float16,
|
|
||||||
)
|
|
||||||
})
|
|
||||||
@ -0,0 +1,57 @@
|
|||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
"""
|
||||||
|
network config setting, will be used in train.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
from easydict import EasyDict as edict
|
||||||
|
import mindspore.common.dtype as mstype
|
||||||
|
from mindspore.model_zoo.Bert_NEZHA import BertConfig
|
||||||
|
bert_train_cfg = edict({
|
||||||
|
'epoch_size': 10,
|
||||||
|
'num_warmup_steps': 0,
|
||||||
|
'start_learning_rate': 1e-4,
|
||||||
|
'end_learning_rate': 0.0,
|
||||||
|
'decay_steps': 1000,
|
||||||
|
'power': 10.0,
|
||||||
|
'save_checkpoint_steps': 2000,
|
||||||
|
'keep_checkpoint_max': 10,
|
||||||
|
'checkpoint_prefix': "checkpoint_bert",
|
||||||
|
# please add your own dataset path
|
||||||
|
'DATA_DIR': "/your/path/examples.tfrecord",
|
||||||
|
# please add your own dataset schema path
|
||||||
|
'SCHEMA_DIR': "/your/path/datasetSchema.json"
|
||||||
|
})
|
||||||
|
bert_net_cfg = BertConfig(
|
||||||
|
batch_size=16,
|
||||||
|
seq_length=128,
|
||||||
|
vocab_size=21136,
|
||||||
|
hidden_size=1024,
|
||||||
|
num_hidden_layers=24,
|
||||||
|
num_attention_heads=16,
|
||||||
|
intermediate_size=4096,
|
||||||
|
hidden_act="gelu",
|
||||||
|
hidden_dropout_prob=0.0,
|
||||||
|
attention_probs_dropout_prob=0.0,
|
||||||
|
max_position_embeddings=512,
|
||||||
|
type_vocab_size=2,
|
||||||
|
initializer_range=0.02,
|
||||||
|
use_relative_positions=True,
|
||||||
|
input_mask_from_dataset=True,
|
||||||
|
token_type_ids_from_dataset=True,
|
||||||
|
dtype=mstype.float32,
|
||||||
|
compute_type=mstype.float16,
|
||||||
|
)
|
||||||
@ -0,0 +1,32 @@
|
|||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
network config setting, will be used in train.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
from easydict import EasyDict as edict
|
||||||
|
|
||||||
|
alexnet_cfg = edict({
|
||||||
|
'num_classes': 10,
|
||||||
|
'learning_rate': 0.002,
|
||||||
|
'momentum': 0.9,
|
||||||
|
'epoch_size': 1,
|
||||||
|
'batch_size': 32,
|
||||||
|
'buffer_size': 1000,
|
||||||
|
'image_height': 227,
|
||||||
|
'image_width': 227,
|
||||||
|
'save_checkpoint_steps': 1562,
|
||||||
|
'keep_checkpoint_max': 10,
|
||||||
|
})
|
||||||
@ -0,0 +1,54 @@
|
|||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
Produce the dataset
|
||||||
|
"""
|
||||||
|
|
||||||
|
from config import alexnet_cfg as cfg
|
||||||
|
import mindspore.dataset as ds
|
||||||
|
import mindspore.dataset.transforms.c_transforms as C
|
||||||
|
import mindspore.dataset.transforms.vision.c_transforms as CV
|
||||||
|
from mindspore.common import dtype as mstype
|
||||||
|
|
||||||
|
|
||||||
|
def create_dataset(data_path, batch_size=32, repeat_size=1, status="train"):
|
||||||
|
"""
|
||||||
|
create dataset for train or test
|
||||||
|
"""
|
||||||
|
cifar_ds = ds.Cifar10Dataset(data_path)
|
||||||
|
rescale = 1.0 / 255.0
|
||||||
|
shift = 0.0
|
||||||
|
|
||||||
|
resize_op = CV.Resize((cfg.image_height, cfg.image_width))
|
||||||
|
rescale_op = CV.Rescale(rescale, shift)
|
||||||
|
normalize_op = CV.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
|
||||||
|
if status == "train":
|
||||||
|
random_crop_op = CV.RandomCrop([32, 32], [4, 4, 4, 4])
|
||||||
|
random_horizontal_op = CV.RandomHorizontalFlip()
|
||||||
|
channel_swap_op = CV.HWC2CHW()
|
||||||
|
typecast_op = C.TypeCast(mstype.int32)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="label", operations=typecast_op)
|
||||||
|
if status == "train":
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=random_crop_op)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=random_horizontal_op)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=resize_op)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=rescale_op)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=normalize_op)
|
||||||
|
cifar_ds = cifar_ds.map(input_columns="image", operations=channel_swap_op)
|
||||||
|
|
||||||
|
cifar_ds = cifar_ds.shuffle(buffer_size=cfg.buffer_size)
|
||||||
|
cifar_ds = cifar_ds.batch(batch_size, drop_remainder=True)
|
||||||
|
cifar_ds = cifar_ds.repeat(repeat_size)
|
||||||
|
return cifar_ds
|
||||||
@ -0,0 +1,58 @@
|
|||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
######################## eval alexnet example ########################
|
||||||
|
eval alexnet according to model file:
|
||||||
|
python eval.py --data_path /YourDataPath --ckpt_path Your.ckpt
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from config import alexnet_cfg as cfg
|
||||||
|
from dataset import create_dataset
|
||||||
|
import mindspore.nn as nn
|
||||||
|
from mindspore import context
|
||||||
|
from mindspore.model_zoo.alexnet import AlexNet
|
||||||
|
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||||
|
from mindspore.train import Model
|
||||||
|
from mindspore.nn.metrics import Accuracy
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description='MindSpore AlexNet Example')
|
||||||
|
parser.add_argument('--device_target', type=str, default="Ascend", choices=['Ascend', 'GPU'],
|
||||||
|
help='device where the code will be implemented (default: Ascend)')
|
||||||
|
parser.add_argument('--data_path', type=str, default="./", help='path where the dataset is saved')
|
||||||
|
parser.add_argument('--ckpt_path', type=str, default="./ckpt", help='if is test, must provide\
|
||||||
|
path where the trained ckpt file')
|
||||||
|
parser.add_argument('--dataset_sink_mode', type=bool, default=False, help='dataset_sink_mode is False or True')
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target, enable_mem_reuse=False)
|
||||||
|
|
||||||
|
network = AlexNet(cfg.num_classes)
|
||||||
|
loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction="mean")
|
||||||
|
repeat_size = cfg.epoch_size
|
||||||
|
opt = nn.Momentum(network.trainable_params(), cfg.learning_rate, cfg.momentum)
|
||||||
|
model = Model(network, loss, opt, metrics={"Accuracy": Accuracy()}) # test
|
||||||
|
|
||||||
|
print("============== Starting Testing ==============")
|
||||||
|
param_dict = load_checkpoint(args.ckpt_path)
|
||||||
|
load_param_into_net(network, param_dict)
|
||||||
|
ds_eval = create_dataset(args.data_path,
|
||||||
|
cfg.batch_size,
|
||||||
|
1,
|
||||||
|
"test")
|
||||||
|
acc = model.eval(ds_eval, dataset_sink_mode=args.dataset_sink_mode)
|
||||||
|
print("============== Accuracy:{} ==============".format(acc))
|
||||||
@ -0,0 +1,58 @@
|
|||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
######################## train alexnet example ########################
|
||||||
|
train alexnet and get network model files(.ckpt) :
|
||||||
|
python train.py --data_path /YourDataPath
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from config import alexnet_cfg as cfg
|
||||||
|
from dataset import create_dataset
|
||||||
|
import mindspore.nn as nn
|
||||||
|
from mindspore import context
|
||||||
|
from mindspore.train import Model
|
||||||
|
from mindspore.nn.metrics import Accuracy
|
||||||
|
from mindspore.model_zoo.alexnet import AlexNet
|
||||||
|
from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description='MindSpore AlexNet Example')
|
||||||
|
parser.add_argument('--device_target', type=str, default="Ascend", choices=['Ascend', 'GPU'],
|
||||||
|
help='device where the code will be implemented (default: Ascend)')
|
||||||
|
parser.add_argument('--data_path', type=str, default="./", help='path where the dataset is saved')
|
||||||
|
parser.add_argument('--ckpt_path', type=str, default="./ckpt", help='if is test, must provide\
|
||||||
|
path where the trained ckpt file')
|
||||||
|
parser.add_argument('--dataset_sink_mode', type=bool, default=False, help='dataset_sink_mode is False or True')
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target, enable_mem_reuse=False)
|
||||||
|
|
||||||
|
network = AlexNet(cfg.num_classes)
|
||||||
|
loss = nn.SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=True, reduction="mean")
|
||||||
|
opt = nn.Momentum(network.trainable_params(), cfg.learning_rate, cfg.momentum)
|
||||||
|
model = Model(network, loss, opt, metrics={"Accuracy": Accuracy()}) # test
|
||||||
|
|
||||||
|
print("============== Starting Training ==============")
|
||||||
|
ds_train = create_dataset(args.data_path,
|
||||||
|
cfg.batch_size,
|
||||||
|
cfg.epoch_size,
|
||||||
|
"train")
|
||||||
|
config_ck = CheckpointConfig(save_checkpoint_steps=cfg.save_checkpoint_steps,
|
||||||
|
keep_checkpoint_max=cfg.keep_checkpoint_max)
|
||||||
|
ckpoint_cb = ModelCheckpoint(prefix="checkpoint_alexnet", directory=args.ckpt_path, config=config_ck)
|
||||||
|
model.train(cfg.epoch_size, ds_train, callbacks=[ckpoint_cb, LossMonitor()],
|
||||||
|
dataset_sink_mode=args.dataset_sink_mode)
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue