m53297601
/
mindspore
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

!11023 Modify DPN about loading ckpt, README and export.py

Browse Source 
From: @penny369
Reviewed-by: @wuxuejian
Signed-off-by:
pull/11023/MERGE
mindspore-ci-bot 5 years ago committed by Gitee
parent f87b5e0cc8 386d2c4d1b
commit edfe70422b
3 changed files with 70 additions and 82 deletions
Show Stats Download Patch File Download Diff File

96
model_zoo/official/cv/dpn/README.md
Unescape View File

@ -22,10 +22,6 @@
- [Model Description](#model-description) - [Model Description](#model-description)
- [Performance](#performance) - [Performance](#performance)
- [Accuracy](#accuracy) - [Accuracy](#accuracy)
- [DPN92 (Pretrain)](#dpn92-pretrain)
- [DPN98 (Pretrain)](#dpn98-pretrain)
- [DPN131 (Pretrain)](#dpn131-pretrain)
- [DPN92 (Fine tune)](#dpn92-fine-tune)
- [DPN92 (Training)](#dpn92-training) - [DPN92 (Training)](#dpn92-training)
- [Efficiency](#efficiency) - [Efficiency](#efficiency)
- [DPN92](#dpn92) - [DPN92](#dpn92)
@ -77,9 +73,7 @@ To run the python scripts in the repository, you need to prepare the environment
- Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend , please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to [ascend@huawei.com](mailto:ascend@huawei.com). Once approved, you can get the resources. - Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend , please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to [ascend@huawei.com](mailto:ascend@huawei.com). Once approved, you can get the resources.
- Python and dependencies - Python and dependencies
- Python3.7 - Python3.7
- Mindspore 1.0.0 - Mindspore 1.1.0
- Easydict
- MXNet 1.6.0 if running the script `param_convert.py`
- For more information, please check the resources below： - For more information, please check the resources below：
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html) - [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html) - [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
@ -124,6 +118,7 @@ The structure of the files in this repository is shown below.
│ └─ lr_scheduler.py // dpn learning rate scheduler │ └─ lr_scheduler.py // dpn learning rate scheduler
├─ eval.py // evaluation script ├─ eval.py // evaluation script
├─ train.py // training script ├─ train.py // training script
├─ export.py // export model
└─ README.md // descriptions about this repository └─ README.md // descriptions about this repository
``` ```
@ -196,7 +191,9 @@ The script will run training in the background, you can view the results through
```text ```text
epoch: 1 step: 40036, loss is 3.6232593 epoch: 1 step: 40036, loss is 3.6232593
Epoch time: 10048893.336, per step time: 250.996 epoch time: 10048893.336 ms, per step time: 250.996 ms
epoch: 2 step: 40036, loss is 3.200775
epoch time: 9306154.456 ms, per step time: 232.445 ms
... ...
``` ```
@ -204,7 +201,7 @@ or as follows (eval_each_epoch = 1):
```text ```text
epoch: 1 step: 40036, loss is 3.6232593 epoch: 1 step: 40036, loss is 3.6232593
Epoch time: 10048893.336, per step time: 250.996 epoch time: 10048893.336 ms, per step time: 250.996 ms
Save the maximum accuracy checkpoint,the accuracy is 0.2629158669225848 Save the maximum accuracy checkpoint,the accuracy is 0.2629158669225848
... ...
``` ```
@ -230,20 +227,10 @@ sh scripts/train_distributed.sh /home/rank_table.json /data/dataset/imagenet/ ..
The above shell script will run distribute training in the background. You can view the results through the file `train_parallel[X]/log.txt` as follows: The above shell script will run distribute training in the background. You can view the results through the file `train_parallel[X]/log.txt` as follows:
```text ```text
train_parallel0/log: epoch: 1 step 5004, loss is 4.5680037
epoch: 1 step 20018, loss is 5.74429988861084 epoch time: 2312519.441 ms, per step time: 462.134 ms
Epoch time: 7908183.789, per step time: 395.054, avg loss: 5.744 epoch: 2 step 5004, loss is 2.964888
train_parallel0/log: Epoch time: 1350398.913 ms, per step time: 369.864 ms
epoch: 2 step 20018, loss is 4.53381872177124
Epoch time: 5036189.547, per step time: 251.583, avg loss: 4.534
...
train_parallel1/log:
poch: 1 step 20018, loss is 5.751555442810059
Epoch time: 7895946.079, per step time: 394.442, avg loss: 5.752
train_parallel1/log:
epoch: 2 step 20018, loss is 4.875896453857422
Epoch time: 5036190.008, per step time: 251.583, avg loss: 4.876
...
... ...
``` ```
@ -262,7 +249,7 @@ sh scripts/eval.sh [device_id] [dataset_dir] [pretrained_ckpt]
For example, you can run the shell command below to launch the validation procedure. For example, you can run the shell command below to launch the validation procedure.
```text ```text
sh scripts/eval.sh 0 /data/dataset/imageNet/ pretrain/dpn92.ckpt sh scripts/eval.sh 0 /data/dataset/imagenet/ pretrain/dpn-180_5004.ckpt
``` ```
The above shell script will run evaluation in the background. You can view the results through the file `eval_log.txt`. The result will be achieved as follows: The above shell script will run evaluation in the background. You can view the results through the file `eval_log.txt`. The result will be achieved as follows:
@ -282,65 +269,14 @@ All results are validated at image size of 224x224. The dataset preprocessing an
### [Accuracy](#contents) ### [Accuracy](#contents)
The `Pretrain` tag in the table above means that the model's weights are converted from MXNet directly without further training. Relatively, the `Fine tune` tag means that the model is fine tuned after converted from MXNet.
#### DPN92 (Pretrain)
| Parameters | Ascend |
| ----------------- | --------------------------- |
| Model Version | DPN92 (Pretrain) |
| Resource | Ascend 910 |
| Uploaded Date | 09/19/2020 (month/day/year) |
| MindSpore Version | 0.5.0 |
| Dataset | ImageNet-1K |
| outputs | probability |
| train performance | Top1:79.12%; Top5:94.49% |
#### DPN98 (Pretrain)
| Parameters | Ascend |
| ----------------- | --------------------------- |
| Model Version | DPN98 (Pretrain) |
| Resource | Ascend 910 |
| Uploaded Date | 09/19/2020 (month/day/year) |
| MindSpore Version | 0.5.0 |
| Dataset | ImageNet-1K |
| outputs | probability |
| train performance | Top1:79.90%; Top5:94.81% |
#### DPN131 (Pretrain)
| Parameters | Ascend |
| ----------------- | --------------------------- |
| Model Version | DPN131 (Pretrain) |
| Resource | Ascend 910 |
| Uploaded Date | 09/19/2020 (month/day/year) |
| MindSpore Version | 0.5.0 |
| Dataset | ImageNet-1K |
| outputs | probability |
| train performance | Top1:79.96%; Top5:94.81% |
#### DPN92 (Fine tune)
| Parameters | Ascend |
| ----------------- | --------------------------- |
| Model Version | DPN92 (Pretrain) |
| Resource | Ascend 910 |
| Uploaded Date | 09/19/2020 (month/day/year) |
| MindSpore Version | 0.5.0 |
| Dataset | ImageNet-1K |
| epochs | 30 |
| outputs | probability |
| train performance | Top1:79.30%; Top5:94.58% |
#### DPN92 (Training) #### DPN92 (Training)
| Parameters | Ascend | | Parameters | Ascend |
| ----------------- | --------------------------- | | ----------------- | --------------------------- |
| Model Version | DPN92 (Train) | | Model Version | DPN92 (Train) |
| Resource | Ascend 910 | | Resource | Ascend 910 |
| Uploaded Date | 11/13/2020 (month/day/year) | | Uploaded Date | 12/20/2020 (month/day/year) |
| MindSpore Version | 1.0.0 | | MindSpore Version | 1.1.0 |
| Dataset | ImageNet-1K | | Dataset | ImageNet-1K |
| epochs | 180 | | epochs | 180 |
| outputs | probability | | outputs | probability |
@ -354,12 +290,12 @@ The `Pretrain` tag in the table above means that the model's weights are convert
| ----------------- | --------------------------------- | | ----------------- | --------------------------------- |
| Model Version | DPN92 | | Model Version | DPN92 |
| Resource | Ascend 910 | | Resource | Ascend 910 |
| Uploaded Date | 09/19/2020 (month/day/year) | | Uploaded Date | 12/20/2020 (month/day/year) |
| MindSpore Version | 0.5.0 | | MindSpore Version | 1.1.0 |
| Dataset | ImageNet-1K | | Dataset | ImageNet-1K |
| batch_size | 32 | | batch_size | 32 |
| outputs | probability | | outputs | probability |
| speed | 1pc:127.90 img/s;8pc:1023.2 img/s | | speed | 1pc:233 ms/step;8pc:240 ms/step |
# [Description of Random Situation](#contents) # [Description of Random Situation](#contents)

4
model_zoo/official/cv/dpn/eval.py
Unescape View File

@ -68,8 +68,8 @@ def dpn_evaluate(args):
# create network # create network
net = dpns[args.backbone](num_classes=args.num_classes) net = dpns[args.backbone](num_classes=args.num_classes)
# load checkpoint # load checkpoint
if os.path.isfile(args.pretrained): load_param_into_net(net, load_checkpoint(args.pretrained))
load_param_into_net(net, load_checkpoint(args.pretrained)) print("load checkpoint from [{}].".format(args.pretrained))
# loss # loss
if args.dataset == "imagenet-1K": if args.dataset == "imagenet-1K":
loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')

52
model_zoo/official/cv/dpn/export.py
Unescape View File

@ -0,0 +1,52 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Export DPN"""
import argparse
import numpy as np
from mindspore import Tensor, context, load_checkpoint, load_param_into_net, export
from src.dpn import dpns
from src.config import config
parser = argparse.ArgumentParser(description="export dpn")
parser.add_argument("--device_id", type=int, default=0, help="device id")
parser.add_argument("--ckpt_file", type=str, required=True, help="dpn ckpt file.")
parser.add_argument("--width", type=int, default=224, help="input width")
parser.add_argument("--height", type=int, default=224, help="input height")
parser.add_argument("--file_name", type=str, default="dpn", help="dpn output file name.")
parser.add_argument("--file_format", type=str, choices=["AIR", "ONNX", "MINDIR"],
default="MINDIR", help="file format")
parser.add_argument("--device_target", type=str, choices=["Ascend", "GPU", "CPU"], default="Ascend",
help="device target")
args = parser.parse_args()
context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target)
if args.device_target == "Ascend":
context.set_context(device_id=args.device_id)
if __name__ == "__main__":
# define net
backbone = config.backbone
num_classes = config.num_classes
net = dpns[backbone](num_classes=num_classes)
# load checkpoint
param_dict = load_checkpoint(args.ckpt_file)
load_param_into_net(net, param_dict)
net.set_train(False)
image = Tensor(np.zeros([config.batch_size, 3, args.height, args.width], np.float32))
export(net, image, file_name=args.file_name, file_format=args.file_format)
Write Preview
Loading…
Cancel
Save