pull/12200/head
wanyiming 4 years ago
parent c2582dcab9
commit 2be036b4e5

@ -70,21 +70,27 @@ Dataset used: [LibriSpeech](<http://www.openslr.org/12>)
.
├── audio
├── deepspeech2
├── train.py // training scripts
├── eval.py // testing and evaluation outputs
├── export.py // convert mindspore model to mindir model
├── labels.json // possible characters to map to
├── README.md // descriptions about DeepSpeech
├── deepspeech_pytorch //
├──decoder.py // decoder from third party codes(MIT License)
├── scripts
│ ├──run_distribute_train_gpu.sh // launch distributed training with gpu platform(8p)
│ ├──run_eval_cpu.sh // launch evaluation with cpu platform
│ ├──run_eval_gpu.sh // launch evaluation with gpu platform
│ ├──run_standalone_train_cpu.sh // launch standalone training with cpu platform
│ └──run_standalone_train_gpu.sh // launch standalone training with gpu platform(1p)
├── train.py // training scripts
├── eval.py // testing and evaluation outputs
├── export.py // convert mindspore model to mindir model
├── labels.json // possible characters to map to
├── README.md // descriptions about DeepSpeech
├── deepspeech_pytorch //
├──decoder.py // decoder from third party codes(MIT License)
├── src
├──__init__.py
├──DeepSpeech.py // DeepSpeech networks
├──dataset.py // generate dataloader and data processing entry
├──config.py // DeepSpeech configs
├──lr_generator.py // learning rate generator
├──greedydecoder.py // modified greedydecoder for mindspore code
└──callback.py // callbacks to monitor the training
├──DeepSpeech.py // DeepSpeech networks
├──dataset.py // generate dataloader and data processing entry
├──config.py // DeepSpeech configs
├──lr_generator.py // learning rate generator
├──greedydecoder.py // modified greedydecoder for mindspore code
└──callback.py // callbacks to monitor the training
```
@ -219,11 +225,14 @@ After installing MindSpore via the official website and finishing dataset proces
```shell
# standalone training
CUDA_VISIBLE_DEVICES='0' python train.py
# standalone training gpu
sh ./scripts/run_standalone_train_gpu.sh [DEVICE_ID]
# distributed training
CUDA_VISIBLE_DEVICES='0,1,2,3,4,5,6,7' mpirun --allow-run-as-root -n 8 python train.py --is_distributed > log 2>&1 &
# standalone training cpu
sh ./scripts/run_standalone_train_cpu.sh
# distributed training gpu
sh ./scripts/run_distribute_train_gpu.sh
```
@ -233,8 +242,12 @@ deepspeech_pytorch into deepspeech2 directory. After that, the file directory wi
```shell
# eval
CUDA_VISIBLE_DEVICES='0' python eval.py --pretrain_ckpt='saved_model_path'
# eval on cpu
sh ./scripts/run_eval_cpu.sh [PATH_CHECKPOINT]
# eval on gpu
sh ./scripts/run_eval_gpu.sh [DEVICE_ID] [PATH_CHECKPOINT]
```
## [Export MindIR](#contents)

@ -0,0 +1,17 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
mpirun --allow-run-as-root -n 8 --output-filename log_output --merge-stderr-to-stdout \
python ./train.py --is_distributed --device_target 'GPU' > train.log 2>&1 &

@ -0,0 +1,17 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
PATH_CHECKPOINT=$1
python ./eval.py --pretrain_ckpt $PATH_CHECKPOINT --device_target 'CPU' > eval.log 2>&1 &

@ -0,0 +1,19 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
DEVICE_ID=$1
PATH_CHECKPOINT=$2
CUDA_VISIBLE_DEVICES=$DEVICE_ID python ./eval.py --pretrain_ckpt $PATH_CHECKPOINT \
--device_target 'GPU' > eval.log 2>&1 &

@ -0,0 +1,17 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
python ./train.py --device_target 'CPU' > train.log 2>&1 &

@ -0,0 +1,18 @@
#!/bin/bash
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
DEVICE_ID=$1
CUDA_VISIBLE_DEVICES=$DEVICE_ID python ./train.py --device_target 'GPU' > train.log 2>&1 &
Loading…
Cancel
Save