mindspore/model_zoo/official/cv/alexnet

Contents

• AlexNet Description
• Model Architecture
• Dataset
• Environment Requirements
• Quick Start
• Script Description
  • Script and Sample Code
  • Script Parameters
  • Training Process
    • Training
  • Evaluation Process
    • Evaluation
• Model Description
  • Performance
    • Evaluation Performance
• ModelZoo Homepage

AlexNet Description

AlexNet was proposed in 2012, one of the most influential neural networks. It got big success in ImageNet Dataset recognition than other models.

Paper: Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep ConvolutionalNeural Networks. Advances In Neural Information Processing Systems. 2012.

Model Architecture

AlexNet composition consists of 5 convolutional layers and 3 fully connected layers. Multiple convolutional kernels can extract interesting features in images and get more accurate classification.

Dataset

Dataset used: CIFAR-10

• Dataset size：175M，60,000 32*32 colorful images in 10 classes
  • Train：146M，50,000 images
  • Test：29.3M，10,000 images
• Data format：binary files
  • Note：Data will be processed in dataset.py
• Download the dataset, the directory structure is as follows:

├─cifar-10-batches-bin
│
└─cifar-10-verify-bin

Environment Requirements

• Hardware（Ascend/GPU）
  • Prepare hardware environment with Ascend or GPU processor.
• Framework
  • MindSpore
• For more information, please check the resources below：
  • MindSpore Tutorials
  • MindSpore Python API

Quick Start

After installing MindSpore via the official website, you can start training and evaluation as follows:

# enter script dir, train AlexNet
sh run_standalone_train_ascend.sh [DATA_PATH] [CKPT_SAVE_PATH]
# enter script dir, evaluate AlexNet
sh run_standalone_eval_ascend.sh [DATA_PATH] [CKPT_NAME]

Script Description

Script and Sample Code

├── cv
    ├── alexnet
        ├── README.md                    // descriptions about alexnet
        ├── requirements.txt             // package needed
        ├── scripts
        │   ├──run_standalone_train_gpu.sh             // train in gpu
        │   ├──run_standalone_train_ascend.sh          // train in ascend
        │   ├──run_standalone_eval_gpu.sh             //  evaluate in gpu
        │   ├──run_standalone_eval_ascend.sh          //  evaluate in ascend
        ├── src
        │   ├──dataset.py             // creating dataset
        │   ├──alexnet.py              // alexnet architecture
        │   ├──config.py            // parameter configuration
        ├── train.py               // training script
        ├── eval.py               //  evaluation script

Script Parameters

Major parameters in train.py and config.py as follows:

--data_path: The absolute full path to the train and evaluation datasets.
--epoch_size: Total training epochs.
--batch_size: Training batch size.
--image_height: Image height used as input to the model.
--image_width: Image width used as input the model.
--device_target: Device where the code will be implemented. Optional values are "Ascend", "GPU".
--checkpoint_path: The absolute full path to the checkpoint file saved after training.
--data_path: Path where the dataset is saved

Training Process

Training

• running on Ascend

  python train.py --data_path cifar-10-batches-bin --ckpt_path ckpt > log.txt 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_train_ascend.sh cifar-10-batches-bin ckpt
  

  After training, the loss value will be achieved as follows:

  # grep "loss is " train.log
  epoch: 1 step: 1, loss is 2.2791853
  ...
  epoch: 1 step: 1536, loss is 1.9366643
  epoch: 1 step: 1537, loss is 1.6983616
  epoch: 1 step: 1538, loss is 1.0221305
  ...
  

  The model checkpoint will be saved in the current directory.

• running on GPU

  python train.py --device_target "GPU" --data_path cifar-10-batches-bin --ckpt_path ckpt > log.txt 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_train_for_gpu.sh cifar-10-batches-bin ckpt
  

  After training, the loss value will be achieved as follows:

  # grep "loss is " train.log
  epoch: 1 step: 1, loss is 2.3125906
  ...
  epoch: 30 step: 1560, loss is 0.6687547
  epoch: 30 step: 1561, loss is 0.20055409
  epoch: 30 step: 1561, loss is 0.103845775
  

Evaluation Process

Evaluation

Before running the command below, please check the checkpoint path used for evaluation.

• running on Ascend

  python eval.py --data_path cifar-10-verify-bin --ckpt_path ckpt/checkpoint_alexnet-1_1562.ckpt > log.txt 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_eval_ascend.sh cifar-10-verify-bin ckpt/checkpoint_alexnet-1_1562.ckpt
  

  You can view the results through the file "log.txt". The accuracy of the test dataset will be as follows:

  # grep "Accuracy: " log.txt
  'Accuracy': 0.8832
  

• running on GPU

  python eval.py --device_target "GPU" --data_path cifar-10-verify-bin --ckpt_path ckpt/checkpoint_alexnet-30_1562.ckpt > log.txt 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_eval_for_gpu.sh cifar-10-verify-bin ckpt/checkpoint_alexnet-30_1562.ckpt
  

  You can view the results through the file "log.txt". The accuracy of the test dataset will be as follows:

  # grep "Accuracy: " log.txt
  'Accuracy': 0.88512
  

Model Description

Performance

Evaluation Performance

Parameters	Ascend	GPU
Resource	Ascend 910; CPU 2.60GHz, 192cores; Memory, 755G	NV SMX2 V100-32G
uploaded Date	06/09/2020 (month/day/year)	17/09/2020 (month/day/year)
MindSpore Version	0.5.0-beta	0.7.0-beta
Dataset	CIFAR-10	CIFAR-10
Training Parameters	epoch=30, steps=1562, batch_size = 32, lr=0.002	epoch=30, steps=1562, batch_size = 32, lr=0.002
Optimizer	Momentum	Momentum
Loss Function	Softmax Cross Entropy	Softmax Cross Entropy
outputs	probability	probability
Loss	0.0016	0.01
Speed	21 ms/step	16.8 ms/step
Total time	17 mins	14 mins
Checkpoint for Fine tuning	445M (.ckpt file)	445M (.ckpt file)
Scripts	https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/alexnet	https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/cv/alexnet

Description of Random Situation

In dataset.py, we set the seed inside create_dataset function.

ModelZoo Homepage

Please check the official homepage.