|
|
# Contents
|
|
|
|
|
|
- [NAML Description](#NAML-description)
|
|
|
- [Dataset](#dataset)
|
|
|
- [Environment Requirements](#environment-requirements)
|
|
|
- [Script Description](#script-description)
|
|
|
- [Script and Sample Code](#script-and-sample-code)
|
|
|
- [Training Process](#training-process)
|
|
|
- [Model Export](#model-export)
|
|
|
- [Model Description](#model-description)
|
|
|
- [Performance](#performance)
|
|
|
- [Evaluation Performance](#evaluation-performance)
|
|
|
- [Inference Performance](#evaluation-performance)
|
|
|
- [Description of Random Situation](#description-of-random-situation)
|
|
|
- [ModelZoo Homepage](#modelzoo-homepage)
|
|
|
|
|
|
# [NAML Description](#contents)
|
|
|
|
|
|
NAML is a multi-view news recommendation approach. The core of NAML is a news encoder and a user encoder. The newsencoder is composed of a title encoder, a abstract encoder, a category encoder and a subcategory encoder. In the user encoder, we learn representations of users from their browsed news. Besides, we apply additive attention to learn more informative news and user representations by selecting important words and news.
|
|
|
|
|
|
[Paper](https://arxiv.org/abs/1907.05576) Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang and Xing Xie: Neural News Recommendation with Attentive Multi-View Learning, IJCAI 2019
|
|
|
|
|
|
# [Dataset](#contents)
|
|
|
|
|
|
Dataset used: [MIND](https://msnews.github.io/)
|
|
|
|
|
|
MIND contains about 160k English news articles and more than 15 million impression logs generated by 1 million users.
|
|
|
|
|
|
You can download the dataset and put the directory in structure as follows:
|
|
|
|
|
|
```path
|
|
|
└─MINDlarge
|
|
|
├─MINDlarge_train
|
|
|
├─MINDlarge_dev
|
|
|
└─MINDlarge_utils
|
|
|
```
|
|
|
|
|
|
# [Environment Requirements](#contents)
|
|
|
|
|
|
- Hardware（Ascend/GPU）
|
|
|
- Prepare hardware environment with Ascend, GPU processor.
|
|
|
- Framework
|
|
|
- [MindSpore](https://www.mindspore.cn/install/en)
|
|
|
- For more information, please check the resources below：
|
|
|
- [MindSpore Tutorials](https://www.mindspore.cn/tutorial/training/en/master/index.html)
|
|
|
- [MindSpore Python API](https://www.mindspore.cn/doc/api_python/en/master/index.html)
|
|
|
|
|
|
# [Script description](#contents)
|
|
|
|
|
|
## [Script and sample code](#contents)
|
|
|
|
|
|
```path
|
|
|
├── naml
|
|
|
├── README.md # descriptions about NAML
|
|
|
├── scripts
|
|
|
│ ├──run_train.sh # shell script for training
|
|
|
│ ├──run_eval.sh # shell script for evaluation
|
|
|
│ ├──run_infer_310.sh # shell script for 310 inference
|
|
|
├── src
|
|
|
│ ├──option.py # parse args
|
|
|
│ ├──callback.py # callback file
|
|
|
│ ├──dataset.py # creating dataset
|
|
|
│ ├──naml.py # NAML architecture
|
|
|
│ ├──config.py # config file
|
|
|
│ ├──utils.py # utils to load ckpt_file for fine tune or incremental learn
|
|
|
├──ascend310_infer #application for 310 inference
|
|
|
├── train.py # training script
|
|
|
├── eval.py # evaluation script
|
|
|
├── export.py # export mindir script
|
|
|
└──postprogress.py # post process for 310 inference
|
|
|
```
|
|
|
|
|
|
## [Training process](#contents)
|
|
|
|
|
|
### Usage
|
|
|
|
|
|
You can start training using python or shell scripts. The usage of shell scripts as follows:
|
|
|
|
|
|
```shell
|
|
|
# train standalone
|
|
|
bash run_train.sh [PLATFORM] [DEVICE_ID] [DATASET] [DATASET_PATH]
|
|
|
# train distribute
|
|
|
bash run_distribute_train.sh [PLATFORM] [DEVICE_NUM] [DATASET] [DATASET_PATH] [RANK_TABLE_FILE]
|
|
|
# evaluation
|
|
|
bash run_eval.sh [PLATFORM] [DEVICE_ID] [DATASET] [DATASET_PATH] [CHECKPOINT_PATH]
|
|
|
```
|
|
|
|
|
|
- `PLATFORM` should be Ascend.
|
|
|
- `DEVICE_ID` is the device id you want to run the network.
|
|
|
- `DATASET` MIND dataset, support large, small and demo.
|
|
|
- `DATASET_PATH` is the dataset path, the structure as [Dataset](#dataset).
|
|
|
- `CHECKPOINT_PATH` is a pre-trained checkpoint path.
|
|
|
- `RANK_TABLE_FILE` is HCCL configuration file when running on Ascend.
|
|
|
|
|
|
## [Model Export](#contents)
|
|
|
|
|
|
```shell
|
|
|
python export.py --platform [PLATFORM] --checkpoint_path [CHECKPOINT_PATH] --file_format [EXPORT_FORMAT] --batch_size [BATCH_SIZE]
|
|
|
```
|
|
|
|
|
|
- `EXPORT_FORMAT` should be in ["AIR", "MINDIR"]
|
|
|
|
|
|
# [Model Description](#contents)
|
|
|
|
|
|
## [Performance](#contents)
|
|
|
|
|
|
### Evaluation Performance
|
|
|
|
|
|
| Parameters | Ascend |
|
|
|
| -------------------------- | ------------------------------------------------------------ |
|
|
|
| Model Version | NAML |
|
|
|
| Resource | Ascend 910 ；CPU 2.60GHz，56cores；Memory，314G |
|
|
|
| uploaded Date | 02/23/2021 (month/day/year) |
|
|
|
| MindSpore Version | 1.2.0 |
|
|
|
| Dataset | MINDlarge |
|
|
|
| Training Parameters | epoch=1, steps=52869, batch_size=64, lr=0.001 |
|
|
|
| Optimizer | Adam |
|
|
|
| Loss Function | Softmax Cross Entropy |
|
|
|
| outputs | probability |
|
|
|
| Speed | 1pc: 62 ms/step |
|
|
|
| Total time | 1pc: 54 mins |
|
|
|
|
|
|
### Inference Performance
|
|
|
|
|
|
| Parameters | Ascend |
|
|
|
| ------------------- | --------------------------- |
|
|
|
| Model Version | NAML |
|
|
|
| Resource | Ascend 910 |
|
|
|
| Uploaded Date | 02/23/2021 (month/day/year) |
|
|
|
| MindSpore Version | 1.2.0 |
|
|
|
| Dataset | MINDlarge |
|
|
|
| batch_size | 64 |
|
|
|
| outputs | probability |
|
|
|
| Accuracy | AUC: 0.66 |
|
|
|
|
|
|
### Inference on Ascend310 Performance
|
|
|
|
|
|
| Parameters | Ascend |
|
|
|
| ------------------- | --------------------------- |
|
|
|
| Model Version | NAML |
|
|
|
| Resource | Ascend 310 |
|
|
|
| Uploaded Date | 03/13/2021 (month/day/year) |
|
|
|
| MindSpore Version | 1.2.0 |
|
|
|
| Dataset | MINDlarge |
|
|
|
| batch_size | 64 |
|
|
|
| outputs | probability |
|
|
|
| Accuracy | AUC: 0.667 |
|
|
|
|
|
|
# [Description of Random Situation](#contents)
|
|
|
|
|
|
<!-- In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py. -->
|
|
|
In train.py, we set the seed which is used by numpy.random, mindspore.common.Initializer, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution.
|
|
|
|
|
|
# [ModelZoo Homepage](#contents)
|
|
|
|
|
|
Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).
|
