- [Create MindDataset By MindRecord](#create-minddataset-by-mindrecord)
- [Create MindDataset By MindRecord](#create-minddataset-by-mindrecord)
<!-- /TOC -->
<!-- /TOC -->
## What does the example do
## What does the example do
This example is used to read data from aclImdb dataset and generate mindrecord. It just transfers the aclImdb dataset to mindrecord without any data preprocessing. You can modify the example or follow the example to implement your own example.
This example is used to read data from aclImdb dataset and generate mindrecord. It just transfers the aclImdb dataset to mindrecord without any data preprocessing. You can modify the example or refer to the example to implement your own example.
1. run.sh: generate MindRecord entry script.
1. run.sh: generate MindRecord entry script.
- gen_mindrecord.py : read the aclImdb data and transfer it to mindrecord.
- gen_mindrecord.py : read the aclImdb data and transfer it to mindrecord.
2. run_read.py: create MindDataset by MindRecord entry script.
2. run_read.py: create MindDataset by MindRecord entry script.
- create_dataset.py: use MindDataset to read MindRecord to generate dataset.
- create_dataset.py: use MindDataset to read MindRecord to generate dataset.
## How to use the example to generate MindRecord
## How to use the example to generate MindRecord
@ -30,6 +29,7 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
> [aclImdb dataset download address](http://ai.stanford.edu/~amaas/data/sentiment/) **-> Large Movie Review Dataset v1.0**
> [aclImdb dataset download address](http://ai.stanford.edu/~amaas/data/sentiment/) **-> Large Movie Review Dataset v1.0**
2. Unzip the training data to dir example/nlp_to_mindrecord/aclImdb/data.
2. Unzip the training data to dir example/nlp_to_mindrecord/aclImdb/data.
```
```
tar -zxvf aclImdb_v1.tar.gz -C {your-mindspore}/example/nlp_to_mindrecord/aclImdb/data/
tar -zxvf aclImdb_v1.tar.gz -C {your-mindspore}/example/nlp_to_mindrecord/aclImdb/data/
```
```
@ -37,11 +37,13 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
### Generate MindRecord
### Generate MindRecord
1. Run the run.sh script.
1. Run the run.sh script.
```bash
```bash
bash run.sh
bash run.sh
```
```
2. Output like this:
2. Output like this:
```
```
...
...
>> begin generate mindrecord by train data
>> begin generate mindrecord by train data
@ -77,6 +79,7 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min