Luo Tao
7 years ago
parent
2f41aaa492
commit
d1d8257fdf
@ -1,61 +1,7 @@
|
||||
GET STARTED
|
||||
============
|
||||
|
||||
.. _quick_install:
|
||||
|
||||
Quick Install
|
||||
----------------------
|
||||
|
||||
You can use pip to install PaddlePaddle with a single command, supports
|
||||
CentOS 6 above, Ubuntu 14.04 above or MacOS 10.12, with Python 2.7 installed.
|
||||
Simply run the following command to install, the version is cpu_avx_openblas:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle
|
||||
|
||||
If you need to install GPU version (cuda7.5_cudnn5_avx_openblas), run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle-gpu
|
||||
|
||||
For more details about installation and build:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
build_and_install/index_en.rst
|
||||
|
||||
|
||||
.. _quick_start:
|
||||
|
||||
Quick Start
|
||||
++++++++
|
||||
|
||||
Create a new file called housing.py, and paste this Python
|
||||
code:
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import paddle.v2 as paddle
|
||||
|
||||
# Initialize PaddlePaddle.
|
||||
paddle.init(use_gpu=False, trainer_count=1)
|
||||
|
||||
# Configure the neural network.
|
||||
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
|
||||
y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
|
||||
|
||||
# Infer using provided test data.
|
||||
probs = paddle.infer(
|
||||
output_layer=y_predict,
|
||||
parameters=paddle.dataset.uci_housing.model(),
|
||||
input=[item for item in paddle.dataset.uci_housing.test()()])
|
||||
|
||||
for i in xrange(len(probs)):
|
||||
print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
|
||||
|
||||
Run :code:`python housing.py` and voila! It should print out a list of predictions
|
||||
for the test housing data.
|
||||
quickstart_en.rst
|
||||
|
||||
@ -0,0 +1,45 @@
|
||||
Quick Start
|
||||
============
|
||||
|
||||
You can use pip to install PaddlePaddle with a single command, supports
|
||||
CentOS 6 above, Ubuntu 14.04 above or MacOS 10.12, with Python 2.7 installed.
|
||||
Simply run the following command to install, the version is cpu_avx_openblas:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle
|
||||
|
||||
If you need to install GPU version (cuda7.5_cudnn5_avx_openblas), run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install paddlepaddle-gpu
|
||||
|
||||
For more details about installation and build: :ref:`install_steps` .
|
||||
|
||||
Create a new file called housing.py, and paste this Python
|
||||
code:
|
||||
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import paddle.v2 as paddle
|
||||
|
||||
# Initialize PaddlePaddle.
|
||||
paddle.init(use_gpu=False, trainer_count=1)
|
||||
|
||||
# Configure the neural network.
|
||||
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
|
||||
y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
|
||||
|
||||
# Infer using provided test data.
|
||||
probs = paddle.infer(
|
||||
output_layer=y_predict,
|
||||
parameters=paddle.dataset.uci_housing.model(),
|
||||
input=[item for item in paddle.dataset.uci_housing.test()()])
|
||||
|
||||
for i in xrange(len(probs)):
|
||||
print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
|
||||
|
||||
Run :code:`python housing.py` and voila! It should print out a list of predictions
|
||||
for the test housing data.
|
||||
@ -0,0 +1,10 @@
|
||||
分布式训练
|
||||
==========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
introduction_cn.md
|
||||
preparations_cn.md
|
||||
cmd_argument_cn.md
|
||||
multi_cluster/index_cn.rst
|
||||
@ -0,0 +1,10 @@
|
||||
Distributed Training
|
||||
====================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
introduction_en.md
|
||||
preparations_en.md
|
||||
cmd_argument_en.md
|
||||
multi_cluster/index_en.rst
|
||||
@ -0,0 +1,13 @@
|
||||
## Introduction
|
||||
|
||||
In this section, we'll explain how to run distributed training jobs with PaddlePaddle on different types of clusters. The diagram below shows the main architecture of a distributed trainning job:
|
||||
|
||||
<img src="https://user-images.githubusercontent.com/13348433/31772146-41523d84-b511-11e7-8a12-a69fd136c283.png" width="500">
|
||||
|
||||
- Data shard: training data will be split into multiple partitions, trainers use the partitions of the whole dataset to do the training job.
|
||||
- Trainer: each trainer reads the data shard, and train the neural network. Then the trainer will upload calculated "gradients" to parameter servers, and wait for parameters to be optimized on the parameter server side. When that finishes, the trainer download optimized parameters and continues its training.
|
||||
- Parameter server: every parameter server stores part of the whole neural network model data. They will do optimization calculations when gradients are uploaded from trainers, and then send updated parameters to trainers.
|
||||
|
||||
PaddlePaddle can support both synchronize stochastic gradient descent (SGD) and asynchronous SGD.
|
||||
|
||||
When training with synchronize SGD, PaddlePaddle uses an internal "synchronize barrier" which makes gradients update and parameter download in strict order. On the other hand, asynchronous SGD won't wait for all trainers to finish upload at a single step, this will increase the parallelism of distributed training: parameter servers do not depend on each other, they'll do parameter optimization concurrently. Parameter servers will not wait for trainers, so trainers will also do their work concurrently. But asynchronous SGD will introduce more randomness and noises in the gradient.
|
||||
@ -0,0 +1,19 @@
|
||||
Use different clusters
|
||||
======================
|
||||
|
||||
PaddlePaddle supports running jobs on several platforms including:
|
||||
- `Kubernetes <http://kubernetes.io>`_ open-source system for automating deployment, scaling, and management of containerized applications from Google.
|
||||
- `OpenMPI <https://www.open-mpi.org>`_ Mature high performance parallel computing framework.
|
||||
- `Fabric <http://www.fabfile.org>`_ A cluster management tool. Write scripts to submit jobs or manage the cluster.
|
||||
|
||||
We'll introduce cluster job management on these platforms. The examples can be found under `cluster_train_v2 <https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/scripts/cluster_train_v2>`_ .
|
||||
|
||||
These cluster platforms provide API or environment variables for training processes, when the job is dispatched to different nodes. Like node ID, IP or total number of nodes etc.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
fabric_en.md
|
||||
openmpi_en.md
|
||||
k8s_en.md
|
||||
k8s_aws_en.md
|
||||
@ -0,0 +1,17 @@
|
||||
## Preparations
|
||||
|
||||
1. Prepare your computer cluster. It's normally a bunch of Linux servers connected by LAN. Each server will be assigned a unique IP address. The computers in the cluster can be called "nodes".
|
||||
2. Install PaddlePaddle on every node. If you are going to take advantage of GPU cards, you'll also need to install proper driver and CUDA libraries. To install PaddlePaddle please read [this build and install](http://www.paddlepaddle.org/docs/develop/documentation/en/getstarted/build_and_install/index_en.html) document. We strongly recommend using [Docker installation](http://www.paddlepaddle.org/docs/develop/documentation/en/getstarted/build_and_install/docker_install_en.html).
|
||||
|
||||
After installation, you can check the version by typing the below command (run a docker container if using docker: `docker run -it paddlepaddle/paddle:[tag] /bin/bash`):
|
||||
|
||||
```bash
|
||||
$ paddle version
|
||||
PaddlePaddle 0.10.0rc, compiled with
|
||||
with_avx: ON
|
||||
with_gpu: OFF
|
||||
with_double: OFF
|
||||
with_python: ON
|
||||
with_rdma: OFF
|
||||
with_timer: OFF
|
||||
```
|
||||
Loading…
Reference in new issue