Yu Yang
9 years ago
committed by
GitHub
commit
34dc87e2f5
@ -1,81 +1,83 @@
|
||||
Using and Building Docker Images
|
||||
================================
|
||||
PaddlePaddle in Docker Containers
|
||||
=================================
|
||||
|
||||
We release PaddlePaddle in the form of `Docker <https://www.docker.com/>`_ images on `dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_. Running as Docker containers is currently the only officially-supported way to running PaddlePaddle.
|
||||
Docker container is currently the only officially-supported way to
|
||||
running PaddlePaddle. This is reasonable as Docker now runs on all
|
||||
major operating systems including Linux, Mac OS X, and Windows.
|
||||
Please be aware that you will need to change `Dockers settings
|
||||
<https://github.com/PaddlePaddle/Paddle/issues/627>`_ to make full use
|
||||
of your hardware resource on Mac OS X and Windows.
|
||||
|
||||
Run Docker images
|
||||
-----------------
|
||||
|
||||
For each version of PaddlePaddle, we release 4 variants of Docker images:
|
||||
CPU-only and GPU Images
|
||||
-----------------------
|
||||
|
||||
+-----------------+-------------+-------+
|
||||
| | CPU AVX | GPU |
|
||||
+=================+=============+=======+
|
||||
| cpu | yes | no |
|
||||
+-----------------+-------------+-------+
|
||||
| cpu-noavx | no | no |
|
||||
+-----------------+-------------+-------+
|
||||
| gpu | yes | yes |
|
||||
+-----------------+-------------+-------+
|
||||
| gpu-noavx | no | yes |
|
||||
+-----------------+-------------+-------+
|
||||
For each version of PaddlePaddle, we release 2 Docker images, a
|
||||
CPU-only one and a CUDA GPU one. We do so by configuring
|
||||
`dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_
|
||||
automatically runs the following commands:
|
||||
|
||||
We run the following command on Linux to check if the CPU supports :code:`AVX`.
|
||||
.. code-block:: base
|
||||
|
||||
.. code-block:: bash
|
||||
docker build -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile .
|
||||
docker build -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu .
|
||||
|
||||
if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
|
||||
|
||||
On Mac OS X, we need to run
|
||||
To run the CPU-only image as an interactive container:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sysctl -a | grep machdep.cpu.leaf7_features
|
||||
|
||||
docker run -it --rm paddledev/paddle:cpu-latest /bin/bash
|
||||
|
||||
Once we determine the proper variant, we can cope with the Docker image tag name by appending the version number. For example, the following command runs the AVX-enabled image of the most recent version:
|
||||
or, we can run it as a daemon container
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -it --rm paddledev/paddle:cpu-latest /bin/bash
|
||||
docker run -d -p 2202:22 paddledev/paddle:cpu-latest
|
||||
|
||||
To run a GPU-enabled image, you need to install CUDA and let Docker knows about it:
|
||||
and SSH to this container using password :code:`root`:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
|
||||
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
|
||||
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest
|
||||
|
||||
The default entry point of all our Docker images starts the OpenSSH server. To run PaddlePaddle and to expose OpenSSH port to 2202 on the host computer:
|
||||
ssh -p 2202 root@localhost
|
||||
|
||||
.. code-block:: bash
|
||||
An advantage of using SSH is that we can connect to PaddlePaddle from
|
||||
more than one terminals. For example, one terminal running vi and
|
||||
another one running Python interpreter. Another advantage is that we
|
||||
can run the PaddlePaddle container on a remote server and SSH to it
|
||||
from a laptop.
|
||||
|
||||
docker run -d -p 2202:22 paddledev/paddle:cpu-latest
|
||||
|
||||
Then we can login to the container using username :code:`root` and password :code:`root`:
|
||||
Above methods work with the GPU image too -- just please don't forget
|
||||
to install CUDA driver and let Docker knows about it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ssh -p 2202 root@localhost
|
||||
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
|
||||
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
|
||||
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest
|
||||
|
||||
|
||||
Build Docker images
|
||||
-------------------
|
||||
Non-AVX Images
|
||||
--------------
|
||||
|
||||
Developers might want to build Docker images from their local commit or from a tagged version. Suppose that your local repo is at :code:`~/work/Paddle`, the following steps builds a cpu variant from your current work:
|
||||
Please be aware that the CPU-only and the GPU images both use the AVX
|
||||
instruction set, but old computers produced before 2008 do not support
|
||||
AVX. The following command checks if your Linux computer supports
|
||||
AVX:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/Paddle
|
||||
./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant.
|
||||
docker build -t paddle:latest -f ./paddle/scripts/docker/Dockerfile.cpu
|
||||
if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
|
||||
|
||||
|
||||
As a release engineer, you might want to build Docker images for a certain version and publish them to dockerhub.com. You can do this by switching to the right Git tag, or create a new tag, before running `docker build`. For example, the following commands build Docker images for v0.9.0:
|
||||
If it doesn't, we will need to build non-AVX images manually from
|
||||
source code:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/Paddle
|
||||
git checkout tags/v0.9.0
|
||||
./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant.
|
||||
docker build -t paddle:cpu-v0.9.0 -f ./paddle/scripts/docker/Dockerfile.cpu
|
||||
cd ~
|
||||
git clone github.com/PaddlePaddle/Paddle
|
||||
cd Paddle
|
||||
docker build --build-arg WITH_AVX=OFF -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile .
|
||||
docker build --build-arg WITH_AVX=OFF -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu .
|
||||
|
||||
@ -0,0 +1,32 @@
|
||||
FROM nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04
|
||||
MAINTAINER PaddlePaddle Authors <paddle-dev@baidu.com>
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install -y cmake libprotobuf-dev protobuf-compiler git \
|
||||
libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \
|
||||
python-protobuf python-numpy python-dev swig openssh-server \
|
||||
wget unzip python-matplotlib tar xz-utils bzip2 gzip coreutils \
|
||||
sed grep graphviz libjpeg-dev zlib1g-dev doxygen && \
|
||||
apt-get clean -y
|
||||
RUN pip install BeautifulSoup docopt PyYAML pillow \
|
||||
'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark
|
||||
|
||||
ARG WITH_AVX
|
||||
ENV WITH_AVX=${WITH_AVX:-ON}
|
||||
ENV WITH_GPU=ON
|
||||
|
||||
RUN mkdir /paddle
|
||||
COPY . /paddle/
|
||||
RUN /paddle/paddle/scripts/docker/build.sh
|
||||
|
||||
RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile
|
||||
RUN pip install /usr/local/opt/paddle/share/wheels/*.whl
|
||||
RUN paddle version # print version after build
|
||||
|
||||
# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service
|
||||
RUN mkdir /var/run/sshd
|
||||
RUN echo 'root:root' | chpasswd
|
||||
RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config
|
||||
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
|
||||
EXPOSE 22
|
||||
CMD ["/usr/sbin/sshd", "-D"]
|
||||
@ -1,24 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
cd `dirname $0`
|
||||
|
||||
m4 -DPADDLE_WITH_GPU=OFF \
|
||||
-DPADDLE_WITH_AVX=ON \
|
||||
-DPADDLE_BASE_IMAGE=ubuntu:14.04 \
|
||||
Dockerfile.m4 > Dockerfile.cpu
|
||||
|
||||
m4 -DPADDLE_WITH_GPU=OFF \
|
||||
-DPADDLE_WITH_AVX=OFF \
|
||||
-DPADDLE_BASE_IMAGE=ubuntu:14.04 \
|
||||
Dockerfile.m4 > Dockerfile.cpu-noavx
|
||||
|
||||
m4 -DPADDLE_WITH_GPU=ON \
|
||||
-DPADDLE_WITH_AVX=ON \
|
||||
-DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \
|
||||
Dockerfile.m4 > Dockerfile.gpu
|
||||
|
||||
m4 -DPADDLE_WITH_GPU=ON \
|
||||
-DPADDLE_WITH_AVX=OFF \
|
||||
-DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \
|
||||
Dockerfile.m4 > Dockerfile.gpu-noavx
|
||||
Loading…
Reference in new issue