Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix-7717
Conflicts: doc/api/v2/fluid/layers.rst python/paddle/v2/fluid/layers/nn.py python/paddle/v2/fluid/tests/test_layers.pyemailweixu-patch-1
yangyaming
8 years ago
commit
8c81439e24
@ -0,0 +1,51 @@
|
||||
# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserve.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
include(ExternalProject)
|
||||
|
||||
set(BOOST_PROJECT "extern_boost")
|
||||
set(BOOST_VER "1.66.0")
|
||||
set(BOOST_TAR "boost_1_66_0")
|
||||
set(BOOST_URL "https://dl.bintray.com/boostorg/release/${BOOST_VER}/source/${BOOST_TAR}.tar.gz")
|
||||
set(BOOST_SOURCES_DIR ${THIRD_PARTY_PATH}/boost)
|
||||
set(BOOST_DOWNLOAD_DIR "${BOOST_SOURCES_DIR}/src/${BOOST_PROJECT}")
|
||||
set(BOOST_INCLUDE_DIR "${BOOST_DOWNLOAD_DIR}/${BOOST_TAR}" CACHE PATH "boost include directory." FORCE)
|
||||
|
||||
include_directories(${BOOST_INCLUDE_DIR})
|
||||
|
||||
ExternalProject_Add(
|
||||
${BOOST_PROJECT}
|
||||
${EXTERNAL_PROJECT_LOG_ARGS}
|
||||
DOWNLOAD_DIR ${BOOST_DOWNLOAD_DIR}
|
||||
DOWNLOAD_COMMAND wget --no-check-certificate ${BOOST_URL} -c -q -O ${BOOST_TAR}.tar.gz
|
||||
&& tar zxf ${BOOST_TAR}.tar.gz
|
||||
DOWNLOAD_NO_PROGRESS 1
|
||||
PREFIX ${BOOST_SOURCES_DIR}
|
||||
CONFIGURE_COMMAND ""
|
||||
BUILD_COMMAND ""
|
||||
INSTALL_COMMAND ""
|
||||
UPDATE_COMMAND ""
|
||||
)
|
||||
|
||||
if (${CMAKE_VERSION} VERSION_LESS "3.3.0")
|
||||
set(dummyfile ${CMAKE_CURRENT_BINARY_DIR}/boost_dummy.c)
|
||||
file(WRITE ${dummyfile} "const char *dummy = \"${dummyfile}\";")
|
||||
add_library(boost STATIC ${dummyfile})
|
||||
else()
|
||||
add_library(boost INTERFACE)
|
||||
endif()
|
||||
|
||||
add_dependencies(boost ${BOOST_PROJECT})
|
||||
list(APPEND external_project_dependencies boost)
|
||||
set(Boost_INCLUDE_DIR ${BOOST_INCLUDE_DIR})
|
||||
@ -1,34 +0,0 @@
|
||||
Introduction
|
||||
==============
|
||||
DataProvider is a module that loads training or testing data into cpu or gpu
|
||||
memory for the following triaining or testing process.
|
||||
|
||||
For simple use, users can use Python :code:`PyDataProvider` to dynamically reads
|
||||
the original data in any format or in any form, and then transfer them into a
|
||||
data format PaddlePaddle requires. The process is extremly flexible and highly
|
||||
customized, with sacrificing the efficiency only a little. This is extremly
|
||||
useful when you have to dynamically generate certain kinds of data according to,
|
||||
for example, the training performance.
|
||||
|
||||
Besides, users also can customize a C++ :code:`DataProvider` for a more
|
||||
complex usage, or for a higher efficiency.
|
||||
|
||||
The following parameters are required to define in the PaddlePaddle network
|
||||
configuration file (trainer_config.py): which DataProvider is chosen to used,
|
||||
and specific parameters for DataProvider, including training file list
|
||||
(train.list) and testing file list (test.list).
|
||||
|
||||
Train.list and test.list are simply two plain text files, which defines path
|
||||
of training or testing data. It is recommended that directly placing them into
|
||||
the training directory, and reference to them by using a relative path (
|
||||
relative to the PaddePaddle program).
|
||||
|
||||
Testing or evaluating will not be performed during training if the test.list is
|
||||
not set or set to None. Otherwise, PaddlePaddle will evaluate the trained model
|
||||
by the specified tesing data while training, every testing period (a user
|
||||
defined command line parameter in PaddlePaddle) to prevent over-fitting.
|
||||
|
||||
Each line of train.list and test.list is an absolute or relative path (relative
|
||||
to the PaddePaddle program runtime) of data file. Fascinatingly more, each line
|
||||
can also be a HDFS file path or a SQL connection string. As long as the user
|
||||
assures how to access each file in DataProvider.
|
||||
@ -1,249 +0,0 @@
|
||||
.. _api_pydataprovider2:
|
||||
|
||||
PyDataProvider2
|
||||
===============
|
||||
|
||||
We highly recommand users to use PyDataProvider2 to provide training or testing
|
||||
data to PaddlePaddle. The user only needs to focus on how to read a single
|
||||
sample from the original data file by using PyDataProvider2, leaving all of the
|
||||
trivial work, including, transfering data into cpu/gpu memory, shuffle, binary
|
||||
serialization to PyDataProvider2. PyDataProvider2 uses multithreading and a
|
||||
fanscinating but simple cache strategy to optimize the efficiency of the data
|
||||
providing process.
|
||||
|
||||
DataProvider for the non-sequential model
|
||||
-----------------------------------------
|
||||
|
||||
Here we use the MNIST handwriting recognition data as an example to illustrate
|
||||
how to write a simple PyDataProvider.
|
||||
|
||||
MNIST is a handwriting classification data set. It contains 70,000 digital
|
||||
grayscale images. Labels of the training sample range from 0 to 9. All the
|
||||
images have been size-normalized and centered into images with the same size
|
||||
of 28 x 28 pixels.
|
||||
|
||||
A small part of the original data as an example is shown as below:
|
||||
|
||||
.. literalinclude:: src/mnist_train.txt
|
||||
|
||||
Each line of the data contains two parts, separated by :code:`;`. The first part is
|
||||
label of an image. The second part contains 28x28 pixel float values.
|
||||
|
||||
Just write path of the above data into train.list. It looks like this:
|
||||
|
||||
.. literalinclude:: src/train.list
|
||||
|
||||
The corresponding dataprovider is shown as below:
|
||||
|
||||
.. literalinclude:: src/mnist_provider.dict.py
|
||||
|
||||
The first line imports PyDataProvider2 package.
|
||||
The main function is the process function, that has two parameters.
|
||||
The first parameter is the settings, which is not used in this example.
|
||||
The second parameter is the filename, that is exactly each line of train.list.
|
||||
This parameter is passed to the process function by PaddlePaddle.
|
||||
|
||||
:code:`@provider` is a Python
|
||||
`Decorator <http://www.learnpython.org/en/Decorators>`_ .
|
||||
It sets some properties to DataProvider, and constructs a real PaddlePaddle
|
||||
DataProvider from a very simple user implemented python function. It does not
|
||||
matter if you are not familiar with `Decorator`_. You can keep it simple by
|
||||
just taking :code:`@provider` as a fixed mark above the provider function you
|
||||
implemented.
|
||||
|
||||
`input_types`_ defines the data format that a DataProvider returns.
|
||||
In this example, it is set to a 28x28-dimensional dense vector and an integer
|
||||
scalar, whose value ranges from 0 to 9.
|
||||
`input_types`_ can be set to several kinds of input formats, please refer to the
|
||||
document of `input_types`_ for more details.
|
||||
|
||||
|
||||
The process method is the core part to construct a real DataProvider in
|
||||
PaddlePaddle. It implements how to open the text file, how to read one sample
|
||||
from the original text file, convert them into `input_types`_, and give them
|
||||
back to PaddlePaddle process at line 23.
|
||||
Note that data yielded by the process function must follow the same order that
|
||||
`input_types`_ are defined.
|
||||
|
||||
|
||||
With the help of PyDataProvider2, user can focus on how to generate ONE traning
|
||||
sample by using keywords :code:`yield`.
|
||||
:code:`yield` is a python keyword, and a concept related to it includes
|
||||
:code:`generator`.
|
||||
|
||||
Only a few lines of codes need to be added into the training configuration file,
|
||||
you can take this as an example.
|
||||
|
||||
.. literalinclude:: src/mnist_config.py
|
||||
|
||||
Here we specify training data by :code:`train.list`, and no testing data is specified.
|
||||
The method which actually provide data is :code:`process`.
|
||||
|
||||
User also can use another style to provide data, which defines the
|
||||
:code:`data_layer`'s name explicitly when `yield`. For example,
|
||||
the :code:`dataprovider` is shown as below.
|
||||
|
||||
.. literalinclude:: src/mnist_provider.dict.py
|
||||
:linenos:
|
||||
|
||||
If user did't give the :code:`data_layer`'s name, PaddlePaddle will use
|
||||
the order of :code:`data_layer` definition roughly to determine which feature to
|
||||
which :code:`data_layer`. This order may be not correct, so TO DEFINE THE
|
||||
:code:`data_layer`'s NAMES EXPLICITLY IS THE RECOMMANDED WAY TO PROVIDER DATA.
|
||||
|
||||
Now, this simple example of using PyDataProvider is finished.
|
||||
The only thing that the user should know is how to generte **one sample** from
|
||||
**one data file**.
|
||||
And PaddlePadle will do all of the rest things\:
|
||||
|
||||
* Form a training batch
|
||||
* Shuffle the training data
|
||||
* Read data with multithreading
|
||||
* Cache the training data (Optional)
|
||||
* CPU-> GPU double buffering.
|
||||
|
||||
Is this cool?
|
||||
|
||||
.. _api_pydataprovider2_sequential_model:
|
||||
|
||||
DataProvider for the sequential model
|
||||
-------------------------------------
|
||||
A sequence model takes sequences as its input. A sequence is made up of several
|
||||
timesteps. The so-called timestep, is not necessary to have something to do
|
||||
with time. It can also be explained to that the order of data are taken into
|
||||
consideration into model design and training.
|
||||
For example, the sentence can be interpreted as a kind of sequence data in NLP
|
||||
tasks.
|
||||
|
||||
Here is an example on data proivider for English sentiment classification data.
|
||||
The original input data are simple English text, labeled into positive or
|
||||
negative sentiment (marked by 0 and 1 respectively).
|
||||
|
||||
A small part of the original data as an example can be found in the path below:
|
||||
|
||||
.. literalinclude:: src/sentimental_train.txt
|
||||
|
||||
The corresponding data provider can be found in the path below:
|
||||
|
||||
.. literalinclude:: src/sentimental_provider.py
|
||||
|
||||
This data provider for sequential model is a little more complex than that
|
||||
for MINST dataset.
|
||||
A new initialization method is introduced here.
|
||||
The method :code:`on_init` is configured to DataProvider by :code:`@provider`'s
|
||||
:code:`init_hook` parameter, and it will be invoked once DataProvider is
|
||||
initialized. The :code:`on_init` function has the following parameters:
|
||||
|
||||
* The first parameter is the settings object.
|
||||
* The rest parameters are passed by key word arguments. Some of them are passed
|
||||
by PaddlePaddle, see reference for `init_hook`_.
|
||||
The :code:`dictionary` object is a python dict object passed from the trainer
|
||||
configuration file, and it maps word string to word id.
|
||||
|
||||
To pass these parameters into DataProvider, the following lines should be added
|
||||
into trainer configuration file.
|
||||
|
||||
.. literalinclude:: src/sentimental_config.py
|
||||
|
||||
The definition is basically same as MNIST example, except:
|
||||
* Load dictionary in this configuration
|
||||
* Pass it as a parameter to the DataProvider
|
||||
|
||||
The `input_types` is configured in method :code:`on_init`. It has the same
|
||||
effect to configure them by :code:`@provider`'s :code:`input_types` parameter.
|
||||
However, the :code:`input_types` is set at runtime, so we can set it to
|
||||
different types according to the input data. Input of the neural network is a
|
||||
sequence of word id, so set :code:`seq_type` to :code:`integer_value_sequence`.
|
||||
|
||||
Durning :code:`on_init`, we save :code:`dictionary` variable to
|
||||
:code:`settings`, and it will be used in :code:`process`. Note the settings
|
||||
parameter for the process function and for the on_init's function are a same
|
||||
object.
|
||||
|
||||
The basic processing logic is the same as MNIST's :code:`process` method. Each
|
||||
sample in the data file is given back to PaddlePaddle process.
|
||||
|
||||
Thus, the basic usage of PyDataProvider is here.
|
||||
Please refer to the following section reference for details.
|
||||
|
||||
Reference
|
||||
---------
|
||||
|
||||
@provider
|
||||
+++++++++
|
||||
|
||||
.. autofunction:: paddle.trainer.PyDataProvider2.provider
|
||||
|
||||
input_types
|
||||
+++++++++++
|
||||
|
||||
PaddlePaddle has four data types, and three sequence types.
|
||||
The four data types are:
|
||||
|
||||
* :code:`dense_vector`: dense float vector.
|
||||
* :code:`sparse_binary_vector`: sparse binary vector, most of the value is 0, and
|
||||
the non zero elements are fixed to 1.
|
||||
* :code:`sparse_float_vector`: sparse float vector, most of the value is 0, and some
|
||||
non zero elements can be any float value. They are given by the user.
|
||||
* :code:`integer`: an integer scalar, that is especially used for label or word index.
|
||||
|
||||
The three sequence types are:
|
||||
|
||||
* :code:`SequenceType.NO_SEQUENCE` means the sample is not a sequence.
|
||||
* :code:`SequenceType.SEQUENCE` means the sample is a sequence.
|
||||
* :code:`SequenceType.SUB_SEQUENCE` means it is a nested sequence, that each timestep of
|
||||
the input sequence is also a sequence.
|
||||
|
||||
Different input type has a defferenct input format. Their formats are shown
|
||||
in the above table.
|
||||
|
||||
+----------------------+---------------------+-----------------------------------+------------------------------------------------+
|
||||
| | NO_SEQUENCE | SEQUENCE | SUB_SEQUENCE |
|
||||
+======================+=====================+===================================+================================================+
|
||||
| dense_vector | [f, f, ...] | [[f, ...], [f, ...], ...] | [[[f, ...], ...], [[f, ...], ...],...] |
|
||||
+----------------------+---------------------+-----------------------------------+------------------------------------------------+
|
||||
| sparse_binary_vector | [i, i, ...] | [[i, ...], [i, ...], ...] | [[[i, ...], ...], [[i, ...], ...],...] |
|
||||
+----------------------+---------------------+-----------------------------------+------------------------------------------------+
|
||||
| sparse_float_vector | [(i,f), (i,f), ...] | [[(i,f), ...], [(i,f), ...], ...] | [[[(i,f), ...], ...], [[(i,f), ...], ...],...] |
|
||||
+----------------------+---------------------+-----------------------------------+------------------------------------------------+
|
||||
| integer_value | i | [i, i, ...] | [[i, ...], [i, ...], ...] |
|
||||
+----------------------+---------------------+-----------------------------------+------------------------------------------------+
|
||||
|
||||
where f represents a float value, i represents an integer value.
|
||||
|
||||
init_hook
|
||||
+++++++++
|
||||
|
||||
init_hook is a function that is invoked once the data provoder is initialized.
|
||||
Its parameters lists as follows:
|
||||
|
||||
* The first parameter is a settings object, which is the same to :code:`settings`
|
||||
in :code:`process` method. The object contains several attributes, including:
|
||||
|
||||
* :code:`settings.input_types`: the input types. Reference `input_types`_.
|
||||
* :code:`settings.logger`: a logging object.
|
||||
|
||||
* The rest parameters are the key word arguments. It is made up of PaddpePaddle
|
||||
pre-defined parameters and user defined parameters.
|
||||
|
||||
* PaddlePaddle-defined parameters including:
|
||||
|
||||
* :code:`is_train` is a bool parameter that indicates the DataProvider is used in
|
||||
training or testing.
|
||||
* :code:`file_list` is the list of all files.
|
||||
|
||||
* User-defined parameters args can be set in training configuration.
|
||||
|
||||
Note, PaddlePaddle reserves the right to add pre-defined parameter, so please
|
||||
use :code:`**kwargs` in init_hook to ensure compatibility by accepting the
|
||||
parameters which your init_hook does not use.
|
||||
|
||||
cache
|
||||
+++++
|
||||
DataProvider provides two simple cache strategy. They are:
|
||||
|
||||
* :code:`CacheType.NO_CACHE` means do not cache any data, then data is read at runtime by
|
||||
the user implemented python module every pass.
|
||||
* :code:`CacheType.CACHE_PASS_IN_MEM` means the first pass reads data by the user
|
||||
implemented python module, and the rest passes will directly read data from
|
||||
memory.
|
||||
@ -1,24 +0,0 @@
|
||||
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserve.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from paddle.trainer_config_helpers import *
|
||||
|
||||
define_py_data_sources2(
|
||||
train_list='train.list',
|
||||
test_list=None,
|
||||
module='mnist_provider',
|
||||
obj='process')
|
||||
|
||||
img = data_layer(name='pixel', size=784)
|
||||
label = data_layer(name='label', size=10)
|
||||
@ -1,38 +0,0 @@
|
||||
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserve.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from paddle.trainer.PyDataProvider2 import *
|
||||
|
||||
|
||||
# Define a py data provider
|
||||
@provider(
|
||||
input_types={'pixel': dense_vector(28 * 28),
|
||||
'label': integer_value(10)})
|
||||
def process(settings, filename): # settings is not used currently.
|
||||
f = open(filename, 'r') # open one of training file
|
||||
|
||||
for line in f: # read each line
|
||||
label, pixel = line.split(';')
|
||||
|
||||
# get features and label
|
||||
pixels_str = pixel.split(' ')
|
||||
|
||||
pixels_float = []
|
||||
for each_pixel_str in pixels_str:
|
||||
pixels_float.append(float(each_pixel_str))
|
||||
|
||||
# give data to paddle.
|
||||
yield {"pixel": pixels_float, 'label': int(label)}
|
||||
|
||||
f.close() # close file
|
||||
File diff suppressed because one or more lines are too long
@ -1,28 +0,0 @@
|
||||
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserve.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from paddle.trainer_config_helpers import *
|
||||
|
||||
dictionary = dict()
|
||||
... # read dictionary from outside
|
||||
|
||||
define_py_data_sources2(
|
||||
train_list='train.list',
|
||||
test_list=None,
|
||||
module='sentimental_provider',
|
||||
obj='process',
|
||||
# above codes same as mnist sample.
|
||||
args={ # pass to provider.
|
||||
'dictionary': dictionary
|
||||
})
|
||||
@ -1,57 +0,0 @@
|
||||
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserve.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from paddle.trainer.PyDataProvider2 import *
|
||||
|
||||
|
||||
def on_init(settings, dictionary, **kwargs):
|
||||
# on_init will invoke when data provider is initialized. The dictionary
|
||||
# is passed from trainer_config, and is a dict object with type
|
||||
# (word string => word id).
|
||||
|
||||
# set input types in runtime. It will do the same thing as
|
||||
# @provider(input_types) will do, but it is set dynamically during runtime.
|
||||
settings.input_types = {
|
||||
# The text is a sequence of integer values, and each value is a word id.
|
||||
# The whole sequence is the sentences that we want to predict its
|
||||
# sentimental.
|
||||
'data': integer_value_sequence(len(dictionary)), # text input
|
||||
'label': integer_value(2) # label positive/negative
|
||||
}
|
||||
|
||||
# save dictionary as settings.dictionary.
|
||||
# It will be used in process method.
|
||||
settings.dictionary = dictionary
|
||||
|
||||
|
||||
@provider(init_hook=on_init)
|
||||
def process(settings, filename):
|
||||
f = open(filename, 'r')
|
||||
|
||||
for line in f: # read each line of file
|
||||
label, sentence = line.split('\t') # get label and sentence
|
||||
words = sentence.split(' ') # get words
|
||||
|
||||
# convert word string to word id
|
||||
# the word not in dictionary will be ignored.
|
||||
word_ids = []
|
||||
|
||||
for each_word in words:
|
||||
if each_word in settings.dictionary:
|
||||
word_ids.append(settings.dictionary[each_word])
|
||||
|
||||
# give data to paddle.
|
||||
yield word_ids, int(label)
|
||||
|
||||
f.close()
|
||||
@ -1,3 +0,0 @@
|
||||
0 I saw this movie at the AFI Dallas festival . It all takes place at a lake house and it looks wonderful .
|
||||
1 This documentary makes you travel all around the globe . It contains rare and stunning sequels from the wilderness .
|
||||
...
|
||||
@ -1 +0,0 @@
|
||||
mnist_train.txt
|
||||
@ -1,37 +0,0 @@
|
||||
API中文手册
|
||||
============
|
||||
|
||||
DataProvider API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
data_provider/dataprovider_cn.rst
|
||||
data_provider/pydataprovider2_cn.rst
|
||||
|
||||
.. _api_trainer_config:
|
||||
|
||||
Model Config API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
trainer_config_helpers/optimizers.rst
|
||||
trainer_config_helpers/data_sources.rst
|
||||
trainer_config_helpers/layers.rst
|
||||
trainer_config_helpers/activations.rst
|
||||
trainer_config_helpers/poolings.rst
|
||||
trainer_config_helpers/networks.rst
|
||||
trainer_config_helpers/evaluators.rst
|
||||
trainer_config_helpers/attrs.rst
|
||||
|
||||
|
||||
Applications API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
predict/swig_py_paddle_cn.rst
|
||||
@ -1,37 +0,0 @@
|
||||
API
|
||||
===
|
||||
|
||||
DataProvider API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
data_provider/dataprovider_en.rst
|
||||
data_provider/pydataprovider2_en.rst
|
||||
|
||||
.. _api_trainer_config:
|
||||
|
||||
Model Config API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
trainer_config_helpers/optimizers.rst
|
||||
trainer_config_helpers/data_sources.rst
|
||||
trainer_config_helpers/layers.rst
|
||||
trainer_config_helpers/activations.rst
|
||||
trainer_config_helpers/poolings.rst
|
||||
trainer_config_helpers/networks.rst
|
||||
trainer_config_helpers/evaluators.rst
|
||||
trainer_config_helpers/attrs.rst
|
||||
|
||||
|
||||
Applications API
|
||||
----------------
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
predict/swig_py_paddle_en.rst
|
||||
@ -1,135 +0,0 @@
|
||||
# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from py_paddle import swig_paddle, DataProviderConverter
|
||||
from paddle.trainer.PyDataProvider2 import dense_vector
|
||||
from paddle.trainer.config_parser import parse_config
|
||||
|
||||
TEST_DATA = [[[
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.215686, 0.533333, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.67451, 0.992157, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0.070588, 0.886275, 0.992157, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.192157,
|
||||
0.070588, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.670588, 0.992157,
|
||||
0.992157, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.117647, 0.933333, 0.858824, 0.313725,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.090196, 0.858824, 0.992157, 0.831373, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0.141176, 0.992157, 0.992157, 0.611765, 0.054902, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0.258824, 0.992157, 0.992157, 0.529412, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0.368627, 0.992157, 0.992157, 0.419608, 0.003922, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0.094118, 0.835294, 0.992157, 0.992157, 0.517647, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0.603922, 0.992157, 0.992157, 0.992157, 0.603922,
|
||||
0.545098, 0.043137, 0, 0, 0, 0, 0, 0, 0, 0.447059, 0.992157, 0.992157,
|
||||
0.956863, 0.062745, 0, 0, 0, 0, 0, 0, 0, 0, 0.011765, 0.666667, 0.992157,
|
||||
0.992157, 0.992157, 0.992157, 0.992157, 0.745098, 0.137255, 0, 0, 0, 0, 0,
|
||||
0.152941, 0.866667, 0.992157, 0.992157, 0.521569, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0.070588, 0.992157, 0.992157, 0.992157, 0.803922, 0.352941, 0.745098,
|
||||
0.992157, 0.945098, 0.317647, 0, 0, 0, 0, 0.580392, 0.992157, 0.992157,
|
||||
0.764706, 0.043137, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.070588, 0.992157, 0.992157,
|
||||
0.776471, 0.043137, 0, 0.007843, 0.27451, 0.882353, 0.941176, 0.176471, 0,
|
||||
0, 0.180392, 0.898039, 0.992157, 0.992157, 0.313725, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0.070588, 0.992157, 0.992157, 0.713725, 0, 0, 0, 0, 0.627451,
|
||||
0.992157, 0.729412, 0.062745, 0, 0.509804, 0.992157, 0.992157, 0.776471,
|
||||
0.035294, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.494118, 0.992157, 0.992157,
|
||||
0.968627, 0.168627, 0, 0, 0, 0.423529, 0.992157, 0.992157, 0.364706, 0,
|
||||
0.717647, 0.992157, 0.992157, 0.317647, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0.533333, 0.992157, 0.984314, 0.945098, 0.603922, 0, 0, 0, 0.003922,
|
||||
0.466667, 0.992157, 0.988235, 0.976471, 0.992157, 0.992157, 0.788235,
|
||||
0.007843, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.686275, 0.882353, 0.364706, 0,
|
||||
0, 0, 0, 0, 0, 0.098039, 0.588235, 0.992157, 0.992157, 0.992157, 0.980392,
|
||||
0.305882, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.101961, 0.67451, 0.321569,
|
||||
0, 0, 0, 0, 0, 0, 0, 0.105882, 0.733333, 0.976471, 0.811765, 0.713725, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.65098, 0.992157, 0.321569, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0.25098, 0.007843, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,
|
||||
0.94902, 0.219608, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0.968627, 0.764706, 0.152941, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.498039, 0.25098, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0
|
||||
]], [[
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0.298039, 0.333333, 0.333333, 0.333333, 0.337255,
|
||||
0.333333, 0.333333, 0.109804, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0.027451, 0.223529, 0.776471, 0.964706, 0.988235, 0.988235, 0.988235,
|
||||
0.992157, 0.988235, 0.988235, 0.780392, 0.098039, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0.14902, 0.698039, 0.988235, 0.992157, 0.988235, 0.901961,
|
||||
0.87451, 0.568627, 0.882353, 0.976471, 0.988235, 0.988235, 0.501961, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.188235, 0.647059, 0.988235, 0.988235,
|
||||
0.745098, 0.439216, 0.098039, 0, 0, 0, 0.572549, 0.988235, 0.988235,
|
||||
0.988235, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2, 0.933333, 0.992157,
|
||||
0.941176, 0.247059, 0, 0, 0, 0, 0, 0, 0.188235, 0.898039, 0.992157,
|
||||
0.992157, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.039216, 0.639216, 0.933333,
|
||||
0.988235, 0.913725, 0.278431, 0, 0, 0, 0, 0, 0, 0, 0.113725, 0.843137,
|
||||
0.988235, 0.988235, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.235294, 0.988235,
|
||||
0.992157, 0.988235, 0.815686, 0.07451, 0, 0, 0, 0, 0, 0, 0, 0.333333,
|
||||
0.988235, 0.988235, 0.552941, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.211765,
|
||||
0.878431, 0.988235, 0.992157, 0.701961, 0.329412, 0.109804, 0, 0, 0, 0, 0,
|
||||
0, 0, 0.698039, 0.988235, 0.913725, 0.145098, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0.188235, 0.890196, 0.988235, 0.988235, 0.745098, 0.047059, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0.882353, 0.988235, 0.568627, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2,
|
||||
0.933333, 0.992157, 0.992157, 0.992157, 0.447059, 0.294118, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0.447059, 0.992157, 0.768627, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0.623529, 0.988235, 0.988235, 0.988235, 0.988235, 0.992157, 0.47451, 0, 0,
|
||||
0, 0, 0, 0, 0, 0.188235, 0.933333, 0.87451, 0.509804, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0.992157, 0.988235, 0.937255, 0.792157, 0.988235, 0.894118,
|
||||
0.082353, 0, 0, 0, 0, 0, 0, 0.027451, 0.647059, 0.992157, 0.654902, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0.623529, 0.988235, 0.913725, 0.329412, 0.376471,
|
||||
0.184314, 0, 0, 0, 0, 0, 0, 0.027451, 0.513725, 0.988235, 0.635294,
|
||||
0.219608, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.196078, 0.929412, 0.988235,
|
||||
0.988235, 0.741176, 0.309804, 0, 0, 0, 0, 0, 0, 0.529412, 0.988235,
|
||||
0.678431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.223529, 0.992157,
|
||||
0.992157, 1, 0.992157, 0.992157, 0.992157, 0.992157, 1, 0.992157, 0.992157,
|
||||
0.882353, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.023529,
|
||||
0.478431, 0.654902, 0.658824, 0.952941, 0.988235, 0.988235, 0.988235,
|
||||
0.992157, 0.988235, 0.729412, 0.278431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0.196078, 0.647059, 0.764706, 0.764706, 0.768627,
|
||||
0.580392, 0.047059, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
|
||||
0, 0, 0, 0, 0, 0, 0
|
||||
]]]
|
||||
|
||||
|
||||
def main():
|
||||
conf = parse_config("./mnist_model/trainer_config.py", "")
|
||||
print conf.data_config.load_data_args
|
||||
network = swig_paddle.GradientMachine.createFromConfigProto(
|
||||
conf.model_config)
|
||||
assert isinstance(network, swig_paddle.GradientMachine) # For code hint.
|
||||
network.loadParameters("./mnist_model/")
|
||||
converter = DataProviderConverter([dense_vector(784)])
|
||||
inArg = converter(TEST_DATA)
|
||||
print network.forwardTest(inArg)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
swig_paddle.initPaddle("--use_gpu=0")
|
||||
main()
|
||||
@ -1,59 +0,0 @@
|
||||
Python Prediction
|
||||
==================
|
||||
|
||||
PaddlePaddle offers a set of clean prediction interfaces for python with the help of
|
||||
SWIG. The main steps of predict values in python are:
|
||||
|
||||
* Parse training configurations
|
||||
* Construct GradientMachine
|
||||
* Prepare data
|
||||
* Predict
|
||||
|
||||
Here is a sample python script that shows the typical prediction process for the
|
||||
MNIST classification problem. A complete sample code could be found at
|
||||
:code:`src_root/doc/ui/predict/predict_sample.py`.
|
||||
|
||||
.. literalinclude:: src/predict_sample.py
|
||||
:language: python
|
||||
:lines: 15-18,90-100,101-104
|
||||
|
||||
The module that does the most of the job is py_paddle.swig_paddle, it's
|
||||
generated by SWIG and has complete documents, for more details you can use
|
||||
python's :code:`help()` function. Let's walk through the above python script:
|
||||
|
||||
* At the beginning, use :code:`swig_paddle.initPaddle()` to initialize
|
||||
PaddlePaddle with command line arguments, for more about command line arguments
|
||||
see :ref:`cmd_detail_introduction` .
|
||||
* Parse the configuration file that is used in training with :code:`parse_config()`.
|
||||
Because data to predict with always have no label, and output of prediction work
|
||||
normally is the output layer rather than the cost layer, so you should modify
|
||||
the configuration file accordingly before using it in the prediction work.
|
||||
* Create a neural network with
|
||||
:code:`swig_paddle.GradientMachine.createFromConfigproto()`, which takes the
|
||||
parsed configuration :code:`conf.model_config` as argument. Then load the
|
||||
trained parameters from the model with :code:`network.loadParameters()`.
|
||||
* Create a data converter object of utility class :code:`DataProviderConverter`.
|
||||
- Note: As swig_paddle can only accept C++ matrices, we offer a utility
|
||||
class DataProviderConverter that can accept the same input data with
|
||||
PyDataProvider2, for more information please refer to document
|
||||
of :ref:`api_pydataprovider2` .
|
||||
* Do the prediction with :code:`forwardTest()`, which takes the converted
|
||||
input data and outputs the activations of the output layer.
|
||||
|
||||
Here is a typical output:
|
||||
|
||||
.. code-block:: text
|
||||
|
||||
[{'id': None, 'value': array([[ 5.53018653e-09, 1.12194102e-05, 1.96644767e-09,
|
||||
1.43630644e-02, 1.51111044e-13, 9.85625684e-01,
|
||||
2.08823112e-10, 2.32777140e-08, 2.00186201e-09,
|
||||
1.15501715e-08],
|
||||
[ 9.99982715e-01, 1.27787406e-10, 1.72296313e-05,
|
||||
1.49316648e-09, 1.36540484e-11, 6.93137714e-10,
|
||||
2.70634608e-08, 3.48565123e-08, 5.25639710e-09,
|
||||
4.48684503e-08]], dtype=float32)}]
|
||||
|
||||
:code:`value` is the output of the output layer, each row represents result of
|
||||
the corresponding row in the input data, each element represents activation of
|
||||
the corresponding neuron in the output layer.
|
||||
|
||||
@ -0,0 +1,96 @@
|
||||
# Design Doc: CSP in PaddlePaddle Fluid
|
||||
|
||||
## Motivation
|
||||
|
||||
Concurrent programming is important for deep learning. Few example applications are:
|
||||
|
||||
1. The main thread keeps reading the next mini-batch while another thread uses the GPU for computing.
|
||||
2. The main thread performs the computation while another thread uploads the local gradients from each trainer to the parameter server.
|
||||
|
||||
Most DL systems, including TensorFlow, Caffe2, and MxNet, can asynchronously execute operators in a graph. However, Fluid doesn't have the concept of a graph at all, as the design goal of Fluid is that of a programming language.
|
||||
|
||||
## Concurrent Programming Models
|
||||
|
||||
There were many concurrent programming models, implemented in various forms:
|
||||
|
||||
| concurrent programming model | implementation |
|
||||
|-----|-----|
|
||||
| mutex | types and functions in standard libraries |
|
||||
| semaphore | types and functions in standard libraries |
|
||||
| communicating sequential processes (CSP) | Go programming language |
|
||||
| actor model | Erlang programming language |
|
||||
| message passing | MPI |
|
||||
| bulk synchronous parallel (BSP) | Pregel distributed programming framework |
|
||||
|
||||
Since Fluid was designed to be a programming language, we would like to implement CSP in Fluid.
|
||||
|
||||
### CSP v.s. Actor Model
|
||||
|
||||
A well-known implementation of Actor Model is the Erlang programming language. In Actor Model, *processes* could send messages to another process and receive messages from another process given the process IDs. We can find the three ingredients, process with ID, send, and recv, in MPI too. Indeed, we can rewrite Erlang programs in Python + MPI with possibly fewer lines of code. Our concern with Actor Model is that it doesn't seem reasonable to implement process management in a programming language's runtime library; instead, it should be the operating systems' responsibility to manage processes and libraries like MPI for send/recv.
|
||||
|
||||
## CSP in Fluid
|
||||
|
||||
Fluid has two fundamental control-flows: *if-else* and *while*. If we are to implement CSP, we need the following:
|
||||
|
||||
1. a new data type: *channel* and operators *send* and *recv*,
|
||||
1. *goroutine* or thread, and
|
||||
1. a new control-flow: select.
|
||||
|
||||
We also need Python wrappers for the above components.
|
||||
|
||||
The type *channel* is conceptually the blocking queue. In Go, its implemented is a [blocking circular queue](https://github.com/golang/go/blob/68ce117cf17b8debf5754bfd476345779b5b6616/src/runtime/chan.go#L31-L50), which supports send and recv.
|
||||
|
||||
The `select` operation has been in OS kernels long before Go language. All Unix kernels implement system calls *poll* and *select*. They monitor multiple file descriptors to see if I/O is possible on any of them. This takes O(N) time. Since Linux 2.6, a new system call, *epoll*, can do the same in O(1) time. In BSD systems, there is a similar system call *kqueue*. Go's Linux implementation uses epoll.
|
||||
|
||||
It might be a good idea to implement Fluid's select using epoll too. In this design doc, we start from the O(N) way, so we could focus on Python binding and the syntax.
|
||||
|
||||
### Type Channel
|
||||
|
||||
Fluid supports many data types:
|
||||
|
||||
1. Tensor,
|
||||
1. Row-sparse Tensor
|
||||
1. LoD Tensor,
|
||||
1. Tensor array, etc
|
||||
|
||||
Each data type is registered in the [`framework.proto`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L117-L127) as an enum value. To add a new type channel, we need to add a new type enum.
|
||||
|
||||
To expose a C++ type to Python, we need to edit the [`pybind.cc`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/pybind/pybind.cc) file. [Here](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/pybind/pybind.cc#L120-L164) is an example how we expose C++ class LoDTensor.
|
||||
|
||||
## Syntax Design
|
||||
|
||||
### Create Channel
|
||||
|
||||
In Go, we create a channel by specifying the element type and buffer size:
|
||||
|
||||
```go
|
||||
ch := make(chan int) // a channel without buffer
|
||||
ch1 := make(chan int, 100) // a channel that can buffer 100 ints.
|
||||
```
|
||||
|
||||
In Fluid, we should be able to do the same:
|
||||
|
||||
```python
|
||||
ch = fluid.make_chan(dtype=INT)
|
||||
ch1 = fluid.make_chan(dtype=INT, 100)
|
||||
```
|
||||
|
||||
In addition to that, we want channels that can hold more complex element types, e.g., Tensors of float16:
|
||||
|
||||
```python
|
||||
ch = fluid.make_chan(dtype=Tensor, etype=float16)
|
||||
```
|
||||
|
||||
or Tensors of Tensors of float16 etc.
|
||||
|
||||
The point here is that we need a consistent way to compose types, like in C++ we can have `Tensor<Tensor<...<float16>...> >`.
|
||||
|
||||
### Send and Recv
|
||||
|
||||
### Select
|
||||
|
||||
## Example Programs
|
||||
|
||||
### 1. RPC between Trainers and Parameter Servers
|
||||
|
||||
### 2. Concurrent Minibatch Loading
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue