Paddle

Commit Graph

Author	SHA1	Message	Date
wanghuancoder	df43905f12	use iwyu clean include (#27267 ) * use iwyu clean include, test=develop, test=win * compilation error, test=develop * fix compilation error2, test=develop * fix compilation error3, test=develop * fix compilation error4, test=develop * fix compilation error5, test=develop * fix compilation error6, test=develop * fix compilation error7, test=develop * fix compilation error8, test=develop * fix compilation error8, test=develop * fix compilation error10, test=develop * fix compilation error11, test=develop	4 years ago
hutuxian	175954d894	PaddleBox Framework Part2 (#22466 ) * Add two types of Metric Calculator: MultiTaskCalculator & CmatchRankCalculator. * Add a config for DynamicAdjustChannelNum function to denote whether we will discard the remaining instances when they are not be distributed evenly. * Remove CPU code in Pull/PushSparse and we will add it back when testing it fully. * Fix some known issues: such as copying persistable vars after one epoch running.	5 years ago
Thunderbrook	1fe468d319	support debug each output of each ins (#19004 ) * dump slot * test * proto * dump slot * test * proto * code style * code style * code style * style * add delete after unseen days * add unseen days * code style * conflict solve test=develop * add clear model * code style test=develop * code style test=develop * support debug tensor of each ins test=develop * support debug tensor of each ins test=develop * learning rate * code style * code style * code style * code style * code style * code style * code style * code style * code style * code style * code style * code style * code style test=develop * code style test=develop * unitest * style * style * multi phase * add channel * code style * style * style * unitest * style * define * define test=develop * style test=develop * rm define test=develop * linux * linux test=develop * style test=develop * output format test=develop * windows ci test=develop	6 years ago
jiaqi	3f8031e256	dataset (#17973 ) (1) use channel instead of vector/BlockingQueue in Dataset，to keep same with existing implementation, and make code more readable and flexible (dataset single output channel or multi output channel). one previous memory out of limit problem is cause by not release memory after training. (2) add Record because MultiSlotType costs too much memory (80B)，fix memory out of limit problem. (3) add Channel, Archive in paddle/fluid/framework (4) change dataset from shared_ptr to unique_ptr in pybind (5) move create/destroy readers from trainer to dataset (6) move shuffle from datafeed to dataset. dataset holds memory, datafeed is only for load data and feed data to network. (7) fix thread num bug of Dataset when filelist size < thread num (8) support set_queue_num in InMemoryDataset	6 years ago
Xin Pan	ddd60581b7	clean up channel test=develop	6 years ago
Abhinav Arora	7bf82f82b1	Fix CPPlint errors in channel.h, channel_impl.h and channel_test.cc (#9628 ) * Fix cpplint issues * Fix cpplint issues in channel.h and channel_impl.h * Fix typo	7 years ago
Abhinav Arora	65534c4762	Fluid channels should match the semantics of Go Channels (#9265 ) * Fluid Channel should match Go Channel in Semantics * Fix Python channel_send * Address code rveiew feedback * Fix open_files_op.cc * Add description to Channel Asserts	7 years ago
Thuan Nguyen	1e4c504e60	Implement Select OP (#9088 ) * Fix old documentation for channel_recv * Initial design of CSP select * Redesign channel implementation for Select Op * Remove unecessary header * Initial checkin of select op, currently will read all the conditional_op in the cases block and also pull out all channels involved in the select. * Init python select op API * Python select bug fix when checking op creates block * Add case_to_execute as (a) input to select, (b) into the passed inputs into the select op * Add in addition code for select op * Init fibonacci test from python * implement fibonnaci sequence test * update fib unit test * Improve select test cases * Shorten non-pep-8-ed lines * Add methods on channel needed by select op * Fix compile issues, finish implementation, still need to debug code * Fix issue with fibonncci test, it works now! * Change QueueMessage callback to take in an ChannelAction enum, fix select unit test * Fix case attributes * Fix issue with select control flow * Make cases - previously on each selectcase conditional_block - attributes to select * Use class constants for type of channel * Change select op to take in "cases" attribute * return boolean from select callback function to tell Channel if this RECV or SEND should be executed * Improve attributes and inputs comments on select op * Fix issues with python unit test * Assert fibonacci final output * Fix issue when channel name / channel var is null for "default" case in select op * Assert base select test output * Make QueueMessage use shared pointer and modify the order of the callback * Fixing the order in which the callback is called * Move channel utility methods to paddle/fluid/operators/concurrency/channel_util * Create channel_util and move channel util methods * Fix crash when calling select_op * Fix deadlock * Fix issue of channel destructor deadlock * Fix precommit issues * Accidentally checked in changes to beam_search_op, reverting change. * Fix dependency issue in concurrency cmake * add device_context dependency for concurrency target	7 years ago
Abhinav Arora	41894da145	Add changes to channel that are needed for select op (#9084 )	7 years ago
Abhinav Arora	78c884d7a7	Redesign channel implementation for Select Op (#8814 ) * Redesign channel implementation for Select Op * Remove unecessary header * Remove unnecessary comments	7 years ago
Abhinav Arora	0d878e4c09	Add Go_op, Channel_create, channel_close, channel_send and channel_receive ops (#8593 ) * Adding Python boilerplate code for Go op * Add very basic test case * Adding the python logic for go routine * Fix syntax * Changing test to notest * Rename Routine to Go * Combining GoGuard and Go in one class * Modify test * Adding fluid close channel * Fixing __init__.py for calling fluid.go() * Adding stubs for channel methods and updating test case * Removing import * * Adding imports from concurrency * Initial commit of GO_OP (for varun) * Creating local scopes and go through them * Updated go op inputs persistability enforcement * Add thread execution; compile failing though * Fix go op * Cleaned up Go op * Fix yapf format issue * Readd warp ctc dir for unit tests * Updated make_channel, channel_send, channel_recv and channel_close * Moved thread function to another method, update unit tests * remove output var * Add stubs for channel operators * Updating concurrency with signatures * Updated the signature with return status * Fixed dtype in variables * Updating stub of ChannelSend + add infershape * Updating stub of ChannelRecv + add infershape * Updated signature * Adding the channel_create operator * Merge channel send+receive ops * Update concurrency tests using all operators * Updating the create op with ChannelHolder * Fix issues with channel_create_op * Add the implementation for channel_close op * Add channel close operator, fix channel close op * Adding the channel_send op * Comment channels C++ and Python code * Concurrency python api comment fix * Update unit test to add Status variable * Adding channel receive operator * Update concurrency test to demonstrate a complete CSP flow * Fix clang-format issues * Fixed "Out" parameter name * Fixing merge conflict in framework.py * Add channel ops to framework.py no_kernel_op_set * Seperating channel_send and channel_recv operators * Documenting capacity type * Update concurrency test to create go block as child block of main program * Changing set status implementation	7 years ago
Abhinav Arora	6e7fee0ee1	Add unit tests for ChannelHolder (#8510 )	7 years ago
kavyasrinet	77ee8fb240	Exposing Channel to be used as a Variable and integrating with Fluid (#8486 ) * Adding set_capacity method support * Adding Python for make_channel * Updating notest_concurrency * Write python for make_channel method * Write python for make_channel method * Fix make_channel and test * Placeholder ops for channel send, recv and close * Adding ToTypeIndex method to var_type.h * Add var_type.h to channel: * Added POD_Type to the method * Add CHANNEL to executor * Updated get and set DataType to accomodate Channels * Updating get and set to incorporate channels * Adding CHANNEL as supported VarType in protobuf * Removing unecessary import * Fixing VarDesc to adapt to Channel as VarType * Add channel.h to executor * Remove innclude from channel * Updated var_type to support Channel as var type * Adding get_channel to pybind * Added ChannelHolder * Adding make_channel as an op * Adding ChannelHolder in channel * Fixing typo * Commenting out operators in concurrency * Removing totypeid right now since we don't need it. * Reverting python changes * Fixing typo in framework.py * Modify comments for ReaderHolder	7 years ago
Yi Wang	fc374821dd	Correct #include path	7 years ago
Yi Wang	90648f336d	Move file to fluid/; Edit CMakeLists.txt	7 years ago

15 Commits (e50bc2c2a6fcfec748d5bb991588a2fdc2ab4caf)