11 Commits (22fa4c2d2440208870eb94de8694c05f2605cfe8)

Author	SHA1	Message	Date
jiaqi	233746d89d	set fleet_send_batch_num a default value according to trainer num ... (1) set fleet_send_batch_num a default value according to trainer num， the previous 80000 is fixed，if trainer num is much less or larger than 100，global shuffle may have timeout error. (2) fix load one table bug, add barrier	7 years ago
Thunderbrook	52c1431eee	add clear_model interface in fleetwrapper (#18815) ... * dump slot * test * proto * dump slot * test * proto * code style * code style * code style * style * add delete after unseen days * add unseen days * code style * conflict solve test=develop * add clear model * code style test=develop * code style test=develop	7 years ago
fuyinno4	c167a4b4dd	Fix shrink-dense and add scale-datanorm (#18746) ... Fix FleetWrapper: 1. fix shrink dense: just scale show 2. add datanorm scale: divide datanorm's gradient by batch_size	7 years ago
Thunderbrook	d8396281ef	add slot to sparse table (#18686) ... The change includes 2 things: 1. save delta model and shrink table are control by the same parameter before, now add delete_after_unseen_days to control shrink table. 2. value in sparse table has no slot before, now add slot in sparse table, and add DownpureCtrAccessor to support the new meta. test=develop	7 years ago
jiaqi	d18aabb472	support patch data, add load_one_table, fix bug (#18509) ... （1）support patch data （merge slots of instances of same line id, modify dense layer which changes its size） （2）add fleet load_one_table interface, support load from paddle model and load from pslib model （3）fix push sparse bug which cause push sparse cost more time（about 10% in my testcase） （4）when some slots are not in one of your network (join/update, etc.)，data feed、collect label info、push/pull sparse will skip these slots， instead of throw error. （5）add more debug info in TrainFilesWithProfiler	7 years ago
tangwei12	101f74cb19	fix save/load in fleet (#17675) ... * fix save/load in Fleet * add UT framework of Fleet	7 years ago
Qiao Longfei	58f7695ab2	Async exe support communicator (#17386) ... Async exe support communicator	7 years ago
jiaqi	05df39ac06	support sparse table get shard_num from TableParameter (#17443) ... test=develop	7 years ago
jiaqi	34369944f5	support config file, cvm, load, save, shrink (#17319) ... * support config file, cvm, load, save, shrink test=develop * fix error of worker_num & add table.compress_in_save test=develop * fix code style test=develop * fix save model bug test=develop	7 years ago
tangwei12	565d309501	Reformat fleet API (#17135) ... * fix some logic in distributed transpiler, test=develop * reformat fleet API, test=develop	7 years ago
tangwei12	1a4a51db2b	Fleet unify distributed training (#16791) ... * implement distributed transpiler with fleet	7 years ago