Fix data provider bug in srl demo

Once encoutering a single sample whose size is larger than batch size,
the provider will yield empty batch and terminate the current pass unexpectedly
if can_over_batch_size=False.
avx_docs
Peng Li 8 years ago
parent b53bdcdc67
commit 9baf7fc479

@ -43,7 +43,7 @@ def get_batch_size(yeild_data):
init_hook=hook, init_hook=hook,
should_shuffle=True, should_shuffle=True,
calc_batch_size=get_batch_size, calc_batch_size=get_batch_size,
can_over_batch_size=False, can_over_batch_size=True,
cache=CacheType.CACHE_PASS_IN_MEM) cache=CacheType.CACHE_PASS_IN_MEM)
def process(settings, file_name): def process(settings, file_name):
with open(file_name, 'r') as fdata: with open(file_name, 'r') as fdata:

Loading…
Cancel
Save