Fix data provider bug in srl demo

Once encoutering a single sample whose size is larger than batch size,
the provider will yield empty batch and terminate the current pass unexpectedly
if can_over_batch_size=False.
avx_docs
Peng Li 8 years ago
parent b53bdcdc67
commit 9baf7fc479

@ -43,7 +43,7 @@ def get_batch_size(yeild_data):
init_hook=hook,
should_shuffle=True,
calc_batch_size=get_batch_size,
can_over_batch_size=False,
can_over_batch_size=True,
cache=CacheType.CACHE_PASS_IN_MEM)
def process(settings, file_name):
with open(file_name, 'r') as fdata:

Loading…
Cancel
Save