L2R DataLoader

DataLoader for L2R applications.

! [ -e /content ] && pip install -Uqq fastai # upgrade fastai on colab

This module contains all the classes and supporting fuctions to build a L2RDataLoader.

The method quantized_score simulates Pandas groupby using Numpy/PyTorch: (Why? Pandas are cute but speed thrills! More importantly: “Memory”)

 PreLoadTrans (df, qnts=None, device=None)

Initialize self. See help(type(self)) for accurate signature.

 PreLoadTrans.pad_split ()

pads the validation set for each label to the nearest multiple of 16

 PreLoadTrans.count_topbins ()

counts the number of top 2 bins in the val set for each label

 PreLoadTrans.datasets ()

prepare the train/val dataset from self.scored_toks

img = Image.open(Path.cwd()/'pics'/'LRdataloader.png').resize((500,250))
img

 L2RDataLoader (*args, **kwargs)

API compatible with PyTorch DataLoader, with a lot more callbacks and flexibility