DataLoader๏ƒ

class dgl.graphbolt.DataLoader(datapipe, num_workers=0, persistent_workers=True)[source]๏ƒ

Bases: DataLoader

Multiprocessing DataLoader.

Iterates over the data pipeline with everything before feature fetching (i.e. dgl.graphbolt.FeatureFetcher) in subprocesses, and everything after feature fetching in the main process. The datapipe is modified in-place as a result.

Only works on single GPU.

Parameters:

  • datapipe (DataPipe) โ€“ The data pipeline.

  • num_workers (int, optional) โ€“ Number of worker processes. Default is 0.

  • persistent_workers (bool, optional) โ€“ If True, the data loader will not shut down the worker processes after a dataset has been consumed once. This allows to maintain the workers instances alive.