Clarification on the image processing rule

NASA Pose Bowl
1

I have a question about one of the rules in the guidelines - the one that says “Images must be processed one at a time. Parallelization of multiple images is not permitted.”

Does this mean I can’t use PyTorch’s DataLoader with num_workers > 0? Setting num_workers > 0 creates multiple worker processes/threads to load and preprocess data in parallel to speed things up.

But based on that rule, it seems like having multiple workers processing multiple images at the same time might not be allowed? I just wanted to double-check.

It would be nice if you can clarify.

2

Hi @Appian,

The rule about parallelization is designed to simulate the situation onboard a spacecraft, in which images would not typically be processed in parallel. So unfortunately we can’t allow parallelization with PyTorch’s DataLoader as that would give a computational advantage that isn’t really relevant to the ultimate use case.

Good question, thank you for asking!

Mike

1 Like