Hello,
Since uploading large models (e.g., LLaMA) is time-consuming and inefficient, could you enable competitors to upload their assets once, with the assets being copied to a folder that is always mounted in the Docker test environment, for example, under ‘/assets’?
Hi @raspberrycup – good question. Unfortunately we don’t have persistent storage for competitors associated with the code execution environment, so assets have to be mounted each time.
If you’re curious for more details – when a submission is made, our code execution Kubernetes cluster creates an inference pod specifically for that submission. All of the submission assets are mounted to that pod. The pod shuts down after the submission code is done running, and the submission assets aren’t stored elsewhere. This blog post has a longer explanation.
Hello @kwetstone, thank you for your reply and sharing the details about the competition infrastructure and workflows. What about the possibility of hosting commonly used models in a shared storage for all competitors, similar to Kaggle datasets? According to the blog post you already have some competitor-independent storage and a similar storage could be used for hosting common open models. This greatly enhances the platform’s user experience and their engagement.
@raspberrycup I see your point – I know it’s frustrating waiting for the long uploads each time. It’s unlikely we’ll be able to make that change for the current competition, but we see your point and will consider changing the setup for future competitions. We tend to see many different approaches across solutions, so it would be difficult to determine which common open models to make available within the limits of our competition storage capacity.
If you think of anything else that we could to do improve the participant experience, please let us know!
@kwetstone I would like to clarify that even there is no Internet connection while executing our submission, we can still load/download model from HuggingFace. We do not need to include a very big model in our submission ?
@cuongk14 There is no access to the internet while executing your submission, so your code will not be able to download models from HuggingFace. To use an open-source model, you’ll need to download the pretrained weights and include those weights in your submission. The benchmark blog post walks through how to do this.