I can’t tell if I can solve this challenge using LLMs like GPT4.
Prompts on top of GPT-4 might be a quick way to get good performance on this problem, but this code would typically be making calls to OpenAI, and I’m not sure if the rules allow for code that reaches out to an external LLM API.
Hi @vaibhav2614, solutions for this challenge must be open-source (per the rules), which prohibits the use of GPT4. In addition, the code execution environment does not have network access.
Open-source LLMs that packaged up as part of the submission assets are allowed.
Open-source LLMs that packaged up as part of the submission assets are allowed.
Let’s use the popular Llama 2 as an example. The code to train Llama 2 is not public, but the weights are.
Many popular open source LLMs are fine tunes based off Llama 2 (all legal to bundle, but may or may not be considered open source, as per different definitions of open source).
Unquantized Llama 2 70b is ~ 130 GB. There are no size constraints to submissions?
I may not even submit, but I am just listing potential issues of interpretation.
I don’t think an off-the-shelf solution will score well enough to make it an issue, personally, but I wouldn’t stake my life on it. While you could technically try using a 7b model, it likely wouldn’t even complete within the hour time limit (the runtime GPU has 80GB of RAM and an hour time limit).
Hi @raviteja_b – if you need more clarity on what constitutes “open source,” you can check out the rules.
As stated in the rules , prize-eligible submissions may not use:
Common OSI licenses are MIT and Apache 2.0. You can find a full list here .