I am unable to run the benchmark submission (30 min etd) through the smoke test. After all the predictions are generated and the outputs are saved it errors when scoring. I get the error: “There was a problem scoring the file your code generated. Check the submissions page for more details.” When done locally through docker, I am able to score the submission csv file using the provided score script.
When I last tried running something through smoke test (a couple weeks ago), it worked fine so I believe something must have changed recently? I also did try recloning the submission repo and followed the steps from start to finish (in case the docker image was updated and I needed to update to the newer one) and that didn’t helper either.
If the smoke test is working for others when uploading the benchmark “submission.zip” file please let me know.
Edit: Worth nothing that the full submission went through on my end. Seems like this is isolated to the smoke test.