Unable to Run Smoke Test in Prescreened Arena

Pushback to the Future Challenge
1

I am unable to run the benchmark submission (30 min etd) through the smoke test. After all the predictions are generated and the outputs are saved it errors when scoring. I get the error: “There was a problem scoring the file your code generated. Check the submissions page for more details.” When done locally through docker, I am able to score the submission csv file using the provided score script.

When I last tried running something through smoke test (a couple weeks ago), it worked fine so I believe something must have changed recently? I also did try recloning the submission repo and followed the steps from start to finish (in case the docker image was updated and I needed to update to the newer one) and that didn’t helper either.

If the smoke test is working for others when uploading the benchmark “submission.zip” file please let me know.

Edit: Worth nothing that the full submission went through on my end. Seems like this is isolated to the smoke test.

1 Like
2

Thanks for bringing this to our attention. It does look like there were issues isolated to the smoke tests; we have a fix incoming. I’ll let you know in this thread when I can confirm things are working as expected once again!

3

The smoke test issue is fixed. You should be able to run smoke test submissions once again. Thanks for your patience!

1 Like