It seems the online evaluation env is broken

After submission, I find the error

 conda run --no-capture-output -n condaenv python /opt/descriptor_eval.py --query_features query_descriptors.npz --ref_features reference_descriptors.npz --candidates_output full_rankings.csv
Traceback (most recent call last):
File "/opt/descriptor_eval.py", line 13, in <module>
from vsc.descriptor_eval_lib import evaluate_descriptor_track
File "/opt/vsc/descriptor_eval_lib.py", line 9, in <module>
from vsc.candidates import CandidateGeneration, MaxScoreAggregation
File "/opt/vsc/candidates.py", line 11, in <module>
from vsc.metrics import CandidatePair
File "/opt/vsc/metrics.py", line 14, in <module>
import matplotlib.pyplot as plt
File "/srv/conda/envs/condaenv/lib/python3.9/site-packages/matplotlib/__init__.py", line 113, in <module>
from . import _api, _version, cbook, _docstring, rcsetup
File "/srv/conda/envs/condaenv/lib/python3.9/site-packages/matplotlib/rcsetup.py", line 27, in <module>
from matplotlib.colors import Colormap, is_color_like
File "/srv/conda/envs/condaenv/lib/python3.9/site-packages/matplotlib/colors.py", line 51, in <module>
from PIL import Image
File "/srv/conda/envs/condaenv/lib/python3.9/site-packages/PIL/Image.py", line 114, in <module>
from . import _imaging as core
ImportError: libtiff.so.5: cannot open shared object file: No such file or directory
ERROR conda.cli.main_run:execute(47): `conda run python /opt/descriptor_eval.py --query_features query_descriptors.npz --ref_features reference_descriptors.npz --candidates_output full_rankings.csv` failed. (See above for error)

which never appeared before. (I only submit features and successfully run two times.)

Investigating, will report back shortly. We recently deployed a new container to address the memory issue and I’m wondering if the version of PIL installed was upgraded to something incompatible since it’s not pinned…

Update: I’ve reproduced the bug, attempting a fix by pinning libtiff in the environment. I’m not quite sure why this issue is only appearing now with the most recent build, perhaps it’s some package that’s not pinned that should be, but pillow and matplotlib are pinned in the env so it’s at the moment unclear.

Another update: It appears the libtiff feedstock was just updated a few days ago. I’m attempting to pin to the previous version (4.4) and build. This wasn’t caught in our automated build testing processes due to a separate bug, also working to fix.

Thanks a lot! Hope this solved soon.

@chrisk-dd is it solved? Hope all things go well before Christmas Day.

@wenhaowang I believe we’ve identified the necessary fix, we are just waiting on some final checks / review before deploying. I’ll ping you here once the new container is deployed and I’ve tested that it works so you can give it a shot!

Thanks for your patience and sorry for the delay.

Thank you very much!

@wenhaowang The container appears to be working again! Give it a try and let me know if you run into any more issues. Thanks again for bringing it to our attention so promptly.

Thanks a lot!. I will try!

anything works thanks again

1 Like