I’ve done a clean install of the rodeo, but it gives me an error when I try doing the bulk command. It looks like geopandas is not in the environment. I’m trying to install geopandas but it seems to be having conflicts. Can you shed some light on the situation?
I can verify that the version from earlier today does throw an unhandled exception (just as he says, the import of geopandas, an undocumented requirement, fails). I didn’t have any trouble installing it, and I got to the help output, at least. Didn’t go past that, but I wanted to point out it is indeed a bug.
There are still some issues. The first is that the code assumes there is a /data directory in the root containing metadata.csv and geospatial.gpkg, I was able to create that directory and copy the files into it and it worked, however, when the code reached the snotel data I received the following error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=‘wcc.sc.egov.usda.gov’, port=443): Read timed out.
I tried it twice and was able to get data up to 2019 before it timed out both times.
Another option to consider is to simply create a repo of all the data that we can clone.
I’m not seeing any connection errors with downloading the SNOTEL data. Let me know if you continue running into issues. If the issue persists or if others have similar issues, we may consider looking into how to make the downloads more robust. As a workaround, you may consider using the dataset-specific commands or modify the YAML config file to download specific years as needed.
Yes, after several more attempts, I got it to work. I’m all set now with the latest data.
The latest update to the repository adds a few things that may help you if you run into additional connection errors:
- The SNOTEL download code now has automatic retries for connection errors.
skip_existingflag has been added to all data sources and is on by default. (This standardizes behavior that was previously not uniform across all data sources.) This means you can rerun a failed download and it will not redownload existing successfully downloaded files.