Data Extraction and Processing Times

[cszc]

Hi @kovila. I can’t speak for others, but our benchmark notebooks take about 1-1.5 hours to run end to end:

New Approved Datasets and Blog Post Tutorials

Hello Airathon Community! We’ve got just over two weeks left on the Airathon competitions. To help you perfect your models, we’ve added some new resources. Approved Datasets Requests for new datasets are now closed, but several datasets were approved last week, including: ECMWF IFS CY41r2 High-Resolution Operational Forecasts GEOS Composition Forecasting (GEOS-CF) CALIPSO Lidar (Particulate Matter Track only) Head to the announcements (pm2.5, no2) and the problem description pages (pm2.5, n…

In addition to code for downloading and processing data, the notebooks include code for training a rudimentary model. They only process MAIAC and OMI data for the pm2.5 and NO2 blog posts, respectively. I imagine TROPOMI will take the longest to download and process given the size of the dataset. FWIW, using a machine in the same region as the bucket greatly increases the speed of download.