Request understanding of the data files

Hello,

The file names of downloaded files from your repository is:
tropomi 2019 :

20190107T042626_tropomi_tpe_0.nc
20190107T060756_tropomi_tpe_0.nc
20190107T074926_tropomi_dl_0.nc
20190107T212126_tropomi_la_0.nc

But, when i download S5P no2 files form Copernicus Open Access Hub then the file name is given as:
S5P_RPRO_L2__NO2____20180816T182917_20180816T201245_04361_01_010202_20190101T194705.nc

*** Why are the file names changed? is it for hiding the timestamp details some for this competition?**

*** Why are their multiple files for same date and location like *_tpe_0.nc, _dl_0.nc, _la_0.nc,etc, as Open Access Hub gives only single .nc file?

Same question is for omi data:
omi 2020 :
20200106T235959_omi_dl-la-tpe_0.he5
20200107T235959_omi_dl-la-tpe_0.he5
20200108T235959_omi_dl-la-tpe_0.he5

*** Also, is the internal structure of .he5 file same as .hdf file?**

*** Can anyone provide me some brief introduction or documentation of the various datasets given for download in this completion (other info than already mentioned in this website)?**

Thanks.

Why are the file names changed? is it for hiding the timestamp details some for this competition?

The file names are changed for ease of use so that competitors can easily look up the appropriate file without having to read in the data. Please note that the aws bucket we’ve provided is the only approved source for data.

Why are their multiple files for same date and location like *_tpe_0.nc, _dl_0.nc, _la_0.nc,etc, as Open Access Hub gives only single .nc file?

In the data_download_instructions.txt file found on the data download page, it says:

The remote sensing data files are named {time_end}_{product}_{location}_{number}.ext, where time_end is formatted as YYYYMMDDTHHmmss. In rare cases, locations and times may have more than one associated file. When this occurs, file number is denoted by number. For instance, 20190101T213357_tropomi_la_0.nc represents first data file from the Los Angeles South Coast Air Basin collected by TROPOMI on January 1, 2019 at 9:33pm UTC. In the case of OMI, the same file is used for each location, so location is always dl-la-tpe.

Also, is the internal structure of .he5 file same as .hdf file?

.he5 refers to an hdf5 file, whereas .hdf could refer to either hdf4 or hdf5. The internal structure of each data set may vary from product to product, e.g. there’s no guarantee that the same kind of metadata will be available across hdf5 files.

Hope that helps!

1 Like