TOLNet Data Format Info

[ASDC - mshook]

TOLNet data consists of both vertical profiles of ozone and in-situ chemical measurements made alongside the TOLNet lidars. The preferred and currently accepted data format for lidar files is HDF-4 GEOMS and for surface analysis files is ICARTT (both described below). Two additional formats have been used in the past (so-called “legacy data”). The legacy data files may get translated to newer formats in the future. Best practices for use of the TOLNet data are to use HDF4-GEOMS and ICARTT files when available. Reaching out the data manager and instrument PI (information in the file attributes or headers) is also strongly encouraged.

Currently accepted and preferred lidar data format:
HDF4-GEOMS
These are Hierarchical Data Format (version 4) files with a prescribed structure, filename pattern, and controlled vocabulary. This format is used by NASA and ESA for ground-based networks and satellite validation products. TOLNet uses the GEOMS LIDAR ozone template for all O3Lidar products. TOLNet also follows the NDACC (Network for the Detection for Stratospheric Change) convention for versioning and product type keywords.

• What is HDF?
• HDF4 Documentation
• General GEOMS documentation and link to latest TAV file
• GEOMS LIDAR template information, including GEOMS LIDAR ozone template documentation
• NDACC versioning system
• NDACC versioning keywords

Currently accepted and preferred surface data format:
ICARTT
These are self-describing CSV (comma-separated variable) text files with a prescribed header format and filename pattern. Generally, files in this format are one-dimensional time series for surface meteorology or trace gas data. However, the format can be used for ozone profile data as well.

• ICARTT data format

Previous legacy or other formats:
TOLNet ASCII Profile Data Format
These are self-describing CSV (comma-separated variable) text files with a prescribed header format, filename pattern, and fixed variable list. Filenames start with “TOLNet-O3Lidar” and end in “.dat”.

• TOLNet Profile Data Format documentation

TOLNet ASCII Surface Data Format
These are self-describing CSV (comma-separated variable) text files with a prescribed header format, filename pattern, and fixed variable list, similar to the Profile Data format. Filenames start with “TOLNet-O3Surface” and end in “.dat”.

• TOLNet Surface Data Format documentation

HDF5
These are Hierarchical Data Format (version 5) files generally containing ancillary information for ozone profile data in other files. They do not have a required structure; contact the relevant PI or data provider for more information on how to read or interpret these files.

• HDF5 documentation