Data Overview¶
LEAP makes use of LOTS of data.
This section explains how data is formatted, stored, transferred, and ultimately shared within LEAP.
- File Formats describe which formats to use (Zarr, NetCDF, etc.) for cloud-ready, reproducible workflows.
- Data Locations discusses the different areas data can be stored depending on the stage of work.
- The Data Lifecycle shows the big picture: ingestion, management, and cleanup.
- Data Tools explains the tools for moving data between systems, including
fsspec/gcsfs
andrclone
. - The Data Catalog describes how to discover datasets once they’ve been published with traceable provenance.