Skip to content

Data Overview

LEAP makes use of LOTS of data.

This section explains how data is formatted, stored, transferred, and ultimately shared within LEAP.

  • File Formats describe which formats to use (Zarr, NetCDF, etc.) for cloud-ready, reproducible workflows.
  • Data Locations discusses the different areas data can be stored depending on the stage of work.
  • The Data Lifecycle shows the big picture: ingestion, management, and cleanup.
  • Data Tools explains the tools for moving data between systems, including fsspec/gcsfs and rclone.
  • The Data Catalog describes how to discover datasets once they’ve been published with traceable provenance.