Metrics

Note: For all models logs, there will be a folder named eval/. This contains individual .csv files for each relevant metric (e.g., RMSE, SpecDiv).

We divide our metrics into 3 classes: (1) Deterministic-based, which cover evaluation used in conventional deterministic forecasting tasks, (2) Physics-based, which are aimed to construct a more physically-faithful and explainable data-driven forecast, and (3) Probabilistic-based, which account for the skillfulness of ensemble forecasts.

1. Deterministic-based:

   • RMSE

   • Bias

   • Anomaly Correlation Coefficient (ACC)

   • Multiscale Structural Similarity Index (MS-SSIM)

2. Physics-based:

   • Spectral Divergence (SpecDiv)

   • Spectral Residual (SpecRes)

3. Probabilistic-based:

   • RMSE Ensemble

   • Bias Ensemble

   • ACC Ensemble

   • MS-SSIM Ensemble

   • SpecDiv Ensemble

   • SpecRes Ensemble

   • Continuous Ranked Probability Score (CRPS)

   • Continuous Ranked Probability Skill Score (CRPSS)

   • Spread

   • Spread/Skill Ratio