Overview

What is the HDP?

The Heatwave Diagnostics Package (HDP) is a collection of Python tools for computing heatwave metrics and generating summary figures. Functions can be imported in Jupyter notebooks and Python scripts or called in the terminal using the command line interface (CLI). All data uses xarray data structures and can be saved to disk as either Zarr stores (default) or netCDF datasets. Summary figures can be generated to describe the output and can be saved to disk in Jupyter notebooks.

The HDP workflow follows three steps:

  1. Format both a baseline and test measure of heat

  2. Generate an extreme heat threshold from the baseline

  3. Compute heatwave metrics by comparing the test measure against the baseline threshold

Throughout this guide, “baseline” refers to a measure of heat that is used to generate the threshold values from. Threshold values are defined as a percentile of all baseline values for each day of the year (see Threshold Calculation for more details). The “test” measure is compared against the threshold values to calculate hot days (days that exceed the threshold). Finally, the “heatwave definition” refers to an integer-sequence that describes what patterns of hot days over time are considered heatwaves. The span of different measures, thresholds, and definitions defines the “heatwave parameter space.”

Statement of Need

Development of the HDP was started in 2023 primarily to address the computational obstacles around handling terabyte-scale large ensembles, but quickly evolved to investigate new scientific questions around how the selection of characteristic heatwave parameters may impact hazard analysis. The HDP can provide insight into how the spatial-temporal response of heatwaves to climate perturbations depends on the choice of heatwave parameters. While other software packages primarly focus on calculating heatwave metrics for individual parameter selections (e.g. ehfheatwaves, heatwave3, nctoolkit), the HDP builds on these tools by optimizing the computation to evaluate metrics across large ranges of the parameter space.

Quick Start

Below is example code that computes heatwave metrics for multiple measures, thresholds, and definitions from sample data generated by the HDP. Heatwave metrics are obtained for the “warming” data by comparing against the thresholds generated from the “control” data.

from hdp.graphics.notebook import create_notebook
import hdp.utils
import hdp.measure
import hdp.threshold
import hdp.metric
import numpy as np

output_dir = "."

sample_control_temp = hdp.utils.generate_test_control_dataarray(add_noise=True)
sample_warming_temp = hdp.utils.generate_test_warming_dataarray(add_noise=True)

baseline_measures = hdp.measure.format_standard_measures(
    temp_datasets=[sample_control_temp]
)
test_measures = hdp.measure.format_standard_measures(
    temp_datasets=[sample_warming_temp]
)

percentiles = np.arange(0.9, 1.0, 0.01)

thresholds_dataset = hdp.threshold.compute_thresholds(
    baseline_measures,
    percentiles
)

definitions = [[3,0,0], [3,1,1], [4,0,0], [4,1,1], [5,0,0], [5,1,1]]

metrics_dataset = hdp.metric.compute_group_metrics(test_measures, thresholds_dataset, definitions, include_threshold=True)
metrics_dataset.to_netcdf(f"{output_dir}/sample_hw_metrics.nc", mode='w')

figure_notebook = create_notebook(metrics_dataset)
figure_notebook.save_notebook(f"{output_dir}/sample_hw_summary_figures.ipynb")

sample_control_temp = sample_control_temp.to_dataset()
sample_control_temp.attrs["description"] = "Mock control temperature dataset generated by HDP for unit testing."
sample_control_temp.to_netcdf(f"{output_dir}/sample_control_temp.nc", mode='w')

sample_warming_temp = sample_warming_temp.to_dataset()
sample_warming_temp.attrs["description"] = "Mock temperature dataset with warming trend generated by HDP for unit testing."
sample_warming_temp.to_netcdf(f"{output_dir}/sample_warming_temp.nc", mode='w')

This code snippet is included in the HDP source code and can be executed via:

$ git clone https://github.com/AgentOxygen/HDP.git
$ cd HDP
$ python docs/sample_data/sample.py docs/sample_data/

The sample data, metric data, and summary figures are all saved to the specified docs/sample_data/ but this path can be changed as needed.