Overview
========

What is the HDP?
----------------
The Heatwave Diagnostics Package (HDP) is a collection of Python tools for computing heatwave metrics and generating summary figures. Functions can be imported in Jupyter notebooks and Python scripts or called in the terminal using the command line interface (CLI). All data uses `xarray <https://docs.xarray.dev/en/stable/>`_ data structures and can be saved to disk as either Zarr stores (default) or netCDF datasets. Summary figures can be generated to describe the output and can be saved to disk in Jupyter notebooks.

The HDP workflow follows three steps:

1. Format both a baseline and test measure of heat

2. Generate an extreme heat threshold from the baseline

3. Compute heatwave metrics by comparing the test measure against the baseline threshold

Throughout this guide, "baseline" refers to a measure of heat that is used to generate the threshold values from. Threshold values are defined as a percentile of all baseline values for each day of the year (see :ref:`Threshold Calculation <threshold_calc>` for more details). The "test" measure is compared against the threshold values to calculate hot days (days that exceed the threshold). Finally, the "heatwave definition" refers to an integer-sequence that describes what patterns of hot days over time are considered heatwaves. The span of different measures, thresholds, and definitions defines the "heatwave parameter space."

Statement of Need
-----------------

Development of the `HDP` was started in 2023 primarily to address the computational obstacles around handling terabyte-scale large ensembles, but quickly evolved to investigate new scientific questions around how the selection of characteristic heatwave parameters may impact hazard analysis. The `HDP` can provide insight into how the spatial-temporal response of heatwaves to climate perturbations depends on the choice of heatwave parameters. While other software packages primarly focus on calculating heatwave metrics for individual parameter selections (e.g. `ehfheatwaves <http://tammasloughran.github.io/ehfheatwaves/>`_, `heatwave3 <https://robwschlegel.github.io/heatwave3/index.html>`_, `nctoolkit <https://nctoolkit.readthedocs.io/en/latest/>`_), the `HDP` builds on these tools by optimizing the computation to evaluate metrics across large ranges of the parameter space.

Quick Start
-----------

.. _quickstart:

Below is example code that computes heatwave metrics for multiple measures, thresholds, and definitions from sample data generated by the HDP. Heatwave metrics are obtained for the "warming" data by comparing against the thresholds generated from the "control" data.

.. code-block:: python

    from hdp.graphics.notebook import create_notebook
    import hdp.utils
    import hdp.measure
    import hdp.threshold
    import hdp.metric
    import numpy as np

    output_dir = "."

    sample_control_temp = hdp.utils.generate_test_control_dataarray(add_noise=True)
    sample_warming_temp = hdp.utils.generate_test_warming_dataarray(add_noise=True)

    baseline_measures = hdp.measure.format_standard_measures(
        temp_datasets=[sample_control_temp]
    )
    test_measures = hdp.measure.format_standard_measures(
        temp_datasets=[sample_warming_temp]
    )

    percentiles = np.arange(0.9, 1.0, 0.01)

    thresholds_dataset = hdp.threshold.compute_thresholds(
        baseline_measures,
        percentiles
    )

    definitions = [[3,0,0], [3,1,1], [4,0,0], [4,1,1], [5,0,0], [5,1,1]]

    metrics_dataset = hdp.metric.compute_group_metrics(test_measures, thresholds_dataset, definitions, include_threshold=True)
    metrics_dataset.to_netcdf(f"{output_dir}/sample_hw_metrics.nc", mode='w')

    figure_notebook = create_notebook(metrics_dataset)
    figure_notebook.save_notebook(f"{output_dir}/sample_hw_summary_figures.ipynb")

    sample_control_temp = sample_control_temp.to_dataset()
    sample_control_temp.attrs["description"] = "Mock control temperature dataset generated by HDP for unit testing."
    sample_control_temp.to_netcdf(f"{output_dir}/sample_control_temp.nc", mode='w')

    sample_warming_temp = sample_warming_temp.to_dataset()
    sample_warming_temp.attrs["description"] = "Mock temperature dataset with warming trend generated by HDP for unit testing."
    sample_warming_temp.to_netcdf(f"{output_dir}/sample_warming_temp.nc", mode='w')

This code snippet is included in the HDP source code and can be executed via:

.. code-block:: console

   $ git clone https://github.com/AgentOxygen/HDP.git
   $ cd HDP
   $ python docs/sample_data/sample.py docs/sample_data/


The sample data, metric data, and summary figures are all saved to the specified `docs/sample_data/` but this path can be changed as needed.