From d1ff33f23c120938df31b335c00560e2a947195d Mon Sep 17 00:00:00 2001 From: George McCabe <23407799+georgemccabe@users.noreply.github.com> Date: Fri, 8 Nov 2024 09:43:14 -0700 Subject: [PATCH 1/4] per #2757, add documentation for allow missing input logic --- docs/Users_Guide/glossary.rst | 277 ++++++++++++++++++++++++++++++++++ docs/Users_Guide/wrappers.rst | 43 ++++++ 2 files changed, 320 insertions(+) diff --git a/docs/Users_Guide/glossary.rst b/docs/Users_Guide/glossary.rst index 4ed26829b..f5a4899b8 100644 --- a/docs/Users_Guide/glossary.rst +++ b/docs/Users_Guide/glossary.rst @@ -12990,3 +12990,280 @@ METplus Configuration Glossary Specify the value for 'obtype_as_group_val_flag' in the MET configuration file for EnsembleStat. | *Used by:* EnsembleStat + + ALLOW_MISSING_INPUTS + If True, report a warning instead of an error if a wrapper cannot find all + of its required input files. Only report an error if the number of runs + that successfully found input files does not meet the threshold defined + with :ref:`INPUT_THRESH`. + Wrapper-specific versions of this variable are supported for many of the + wrappers. Defaults to False. + + | *Used by:* Many + + INPUT_THRESH + A decimal number between 0 and 1 that defines the threshold of successful + run times where all of the required input files were found. + If INPUT_THRESH is set to 0.5 and 3 out of the 4 run times (75% or 0.75) + successfully found the required input files, no errors will be reported. + However, if INPUT_THRESH is set to 0.5 and only 1 out of 4 run times + (25% or 0.25) found the required input files, an error will be reported + at the end of the run. The threshold is inclusive, so 2 out of 4 successful + run times (50% or 0.5) will not report an error. + This is only used if :ref:`ALLOW_MISSING_INPUTS` is True. + Wrapper-specific versions of this variable are supported for many of the + wrappers. + + | *Used by:* Many + + ASCII2NC_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for ASCII2NC only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* ASCII2NC + + ASCII2NC_INPUT_THRESH + Defines input threshold for ASCII2NC only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* ASCII2NC + + ENSEMBLE_STAT_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for EnsembleStat only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* EnsembleStat + + ENSEMBLE_STAT_INPUT_THRESH + Defines input threshold for EnsembleStat only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* EnsembleStat + + GEN_ENS_PROD_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for GenEnsProd only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* GenEnsProd + + GEN_ENS_PROD_INPUT_THRESH + Defines input threshold for GenEnsProd only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* GenEnsProd + + GEN_VX_MASK_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for GenVxMask only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* GenVxMask + + GEN_VX_MASK_INPUT_THRESH + Defines input threshold for GenVxMask only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* GenVxMask + + GRID_DIAG_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for GridDiag only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* GridDiag + + GRID_DIAG_INPUT_THRESH + Defines input threshold for GridDiag only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* GridDiag + + GRID_STAT_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for GridStat only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* GridStat + + GRID_STAT_INPUT_THRESH + Defines input threshold for GridStat only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* GridStat + + IODA2NC_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for IODA2NC only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* IODA2NC + + IODA2NC_INPUT_THRESH + Defines input threshold for IODA2NC only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* IODA2NC + + MADIS2NC_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for MADIS2NC only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* MADIS2NC + + MADIS2NC_INPUT_THRESH + Defines input threshold for MADIS2NC only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* MADIS2NC + + MODE_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for MODE only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* MODE + + MODE_INPUT_THRESH + Defines input threshold for MODE only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* MODE + + MTD_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for MTD only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* MTD + + MTD_INPUT_THRESH + Defines input threshold for MTD only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* MTD + + PB2NC_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for PB2NC only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* PB2NC + + PB2NC_INPUT_THRESH + Defines input threshold for PB2NC only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* PB2NC + + PCP_COMBINE_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for PCPCombine only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* PCPCombine + + PCP_COMBINE_INPUT_THRESH + Defines input threshold for PCPCombine only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* PCPCombine + + PLOT_DATA_PLANE_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for PlotDataPlane only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* PlotDataPlane + + PLOT_DATA_PLANE_INPUT_THRESH + Defines input threshold for PlotDataPlane only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* PlotDataPlane + + PLOT_POINT_OBS_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for PlotPointObs only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* PlotPointObs + + PLOT_POINT_OBS_INPUT_THRESH + Defines input threshold for PlotPointObs only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* PlotPointObs + + POINT2GRID_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for Point2Grid only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* Point2Grid + + POINT2GRID_INPUT_THRESH + Defines input threshold for Point2Grid only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* Point2Grid + + POINT_STAT_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for PointStat only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* PointStat + + POINT_STAT_INPUT_THRESH + Defines input threshold for PointStat only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* PointStat + + REGRID_DATA_PLANE_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for RegridDataPlane only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* RegridDataPlane + + REGRID_DATA_PLANE_INPUT_THRESH + Defines input threshold for RegridDataPlane only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* RegridDataPlane + + SERIES_ANALYSIS_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for SeriesAnalysis only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* SeriesAnalysis + + SERIES_ANALYSIS_INPUT_THRESH + Defines input threshold for SeriesAnalysis only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* SeriesAnalysis + + TC_DIAG_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for TCDiag only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* TCDiag + + TC_DIAG_INPUT_THRESH + Defines input threshold for TCDiag only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* TCDiag + + TC_GEN_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for TCGen only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* TCGen + + TC_GEN_INPUT_THRESH + Defines input threshold for TCGen only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* TCGen + + WAVELET_STAT_ALLOW_MISSING_INPUTS + Activates allow missing inputs logic for WaveletStat only. + See :ref:`ALLOWING_MISSING_INPUTS` for details. + + | *Used by:* WaveletStat + + WAVELET_STAT_INPUT_THRESH + Defines input threshold for WaveletStat only. + See :ref:`INPUT_THRESH` for details. + + | *Used by:* WaveletStat diff --git a/docs/Users_Guide/wrappers.rst b/docs/Users_Guide/wrappers.rst index 5ca93e9dd..dc3106fd0 100644 --- a/docs/Users_Guide/wrappers.rst +++ b/docs/Users_Guide/wrappers.rst @@ -64,6 +64,9 @@ METplus Configuration | :term:`ASCII2NC_INC_VALID_TIMES` | :term:`ASCII2NC_SKIP_INIT_TIMES` | :term:`ASCII2NC_INC_INIT_TIMES` +| :term:`ASCII2NC_ALLOW_MISSING_INPUTS` +| :term:`ASCII2NC_INPUT_THRESH` + | .. _ascii2nc-met-conf: @@ -380,6 +383,8 @@ METplus Configuration | :term:`ENSEMBLE_STAT_INC_VALID_TIMES` | :term:`ENSEMBLE_STAT_SKIP_INIT_TIMES` | :term:`ENSEMBLE_STAT_INC_INIT_TIMES` +| :term:`ENSEMBLE_STAT_ALLOW_MISSING_INPUTS` +| :term:`ENSEMBLE_STAT_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -1352,6 +1357,8 @@ METplus Configuration | :term:`GEN_ENS_PROD_INC_VALID_TIMES` | :term:`GEN_ENS_PROD_SKIP_INIT_TIMES` | :term:`GEN_ENS_PROD_INC_INIT_TIMES` +| :term:`GEN_ENS_PROD_ALLOW_MISSING_INPUTS` +| :term:`GEN_ENS_PROD_INPUT_THRESH` .. _gen-ens-prod-met-conf: @@ -1746,6 +1753,8 @@ Configuration | :term:`GEN_VX_MASK_INC_VALID_TIMES` | :term:`GEN_VX_MASK_SKIP_INIT_TIMES` | :term:`GEN_VX_MASK_INC_INIT_TIMES` +| :term:`GEN_VX_MASK_ALLOW_MISSING_INPUTS` +| :term:`GEN_VX_MASK_INPUT_THRESH` | .. _gfdl_tracker_wrapper: @@ -3054,6 +3063,8 @@ METplus Configuration | :term:`GRID_DIAG_INC_VALID_TIMES` | :term:`GRID_DIAG_SKIP_INIT_TIMES` | :term:`GRID_DIAG_INC_INIT_TIMES` +| :term:`GRID_DIAG_ALLOW_MISSING_INPUTS` +| :term:`GRID_DIAG_INPUT_THRESH` | .. _grid-diag-met-conf: @@ -3399,6 +3410,8 @@ METplus Configuration | :term:`GRID_STAT_UGRID_COORDINATES_FILE` | :term:`GRID_STAT_UGRID_CONFIG_FILE` | :term:`GRID_STAT_TIME_OFFSET_WARNING` +| :term:`GRID_STAT_ALLOW_MISSING_INPUTS` +| :term:`GRID_STAT_INPUT_THRESH` | .. warning:: **DEPRECATED** @@ -4169,6 +4182,8 @@ METplus Configuration | :term:`IODA2NC_INC_VALID_TIMES` | :term:`IODA2NC_SKIP_INIT_TIMES` | :term:`IODA2NC_INC_INIT_TIMES` +| :term:`IODA2NC_ALLOW_MISSING_INPUTS` +| :term:`IODA2NC_INPUT_THRESH` .. _ioda2nc-met-conf: @@ -4451,6 +4466,8 @@ METplus Configuration | :term:`MADIS2NC_INC_VALID_TIMES` | :term:`MADIS2NC_SKIP_INIT_TIMES` | :term:`MADIS2NC_INC_INIT_TIMES` +| :term:`MADIS2NC_ALLOW_MISSING_INPUTS` +| :term:`MADIS2NC_INPUT_THRESH` | .. _madis2nc-met-conf: @@ -4878,6 +4895,8 @@ METplus Configuration | :term:`MODE_SKIP_INIT_TIMES` | :term:`MODE_INC_INIT_TIMES` | :term:`MODE_TIME_OFFSET_WARNING` +| :term:`MODE_ALLOW_MISSING_INPUTS` +| :term:`MODE_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -5638,6 +5657,8 @@ METplus Configuration | :term:`MTD_INC_VALID_TIMES` | :term:`MTD_SKIP_INIT_TIMES` | :term:`MTD_INC_INIT_TIMES` +| :term:`MTD_ALLOW_MISSING_INPUTS` +| :term:`MTD_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -5930,6 +5951,8 @@ METplus Configuration | :term:`PB2NC_INC_VALID_TIMES` | :term:`PB2NC_SKIP_INIT_TIMES` | :term:`PB2NC_INC_INIT_TIMES` +| :term:`PB2NC_ALLOW_MISSING_INPUTS` +| :term:`PB2NC_INPUT_THRESH` .. warning:: **DEPRECATED:** @@ -6275,6 +6298,8 @@ METplus Configuration | :term:`FCST_PCP_COMBINE_VLD_THRESH` | :term:`OBS_PCP_COMBINE_INPUT_THRESH` | :term:`OBS_PCP_COMBINE_VLD_THRESH` +| :term:`PCP_COMBINE_ALLOW_MISSING_INPUTS` +| :term:`PCP_COMBINE_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -6339,6 +6364,8 @@ Configuration | :term:`PLOT_DATA_PLANE_INC_VALID_TIMES` | :term:`PLOT_DATA_PLANE_SKIP_INIT_TIMES` | :term:`PLOT_DATA_PLANE_INC_INIT_TIMES` +| :term:`PLOT_DATA_PLANE_ALLOW_MISSING_INPUTS` +| :term:`PLOT_DATA_PLANE_INPUT_THRESH` .. _plot_point_obs_wrapper: @@ -6406,6 +6433,8 @@ Configuration | :term:`PLOT_POINT_OBS_INC_VALID_TIMES` | :term:`PLOT_POINT_OBS_SKIP_INIT_TIMES` | :term:`PLOT_POINT_OBS_INC_INIT_TIMES` +| :term:`PLOT_POINT_OBS_ALLOW_MISSING_INPUTS` +| :term:`PLOT_POINT_OBS_INPUT_THRESH` .. _plot-point-obs-met-conf: @@ -6793,6 +6822,8 @@ METplus Configuration | :term:`POINT2GRID_OBS_QUALITY_INC` | :term:`POINT2GRID_OBS_QUALITY_EXC` | :term:`POINT2GRID_MET_CONFIG_OVERRIDES` +| :term:`POINT2GRID_ALLOW_MISSING_INPUTS` +| :term:`POINT2GRID_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -7125,6 +7156,8 @@ Configuration | :term:`POINT_STAT_UGRID_COORDINATES_FILE` | :term:`POINT_STAT_UGRID_CONFIG_FILE` | :term:`POINT_STAT_POINT_WEIGHT_FLAG` +| :term:`POINT_STAT_ALLOW_MISSING_INPUTS` +| :term:`POINT_STAT_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -7926,6 +7959,8 @@ METplus Configuration | :term:`REGRID_DATA_PLANE_INC_VALID_TIMES` | :term:`REGRID_DATA_PLANE_SKIP_INIT_TIMES` | :term:`REGRID_DATA_PLANE_INC_INIT_TIMES` +| :term:`REGRID_DATA_PLANE_ALLOW_MISSING_INPUTS` +| :term:`REGRID_DATA_PLANE_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -8107,6 +8142,8 @@ METplus Configuration | :term:`SERIES_ANALYSIS_SKIP_INIT_TIMES` | :term:`SERIES_ANALYSIS_INC_INIT_TIMES` | :term:`SERIES_ANALYSIS_TIME_OFFSET_WARNING` +| :term:`SERIES_ANALYSIS_ALLOW_MISSING_INPUTS` +| :term:`SERIES_ANALYSIS_INPUT_THRESH` | .. warning:: **DEPRECATED:** @@ -9442,6 +9479,8 @@ METplus Configuration | :term:`TC_DIAG_INC_VALID_TIMES` | :term:`TC_DIAG_SKIP_INIT_TIMES` | :term:`TC_DIAG_INC_INIT_TIMES` +| :term:`TC_DIAG_ALLOW_MISSING_INPUTS` +| :term:`TC_DIAG_INPUT_THRESH` | .. _tc-diag-met-conf: @@ -9974,6 +10013,8 @@ METplus Configuration | :term:`TC_GEN_DLAND_FILE` | :term:`TC_GEN_BASIN_FILE` | :term:`TC_GEN_NC_PAIRS_GRID` +| :term:`TC_GEN_ALLOW_MISSING_INPUTS` +| :term:`TC_GEN_INPUT_THRESH` .. warning:: **DEPRECATED:** @@ -12147,6 +12188,8 @@ METplus Configuration | :term:`WAVELET_STAT_WVLT_PLOT_PLOT_MAX` | :term:`WAVELET_STAT_OUTPUT_PREFIX` | :term:`WAVELET_STAT_TIME_OFFSET_WARNING` +| :term:`WAVELET_STAT_ALLOW_MISSING_INPUTS` +| :term:`WAVELET_STAT_INPUT_THRESH` .. _wavelet-stat-met-conf: From 1ba4351ddd758f039b70e2c30992fd4c374544f7 Mon Sep 17 00:00:00 2001 From: George McCabe <23407799+georgemccabe@users.noreply.github.com> Date: Fri, 8 Nov 2024 09:46:53 -0700 Subject: [PATCH 2/4] fix incorrect link to glossary item --- docs/Users_Guide/glossary.rst | 88 +++++++++++++++++------------------ 1 file changed, 44 insertions(+), 44 deletions(-) diff --git a/docs/Users_Guide/glossary.rst b/docs/Users_Guide/glossary.rst index f5a4899b8..dabfa9786 100644 --- a/docs/Users_Guide/glossary.rst +++ b/docs/Users_Guide/glossary.rst @@ -12995,7 +12995,7 @@ METplus Configuration Glossary If True, report a warning instead of an error if a wrapper cannot find all of its required input files. Only report an error if the number of runs that successfully found input files does not meet the threshold defined - with :ref:`INPUT_THRESH`. + with :term:`INPUT_THRESH`. Wrapper-specific versions of this variable are supported for many of the wrappers. Defaults to False. @@ -13010,7 +13010,7 @@ METplus Configuration Glossary (25% or 0.25) found the required input files, an error will be reported at the end of the run. The threshold is inclusive, so 2 out of 4 successful run times (50% or 0.5) will not report an error. - This is only used if :ref:`ALLOW_MISSING_INPUTS` is True. + This is only used if :term:`ALLOW_MISSING_INPUTS` is True. Wrapper-specific versions of this variable are supported for many of the wrappers. @@ -13018,252 +13018,252 @@ METplus Configuration Glossary ASCII2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for ASCII2NC only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* ASCII2NC ASCII2NC_INPUT_THRESH Defines input threshold for ASCII2NC only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* ASCII2NC ENSEMBLE_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for EnsembleStat only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* EnsembleStat ENSEMBLE_STAT_INPUT_THRESH Defines input threshold for EnsembleStat only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* EnsembleStat GEN_ENS_PROD_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GenEnsProd only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* GenEnsProd GEN_ENS_PROD_INPUT_THRESH Defines input threshold for GenEnsProd only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* GenEnsProd GEN_VX_MASK_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GenVxMask only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* GenVxMask GEN_VX_MASK_INPUT_THRESH Defines input threshold for GenVxMask only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* GenVxMask GRID_DIAG_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GridDiag only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* GridDiag GRID_DIAG_INPUT_THRESH Defines input threshold for GridDiag only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* GridDiag GRID_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GridStat only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* GridStat GRID_STAT_INPUT_THRESH Defines input threshold for GridStat only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* GridStat IODA2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for IODA2NC only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* IODA2NC IODA2NC_INPUT_THRESH Defines input threshold for IODA2NC only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* IODA2NC MADIS2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MADIS2NC only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* MADIS2NC MADIS2NC_INPUT_THRESH Defines input threshold for MADIS2NC only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* MADIS2NC MODE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MODE only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* MODE MODE_INPUT_THRESH Defines input threshold for MODE only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* MODE MTD_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MTD only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* MTD MTD_INPUT_THRESH Defines input threshold for MTD only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* MTD PB2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PB2NC only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* PB2NC PB2NC_INPUT_THRESH Defines input threshold for PB2NC only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* PB2NC PCP_COMBINE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PCPCombine only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* PCPCombine PCP_COMBINE_INPUT_THRESH Defines input threshold for PCPCombine only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* PCPCombine PLOT_DATA_PLANE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PlotDataPlane only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* PlotDataPlane PLOT_DATA_PLANE_INPUT_THRESH Defines input threshold for PlotDataPlane only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* PlotDataPlane PLOT_POINT_OBS_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PlotPointObs only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* PlotPointObs PLOT_POINT_OBS_INPUT_THRESH Defines input threshold for PlotPointObs only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* PlotPointObs POINT2GRID_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for Point2Grid only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* Point2Grid POINT2GRID_INPUT_THRESH Defines input threshold for Point2Grid only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* Point2Grid POINT_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PointStat only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* PointStat POINT_STAT_INPUT_THRESH Defines input threshold for PointStat only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* PointStat REGRID_DATA_PLANE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for RegridDataPlane only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* RegridDataPlane REGRID_DATA_PLANE_INPUT_THRESH Defines input threshold for RegridDataPlane only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* RegridDataPlane SERIES_ANALYSIS_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for SeriesAnalysis only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* SeriesAnalysis SERIES_ANALYSIS_INPUT_THRESH Defines input threshold for SeriesAnalysis only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* SeriesAnalysis TC_DIAG_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for TCDiag only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* TCDiag TC_DIAG_INPUT_THRESH Defines input threshold for TCDiag only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* TCDiag TC_GEN_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for TCGen only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* TCGen TC_GEN_INPUT_THRESH Defines input threshold for TCGen only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* TCGen WAVELET_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for WaveletStat only. - See :ref:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOWING_MISSING_INPUTS` for details. | *Used by:* WaveletStat WAVELET_STAT_INPUT_THRESH Defines input threshold for WaveletStat only. - See :ref:`INPUT_THRESH` for details. + See :term:`INPUT_THRESH` for details. | *Used by:* WaveletStat From 4a1c137c0a312c8384d8ab0ded4db232e309592c Mon Sep 17 00:00:00 2001 From: George McCabe <23407799+georgemccabe@users.noreply.github.com> Date: Fri, 8 Nov 2024 09:56:18 -0700 Subject: [PATCH 3/4] fix typo --- docs/Users_Guide/glossary.rst | 42 +++++++++++++++++------------------ 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/docs/Users_Guide/glossary.rst b/docs/Users_Guide/glossary.rst index dabfa9786..94b11cc28 100644 --- a/docs/Users_Guide/glossary.rst +++ b/docs/Users_Guide/glossary.rst @@ -13018,7 +13018,7 @@ METplus Configuration Glossary ASCII2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for ASCII2NC only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* ASCII2NC @@ -13030,7 +13030,7 @@ METplus Configuration Glossary ENSEMBLE_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for EnsembleStat only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* EnsembleStat @@ -13042,7 +13042,7 @@ METplus Configuration Glossary GEN_ENS_PROD_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GenEnsProd only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* GenEnsProd @@ -13054,7 +13054,7 @@ METplus Configuration Glossary GEN_VX_MASK_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GenVxMask only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* GenVxMask @@ -13066,7 +13066,7 @@ METplus Configuration Glossary GRID_DIAG_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GridDiag only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* GridDiag @@ -13078,7 +13078,7 @@ METplus Configuration Glossary GRID_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for GridStat only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* GridStat @@ -13090,7 +13090,7 @@ METplus Configuration Glossary IODA2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for IODA2NC only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* IODA2NC @@ -13102,7 +13102,7 @@ METplus Configuration Glossary MADIS2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MADIS2NC only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* MADIS2NC @@ -13114,7 +13114,7 @@ METplus Configuration Glossary MODE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MODE only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* MODE @@ -13126,7 +13126,7 @@ METplus Configuration Glossary MTD_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for MTD only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* MTD @@ -13138,7 +13138,7 @@ METplus Configuration Glossary PB2NC_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PB2NC only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* PB2NC @@ -13150,7 +13150,7 @@ METplus Configuration Glossary PCP_COMBINE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PCPCombine only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* PCPCombine @@ -13162,7 +13162,7 @@ METplus Configuration Glossary PLOT_DATA_PLANE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PlotDataPlane only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* PlotDataPlane @@ -13174,7 +13174,7 @@ METplus Configuration Glossary PLOT_POINT_OBS_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PlotPointObs only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* PlotPointObs @@ -13186,7 +13186,7 @@ METplus Configuration Glossary POINT2GRID_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for Point2Grid only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* Point2Grid @@ -13198,7 +13198,7 @@ METplus Configuration Glossary POINT_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for PointStat only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* PointStat @@ -13210,7 +13210,7 @@ METplus Configuration Glossary REGRID_DATA_PLANE_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for RegridDataPlane only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* RegridDataPlane @@ -13222,7 +13222,7 @@ METplus Configuration Glossary SERIES_ANALYSIS_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for SeriesAnalysis only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* SeriesAnalysis @@ -13234,7 +13234,7 @@ METplus Configuration Glossary TC_DIAG_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for TCDiag only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* TCDiag @@ -13246,7 +13246,7 @@ METplus Configuration Glossary TC_GEN_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for TCGen only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* TCGen @@ -13258,7 +13258,7 @@ METplus Configuration Glossary WAVELET_STAT_ALLOW_MISSING_INPUTS Activates allow missing inputs logic for WaveletStat only. - See :term:`ALLOWING_MISSING_INPUTS` for details. + See :term:`ALLOW_MISSING_INPUTS` for details. | *Used by:* WaveletStat From cc1f40a50c5eea0399e3286f95b4ffe90f31f77e Mon Sep 17 00:00:00 2001 From: George McCabe <23407799+georgemccabe@users.noreply.github.com> Date: Wed, 13 Nov 2024 12:05:45 -0700 Subject: [PATCH 4/4] Per #2757, add info and examples for allow missing input logic --- docs/Users_Guide/systemconfiguration.rst | 90 ++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/docs/Users_Guide/systemconfiguration.rst b/docs/Users_Guide/systemconfiguration.rst index cdffb1ffc..ac84ffb02 100644 --- a/docs/Users_Guide/systemconfiguration.rst +++ b/docs/Users_Guide/systemconfiguration.rst @@ -2138,6 +2138,96 @@ can be simplified as:: INPUT_TEMPLATE = ensbegin_end_incr(1,8,1,2).nc +.. _allow-missing-inputs: + +Allow Missing Inputs +-------------------- + +When any of the required input files for a given METplus run time are not found, +an error is reported. In result, the entire METplus run fails. +In some cases, users may expect a certain number of inputs to be unavailable +and do not want the entire run to fail when this happens. + +The :term:`ALLOW_MISSING_INPUTS` config variable can be set to **True** to +report a warning when required inputs are not found for a given run time. +An error at the end of the METplus run will only be reported if the number +of successful runs does not meet the value defined by :term:`INPUT_THRESH`. +The value of :term:`INPUT_THRESH` should be a decimal number between 0 and 1. +The default value is 0.0, so any missing input files in a run will still report +an error unless this value is changed. + +The threshold is compared to the results of each item in the +:ref:`Process_List`, so each wrapper listed in the **PROCESS_LIST** must meet +the threshold to prevent an error. + +There are wrapper-specific versions of both :term:`ALLOW_MISSING_INPUTS` and +:term:`INPUT_THRESH` for most of the wrappers, +e.g. :term:`GRID_STAT_ALLOW_MISSING_INPUTS` and :term:`GRID_STAT_INPUT_THRESH`. +Refer to the :ref:`python_wrappers` chapter or the :ref:`METplus_glossary` +to see which variables are supported. + +**Example 1**:: + + [config] + PROCESS_LIST = RegridDataPlane, GridStat + VALID_TIME_FMT = %Y%m%d%H + VALID_BEG = 2024020301 + VALID_BEG = 2024020310 + VALID_INCREMENT = 1H + LEAD_SEQ = 0 + + ALLOW_MISSING_INPUTS = True + INPUT_THRESH = 0.6 + +In this example, 10 valid times will be run, so there will be 10 calls to +RegridDataPlane and 10 calls to GridStat. The input threshold is set +to 60%, so if 6 or more of the RegridDataPlane runs *and* +6 or more of the GridStat runs successfully find all of the required files, +an error will not be reported. If 5 or fewer runs for either wrapper succeed, +then an error will be reported. + +**Example 2**:: + + + [config] + PROCESS_LIST = RegridDataPlane, GridStat + VALID_TIME_FMT = %Y%m%d%H + VALID_BEG = 2024020301 + VALID_BEG = 2024020310 + VALID_INCREMENT = 1H + LEAD_SEQ = 0 + + GRID_STAT_ALLOW_MISSING_INPUTS = True + INPUT_THRESH = 0.6 + +In this case, only GridStat wrapper will allow missing inputs. +At least 60% of the GridStat runs +must successfully find the required input files to prevent an error. +Any missing inputs for RegridDataPlane will result in an error. + +**Example 3**:: + + + [config] + PROCESS_LIST = RegridDataPlane, GridStat + VALID_TIME_FMT = %Y%m%d%H + VALID_BEG = 2024020301 + VALID_BEG = 2024020310 + VALID_INCREMENT = 1H + LEAD_SEQ = 0 + + ALLOW_MISSING_INPUTS = True + REGRID_DATA_PLANE_INPUT_THRESH = 0.9 + GRID_STAT_INPUT_THRESH = 0.6 + +In this case, both GridStat and RegridDataPlane wrappers allow missing inputs, +but the threshold to prevent an error differs between wrappers. +At least 90% of the RegridDataPlane runs +must successfully find the required input files to prevent an error. +At least 60% of the GridStat runs +must successfully find the required input files to prevent an error. +Any missing inputs for RegridDataPlane will result in an error. + .. _metplus-control-met: How METplus controls MET configuration variables