Skip to content

Commit

Permalink
Feature #2735 v6.0.0 beta6 (#2736)
Browse files Browse the repository at this point in the history
* change version for beta6 release

* added script to help generate release notes for development releases

* add default title for issues and added documentation issue template

* sort issues by number, improve formatting for release notes title, add logging to alert users what is happening when timely github queries are running

* added release notes for beta6 release

* Add option to component version script to return 'develop' if the input version is a beta or rc version to preserve previous behavior of GHA scripts. Update GHA scripts to use new option

* applied suggestions from feedback in PR #2736

* bold some release notes
  • Loading branch information
georgemccabe authored Oct 18, 2024
1 parent 86327e0 commit 5ba60f7
Show file tree
Hide file tree
Showing 9 changed files with 288 additions and 9 deletions.
68 changes: 68 additions & 0 deletions .github/ISSUE_TEMPLATE/documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
name: Documentation
about: Update the documentation
title: 'Documentation: '
labels: 'alert: NEED ACCOUNT KEY, alert: NEED MORE DEFINITION, alert: NEED CYCLE ASSIGNMENT, type: documentation'
assignees: ''

---

*Replace italics below with details for this issue.*

## Describe the Task ##
*Provide a description of the task here.*

### Time Estimate ###
*Estimate the amount of work required here.*
*Issues should represent approximately 1 to 3 days of work.*

### Sub-Issues ###
Consider breaking the task down into sub-issues.
- [ ] *Add a checkbox for each sub-issue here.*

### Relevant Deadlines ###
*List relevant project deadlines here or state NONE.*

### Funding Source ###
*Define the source of funding and account keys here or state NONE.*

## Define the Metadata ##

### Assignee ###
- [ ] Select appropriate **assignee** for this issue

### Labels ###
- [ ] Review default **alert** labels
- [ ] Select **component(s)**
- [ ] Select **priority**
- [ ] Select **requestor(s)**

### Milestone and Projects ###
- [ ] Select **Milestone** as a **METplus-Wrappers-X.Y.Z** version, **Consider for Next Release**, or **Backlog of Development Ideas**
- [ ] For a **METplus-Wrappers-X.Y.Z** version, select the **METplus-Wrappers-X.Y.Z Development** project

## Define Related Issue(s) ##
Consider the impact to the other METplus components.
- [ ] [METplus](https://github.com/dtcenter/METplus/issues/new/choose), [MET](https://github.com/dtcenter/MET/issues/new/choose), [METdataio](https://github.com/dtcenter/METdataio/issues/new/choose), [METviewer](https://github.com/dtcenter/METviewer/issues/new/choose), [METexpress](https://github.com/dtcenter/METexpress/issues/new/choose), [METcalcpy](https://github.com/dtcenter/METcalcpy/issues/new/choose), [METplotpy](https://github.com/dtcenter/METplotpy/issues/new/choose)

## Task Checklist ##
See the [METplus Workflow](https://metplus.readthedocs.io/en/latest/Contributors_Guide/github_workflow.html) for details.
- [ ] Complete the issue definition above, including the **Time Estimate** and **Funding Source**.
- [ ] Fork this repository or create a branch of **develop**.
Branch name: `feature_<Issue Number>_<Description>`
- [ ] Complete the development and test your changes.
- [ ] Add/update log messages for easier debugging.
- [ ] Add/update unit tests.
- [ ] Add/update documentation.
- [ ] Add any new Python packages to the [METplus Components Python Requirements](https://metplus.readthedocs.io/en/develop/Users_Guide/appendixA.html#metplus-components-python-packages) table.
- [ ] For any new datasets, an entry to the [METplus Verification Datasets Guide](https://metplus.readthedocs.io/en/latest/Verification_Datasets/index.html).
- [ ] Push local changes to GitHub.
- [ ] Submit a pull request to merge into **develop**.
Pull request: `feature <Issue Number> <Description>`
- [ ] Define the pull request metadata, as permissions allow.
Select: **Reviewer(s)** and **Development** issue
Select: **Milestone** as the next official version
Select: **METplus-Wrappers-X.Y.Z Development** project for development toward the next official release
- [ ] Iterate until the reviewer(s) accept and merge your changes.
- [ ] Delete your fork or branch.
- [ ] Close this issue.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/enhancement_request.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: Enhancement request
about: Improve something that it's currently doing
title: ''
title: 'Enhancement: '
labels: 'alert: NEED ACCOUNT KEY, alert: NEED MORE DEFINITION, alert: NEED CYCLE ASSIGNMENT, type: enhancement'
assignees: ''

Expand Down
2 changes: 1 addition & 1 deletion .github/jobs/docker_build_metplus_images.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ fi
metplus_version=${SOURCE_BRANCH:1}

# Get MET tag and adjust MET Docker repo if develop
met_tag=$("${GITHUB_WORKSPACE}"/metplus/component_versions.py -v "${metplus_version}" -o MET -f "{X}.{Y}-latest")
met_tag=$("${GITHUB_WORKSPACE}"/metplus/component_versions.py -v "${metplus_version}" -o MET -f "{X}.{Y}-latest" --no-get_dev_version)
echo "$met_tag"

MET_DOCKER_REPO=met
Expand Down
2 changes: 1 addition & 1 deletion .github/jobs/docker_setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ echo "TIMING: docker pull ${DOCKERHUB_TAG} took `printf '%02d' $(($duration / 60
export DOCKERFILE_PATH=${GITHUB_WORKSPACE}/internal/scripts/docker/Dockerfile

metplus_version=$(head -n 1 "${GITHUB_WORKSPACE}/metplus/VERSION")
MET_TAG=$("${GITHUB_WORKSPACE}"/metplus/component_versions.py -v "${metplus_version}" -o MET -f "{X}.{Y}-latest")
MET_TAG=$("${GITHUB_WORKSPACE}"/metplus/component_versions.py -v "${metplus_version}" -o MET -f "{X}.{Y}-latest" --no-get_dev_version)

MET_DOCKER_REPO=met-dev
if [ "${MET_TAG}" != "develop" ]; then
Expand Down
57 changes: 57 additions & 0 deletions docs/Users_Guide/release-notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,63 @@ METplus Wrappers Release Notes
When applicable, release notes are followed by the
`GitHub issue <https://github.com/dtcenter/METplus/issues>`__ number which
describes the bugfix, enhancement, or new feature.
Important issues are listed **in bold** for emphasis.

METplus Version 6.0.0 beta6 Release Notes (2024-10-18)
------------------------------------------------------

.. dropdown:: Enhancement

* Support for setting Point2Grid MET config variables
(`#2540 <https://github.com/dtcenter/METplus/issues/2540>`_)
* Support processing groups of forecast leads
(`#2612 <https://github.com/dtcenter/METplus/issues/2612>`_)
* Support separate climatology datasets for both the forecast and observation inputs
(`#2622 <https://github.com/dtcenter/METplus/issues/2622>`_)
* Support the new `-aggr` command line option in SeriesAnalysis wrapper
(`#2651 <https://github.com/dtcenter/METplus/issues/2651>`_)
* **Deprecate master_metplus.py**
(`#2714 <https://github.com/dtcenter/METplus/issues/2714>`_)
* Support for setting point_weight_flag and obtype_as_group_val_flag in PointStat and EnsembleStat
(`#2727 <https://github.com/dtcenter/METplus/issues/2727>`_)

.. dropdown:: Bugfix

* ASCII2NC file window bad default value and redundant initialization of wrappers
(`#2520 <https://github.com/dtcenter/METplus/issues/2520>`_)
* Inconsistent RUN_ID values when using instances
(`#2596 <https://github.com/dtcenter/METplus/issues/2596>`_)
* Fix GridStat_SeriesAnalysis _fcstNMME_obsCPC _seasonal_forecast use cases with poorly configured climatology settings
(`#2695 <https://github.com/dtcenter/METplus/issues/2695>`_)
* Improve SeriesAnalysis field info generation with regards to time
(`#2705 <https://github.com/dtcenter/METplus/issues/2705>`_)
* Clean up existing use cases wrt SonarQube
(`#2710 <https://github.com/dtcenter/METplus/issues/2710>`_)

.. dropdown:: New Wrapper

NONE

.. dropdown:: New Use Case

* Multivariate MODE for RRFS
(`#2647 <https://github.com/dtcenter/METplus/issues/2647>`_)

.. dropdown:: Documentation

* Add information to Contributor's Guide for adding new use cases that utilize METplotpy/METcalcpy/METdataio
(`#1882 <https://github.com/dtcenter/METplus/issues/1882>`_)
* Update Release Guide for MET releases to update version numbers in the installation.rst in the MET User's Guide
(`#2452 <https://github.com/dtcenter/METplus/issues/2452>`_)
* Update Documentation Overview and Conventions
(`#2489 <https://github.com/dtcenter/METplus/issues/2489>`_)
* Update the User Support section in the Contributor's Guide
(`#2679 <https://github.com/dtcenter/METplus/issues/2679>`_)

.. dropdown:: Internal

NONE


METplus Version 6.0.0 Beta 5 Release Notes (2024-07-10)
-------------------------------------------------------
Expand Down
129 changes: 129 additions & 0 deletions internal/scripts/dev_tools/generate_release_notes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
#! /usr/bin/env python3

import sys
import os

from github import Github
from datetime import datetime, timezone

GITHUB_ORG = 'dtcenter'

CATEGORIES = (
'Enhancement',
'Bugfix',
'New Wrapper',
'New Use Case',
'Documentation',
'Internal',
)


def main(dev_name, dev_start_date, dev_end_date=datetime.today(), repo_name='METplus'):
token = os.getenv('GITHUB_TOKEN')
if not token:
print("ERROR: Must set GITHUB_TOKEN environment variable")
sys.exit(1)

all_issues = get_all_issues_since_dev_start(token, repo_name, dev_start_date)
issues_by_category = get_issues_by_category(all_issues)

print_banner('ADD THIS TO docs/Users_Guide/release-notes.rst')

print_header(repo_name, dev_name, dev_end_date)
print_issues_by_category(repo_name, issues_by_category)

print_banner('ADD THIS TO METplus Coordinated Release Acceptance Testing')

print_release_testing(repo_name, dev_name, all_issues)


def get_all_issues_since_dev_start(token, repo_name, dev_start_date):
print(f"Finding issues in {GITHUB_ORG}/{repo_name} that were closed after {dev_start_date.strftime('%Y-%m-%d')}...")
github_obj = Github(token)
org = github_obj.get_organization(GITHUB_ORG)
repo = org.get_repo(repo_name)

all_issues = repo.get_issues(state='closed', since=dev_start_date)
all_issues = [issue for issue in all_issues if issue.pull_request is None]
all_issues = [issue for issue in all_issues if not issue.title.startswith('Update Truth')]
all_issues.sort(key=lambda x: x.number)
return all_issues


def print_banner(msg):
print(f"\n{'*' * len(msg)}\n{msg}\n{'*' * len(msg)}\n")


def print_header(repo_name, dev_name, dev_end_date):
dev_fmt = dev_name.replace('-', '').replace('beta', 'Beta ').replace('rc', 'RC ')
header = f"{repo_name} Version {dev_fmt} Release Notes ({dev_end_date.strftime('%Y-%m-%d')})"
print(header)
print('-' * len(header))


def print_issues_by_category(repo_name, issues_by_category):
for category, issues in issues_by_category.items():
print()
if category != 'none':
print(f" .. dropdown:: {category}\n")
else:
print('COULD NOT PARSE CATEGORY FROM THESE:\n')
if issues is None:
print(' NONE')
continue
for issue in issues:
title = issue.title.removeprefix(category).lstrip(' :')
num = issue.number
print(f" * {title}")
print(f" (`#{num} <https://github.com/{GITHUB_ORG}/{repo_name}/issues/{num}>`_)")


def get_issues_by_category(all_issues):
issues_by_category = dict.fromkeys(CATEGORIES)
issues_by_category['none'] = []
for issue in all_issues:
found_cat = False
for category in CATEGORIES:
if issue.title.startswith(f"{category}:"):
if issues_by_category[category] is None:
issues_by_category[category] = []
issues_by_category[category].append(issue)
found_cat = True
break
if not found_cat:
if issues_by_category['none'] is None:
issues_by_category['none'] = []
issues_by_category['none'].append(issue)
return issues_by_category


def print_release_testing(repo_name, dev_name, all_issues):
dev_info = dev_name.split('-')[1]
for issue in all_issues:
num = issue.number
print(f"| **OPEN** || [#{num}](https://github.com/{GITHUB_ORG}/{repo_name}/issues/{num}) | {dev_info} |||")


if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('dev_name',
help="name of development cycle, e.g. 6.0.0-beta6")
parser.add_argument('start_date',
help="YYYYMMDD date when development cycle started")
parser.add_argument('-r', '--repo',
default='METplus',
help="Repository to parse, default is METplus")
args = parser.parse_args()

ymd = args.start_date
try:
year = int(ymd[:4])
month = int(ymd[4:6])
day = int(ymd[6:8])
except ValueError:
print('ERROR: Argument must be YYYYMMDD')
sys.exit(1)

start_date = datetime(year, month, day, tzinfo=timezone.utc)
main(repo_name=args.repo, dev_name=args.dev_name, dev_start_date=start_date)
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,18 @@ def test_get_coordinated_version(component, version, expected_result):
@pytest.mark.util
def test_get_component_version(input_component, input_version, output_component, output_format, expected_result):
assert component_versions.get_component_version(input_component, input_version, output_component, output_format) == expected_result


@pytest.mark.parametrize(
'input_version, get_dev, expected_result', [
('5.1.0', True, 'v5.1.0'),
('5.1.0', False, 'v5.1.0'),
('5.1.0-beta3', True, 'v5.1.0-beta3'),
('5.1.0-beta3', False, 'develop'),
('5.1.0-rc1', True, 'v5.1.0-rc1'),
('5.1.0-rc1', False, 'develop'),
]
)
@pytest.mark.util
def test_get_component_version_get_dev(input_version, get_dev, expected_result):
assert component_versions.get_component_version('METplus', input_version, 'METplus', get_dev=get_dev) == expected_result
2 changes: 1 addition & 1 deletion metplus/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6.0.0-beta6-dev
6.0.0-beta6
20 changes: 15 additions & 5 deletions metplus/component_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
DEFAULT_OUTPUT_FORMAT = "v{X}.{Y}.{Z}{N}"

def get_component_version(input_component, input_version, output_component,
output_format=DEFAULT_OUTPUT_FORMAT):
output_format=DEFAULT_OUTPUT_FORMAT, get_dev=True):
"""!Get the version of a requested METplus component given another METplus
component and its version. Parses out X.Y version numbers of input version
to find desired version. Optionally specific format of output content.
Expand All @@ -51,10 +51,13 @@ def get_component_version(input_component, input_version, output_component,
{X}, {Y}, and {Z} will be replaced with x, y, and z version numbers from
X.Y.Z. {N} will be replaced with development version if found in the
input version, e.g. "-beta3" or "-rc1"
@param get_dev (optional) if True, get corresponding -beta or -rc version.
If False, return "develop" if input is beta or rc.
@returns string of requested version number, or "develop" if input version
ends with "-dev", or None if version number could not be determined.
"""
if input_version.endswith('-dev'):
if ('-dev' in input_version or
(not get_dev and any(ext in input_version for ext in ['-beta', '-rc']))):
return 'develop'
coord_version = get_coordinated_version(input_component, input_version)
versions = VERSION_LOOKUP.get(coord_version)
Expand Down Expand Up @@ -93,10 +96,12 @@ def main():
parser = argparse.ArgumentParser()
parser.add_argument('-i', '--input_component',
default='metplus',
help='Name of METplus component to use to find version')
help='Name of METplus component to use to find version,'
' default is METplus.')
parser.add_argument('-v', '--input_version',
default=next(iter(VERSION_LOOKUP)),
help='version of input_component to search')
help='version of input_component to search,'
' default is upcoming version')
parser.add_argument('-o', '--output_component', required=True,
help='name of METplus component to obtain version')
parser.add_argument('-f', '--output_format',
Expand All @@ -106,9 +111,14 @@ def main():
' z version numbers from X.Y.Z. {N} will be '
'replaced with development version if found in the'
'input version, e.g. "-beta3" or "-rc1"')
parser.add_argument('--get_dev_version', action=argparse.BooleanOptionalAction,
default=True,
help='If True, get corresponding -beta or -rc version. '
'If False, return develop if development version.')
args = parser.parse_args()
return get_component_version(args.input_component, args.input_version,
args.output_component, args.output_format)
args.output_component, args.output_format,
args.get_dev_version)


if __name__ == "__main__":
Expand Down

0 comments on commit 5ba60f7

Please sign in to comment.