Release scorecard definitions

• Spreadsheet

Release Hearbeat

Metric	How to Measure
Date/time RC1 cut (PST)	Check Release Self-Hosted/Cloud channel history for date/time RC1 was cut.
Total number of RCs cut	Check Release Self-Hosted/Cloud channel history for how many RCs were cut for that release.
Number of days the final build is cut before the 16th	Check Release Self-Hosted/Cloud channel for post with official release build. Oxygen = 16th - Day Final RC is cut
Community + customer bugs reported during release timeframe (17th to 16th)	With a new or existing Jira filter, with: Project = Mattermost Fix Versions = Latest released version Issue Type = Bug Label = Customer-bug and Community-bug Created Date = 17th of the previous month Release Date = 16th of the current month
Number of bugs reported within a week after the release	With a new or existing Jira filter, with: Project = Mattermost Issue Type = Bug Status = Open Label = Customer-bug and Community-bug Created Date = Between the 16th and 23rd of the month
Number of customer bugs fixed during release	With a new or existing Jira filter, with: Project = Mattermost Fix Versions = Latest released version Issue Type = Bug Status = Closed and Resolved Label = Customer-bug
Total valid bug fixes in fix version	After closing current release: project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion()
Total valid regression fixes in fix version	After closing current release: project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion() AND label = rc2, rc3
Valid bugs found after RC1 is pushed to next release	After closing current release, adjust dates as per above, and use this Jira query: project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND created > "START" AND created < "END" AND fixVersion = earliestUnreleasedVersion()
Valid bugs found after RC1 fix version = other (eg unscheduled, not set)	After closing current release, adjust dates as per above, and use this Jira query: project = Mattermost AND issuetype = Bug AND created > "START" AND created < "END" AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND (fixVersion not in (latestReleasedVersion(), earliestUnreleasedVersion()) OR fixVersion is EMPTY)
Total valid bugs found after RC1 is cut	After closing current release, adjust dates as per above, and use this Jira query: Check Jira timezone + Community server timezone and make sure times match Replace START with date (yyyy-MM-dd HH:mm) RC1 was cut Replace END with date (yyyy-MM-dd HH:mm) release was published project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't fix") AND created > "START" AND created < "END"
Number of PRs reverted	Check recently merged GitHub PRs (with the word "Revert" in the PR title).
(Non-security) Bugs requiring patch release	After any patch release goes out (after the normal release date): Check Changelog for total number of non-security patch releases.
Total features/improvements in fix version	With a new or existing Jira filter, with: Project = Mattermost Fix Versions = Latest released version Issue Type = Story Status = Closed and Resolved