# Release scorecard definitions

* [Spreadsheet](https://docs.google.com/spreadsheets/d/1Aoj4OTaWoyrKIcQNiHH1MVoRG51T20Y_0w2tg5oVw-M/edit#gid=825551144)

## Release Hearbeat

| Metric                                                                     | How to Measure                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Date/time RC1 cut (PST)                                                    | Check Release Self-Hosted/Cloud channel history for date/time RC1 was cut.                                                                                                                                                                                                                                                                                                                                                                                                             |
| Total number of RCs cut                                                    | Check Release Self-Hosted/Cloud channel history for how many RCs were cut for that release.                                                                                                                                                                                                                                                                                                                                                                                            |
| Number of days the final build is cut before the 16th                      | Check Release Self-Hosted/Cloud channel for post with official release build. Oxygen = 16th - Day Final RC is cut                                                                                                                                                                                                                                                                                                                                                                      |
| Community + customer bugs reported during release timeframe (17th to 16th) | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Bug</li><li>Label = Customer-bug and Community-bug</li><li>Created Date = 17th of the previous month</li><li>Release Date = 16th of the current month</li></ol>                                                                                                                                                                       |
| Number of bugs reported within a week after the release                    | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Issue Type = Bug</li><li>Status = Open</li><li>Label = Customer-bug and Community-bug</li><li>Created Date = Between the 16th and 23rd of the month</li></ol>                                                                                                                                                                                                                                     |
| Number of customer bugs fixed during release                               | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Bug</li><li>Status = Closed and Resolved</li><li>Label = Customer-bug</li></ol>                                                                                                                                                                                                                                                       |
| Total valid bug fixes in fix version                                       | <p>After closing current release:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion()</p>                                                                                                                                                                                                                                                                                       |
| Total valid regression fixes in fix version                                | <p>After closing current release:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion() AND label = rc2, rc3</p>                                                                                                                                                                                                                                                                  |
| Valid bugs found after RC1 is pushed to next release                       | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND created > "START" AND created < "END" AND fixVersion = earliestUnreleasedVersion()</p>                                                                                                                                                                                     |
| Valid bugs found after RC1 fix version = other (eg unscheduled, not set)   | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><p>project = Mattermost AND issuetype = Bug AND created > "START" AND created < "END" AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND (fixVersion not in (latestReleasedVersion(), earliestUnreleasedVersion()) OR fixVersion is EMPTY)</p>                                                                                                                            |
| Total valid bugs found after RC1 is cut                                    | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><ol><li>Check Jira timezone + Community server timezone and make sure times match</li><li>Replace START with date (yyyy-MM-dd HH:mm) RC1 was cut</li><li>Replace END with date (yyyy-MM-dd HH:mm) release was published</li></ol><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't fix") AND created > "START" AND created < "END"</p> |
| Number of PRs reverted                                                     | Check recently merged GitHub PRs (with the word "Revert" in the PR title).                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                                                            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| (Non-security) Bugs requiring patch release                                | After any patch release goes out (after the normal release date): Check Changelog for total number of non-security patch releases.                                                                                                                                                                                                                                                                                                                                                     |
| Total features/improvements in fix version                                 | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Story</li><li>Status = Closed and Resolved</li></ol>                                                                                                                                                                                                                                                                                  |