# Release scorecard definitions

* [Spreadsheet](https://docs.google.com/spreadsheets/d/1Aoj4OTaWoyrKIcQNiHH1MVoRG51T20Y_0w2tg5oVw-M/edit#gid=825551144)

## Release Hearbeat

| Metric                                                                     | How to Measure                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Date/time RC1 cut (PST)                                                    | Check Release Self-Hosted/Cloud channel history for date/time RC1 was cut.                                                                                                                                                                                                                                                                                                                                                                                                             |
| Total number of RCs cut                                                    | Check Release Self-Hosted/Cloud channel history for how many RCs were cut for that release.                                                                                                                                                                                                                                                                                                                                                                                            |
| Number of days the final build is cut before the 16th                      | Check Release Self-Hosted/Cloud channel for post with official release build. Oxygen = 16th - Day Final RC is cut                                                                                                                                                                                                                                                                                                                                                                      |
| Community + customer bugs reported during release timeframe (17th to 16th) | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Bug</li><li>Label = Customer-bug and Community-bug</li><li>Created Date = 17th of the previous month</li><li>Release Date = 16th of the current month</li></ol>                                                                                                                                                                       |
| Number of bugs reported within a week after the release                    | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Issue Type = Bug</li><li>Status = Open</li><li>Label = Customer-bug and Community-bug</li><li>Created Date = Between the 16th and 23rd of the month</li></ol>                                                                                                                                                                                                                                     |
| Number of customer bugs fixed during release                               | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Bug</li><li>Status = Closed and Resolved</li><li>Label = Customer-bug</li></ol>                                                                                                                                                                                                                                                       |
| Total valid bug fixes in fix version                                       | <p>After closing current release:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion()</p>                                                                                                                                                                                                                                                                                       |
| Total valid regression fixes in fix version                                | <p>After closing current release:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion() AND label = rc2, rc3</p>                                                                                                                                                                                                                                                                  |
| Valid bugs found after RC1 is pushed to next release                       | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND created > "START" AND created < "END" AND fixVersion = earliestUnreleasedVersion()</p>                                                                                                                                                                                     |
| Valid bugs found after RC1 fix version = other (eg unscheduled, not set)   | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><p>project = Mattermost AND issuetype = Bug AND created > "START" AND created < "END" AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND (fixVersion not in (latestReleasedVersion(), earliestUnreleasedVersion()) OR fixVersion is EMPTY)</p>                                                                                                                            |
| Total valid bugs found after RC1 is cut                                    | <p>After closing current release, adjust dates as per above, and use this Jira query:</p><ol><li>Check Jira timezone + Community server timezone and make sure times match</li><li>Replace START with date (yyyy-MM-dd HH:mm) RC1 was cut</li><li>Replace END with date (yyyy-MM-dd HH:mm) release was published</li></ol><p>project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't fix") AND created > "START" AND created < "END"</p> |
| Number of PRs reverted                                                     | Check recently merged GitHub PRs (with the word "Revert" in the PR title).                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                                                            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| (Non-security) Bugs requiring patch release                                | After any patch release goes out (after the normal release date): Check Changelog for total number of non-security patch releases.                                                                                                                                                                                                                                                                                                                                                     |
| Total features/improvements in fix version                                 | <p>With a new or existing Jira filter, with:</p><ol><li>Project = Mattermost</li><li>Fix Versions = Latest released version</li><li>Issue Type = Story</li><li>Status = Closed and Resolved</li></ol>                                                                                                                                                                                                                                                                                  |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://handbook.mattermost.com/operations/research-and-development/product/release-process/release-scorecard-definitions.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.