Release scorecard definitions
Last updated
Was this helpful?
Last updated
Was this helpful?
Date/time RC1 cut (PST)
Check Release Self-Hosted/Cloud channel history for date/time RC1 was cut.
Total number of RCs cut
Check Release Self-Hosted/Cloud channel history for how many RCs were cut for that release.
Number of days the final build is cut before the 16th
Check Release Self-Hosted/Cloud channel for post with official release build. Oxygen = 16th - Day Final RC is cut
Community + customer bugs reported during release timeframe (17th to 16th)
With a new or existing Jira filter, with:
Project = Mattermost
Fix Versions = Latest released version
Issue Type = Bug
Label = Customer-bug and Community-bug
Created Date = 17th of the previous month
Release Date = 16th of the current month
Number of bugs reported within a week after the release
With a new or existing Jira filter, with:
Project = Mattermost
Issue Type = Bug
Status = Open
Label = Customer-bug and Community-bug
Created Date = Between the 16th and 23rd of the month
Number of customer bugs fixed during release
With a new or existing Jira filter, with:
Project = Mattermost
Fix Versions = Latest released version
Issue Type = Bug
Status = Closed and Resolved
Label = Customer-bug
Total valid bug fixes in fix version
After closing current release:
project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion()
Total valid regression fixes in fix version
After closing current release:
project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND fixVersion = latestReleasedVersion() AND label = rc2, rc3
Valid bugs found after RC1 is pushed to next release
After closing current release, adjust dates as per above, and use this Jira query:
project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND created > "START" AND created < "END" AND fixVersion = earliestUnreleasedVersion()
Valid bugs found after RC1 fix version = other (eg unscheduled, not set)
After closing current release, adjust dates as per above, and use this Jira query:
project = Mattermost AND issuetype = Bug AND created > "START" AND created < "END" AND resolution not in (Duplicate, "Cannot Reproduce", "Won't Fix") AND (fixVersion not in (latestReleasedVersion(), earliestUnreleasedVersion()) OR fixVersion is EMPTY)
Total valid bugs found after RC1 is cut
After closing current release, adjust dates as per above, and use this Jira query:
Check Jira timezone + Community server timezone and make sure times match
Replace START with date (yyyy-MM-dd HH:mm) RC1 was cut
Replace END with date (yyyy-MM-dd HH:mm) release was published
project = Mattermost AND issuetype = Bug AND resolution not in (Duplicate, "Cannot Reproduce", "Won't fix") AND created > "START" AND created < "END"
Number of PRs reverted
Check recently merged GitHub PRs (with the word "Revert" in the PR title).
(Non-security) Bugs requiring patch release
After any patch release goes out (after the normal release date): Check Changelog for total number of non-security patch releases.
Total features/improvements in fix version
With a new or existing Jira filter, with:
Project = Mattermost
Fix Versions = Latest released version
Issue Type = Story
Status = Closed and Resolved