Releasing Process

Versioning

The ZINC Grader uses a versioning scheme of YYYY.MM.m, where:

  • YYYY: Year of the release

  • MM: Month of the release

  • m: Minor version of the release

The release cadence of the Grader is currently set as follows:

  • Before the beginning of a regular semester (i.e. September and February), a new major release will be created based on the date of the release.

  • All subsequent releases during the semester can only be patch releases. All breaking changes must be held off until the semester is over.

Major Releases

Monthly releases are the first release of the month. YYYY.MM will be updated to reflect the year and month of the release, and m will be reset to zero.

For example, the first release of January 2021 will be versioned as 2021.01.0.

A new monthly release can only occur before or after a regular semester.

Patch Releases

Minor releases are releases which occur during a month, after a monthly release has been made. YYYY.MM will remain the same to indicate that the release is within the same year/month, and m will be incremented by one.

For example, the second/third release of January 2021 will be versioned as 2021.01.1 and 2021.01.2 respectively.

Release Branches

There are three branches which are used when making a release.

  • master: The default branch; contains all latest changes.

  • stable: The branch where the current stable version is deployed.

  • staging/<version>: Staging area for cherry-picking master patches into stable.

Making a New Release

Prerequisites

Before making a new release, ensure the following requirements are met:

  • There must be some user-visible changes/bugfixes between the last release and the expected release

    • Documentation update (regardless of KDoc/inline/user documentation) do not constitute a new release

    • Feature deprecation do not qualify for a new release

    • Feature addition/removal and bugfixes which causes user-visible behavioral changes do qualify for a new release

  • No unit/integration test failures in master or staging/<version>

Changes which do not qualify for a new release should await for the next release to be aggregated altogether.

Release Creation

Major Release

First, re-run all tests (unit and integration) locally to make sure there are no errors in any tests. Once this is verified, push to origin, and ensure that the CI pipeline of master is executed correctly. If any errors arise from the CI pipeline, they must be fixed before the next step. Tasks like pages where it is non-deterministically prone to failure can be re-run.

After all verification is complete, create a new branch named staging/<version> from master, where <version> is the version of the major release, and push this branch to origin.

While the CI is running, bump the version number in gradle.properties as specified in the Release Process Versioning section, change the CURRENT_STABLE variable to <version> in .gitlab-ci.yml, and commit the version bump. Update and commit the ChangeLog in the same commit, if applicable.

If any errors are discovered, undo the commit of the version bump before investigating and/or patching the bug.

After all verification is complete, create a new branch <version> from staging/<version>, and push the new branch. Wait for the pipeline to succeed, and make sure that the Grader in the deployment environment can startup as expected.

Tag the release in <version> with the version number. Do not prepend v to the tag. Finally, cherry-pick the changelog commit (and not the version bump) back to master, and update the version in the master branch to x.y.99, where x.y is the current major release version.

Patch Release

Since patch releases can only contain cherry-picked commits, care must be taken to ensure that the picked commits can be applied on top of a previous release, and that no breaking or major changes in functionality should be accepted.

First, cherry-pick all relevant commits from master to staging/<version>. If changes need to be made to allow for backporting, cherry-pick the commit, make the minimal necessary changes, and squash the commits. Re-run all tests locally before pushing to origin.

While the CI pipeline of staging/<version> is running, bump the version number in gradle.properties as specified in the Release Process Versioning section, commit the version bump. Update and commit the ChangeLog in the same commit, if applicable.

If any errors are discovered, undo the commit of the version bump and remove the newly-created tag before investigating and/or patching the bug.

After all verification is complete, create a merge request to merge staging/<version> into <version>. Wait for the pipeline to succeed before approving the merge. Approval from other developers are optional.

Once the merge is successful, wait for the <version> branch CI to execute, and make sure that the Grader in the deployment environment can startup as expected. Tag the release in <version> with the version number. Do not prepend v to the tag. Finally, cherry-pick the changelog commit (and not the version bump) back to master.

Release ChangeLog

You may also want to write-up some release notes (although this will only be visible for developers, as users generally have no notion of the Grader’s version). This should be added for the ChangeLog and the releases page of the repository.

The ChangeLog.rst file should be committed with the version bump.

Release Notes

In general, this should consist of four sections:

  • Additions: New features/APIs

  • Changes: Changes in behavior for existing features

  • Removals: Feature/API removals

  • Deprecations: Features/APIs which are slated to be removed in the future

Optional sections may be added if required by the release.

In general, follow the guidelines below:

  • For release headers:

    • Use h1 (i.e. =)

    • The header content should be <version> (<date in ISO-8601>)

  • For release sections

    • Use h2 (i.e. -)

  • Use the following format for each point if possible:

    [TAGS...] component: feature description (#issue, !merge request) - If the description cannot be fully explained in one line, use sub bullet points

  • Use prefixed tags to indicate important information regarding a feature/bugfix

    • [BREAKING]: Should be used to denote an unanticipated breaking change in behavior

    • [API]: Should be used to denote changes to the API, specifically changes to the public API in :*:*-api modules, or changes to the communication between the Grader and other components

    • [Incubating]: Features which are implemented but not yet stable

Release Notes Example

2021.01.0 (2021-01-30)
----------------------

Additions
^^^^^^^^^

- Scoring: Added ability to specify score weighting for XUnit-based pipeline stages
- [API] Daemon: Added HTTP API for status checking
- [Incubating] Scoring: Added new score generation algorithm
    - Users can opt-in by setting ``pipelineStage.scoring.useNewAlgorithm=true`` in ``config.properties``

Removals
^^^^^^^^

- **[BREAKING]** Report: Removed generation of ``scoreUnits`` by default
    - Users must now opt-in in ``config.properties`` using the key ``pipelineStage.generateScoreUnits``

Deprecations
^^^^^^^^^^^^

- ``score`` field for all pipeline stages are deprecated
    - Users are encouraged to migrate to use ``totalScore`` field instead
    - This field will be removed in a version after version series 2021.02