Skip to content

Development

See Contributing for how to contribute; this page describes the development setup, the checks, the documentation and the versions.

Development needs Python 3.13 or later and uv (e.g., pipx install uv; see its installation guide). To set up a development environment with the versions of the lock (uv.lock, see Dependencies), as CI does, and run the checks that CI runs:

uv sync --locked --extra dev  # the app, with the dev dependencies, in `.venv`
source .venv/bin/activate  # or `activate.fish` etc.
pre-commit install  # run the checks on every commit
pre-commit run --all-files  # ruff, mypy, semgrep, ast-grep, actionlint
pre-commit run --all-files --hook-stage manual gitleaks-all  # secrets
coverage run -m pytest  # the tests, with coverage
coverage report

Instead of activating .venv, commands can be run with uv run, which syncs the environment first, so give it the extra (e.g., uv run --locked --extra dev pytest). pip install -e ".[dev]" also works, but installs the newest versions in the ranges of pyproject.toml instead of the locked ones, so its results can differ from CI's.

The tests (tests/, mirroring the package) need no database: app tests create the app from the base config (trs_filer/config.yaml) and the test overlay (tests/config_overlay.yaml, with the settings each deployment must make, for a fictitious identity provider that the tests mock), and send requests through Flask's test client, with access tokens that the tests sign (make_headers() in tests/conftest.py), and with an in-memory database (mongomock); they have no network access (see tests/conftest.py). CI also runs them against MongoDB 7.0; to do so locally, set MONGODB_TEST_URI to a single-node MongoDB without other data, one per test run (the tests drop the database trsStore before each test, and refuse to start if it exists), e.g.:

docker run -d --rm --name mongodb-test -p 127.0.0.1:27017:27017 mongo:7.0
until docker exec mongodb-test mongosh --quiet --eval 'db.adminCommand({ping: 1}).ok'; do sleep 1; done
MONGODB_TEST_URI=mongodb://localhost:27017 pytest
docker stop mongodb-test

The tests of object storage (tests/test_storage.py) use an in-memory S3 (moto). CI also runs them against SeaweedFS (with MongoDB, step "Start S3-compatible server" of .github/workflows/ci.yml, whose image is the version of the Docker Compose example in File storage; bump both by hand); to do so locally, set S3_TEST_ENDPOINT_URL to an S3-compatible server without other data (the tests create and delete the bucket trs-filer-test), and, if it checks credentials, S3_TEST_ACCESS_KEY_ID and S3_TEST_SECRET_ACCESS_KEY (test by default), e.g.:

docker run -d --rm --name s3-test -p 127.0.0.1:8333:8333 chrislusf/seaweedfs:4.51 server -s3 -ip.bind=0.0.0.0
until curl -s -o /dev/null http://127.0.0.1:8333; do sleep 1; done
S3_TEST_ENDPOINT_URL=http://127.0.0.1:8333 pytest tests/test_storage.py
docker stop s3-test

Coverage is measured for trs_filer, scripts and the docs hooks (docs/hooks.py), and must stay at 100%: coverage report and coverage xml fail below it (fail_under in pyproject.toml), and so do Codecov's checks of the whole code and of each pull request's changes (codecov.yml; CI reports coverage to Codecov, see the badge in the README). New code, also new scripts, needs tests that cover it completely; add tests that fail without your change, and exclude unreachable code only with a reason (# pragma: no cover). CI also runs the app with Docker Compose and sends requests to it (job "Smoke test" of .github/workflows/ci.yml: public reads, writes refused without a token, then writes with authentication disabled, see Running without authentication); this test runs in CI only, with no local equivalent other than docker compose up --build --wait and requests by hand.

To try changes in the running app, rebuild it with docker compose up --build -d. python -m trs_filer.app runs the app with Flask's development server instead of Gunicorn (see Server); it needs a MongoDB at db.host of the config (e.g., set MONGO_HOST=localhost for a MongoDB published on the host).

Run pre-commit in the environment with the dev dependencies: the mypy and ast-grep hooks use the installed packages. The first run sets up the other hooks, which takes a few minutes.

The checks are configured in .pre-commit-config.yaml and pyproject.toml. Project-specific rules are in .semgrep/ (security) and .ast-grep/ (anti-patterns such as HTTP status codes as strings, print() calls or calls of the root logger), each with tests. actionlint checks the workflows in .github/workflows/, and their shell scripts with ShellCheck if it is installed (as on GitHub-hosted runners, so CI may find more than a local run without it). To suppress a false positive, add a comment with the reason to the line: # noqa: <code> (ruff), # type: ignore[<code>] (mypy), # nosemgrep: <rule-id>, # ast-grep-ignore: <rule-id> or # gitleaks:allow.

Running without authentication

Writes need an access token (see Access control), also with Docker Compose, whose development overlay (config.dev.yaml) trusts a fictitious identity provider. To set your own, see Local overrides. To try writes locally without an identity provider, disable authentication with the overlay config.noauth.yaml, which sets FOCA's api.specs.trs.disable_auth (false in the base config), after config.dev.yaml, via TRS_FILER_CONFIG_OVERLAY (which docker-compose.yaml passes to the app as FOCA_CONFIG_OVERLAY):

TRS_FILER_CONFIG_OVERLAY=/etc/trs-filer/config.dev.yaml:/etc/trs-filer/config.noauth.yaml \
  docker compose up --build --wait

All operations are then public, writes too, and FOCA warns on startup that authentication is disabled (Authentication is disabled for spec 'trs'). Tools then get no owners, and the roles on tools can't be managed (/tools/{id}/permissions answers 403 Forbidden). Run docker compose up --wait without the variable to enable it again. Only use this on your machine (Docker Compose publishes the app on the loopback interface only), never for deployments.

Dependencies

The dependencies are declared, with version ranges, in pyproject.toml (which installs of the package, e.g., with pip, use) and locked in uv.lock, with exact versions and hashes, by uv (also a dev dependency, so that the lock pins the version of uv that CI uses). CI's lint, test and docs jobs install the locked versions, checked against their hashes (uv sync --locked, with uv of a pinned astral-sh/setup-uv, at the version of the lock), so that Dependabot's pull requests, which change only the lock, are tested on the versions they lock; the lint job also checks that the lock matches pyproject.toml (uv lock --check). The app itself is installed editable there, built with build requirements from the package index (not the lock), as pip install -e does; those jobs hold no publishing credentials. The build requirements of the app (build-system.requires) are locked the same way, as the dependency group build-system: CI's push job derives the version of dev images with them, exported by the uv image of the Dockerfile (pinned by digest) and installed with hash checks (by the pip of the runner's Python, ensurepip, nothing downloaded), so that no workflow step of a job that holds publishing credentials installs a package without hash checks (tests/test_workflows.py checks this, and that the group and build-system.requires are the same: keep them so). Two uv versions are in use: that of uv.lock (astral-sh/setup-uv, updated with the uv entry of Dependabot) and that of the Dockerfile stage (docker entry).

The image build, which these jobs run too, installs the locked runtime dependencies only (uv export --frozen --no-dev --no-emit-project, which leaves out the extras and the group build-system, then uv pip install --system --require-hashes --only-binary :all: --no-deps, see the Dockerfile), so two builds of the same commit get the same packages, and a new release of a dependency gets into the image only through a reviewed change of the lock. The image build builds the app's wheel with the locked build requirements (group build-system), installed with hash checks, too, and without network, so it downloads nothing that isn't hash-checked; it fails unless the base image has the locked FOCA release (scripts/image_requirements.py), so update the foca requirement, the lock and the base image together, by hand (Dependabot updates neither FOCA nor its image). The image contains no pip and no uv: the FOCA base image has none (since FOCA 0.20), and uv is mounted from the Dockerfile stage pinned by digest into the steps that use it, which the smoke test of CI checks (step "Check that image has no installer"; tests/test_image_build.py checks the Dockerfile). After adding, removing or changing a dependency in pyproject.toml, update the lock and commit both:

uv lock  # update `uv.lock` after changes to `pyproject.toml`
uv lock --upgrade-package <name>  # upgrade one locked package
uv lock --check  # what CI checks: the lock matches `pyproject.toml`

Dependabot updates pyproject.toml and uv.lock together, including the build-system group (uv entry of .github/dependabot.yml), and the uv image of the Dockerfile (docker entry).

Helm chart

The Helm chart is in deployment/, a thin chart over FOCA's library chart foca-common (see Helm library chart in FOCA's docs). deployment/Chart.yaml pins its version and deployment/Chart.lock locks it; helm dependency build deployment downloads it into deployment/charts/ (not committed), and deployment/charts.sha256 has its digest, which CI and the release workflow check, as OCI tags can be pushed again. Check changes with Helm, e.g.:

helm dependency build deployment
(cd deployment && sha256sum -c charts.sha256)
helm lint deployment --strict -f deployment/ci/test-values.yaml
helm template trs-filer deployment -f deployment/ci/test-values.yaml

To update the library chart, change its version in deployment/Chart.yaml (and the $comment and the copied parts of deployment/values.schema.json, which follows the library's app-values.schema.json), then run helm dependency update deployment and replace the digest in deployment/charts.sha256 with that of the new package (cd deployment && sha256sum charts/foca-common-<version>.tgz > charts.sha256). The chart renders the library's templates with the app's settings (trs-filer.settings in deployment/templates/_helpers.tpl), which also checks for values of earlier charts (trs-filer.checkLegacyValues) and, on upgrades, for releases of earlier charts (trs-filer.checkUpgrade).

CI's job "Chart" (.github/workflows/ci.yml) lints the chart as the release workflow does, with the values that users must pass (e.g., the name of MongoDB's Secret and a fictitious identity provider): helm lint deployment --strict -f deployment/ci/test-values.yaml. It also runs the tests that render it (tests/test_chart.py; skipped locally without Helm, failing without the library chart in deployment/charts/):

pytest tests/test_chart.py

CI's job "Chart install" installs the chart in a cluster: on kind, with the image built from the commit (loaded into the cluster's node, so that nothing is pushed), the values of deployment/ci/test-values.yaml and a Secret with MongoDB's credentials. It waits for the rollout, then checks the API through a port forward: the service info, public reads, and that writes need a token (401, as the test values trust a fictitious identity provider). It then upgrades the release with authentication disabled (config.api.specs.trs.disable_auth, see Running without authentication; for this test only) and checks the registration and retrieval of a tool, whose URL must have the host name of the values (hostName). To do the same locally, with Docker, kind, kubectl and Helm (in Bash):

kind create cluster --name trs-filer
docker build --build-arg SETUPTOOLS_SCM_PRETEND_VERSION=0.0.0+kind \
    --tag trs-filer:ci .
kind load docker-image trs-filer:ci --name trs-filer
kubectl create namespace trs-filer
kubectl create secret generic trs-filer-mongodb-credentials -n trs-filer \
    --from-literal=root-username=root \
    --from-literal=root-password="$(openssl rand -hex 24)" \
    --from-literal=username=trs-filer \
    --from-literal=password="$(openssl rand -hex 24)"
helm dependency build deployment
helm install trs-filer deployment -n trs-filer \
    -f deployment/ci/test-values.yaml \
    --set trsFiler.image=trs-filer:ci \
    --set trsFiler.imagePullPolicy=Never \
    --wait --timeout 600s
kubectl port-forward -n trs-filer service/trs-filer 8080:8080 &
until curl --fail http://localhost:8080/ga4gh/trs/v2/service-info; do
  sleep 2  # the port forward may still be starting
done
kind delete cluster --name trs-filer  # tear down

trsFiler.imagePullPolicy=Never makes the cluster use the loaded image (with the chart's default, Always, it would pull it from a registry); the image names its tag, so trsFiler.tag stays empty. Send further requests as in the job's step "Check API" (.github/workflows/ci.yml).

Documentation

This documentation is built with MkDocs and Material for MkDocs, from docs/ and mkdocs.yml, by Read the Docs (configured in .readthedocs.yaml) and in CI; warnings, e.g., of broken links, fail the build. The API reference is generated from the docstrings with mkdocstrings. To build it locally, or serve it with live reloading, in the dev environment set up as above, with the locked docs dependencies too (as CI does; uv sync removes the extras that it isn't given, so dev stays in the command; Read the Docs installs the docs extra with pip, from the ranges of pyproject.toml):

uv sync --locked --extra dev --extra docs
mkdocs build --strict  # into `site/`, ignored by Git
mkdocs serve

Parts of the docs are kept elsewhere, so that they are written only once: the introduction, the known deployments and the quick start of the home and getting started pages are sections of README.md (between --8<-- [start:<name>] and --8<-- [end:<name>] comments), and the deployment page is the README of the Helm chart (deployment/README.md), which docs/hooks.py includes, with its links to pages of the docs and to files of the repository adapted (tested in tests/test_docs_hooks.py). Links in the chart's README must be absolute (to https://trs-filer.readthedocs.io/en/latest/ and to files on branch dev on GitHub), as they must also work in the packaged chart.

Versioning and releases

The project adopts the semantic versioning scheme for versioning. Currently the service is in beta stage, so the API may change without further notice.

The version is stored in Git tags (vX.Y.Z, pre-releases vX.Y.ZrcN): the package version is derived from them (via setuptools-scm), and the service info reports it. Builds without Git metadata, such as local Docker builds (e.g., with Docker Compose), report 0.0.0.dev0; the workflows pass the version to the image build (build argument SETUPTOOLS_SCM_PRETEND_VERSION). Images on Docker Hub (elixircloud/trs-filer) are tagged:

  • X.Y.Z: release X.Y.Z; latest: the highest stable release (not an alpha, beta or release candidate) on main;
  • dev: the latest commit on dev; sha-<commit>: commit <commit> on dev (full SHA), to deploy a fixed commit. Only commits that are still the head of dev when their CI run reaches the image job get images.

The current release version is taken from one source, so that a release doesn't need to bump it in many places: the release tag and, on dev, the version in CITATION.cff (set before tagging, see below; from then on, the latest docs show the upcoming version, and before the first release, they show X.Y.Z). The docs and the chart's README (deployment/README.md) write it as a placeholder, version in double curly braces with a space inside each pair, and the chart version (e.g., 1.2.3-rc1 for 1.2.3rc1, as in the names of the packaged charts) as chart_version in the same braces (VARIABLE and CHART_VARIABLE in scripts/chart_readme.py; this page can't show them, as they would be replaced), also in commands and links. The docs hooks (docs/hooks.py) replace them with the version of CITATION.cff, and scripts/chart_readme.py with the release version when the chart is packaged; files read on GitHub (e.g., deployment/README.md) show the placeholders, and sections of README.md included as snippets can't use them. Comments, templates and tests use placeholders (e.g., 1.2.3) and refer to sections by anchor. tests/test_version_mentions.py fails if the current release version is mentioned as a release of the app (as a tag, image or chart, or as "release" or "version" in prose; not, e.g., as the version of a dependency) anywhere else than in CITATION.cff, docs/releases/, the tests of the release tooling, links to the notes and assets of a release, and migration headings (e.g., "From versions before X"), which are about that release itself.

Features and fixes are merged into dev. Branch main only ever moves forward to commits on dev that are released. First, shortly before the release, set its version and date in the citation metadata, which Zenodo archives with each release, and merge the change into dev; each version, also each pre-release (e.g., X.Y.Zrc1), needs its own such change:

python scripts/citation.py set X.Y.Z  # date: today (UTC), or --date YYYY-MM-DD

scripts/citation.py uses the standard library only and is tested in tests/test_citation.py; check (below) fails unless version is X.Y.Z and date-released is a date that is not in the future (UTC).

Then, to release version X.Y.Z from commit <sha> on dev (with that change), after CI passed for it:

git switch --detach <sha>
# The release workflow's checks that need no tag, as it runs them
python scripts/release_version.py outputs vX.Y.Z
python scripts/citation.py check X.Y.Z
python scripts/check_release_pins.py
! grep -n TODO docs/releases/X.Y.Z.md  # curated notes, if any
[ ! -f docs/releases/X.Y.Z.md ] || python scripts/release_notes.py check docs/releases/X.Y.Z.md
readme="$(mktemp -d)/README.md" && cp deployment/README.md "${readme}"
python scripts/chart_readme.py "${readme}" vX.Y.Z  # links of the chart
helm lint deployment --strict -f deployment/ci/test-values.yaml  # as `prepare`
helm package deployment --version X.Y.Z --app-version X.Y.Z -d "$(mktemp -d)"
git push origin <sha>:main  # fails unless it is a fast-forward
git tag -s vX.Y.Z <sha> -m "vX.Y.Z"
git push origin vX.Y.Z

Tags can't be moved or deleted (see the repository settings below), and the release workflow checks the tagged commit, so a check that fails after the tag is pushed fails again on re-runs: the release then needs a new patch version, with its own citation metadata and notes. Run these checks before tagging, therefore. The CI job "Release readiness" runs the pin and the notes checks, too, for the version in CITATION.cff while it has no tag (or its tag is on the checked commit): on pull requests and in forks (which often lack upstream's later tags) as warnings, on pushes to dev as errors, which fail the CI run that the release workflow requires for the release commit. The image scan of the release workflow can still fail after the tag, e.g., on a vulnerability published since CI last scanned the image; the remedy is the same.

The tag must be annotated and signed (git tag -s), with a key that GitHub can verify (added to the tagger's GitHub account). The release workflow, triggered by the tag, checks that GitHub verified the signature, that the signed tag object has the name of the tag, that the tag's commit is on main and its CITATION.cff has the release version and a date that is not in the future (UTC), that the dependencies are releases (see below), and that the CI workflow passed for it (a run for the push of the commit to dev; if CI hasn't finished, failed or was cancelled, e.g., the image job of a commit that a later one superseded, re-run CI, then the release workflow). It packages the Helm chart with the release version as version and appVersion, so that it deploys the release image (elixircloud/trs-filer:X.Y.Z; the repository's appVersion is dev), with the library chart (checked against deployment/charts.sha256). In the packaged README, links to the latest docs and to files on branch dev point to the release (/en/vX.Y.Z/, blob/vX.Y.Z/); the release fails if any link to a version that moves is left (scripts/chart_readme.py). The chart in the repository is unchanged. After approval in the release environment, it builds the image, pushes it by digest (untagged, with SBOM and provenance attestations) and checks that the app reports the release version; it then scans exactly that digest for vulnerabilities, in a job without credentials. After a second approval in the release environment (once the scan passed), it checks that the chart it downloads is the one it packaged (a single file with the name and SHA-256 recorded when packaging; if not, the job fails before it attests, signs or tags anything), attests the build provenance of the scanned digest and signs it (see Verifying images and charts), tags exactly that image X.Y.Z (and latest, if it is the highest stable release on main), attests and signs the chart, and creates a GitHub release with the chart (.tgz), its Sigstore bundle (.tgz.sigstore.json) and the release notes (see below). Releases are immutable (see the repository settings below): once published, their assets can't be added, replaced or deleted. The workflow therefore creates the release as a draft, uploads the assets and publishes it last. A re-run finishes a draft left by an earlier attempt; if the release is already published (e.g., by hand), it uploads nothing and fails unless the release has exactly the assets that the run built (only the Sigstore bundles may differ, as every attempt signs anew). The error says what to do: a new patch version, or, if releases are not immutable, deleting the release and re-running. Add curated notes to the changelog in mkdocs.yml (nav; the docs build fails otherwise), and refer to pages of the docs by name rather than by relative links, which don't work in GitHub releases. No packages are published to PyPI. Tags of alpha, beta and release candidate versions (e.g., vX.Y.Zrc1, also aN and bN) get chart version X.Y.Z-rc1 (or -aN, -bN) and never update the latest image. These and, until 1.0.0, all other releases are GitHub pre-releases (never marked as the latest release). Digests of runs that fail before publish (also in the scan) stay on Docker Hub, untagged and unsigned; if publish fails after signing, the digest stays signed (and possibly tagged), and a re-run of the failed jobs finishes it. Re-running all jobs of the release workflow builds, scans, signs and tags a new image digest (the image build is not reproducible); an earlier digest that was published stays signed. Re-running only the failed jobs reuses the digest already pushed (and scans it again if the scan failed). Re-running all jobs also packages the chart again and replaces the chart artifact; the release checks the new chart against its new SHA-256. Images of commits on dev are published by the CI workflow (environment dev-images): it pushes the image by digest, scans it, and attests, signs and tags exactly that digest once all other jobs passed.

The CI workflow builds the image of every pull request and scans it with Trivy (the shared action scan-image of the organisation, pinned to the commit of a release), and a weekly workflow (scan.yml) scans latest. For pushes to dev and for releases, the workflows push the image by digest, untagged, scan exactly that digest in a job without credentials or signing identity, and only if the scan passes attest, sign and tag it, so that the published images are the scanned ones. The scans fail on critical and high vulnerabilities that have a fix, and upload the findings that have a fix (all severities) to code scanning (Security tab; only in this repository, not for pull requests from forks, nor in the release workflow); findings without a fix appear there once a fix is released. The weekly scan also uploads a full report, findings without a fix included, as the workflow artifact trivy-report-release-image of each run (trivy-report.json and trivy-report.txt, a table; Actions tab, workflow "Scan"); it is not uploaded to code scanning. Accepted findings are left out of all results. To accept a finding (e.g., if it doesn't affect the app), add it to .trivyignore.yaml with the affected package versions (purls), the reason (statement) and an expiry date (expired_at), after which the scans fail again. Trivy also scans for secrets (critical and high ones fail the scans too); accept those under secrets: in the same file. Dependabot updates the actions, including the pins of the shared actions (@<commit> # vX.Y.Z, all on the same release, see tests/test_workflows.py), to new releases of elixir-cloud-aai/.github; the Trivy version is the default of the shared action's trivy-version input, which Dependabot doesn't bump here: change it in the shared action, whose release then reaches this repository as a Dependabot update of the pins. Signing needs no secrets beyond the Docker Hub ones: it uses the workflows' OIDC tokens (permission id-token: write).

The release workflow derives the version, the chart version, the kind of release (stable, GitHub pre-release), whether the image is tagged latest and the previous release (for the generated notes) from the tag with scripts/release_version.py, which is tested over a table of tags (tests/test_release_version.py) and checked like the app (ruff, mypy, coverage); change these rules there, not in the workflow. It finds the GitHub release of the tag, if any, and checks the assets of a published one with scripts/release_assets.py (tested in tests/test_release_assets.py).

To test unreleased changes of FOCA, pyproject.toml may pin a FOCA commit (foca @ git+https://github.com/elixir-cloud-aai/foca@<sha>), with FOCA's development image of that commit as the base image (tagged by date, e.g., elixircloud/foca:YYYYMMDD-py3.13, pinned by digest, as the tag moves with later commits of that day). The Dockerfile records the commit as ## FOCA commit: <sha>, which the tests check (tests/test_config.py); the image build doesn't reinstall FOCA, but fails unless the base image has FOCA of that commit (scripts/image_requirements.py). Such pins are temporary: the release workflow fails while a requirement (also an optional one, a dependency group or a build requirement) is a direct reference or names a pre-release or a local version (e.g., +g<commit>), or a FOCA image has no release tag (scripts/check_release_pins.py, tested in tests/test_check_release_pins.py), so replace them by the released versions before tagging. This applies to pre-releases of the app, too: they are built on FOCA releases.

Release notes follow the format agreed for all repositories of ELIXIR On-Cloud:

  1. An intro paragraph saying what the release brings, with a link to the upgrade steps (e.g., on the Upgrading page) if there are any.
  2. Grouped sections, with empty ones left out, in this order: ## Breaking changes, ## Features, ## Security, ## Fixes, ## Documentation, ## Maintenance. They hold bullets only, and each bullet ends with its issue and pull request numbers, e.g., (#12, #34) (several pairs separated by semicolons, e.g., (#12, #34; #56, #78)).
  3. ## Commits: GitHub's generated notes, verbatim (including "New Contributors"), without their ## What's Changed header, as they credit the authors.
  4. Last, GitHub's generated **Full Changelog**: ... line (no compare link of our own).

The curated notes in docs/releases/X.Y.Z.md (for pre-releases, e.g., X.Y.Zrc1.md) hold parts 1 and 2, without a title (the release's title is its tag); the release workflow adds parts 3 and 4 from GitHub's generate-notes API, for the changes since the previous published release in the history of the release commit (for stable releases the previous stable release, or the previous pre-release if there is no stable one, as the generated notes can't start before the first tag; for alpha, beta and release candidate versions the previous release of any kind). For the first release (no published release before it, even if older tags exist, such as v0.3.0), the API would start at an older tag of its choice, so the workflow lists all merged pull requests in the history of the release commit itself, in the API's format: scripts/release_notes.py first reads the closed pull requests (gh api --paginate --slurp "repos/<owner>/<repo>/pulls?state=closed&per_page=100", for which the job has pull-requests: read) and the commits (git rev-list HEAD at the release commit); run these commands locally to preview the notes. Generated notes in an unexpected format (without the **Full Changelog** line) are kept as they are, with a warning, as the image is published by then. Without curated notes, the release gets GitHub's generated notes alone. Curated notes are published as written, so the release workflow fails if they contain TODO, are not in this format or link to versions that move (the latest docs, /en/latest/, or files on branch dev, blob/dev/; link the release's, e.g., /en/vX.Y.Z/, blob/vX.Y.Z/, as the published notes don't change) or have relative links (see below) (scripts/release_notes.py check, tested in tests/test_release_notes.py, which also checks the notes in docs/releases/), and so does the CI job "Release readiness" on pushes (see above); on re-runs, it replaces the notes of an existing release (also overwriting manual edits). Relative links, e.g., ../upgrading.md#from-040, work in the docs built from docs/releases/, but not in the release body: link the release's docs with their full URL, e.g., https://trs-filer.readthedocs.io/en/vX.Y.Z/upgrading/#from-040, or within the page with #.... The check reports every link target without a scheme (such as https: or mailto:) of inline links, images, reference definitions and the HTML attributes href, src and srcset (each of its URLs), anywhere in the notes, code blocks included (and lines of paragraphs that start like a definition, [label]: ...). Detailed upgrade steps belong on the Upgrading page, linked from the intro.

Maintainers: the workflows rely on these repository settings:

  • environment dev-images, limited to branch dev;
  • environment release, with a required reviewer, "Prevent self-review" and a deployment policy that allows only tags v*;
  • the Docker Hub secrets (DOCKERHUB_LOGIN, DOCKERHUB_TOKEN) only in the two environments, not as repository secrets;
  • a tag ruleset for v* that restricts who may create them and prevents moving or deleting them;
  • immutable releases;
  • main protected against non-fast-forward pushes;
  • a ruleset for dev that requires the check CI passed (the gate of the CI workflow) only.

While CITATION.cff names a version that has no tag yet, findings of "Release readiness" (e.g., a pin to an unreleased commit) also stop the dev image from being published; fix them or push the tag.

The Docker Hub organization is the repository variable DOCKERHUB_ORG (by default elixircloud; forks set it to the organization they publish to, with the Docker Hub secrets in an environment dev-images), so that the scan jobs, which have no secrets, know the image name too, and the released chart's default image (trsFiler.image) is the published one. Forks without DOCKERHUB_ORG publish nothing: their CI runs the same checks (the scan of a local build included, without the upload to code scanning) and passes without the secrets, and their weekly scan is skipped.

On Read the Docs, the default version is stable (the highest release) from the first release on, and the default branch is left empty, so that latest follows the repository's default branch, dev; an automation rule activates the versions of tags (v*), so that the docs linked from packaged charts exist.

The release workflow and CI's jobs "Chart" and "Chart install" install a fixed Helm version (version: of azure/setup-helm in release.yml and ci.yml), and "Chart install" a fixed kind version (version: of helm/kind-action). Dependabot updates the actions, but not these versions: bump them by hand, Helm in all three places.