Skip to content

0.4.0

This is the first release of TRS-Filer since v0.3.0 (2023), and the first with a release image that is scanned for vulnerabilities, attested and signed, and with an attested and signed Helm chart. It moves the app to FOCA 0.19 and Python 3.13, hardens the retrieval of remote files, fixes listing and registration bugs, adds zip downloads of a version's files and reworks the Helm chart, the image and the Docker Compose setup. Many of these changes are breaking (see below) for configs, clients, images and existing chart installs: read the upgrade steps before upgrading, in particular those for the Helm chart, as upgrading a chart install from v0.3.0 can delete MongoDB's data. See the documentation for details.

Breaking changes

  • Config: TRS-Filer runs on FOCA 0.19 (from 0.12) and Python 3.13. Custom configs must apply FOCA's config changes of 0.14 to 0.19 (e.g., exceptions.mapping, no server.environment), set server.max_content_length (1 MiB in the base config), and give the identifier character sets (custom.*.id.charset) as the characters themselves, not as expressions; custom.service defaults to http://localhost:8080 (#88, #139; #187, #190)
  • Service info: POST /service-info is removed, and the app no longer uses the service_info collection (it stays in existing databases); the service info is configured in FOCA's service_info.service (custom.service_info is rejected on startup), with placeholders that FOCA warns about (#88, #139)
  • Errors: errors are problem details (title, status, detail) instead of message and code; requests that the validation of the API specification rejects get the reason as detail (exceptions.validation_details: false keeps the generic text) (#88, #139; #187, #190)
  • Database: without MONGO_DBNAME, the app uses the database trsStore (was trs-filer-db), and MongoDB 4.4 or later is required; MongoDB 7.0 (Docker Compose, Helm chart) can't open the 3.6 data of earlier setups, so migrate it by dump and restore (#88, #139)
  • Remote files: file contents are retrieved from https URLs only by default, never from hosts that resolve to non-public addresses unless allowed, and failing or oversized responses fail the registration with 400 (they were stored as content); see custom.remote_files (#140, #142)
  • API: GET /tools refuses limit=0 (it gave all tools) and invalid offset and limit values with 400 (was 422), caps limit at 1000, omits next_page on the last page, applies the version and image filters to the same version and image, matches an empty author only to empty authors and refuses the unsupported filter tags with 400 (#126, #172; #179, #181)
  • API: registering a tool no longer changes an existing tool class (the tool stores a copy of the stored one); with custom.toolclass.validation, tools with unknown tool classes are no longer stored; identifiers of tool classes follow custom.toolclass.id (was custom.tool.id); deleting the last version of a tool gives 400 (#126, #172; #88, #139)
  • API: replacing tools and versions increments their meta_version, and adding, replacing or deleting a version increments its tool's (custom.tool.meta_version.on_version_change: false turns the latter off) (#126, #172; #176, #185)
  • Image: the image runs as user 1000 (also with a read-only root file system), installs the app as a package (the base config is no longer at /app/trs_filer/config.yaml; use an overlay via FOCA_CONFIG_OVERLAY), configures Gunicorn via GUNICORN_CMD_ARGS (GUNICORN_PROCESSES, GUNICORN_THREADS and TRS_CONFIG are no longer read), and trusts forwarded headers from 127.0.0.1 only (set FORWARDED_ALLOW_IPS behind a proxy) (#111, #123; #143)
  • Image tags: latest is the highest stable release (it was the head of dev, now tagged dev and sha-<commit>); dated tags are no longer published. Pin a release in deployments (#107, #133; #174)
  • Docker Compose: the service is trs-filer (was trs), published on 127.0.0.1:8080 (was port 80 on all interfaces), with MongoDB 7.0 and its data in the named volume mongodb-data (was ../data/trs/db) (#111, #123; #143)
  • Helm chart: MongoDB's Deployment, Service and claim are named mongodb-<appName> (were mongodb), so upgrading a chart install from v0.3.0 deletes the old claim and, usually, its data; MongoDB is pinned to mongo:7.0 (was mongo:3.6 in v0.3.0, the floating mongo:noble on dev). Follow the upgrade steps, or the database is lost (#114; #121; #197, #205)
  • Helm chart: the app's memory request and limit are now 256Mi and 384Mi (were 128Mi each; QoS class Burstable instead of Guaranteed). Pods may no longer be scheduled on clusters or namespaces with tight memory quotas; lower trs_filer.resources there, as described in the chart's README (#200, #206)
  • Helm chart: values changed: clusterType is removed; the chart creates an Ingress if trs_filer.ingress.enabled (by default true), otherwise an OpenShift Route where the cluster has the Route API. The image is elixircloud/trs-filer, and host_name is empty by default (#97; #112; #116; #121)

Features

  • API: GET /tools/{id}/versions/{version_id}/{type}/files?format=zip serves the files of a version as a zip archive (#72, #119)
  • Config: custom.tool.meta_version.on_version_change and the increments of meta versions (custom.tool.meta_version.increment, custom.version.meta_version.increment) (#126, #172; #176, #185)
  • Helm chart: a Kubernetes Ingress next to the OpenShift Route, the config Job sets MongoDB's host to the chart's MongoDB Service, appName on all resources (so that several releases fit in one namespace), MongoDB's database name as a value (mongodb.dbName), resource requests and limits, and a MongoDB init script (#97; #112; #113; #114; #121)
  • Helm chart: trs_filer.ingress.className and trs_filer.ingress.annotations, e.g., to restrict write methods at the ingress controller (#198, #204)
  • Helm chart: trs_filer.route.annotations, e.g., to allow only trusted clients at the OpenShift router, and trs_filer.route.enabled (by default true), to turn the Route off, e.g., for a Route of your own; NOTES.txt points to the planned authentication (#7) and, if the chart exposes nothing, gives a kubectl port-forward hint (#210, #213)
  • Helm chart: mongodb.imagePullPolicy; MongoDB's claim is kept on uninstall (helm.sh/resource-policy: keep), and MongoDB uses the Recreate strategy (#197, #205)
  • Releases: the release image is scanned, attested and signed, and the release has the packaged Helm chart (trs-filer-X.Y.Z.tgz) with its Sigstore bundle; images of dev are scanned, attested and signed too (#107, #133; #174; #134, #180; #184, #186)

Security

  • Remote files: the retrieval of file contents from URLs (on registration and for zip downloads) allows only the configured schemes and hosts with public addresses (checked on every redirect), with limits on size, time and redirects, against server-side request forgery (#140, #142)
  • Image: base image pinned by digest, non-root user, an allowlist .dockerignore, and forwarded headers trusted from 127.0.0.1 only by default (#111, #123; #143)
  • Helm chart: warnings in the chart's README and NOTES.txt that the API has no authentication, with a method-restriction example for ingress-nginx; see also the new page "Access control" of the documentation (#198, #204; #175, #178)

Fixes

  • API: production versions need checksums for all files, other versions don't (the check was applied to all versions) (#118)
  • API: replacing a tool, version or tool class with an identical one succeeds (it failed with 500); versions without is_production are registered as non-production versions (they failed with 500); DELETE /toolClasses/{id} checks all tools, not only the first 1000; retries of generated identifiers use new identifiers (#126, #148, #172)
  • API: the page links next_page and last_page keep the filters of the request, and concurrent registrations of tools with the same new tool class store the same tool class (#179, #181)
  • Image: the image starts the server by default (it exited like a one-off job) (#117)
  • Helm chart: NOTES.txt prints http or https URLs to match how the app is exposed, warns when nothing exposes it, and no longer uses the deprecated "Elixir Cloud" name (#189, #198, #204)
  • Helm chart: the app gets MONGO_HOST directly, so it reaches MongoDB even if the config Job fails; the init script creates only the app database's collections, under a ConfigMap name scoped by appName (#202, #207)

Documentation

  • A documentation site (MkDocs Material, on Read the Docs) with getting started, configuration, API, access control, deployment, operations, upgrading, development and API reference pages (#129, #170; #175, #178; #130, #138)
  • Upgrade guide for configs, clients, images, Docker Compose and Helm chart installs from v0.3.0 and dev; MongoDB version and persistence in the chart's README (#197, #205)
  • The Helm chart can't set the app config yet (the service info stays a placeholder), with a workaround until #135 (#199, #203)
  • Memory sizing of the app (#200, #206); TRS version labels explained on the page "API" (#202, #207)
  • Citation metadata (CITATION.cff, archived on Zenodo with each release) (#132, #171)
  • Community files and security policy of the organisation, a new logo and an updated README (#131, #137; #157, #158; #188, #194; #182, #183)

Maintenance

  • Packaging with pyproject.toml only (version from Git tags via setuptools-scm); setup.py, setup.cfg and requirements*.txt are removed (#111, #123; #143)
  • Tests: an app test harness (mongomock or a real MongoDB), typed tests and 100% coverage required (#125, #169; #106, #147, #150; #127, #173)
  • Linting and formatting with ruff, mypy, pre-commit, ast-grep and semgrep, shared with the other repositories of the organisation (#124, #145; #146, #149; #153, #154)
  • CI: workflow and job names aligned with the organisation, a single gate check, forks supported, shared actions pinned, and the Docker Hub organisation as a variable (#155, #156; #159, #160; #161, #162; #163, #164; #165, #168; #191, #195)
  • Removed the unused 2020 TRS specifications; the chart in the repository has a placeholder version (0.0.0, appVersion: dev) and a description (#202, #207)
  • Dependency updates (#115; #151; #152)