Skip to content

Operations

Components

A deployment of TRS-Filer consists of:

  • the app (the TRS API), served by Gunicorn (see Server);
  • MongoDB, which stores the tools, with their versions and files, and the tool classes (database trsStore, or trs-filer-db with the Helm chart, collections tools and toolclasses; see Database).

The Docker Compose setup and the Helm chart run both.

Logs

The app logs to standard error (section log of the config, by default at level INFO): each request to the API (method, route and client address, not the body; the path at DEBUG) and the status of its response (except for errors, see below), registered and replaced objects, and errors, e.g., the reasons why the content of a file could not be retrieved from its URL (which clients don't get, see Remote files). Read them with docker compose logs or kubectl logs.

Requests that fail with an error have no line with the status of their response. Requests that don't match the API specification have no line of the request either: they are logged at INFO as Request rejected:, with the locations and codes of the invalid fields where known (otherwise a general reason, e.g., The request is malformed.), and the reason of the validation only at DEBUG. Other errors that FOCA's error handler answers (e.g., 404 Not Found, 500 Internal Server Error, and 400 Bad Request for files whose content can't be retrieved) are logged at ERROR with the problem details sent, which include the status; unknown routes have no line of the request. TRS-Filer's own 400 Bad Request responses to requests it refuses (see Errors) are logged only with the reason, at ERROR (e.g., Invalid limit 0.), after the line of the request.

Health checks

The image checks its health by requesting the Swagger UI (/ga4gh/trs/v2/ui/), which needs neither the database nor authentication; set HEALTHCHECK_URL if Gunicorn's port, the API base path or the Swagger UI setting changes. On startup, each Gunicorn worker creates the app and checks the config (e.g., the request size limit and the custom config); if this fails, Gunicorn exits.

Backups

All state is in MongoDB: back up the database (trsStore, or the one set in MONGO_DBNAME, e.g., trs-filer-db with the Helm chart) with mongodump, e.g., in the Docker Compose setup:

docker compose exec -T mongodb mongodump --db trsStore --archive > trs-filer.archive
docker compose exec -T mongodb mongorestore --archive < trs-filer.archive

mongodump reads the collections one after the other, without a point-in-time snapshot, so a registration during the dump can leave the archive inconsistent: stop writes (e.g., stop the app with docker compose stop trs-filer) while dumping, or use a backup method of MongoDB that takes consistent snapshots.

The base config creates unique indexes on the identifiers (id) of tools and tool classes on startup.

Verifying images and charts

From the first release on, images on Docker Hub (elixircloud/trs-filer) are built by this repository's workflows on GitHub Actions, with an SBOM and the build provenance attached as attestations, a GitHub artifact attestation of their provenance, and a keyless Sigstore signature. To check that an image was built by the release workflow, run (with the GitHub CLI or cosign 3 or later, which reads the signatures in the Sigstore bundle format):

gh attestation verify oci://docker.io/elixircloud/trs-filer:0.5.0 \
    --repo elixir-cloud-aai/trs-filer \
    --signer-workflow elixir-cloud-aai/trs-filer/.github/workflows/release.yml \
    --source-ref refs/tags/v0.5.0
cosign verify docker.io/elixircloud/trs-filer:0.5.0 \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com \
    --certificate-identity https://github.com/elixir-cloud-aai/trs-filer/.github/workflows/release.yml@refs/tags/v0.5.0

These are the commands for the latest release, 0.5.0: replace it with the version to verify. --source-ref and the ref in --certificate-identity check that the workflow ran for the release tag, not for another branch. Images of commits on dev (dev, sha-<commit>) are signed and attested by the CI workflow instead: use --signer-workflow elixir-cloud-aai/trs-filer/.github/workflows/ci.yml, --source-ref refs/heads/dev and --certificate-identity https://github.com/elixir-cloud-aai/trs-filer/.github/workflows/ci.yml@refs/heads/dev. Images published before the workflows signed them (the dated tags and the latest image of earlier versions, and dev and sha-<commit> images built before signing was added, see #134) have no signature or attestation. To show an image's SBOM or provenance, run:

docker buildx imagetools inspect docker.io/elixircloud/trs-filer:0.5.0 \
    --format '{{ json .SBOM }}'
docker buildx imagetools inspect docker.io/elixircloud/trs-filer:0.5.0 \
    --format '{{ json .Provenance }}'

The Helm chart attached to a release (trs-filer-0.5.0.tgz) has a GitHub artifact attestation of its provenance and a Sigstore bundle with its signature (trs-filer-0.5.0.tgz.sigstore.json, attached too):

gh release download v0.5.0 --repo elixir-cloud-aai/trs-filer \
    --pattern 'trs-filer-0.5.0.tgz*'
gh attestation verify trs-filer-0.5.0.tgz \
    --repo elixir-cloud-aai/trs-filer \
    --signer-workflow elixir-cloud-aai/trs-filer/.github/workflows/release.yml \
    --source-ref refs/tags/v0.5.0
cosign verify-blob trs-filer-0.5.0.tgz \
    --bundle trs-filer-0.5.0.tgz.sigstore.json \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com \
    --certificate-identity https://github.com/elixir-cloud-aai/trs-filer/.github/workflows/release.yml@refs/tags/v0.5.0

Images are scanned for known vulnerabilities with Trivy in CI and again before a release is published (both fail on fixable critical and high vulnerabilities), and the latest image weekly. The repository's code scanning alerts show the findings that have a fix, which the maintainers can act on; findings without a fix (often OS packages of the base image) appear there once a fix is released. The weekly scan also keeps a full report, those findings included, as a workflow artifact of each run. The published images (releases and dev) are the scanned ones: the workflows push each image by digest, untagged, scan exactly that digest, and only if the scan passes (no fixable critical or high vulnerabilities, except accepted ones) sign and tag it (see Versioning and releases).

Images published to another Docker Hub organization (forks that set DOCKERHUB_ORG) are verified the same way, with that organization and the fork's repository in place of elixircloud and elixir-cloud-aai/trs-filer.