Skip to content

Configuration

The app is configured by trs_filer/config.yaml, the base config, which documents its values. It is a FOCA config: the sections server, db, api, service_info, log and exceptions are described in the FOCA documentation, the app's own settings are in section custom (validated on startup by trs_filer.custom_config.CustomConfig). To override some values, write them to a config overlay file and list it in environment variable FOCA_CONFIG_OVERLAY (several files separated by :); FOCA merges overlays into the config on startup (see the FOCA deployment guide). Mappings are merged recursively; all other values, including lists, replace those of the base config.

Deployment configuration

TRS-Filer works with any OpenID Connect provider that issues access tokens as per RFC 9068 (JWTs). The identity provider, its claims and the groups mapped to roles are deployment configuration: the base config has no defaults for them, so each deployment sets them in an overlay (or in the Helm chart's config value), otherwise the app refuses to start:

  • security.auth.issuers: the trusted token issuers (the iss claim of their tokens, e.g., https://login.example.org/oidc/); tokens of other issuers are rejected (see Authentication);
  • security.auth.audience: the client IDs of the apps whose tokens are accepted (the aud claim, e.g., of a portal or gateway that registers tools), unless security.auth.verify_audience is false, which accepts tokens for any audience (not recommended);
  • security.access_control.claim_groups: the claim with the groups of users (e.g., groups), needed for admins; add it to security.auth.userinfo_claims if the identity provider releases it via user info only, not in access tokens;
  • security.access_control.group_mapping: the group(s) whose members are admins (values of the group claim, see Access control), or security.access_control.require_admin: false to run without admins (nobody can write then).

For example:

security:
    auth:
        issuers:
            - https://login.example.org/oidc/
        audience:
            - trs-portal
        userinfo_claims:
            - groups
    access_control:
        claim_groups: groups
        group_mapping:
            trs-admins: admin

Entitlement-style groups

Some identity providers release groups as entitlements, i.e., URNs, in a multi-valued claim (here entitlement) other than groups, and only via user info:

security:
    auth:
        issuers:
            - https://idp.example.org/
        audience:
            - <client ID of the portal>
        userinfo_claims:
            - entitlement
    access_control:
        claim_groups: entitlement
        group_mapping:
            "urn:example:group:trs-admins#issuer.example.org": admin

Settings for specific identity providers belong to the documentation of the deployment, not here.

Notes:

  • The group (the URN above is illustrative) is a value of the claim. Quote such keys in YAML, as they contain : and #.
  • Without the claim in userinfo_claims, no user gets the admin role.
  • Clients must request the scope that releases the claim when logging in (here the one for entitlement, besides openid), and the identity provider must allow it for the client.
  • TRS-Filer accepts only access tokens with the typ header at+jwt (security.auth.verify_token_type, enabled by default). Before going live, check that the identity provider issues such tokens to the clients, and that their aud matches audience: otherwise writes fail with 401 Unauthorized.
  • To accept tokens of some clients only (e.g., of a portal or gateway in front of TRS-Filer), list their client IDs in security.auth.clients (FOCA compares them with the client_id claim of tokens, or azp without it); tokens of other clients, or without either claim, get 401 Unauthorized. With several issuers, make sure that clients of one issuer can't claim the IDs of another's.

The policies (security.access_control.policies, see FOCA's access control) say what each role may do. They are named, so that an overlay can change, add or remove single rules by name (FOCA's named entries); the base config has rules for reads only, which apply if login is required for them (see Requiring login for reads): writes are for admins. An overlay that gives the policies as a list replaces all rules of the base config instead; FOCA refuses to start if an overlay gives named rules after one that gives a list.

The API specification (api.specs) is named trs, too, so that an overlay can change it by name, e.g., to require login for reads.

FOCA's service clients (security.auth.service_clients, security.auth.on_behalf_of_header) are not supported; the app refuses to start with them (see Authentication).

Local overrides

Docker Compose mounts config.dev.yaml, an overlay for local development that trusts a fictitious token issuer (https://login.example.org/oidc/; set your own in a local override), sets an example group claim (groups), accepts tokens for any audience and requires no admins, so writes are refused until you set your identity provider and admin group (see Getting started, and Running without authentication for development without an identity provider). To override values locally without changing it, put them in config.local.yaml and mount it in a docker-compose.override.yaml (both are ignored by Git):

services:
    trs-filer:
        environment:
            FOCA_CONFIG_OVERLAY: /etc/trs-filer/config.dev.yaml:/etc/trs-filer/config.local.yaml
        volumes:
            - ./config.local.yaml:/etc/trs-filer/config.local.yaml:ro

Public URL

The TRS API includes the URLs of tools and tool versions in its responses (url), which TRS-Filer builds from custom.service: <url_prefix>://<external_host>:<external_port>/<api_path>/tools/<id>. The base config has the URL of the Docker Compose setup (http://localhost:8080/ga4gh/trs/v2); deployments set their own, e.g.:

custom:
  service:
    url_prefix: https
    external_host: trs.example.org
    external_port: 443
    api_path: ga4gh/trs/v2

The Helm chart sets it from its values (see Deployment); other parts of the config go into its config value (see Configuration there).

Identifiers

TRS-Filer generates the identifiers of tools, tool versions and tool classes that are registered without one (POST): random strings of length characters from charset, in custom.tool.id, custom.version.id and custom.toolclass.id. By default, tool identifiers have six upper-case letters and digits, and version and tool class identifiers six lower-case letters and digits (tool classes created along with a tool, see Tool classes, get identifiers like tools). charset lists the characters themselves (e.g., ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789); expressions such as string.ascii_uppercase + string.digits are rejected on startup (see Upgrading). Identifiers can also be chosen by the client (PUT, see API).

Meta versions

Tools and versions have a meta_version (a field of the TRS API that tells clients that an object changed), set in custom.tool.meta_version and custom.version.meta_version:

  • New tools and versions get the init value (default 1).
  • Replacing a tool (PUT /tools/{id} with the identifier of a stored tool) adds custom.tool.meta_version.increment (default 1) to its meta version, and custom.version.meta_version.increment (default 1) to those of its versions that replace stored versions (same identifier); its other versions are new and get the init value.
  • Replacing a version (PUT /tools/{id}/versions/{version_id}) adds custom.version.meta_version.increment to the version's meta version.
  • Adding, replacing or deleting a version with the version endpoints (POST /tools/{id}/versions, PUT and DELETE /tools/{id}/versions/{version_id}) also adds custom.tool.meta_version.increment to the tool's meta version, as the tool as served (its versions) changes. Set custom.tool.meta_version.on_version_change: false to change the tool's meta version only when the tool itself is replaced. Requests that fail change no meta version.
  • Stored meta versions that are not integers restart at the init value.

Replacements with identical content increment the meta versions, too. Concurrent changes of the same object may give it the same meta version, or, for tools whose versions change at the same time, set it back to an earlier value. Tool classes have none (custom.toolclass.meta_version is unused).

Tool classes

Each tool has a tool class (e.g., workflow or command-line tool), which is stored with the tool and in the list of tool classes (GET /toolClasses). When a tool is registered or replaced (POST /tools, PUT /tools/{id}):

  • If a tool class with the identifier of the tool's toolclass.id exists (also if it is pending), the tool stores a copy of that tool class, not the one it was sent with, and the tool class is not changed.
  • Otherwise, if the user may add tool classes (see Adding tool classes), the tool class is created as sent with the tool (with a generated identifier if it has none); if not, the registration fails with 400 Bad Request and nothing is stored.

Tool classes are changed only with the tool class endpoints. The copies stored with tools are not updated when a tool class is replaced (PUT /toolClasses/{id}); register the tools again to update them. The toolClass filter of GET /tools uses the copies.

If tools with the same new tool class identifier but different tool classes are registered at the same time, the tool class of the first one is created, and the others store a copy of it, too.

Adding tool classes

custom.access_control.tool_classes sets who may add tool classes, with the modes of the service types of Cloud Registry:

  • open (the default): any authenticated user, explicitly (POST /toolClasses, or PUT /toolClasses/{id} with the identifier of no tool class) or by registering a tool with a new tool class (see above);
  • controlled: only admins, both ways; tools of other users must have a registered tool class (400 Bad Request otherwise), and their POST /toolClasses and PUT /toolClasses/{id} with new identifiers get 403 Forbidden;
  • moderated: any authenticated user, both ways, but the tool classes that users other than admins add are pending until an admin approves them (PUT /toolClasses/{id}/approval), and so are the tools that have them.

In every mode, only admins replace (PUT /toolClasses/{id} with the identifier of a tool class), approve and delete tool classes; replacing a pending tool class doesn't approve it. Tool classes store who added them (internal field _createdBy, not served).

Pending tool classes and tools. In mode moderated, pending tool classes are listed (GET /toolClasses) only to admins and the user who added them, and pending tools are served (all reads of tools, also GET /tools) only to admins and the users with a role on them (see Roles on tools): the reads accept a token for that (anonymous reads, and reads with an invalid token, see neither). Others can still register tools with a pending tool class, whose tools are pending too. GET /approvals lists the pending tool classes and tools that the user may see (see Approving tool classes). An admin's approval of a tool class approves all tools that have it. There is no way back to pending: admins delete unwanted tool classes instead (DELETE /toolClasses/{id}, once no tool has them). Pending tool classes stay pending if the mode changes, and tools registered with them are pending then too. Tools store whether they are pending (internal field _pending, not served); the app sets it on registration, clears it on approval and corrects it on startup. Without authentication (see Running without authentication), there are no admins: nothing is pending, and all tools are served.

Unused tool classes per user. Users other than admins may only have added custom.access_control.max_unused_tool_classes tool classes that no tool has (default: 10; null for no limit), as for the service types of Cloud Registry (max_unused_service_types), in modes open and moderated (in mode controlled, they can't add tool classes at all): further tool classes are refused with 409 Conflict ("You have added 10 tool classes that no tool has, as many as allowed; ..."), whether added explicitly or by registering a tool with a new tool class, until tools have them or admins delete them (DELETE /toolClasses/{id}, see Access control); pending and approved tool classes both count. Tool classes become unused when their tools are deleted or change their tool class, so the limit bounds how many tool classes each user can add without tools (users can still leave more unused by registering tools with new tool classes and deleting them, one registration per tool class). Set 0 to let only admins add tool classes: users other than admins can then only register tools with stored tool classes, as in mode controlled. Tool classes stored before TRS-Filer recorded who added them count for nobody. Concurrent requests of a user can exceed the limit by their number. Without authentication, there are no users, and no limit.

GET /toolClasses is not paginated: the TRS specification defines no paging for it, and the limit bounds how fast users can grow the list.

Policy rules. Keep the rule add-tool-classes (security.access_control.policies) and set the mode instead of removing the rule: without it, POST /toolClasses and PUT /toolClasses/{id} are for admins only, but registering tools with new tool classes still adds them, unless the mode is controlled. Without the rule for adding to a collection (register-tools for tools, add-tool-classes for tool classes), PUT with a new identifier is refused (403) also for users whom other rules allow to replace objects (e.g., editors, see Admins): such rules let them replace stored objects, not add new ones.

For example, in a config overlay:

custom:
    access_control:
        tool_classes: moderated
        max_unused_tool_classes: 5

Deprecated: custom.toolclass.validation. custom.toolclass.validation: true required stored tool classes for all tools; it now sets the mode controlled, which replaces it, and the app warns of it on startup (`custom.toolclass.validation` is deprecated ...). It is removed in the next breaking release: set custom.access_control.tool_classes: controlled instead. With the mode moderated, the app refuses to start (the two contradict each other). Unlike before, admins add tool classes by registering tools in mode controlled, and other users can't add tool classes explicitly either (see Upgrading). The TRS specification leaves who adds tool classes to registries (GET /toolClasses lists "all tool-classes available"); in mode controlled, every tool has a listed tool class, as with the old setting.

Remote files

Files of tool versions are registered with their content (content) or a URL (url) that TRS-Filer retrieves the content from: on registration, and when it serves the files of a version as a ZIP archive or a descriptor as plain text (PLAIN_ types) and has no stored content for one of them. custom.remote_files sets how URLs are retrieved:

Option Default Description
schemes [https] Allowed URL schemes (http, https).
allowed_hosts [] Host names, IP addresses and CIDR networks that may be retrieved although they resolve to addresses that are not public (e.g., for local setups).
connect_timeout 5 Seconds to connect, per address of a host.
read_timeout 10 Seconds per read.
total_timeout 30 Seconds per file, including redirects.
max_size 1048576 Maximum size of a file, in bytes (1 MiB).
max_redirects 5 Redirects followed per file.

Hosts must resolve to public addresses only (also after redirects), unless they are listed in allowed_hosts; proxy settings of the environment are not used. Retrievals that fail (e.g., a refused URL, a timeout, a response other than 2xx or a file that is too large) fail the registration with 400 Bad Request; for ZIP archives and plain descriptors, 404 Not Found. The app logs the reason, which names the problem but not the URL (e.g., InvalidURL: The URL has user info; for errors of lower layers, only their types, such as ConnectionError (caused by MaxRetryError, NameResolutionError, gaierror)), next to the URL without its user info. Name resolution is not limited by the timeouts. For example, to register files of a web server on the Docker Compose network in a local setup:

custom:
  remote_files:
    schemes: [http, https]
    allowed_hosts: [files.local]

I/O Schemas

custom.io_schema.validation (experimental, false by default) checks GA4GH I/O Schema files (files of type OTHER named ga4gh-io.schema.json) on registration against the meta-schema of the GA4GH I/O Profile 0.3.0, and rejects invalid ones with 400 Bad Request (see Checking I/O Schemas); custom.io_schema.timeout (default: 10) is the time limit of checking a file, in seconds:

custom:
    io_schema:
        validation: true
        timeout: 10

File storage

By default, TRS-Filer stores the content of files in MongoDB, with the other data of tools. Optionally, it stores the content of files in S3-compatible object storage instead, and keeps only their metadata in MongoDB (custom.storage):

Option Default Description
backend mongodb Where the content of files registered from now on is stored: mongodb or s3.
s3.bucket Name of the bucket, which must exist; required if s3 is set.
s3.endpoint_url URL of the S3 API, e.g., https://s3.example.org; boto3's default if not set (AWS, or the environment variable AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL).
s3.region Region of the bucket; boto3's default if not set (e.g., the environment variable AWS_REGION).
s3.prefix "" Prefix of the keys of the objects (e.g., trs-filer/), so that a bucket can be shared.
s3.addressing_style path path (<endpoint>/<bucket>/<key>, which most self-hosted servers need), virtual (<bucket>.<endpoint>/<key>) or auto (boto3's choice).
s3.presigned_urls false Serve files via presigned URLs of their objects instead of through the app (see below).
s3.presigned_url_expiry 300 Validity of presigned URLs, in seconds (at most 604800, 7 days).
s3.connect_timeout 5 Seconds to connect to the S3 API.
s3.read_timeout 10 Seconds per read from the S3 API.

TRS-Filer talks to the storage through the S3 API only (with boto3), so any S3-compatible server or service works, e.g., SeaweedFS or Zenko CloudServer (both Apache-2.0), Ceph RGW, or the S3 services of cloud and HPC providers. CI tests with SeaweedFS.

Credentials are never part of the config (which rejects unknown settings): boto3 takes them from the environment with its default credential chain, e.g., from the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (set from a Secret, see Object storage for the Helm chart), or a web identity token. The app needs the S3 permissions to put, get and delete objects in the bucket (with the prefix, if set); the bucket isn't created by the app.

For example:

custom:
  storage:
    backend: s3
    s3:
      endpoint_url: https://s3.example.org
      bucket: trs-filer
      region: us-east-1
      prefix: trs-filer/

How files are stored:

  • When a version is registered (also with its tool, and when it is replaced), the content of each of its files, once all files are valid (and retrieved from their URLs, see Remote files), is written once as a new object, under a random key with the prefix (as text/plain; charset=utf-8). MongoDB keeps the file's metadata and a reference to the object, with the SHA-256 checksum and the size of the content; the reference is never served.
  • When files are served, their content is read from the object and checked against the checksum and size; changed or missing objects fail the request with 500 Internal Server Error, and the app logs the key. A missing object is first read again from the stored files once, as the version may have been replaced or deleted meanwhile (this needs reads from the primary: with a readPreference of secondaries in MONGO_URI, the retry may still see the old files).
  • Objects that no stored file refers to any more are deleted at the end of the request that released them: those of replaced and deleted versions and tools, and those of registrations that failed. If deleting an object fails, or the app stops during a request, objects can be left without a file that refers to them (logged at ERROR); they are not served.
  • The backend only decides where new files go: files stay where they were stored. With backend: mongodb and s3 set, files in object storage are still served (e.g., to switch back); without s3, they fail with 500 Internal Server Error. Existing files are not moved to object storage.

With presigned URLs (s3.presigned_urls: true), clients get files from the object storage directly, which then must be reachable by them: JSON responses with files (descriptors, tests, containerfiles) have the presigned URL of the object as url, instead of the content and of the URL given on registration, and the SHA-256 checksum of the content (sha-256, unless given on registration); plain responses (PLAIN_ types) redirect to the presigned URL (302 Found). Presigned URLs serve the content as text/plain and as a download (Content-Disposition: attachment), so that browsers don't display it. Presigned URLs expire after s3.presigned_url_expiry seconds, or earlier, when the version is replaced or deleted (its objects are deleted). ZIP archives are still built by the app.

A presigned URL lets anybody who has it read the file until it expires, without a token, also if the deployment requires login for reads (see Requiring login for reads): keep the expiry short there, and keep the URLs out of logs. Serve the S3 API on another host than TRS-Filer, so that clients don't send their access tokens along with the redirect.

Docker Compose doesn't bundle an object store. To try object storage locally, add SeaweedFS (here with the S3 API without authentication, for tests only) in a docker-compose.override.yaml, and set the backend in a config overlay (e.g., config.s3.yaml, after config.dev.yaml):

# docker-compose.override.yaml
services:
  trs-filer:
    environment:
      FOCA_CONFIG_OVERLAY: /etc/trs-filer/config.dev.yaml:/etc/trs-filer/config.s3.yaml
      # Any values: SeaweedFS without authentication doesn't check them
      AWS_ACCESS_KEY_ID: test
      AWS_SECRET_ACCESS_KEY: test
    volumes:
      - ./config.s3.yaml:/etc/trs-filer/config.s3.yaml:ro
  seaweedfs:
    image: docker.io/chrislusf/seaweedfs:4.51
    command: server -s3 -ip.bind=0.0.0.0
    ports:
      - "127.0.0.1:8333:8333"
# config.s3.yaml
custom:
  storage:
    backend: s3
    s3:
      endpoint_url: http://seaweedfs:8333
      bucket: trs-filer
      region: us-east-1

Then create the bucket, e.g., with the AWS CLI on the host: aws --endpoint-url http://127.0.0.1:8333 s3 mb s3://trs-filer.

Request size limit

Request bodies are limited to server.max_content_length bytes (default: 1048576, i.e., 1 MiB), by FOCA: larger requests, also chunked ones, are rejected with 413 Content Too Large before they are parsed. The app refuses to start without a limit. Raise it to register tools with larger inline file contents, together with the limits of any proxies in front of the app; prefer files with URLs.

Rate limits

TRS-Filer has no rate limits of its own. In production, limit the rate of requests per client at the reverse proxy or ingress controller in front of it, so that single clients can't overload the app or its database.

Server

The image serves the app with Gunicorn, with the settings of trs_filer/gunicorn_config.py: it listens on port 8080, with one worker process with four threads. Override settings with the environment variable GUNICORN_CMD_ARGS (e.g., --workers 2 --threads 8), but not --bind: the image's health check and the Helm chart expect port 8080. Behind a reverse proxy, set its addresses in FORWARDED_ALLOW_IPS, so that Gunicorn trusts its X-Forwarded-* headers (with the Helm chart, in trsFiler.extraEnv, see Forwarded headers). server.host and server.port of the config only apply to Flask's development server (python -m trs_filer.app). server.debug must stay off outside local development: Flask's debugger allows executing arbitrary code.

Database

TRS-Filer stores tools (with their versions and files; the content of files optionally in object storage, see File storage) and tool classes in the collections tools and toolclasses of the MongoDB database trsStore at db.host and db.port (in the base config: mongodb:27017, the service of Docker Compose). FOCA reads the environment variables MONGO_HOST, MONGO_PORT, MONGO_DBNAME, MONGO_USERNAME and MONGO_PASSWORD, or a complete connection string in MONGO_URI, which take precedence over the config (see the FOCA documentation); with MONGO_DBNAME set, the app uses that database instead of trsStore. FOCA keeps the rules of access control in the collection policy_rules of the same database (security.access_control.db_name and collection_name), and its changes in policy_rules_changes.

Service info

GET /service-info is served by FOCA from section service_info of the config, according to the GA4GH service info schema (validated on startup). Its version is the version of the installed package (service_info.package). The base config describes the app, not a deployment: its id (org.example.trs-filer) and organization are placeholders in reserved example domains (RFC 2606). FOCA warns on startup while id, organization.url, contactUrl or documentationUrl are in such a domain (as in the example below), not of other fields (e.g., organization.name, which overlays keep unless they set it). Deployments set their own id, name, description, organization, contactUrl and environment in service_info.service, e.g.:

service_info:
  service:
    id: org.example.trs
    name: Example TRS
    description: Tools and workflows of the Example Organization.
    organization:
      name: Example Organization
      url: https://example.org
    contactUrl: mailto:trs-admins@example.org
    environment: prod

The Helm chart has no value for the service info yet, so chart deployments serve the placeholders unless they deploy an overlay of their own (see Configuration in the deployment docs).

Errors

Errors are problem details, sent as application/json (section exceptions, see Errors). Requests that the validation of the API specification rejects (e.g., an unknown descriptor type) get the reason as detail, shortened to exceptions.max_detail_length characters (500 by default); set exceptions.validation_details: false to send a generic detail instead (see FOCA's documentation). The base config sets exceptions.validation_reasons: true, so that this detail is the validator's reason, as up to FOCA 0.21, which may quote the rejected values; set it to false for FOCA's default, which names the invalid fields by location and code instead. Requests that TRS-Filer itself refuses (e.g., deleting the last version of a tool) get their reason as detail whatever exceptions.validation_details is set to; it names the rule, not the values sent (see Errors). Like all details, it is shortened to exceptions.max_detail_length characters, and not sent if exceptions.public_members doesn't list detail.