API reference¶
Reference of the app's Python code, generated from its docstrings. For the HTTP API, see API and the Swagger UI of a deployment.
App¶
app ¶
App factory, and entry point for development.
The image serves the app with Gunicorn (trs_filer.app:create_app(), see
trs_filer.gunicorn_config); python -m trs_filer.app runs it with Flask's
development server.
create_app ¶
Create the app from the config, with the app's checks.
FOCA validates the config, including the custom config
(trs_filer.custom_config.CustomConfig) and the service info
(service_info), and limits the size of request bodies
(server.max_content_length). FOCA also warns of placeholders in the
service info (values in reserved example domains, RFC 2606, as those of
config.yaml). For access control, FOCA requires trusted token issuers
(security.auth.issuers) with an audience, and refuses to start without
admins (security.access_control.require_admin). The app additionally
requires a size limit, refuses service clients (FOCA's
require_no_service_clients()), as service clients (or the users they
act for) would need their own roles on tools, and answers invalid
requests
(trs_filer.errors.exceptions.InvalidRequest) with their reason as
detail. It warns of the deprecated custom.toolclass.validation, and
corrects the pending flags of tools (cf.
trs_filer.ga4gh.trs.tool_classes.sync_pending_tools()).
Gunicorn calls this in each worker, after forking (MongoDB clients must not be shared across forks), so the checks run in each worker; if they fail, the worker fails to boot and Gunicorn exits.
Returns:
| Type | Description |
|---|---|
App
|
Connexion app. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
Request bodies are not limited in size
( |
ValueError
|
The custom config is invalid, e.g., it still sets
|
ValidationError
|
FOCA's config is invalid, e.g., the service info, or no token issuers are trusted, or they have no audience. |
main ¶
Run the app with Flask's development server, on the configured port.
For development; not for production.
gunicorn_config ¶
Gunicorn settings of the image (production WSGI server).
The image runs gunicorn --config python:trs_filer.gunicorn_config
'trs_filer.app:create_app()'. The defaults suit a small container
(about 115 MB at rest and 175 MB at peak, within the Helm chart's 256 MiB
request and 384 MiB limit); override them via
GUNICORN_CMD_ARGS (e.g., --workers 2 --threads 8), which takes
precedence over this module (cf. https://gunicorn.org/reference/settings/).
The app is not preloaded (no preload_app): each worker creates it after
forking, as MongoDB clients must not be shared across forks, and runs the
startup checks; if they fail, the worker fails to boot and Gunicorn exits.
Configuration¶
custom_config ¶
Custom app config models.
ServiceConfig ¶
Bases: FOCABaseConfig
Model for the public URL of the app.
The URL is used for building the url properties of tools and tool
versions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url_prefix
|
URL scheme of application. |
required | |
external_host
|
Name of application host. |
required | |
external_port
|
Port at which application is served. |
required | |
api_path
|
Base path at which API endpoints can be reached. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
url_prefix |
str
|
URL scheme of application. |
external_host |
str | None
|
Name of application host. |
external_port |
int | None
|
Port at which application is served. |
api_path |
str | None
|
Base path at which API endpoints can be reached. |
Example
ServiceConfig( ... url_prefix='https', ... external_host='trs.example.org', ... external_port=443, ... api_path='ga4gh/trs/v2' ... ) ServiceConfig(url_prefix='https', external_host='trs.example.org', ext ernal_port=443, api_path='ga4gh/trs/v2')
IdConfig ¶
Bases: FOCABaseConfig
Model for generating object identifiers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
charset
|
The allowed characters (duplicates are ignored); not an
expression (e.g., |
required | |
length
|
Length of generated identifiers. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
charset |
str
|
The allowed characters. |
length |
int
|
Length of generated identifiers. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
The character set is empty or an expression, or the length is not positive. |
Example
IdConfig( ... charset='ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', ... length=6 ... ) IdConfig(charset='ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789', length=6)
check_charset
classmethod
¶
Check that identifiers can be generated from the character set.
Uses the checks of FOCA's generate_id(), so that invalid character
sets fail on startup instead of on requests.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str
|
Character set. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The character set. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The character set is empty, or an expression. |
MetaVersionConfig ¶
Bases: FOCABaseConfig
Model for defining meta version control configurations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
init
|
Start index for the versioning. Defaults to |
required | |
increment
|
Incremental delta for the next version. Defaults to |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
init |
int
|
Start index for the versioning. Defaults to |
increment |
int
|
Incremental delta for the next version. Defaults to |
ToolMetaVersionConfig ¶
Bases: MetaVersionConfig
Model for the meta version configuration of tools.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
init
|
Start index for the versioning. Defaults to |
required | |
increment
|
Incremental delta for the next version. Defaults to |
required | |
on_version_change
|
Whether adding, replacing or deleting a version of
a tool increments the tool's meta version, too. Defaults to
|
required |
Attributes:
| Name | Type | Description |
|---|---|---|
init |
int
|
Start index for the versioning. Defaults to |
increment |
int
|
Incremental delta for the next version. Defaults to |
on_version_change |
bool
|
Whether adding, replacing or deleting a version of
a tool increments the tool's meta version, too. Defaults to
|
ToolConfig ¶
Bases: FOCABaseConfig
Model for tool config parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
Unique identifier definition for tools. |
required | |
meta_version
|
Version control config params. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
id |
IdConfig
|
Unique identifier definition for tools. |
meta_version |
ToolMetaVersionConfig
|
Version control config params. |
Example
ToolConfig( ... id=IdConfig( ... length=6, ... charset='abcdefghijklmnopqrstuvwxyz0123456789' ... ), ... meta_version=ToolMetaVersionConfig( ... init=1, ... increment=1, ... on_version_change=True ... ) ... ) ToolConfig(id=IdConfig(charset='abcdefghijklmnopqrstuvwxyz0123456789', length=6), meta_version=ToolMetaVersionConfig(init=1, increment=1, on_version_change=True))
VersionConfig ¶
Bases: FOCABaseConfig
Model for version config parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
Unique identifier definition for tool versions. |
required | |
meta_version
|
Version control config params. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
id |
IdConfig
|
Unique identifier definition for tool versions. |
meta_version |
MetaVersionConfig
|
Version control config params. |
Example
VersionConfig( ... id=IdConfig( ... length=6, ... charset='abcdefghijklmnopqrstuvwxyz0123456789' ... ), ... meta_version=MetaVersionConfig( ... init=1, ... increment=1 ... ) ... ) VersionConfig(id=IdConfig(charset='abcdefghijklmnopqrstuvwxyz012345678 9', length=6), meta_version=MetaVersionConfig(init=1, increment=1))
ToolClassConfig ¶
Bases: FOCABaseConfig
Model for tool class config parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
Unique identifier definition for tool classes. |
required | |
meta_version
|
Version control config params. |
required | |
validation
|
Deprecated: whether tools must have a stored tool
class; |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
id |
IdConfig
|
Unique identifier definition for tool classes. |
meta_version |
MetaVersionConfig
|
Version control config params. |
validation |
bool
|
Deprecated: whether tools must have a stored tool class. |
Example
ToolClassConfig( ... id=IdConfig( ... length=6, ... charset='abcdefghijklmnopqrstuvwxyz0123456789' ... ), ... meta_version=MetaVersionConfig( ... init=1, ... increment=1 ... ), ... validation=False ... ) ToolClassConfig(id=IdConfig(charset='abcdefghijklmnopqrstuvwxyz0123456 789', length=6), meta_version=MetaVersionConfig(init=1, increment=1), validation=False)
RemoteFilesConfig ¶
Bases: FOCABaseConfig
Model for retrieving the content of files from their URLs.
When a tool version is registered with files that have a URL but no content, the content is retrieved from the URL and stored; files without stored content are retrieved when served as a zip archive.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schemes
|
Allowed URL schemes: |
required | |
allowed_hosts
|
Host names (compared case-insensitively, without a trailing dot), IP addresses (IPv6 addresses also in brackets) and IP networks (CIDR notation) that may be retrieved from even though they are not public (e.g., for local setups). Other destinations must resolve to public addresses only: loopback, private, link-local, reserved and multicast addresses are refused. |
required | |
connect_timeout
|
Timeout for connecting to each address of a host, in seconds. |
required | |
read_timeout
|
Timeout for each read from a connection, in seconds. |
required | |
total_timeout
|
Timeout for retrieving a file, including redirects, in seconds; name resolution isn't limited by the timeouts. |
required | |
max_size
|
Maximum size of retrieved content, in bytes. |
required | |
max_redirects
|
Maximum number of redirects followed per file. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
schemes |
list[Literal['http', 'https']]
|
Allowed URL schemes. |
allowed_hosts |
list[str]
|
Host names, IP addresses and IP networks exempt from the destination checks, normalized. |
connect_timeout |
float
|
Timeout for connecting, in seconds. |
read_timeout |
float
|
Timeout for each read from a connection, in seconds. |
total_timeout |
float
|
Timeout for retrieving a file, in seconds. |
max_size |
Annotated[int, Field(strict=True, gt=0)]
|
Maximum size of retrieved content, in bytes. |
max_redirects |
Annotated[int, Field(strict=True, ge=0)]
|
Maximum number of redirects followed per file. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
No scheme is allowed, an allowed host is empty, a timeout is out of range or not finite, or a limit is out of range or not an integer. |
Example
RemoteFilesConfig(allowed_hosts=['files.local', '10.0.0.0/8']) RemoteFilesConfig(schemes=['https'], allowed_hosts=['files.local', '1 0.0.0.0/8'], connect_timeout=5.0, read_timeout=10.0, total_timeout=30 .0, max_size=1048576, max_redirects=5)
allowed_networks
property
¶
IP networks of the allowed hosts.
Returns:
| Type | Description |
|---|---|
list[IPNetwork]
|
Allowed IP networks (IP addresses as networks of one address). |
check_timeout
classmethod
¶
Check that a timeout is finite.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
float
|
Timeout, in seconds. |
required |
Returns:
| Type | Description |
|---|---|
float
|
Timeout, in seconds. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The timeout is not finite. |
check_schemes
classmethod
¶
Check that at least one scheme is allowed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
list[str]
|
Allowed schemes. |
required |
Returns:
| Type | Description |
|---|---|
list[str]
|
Allowed schemes. |
Raises:
| Type | Description |
|---|---|
ValueError
|
No scheme is allowed. |
normalize_allowed_hosts
classmethod
¶
Normalize allowed hosts.
IP networks are put in canonical form (without brackets), host names in lowercase and without a trailing dot.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
list[str]
|
Allowed hosts. |
required |
Returns:
| Type | Description |
|---|---|
list[str]
|
Normalized allowed hosts. |
Raises:
| Type | Description |
|---|---|
ValueError
|
An allowed host is empty. |
AccessControlConfig ¶
Bases: FOCABaseConfig
Model for access control settings of the app.
Modelled on the service types of Cloud Registry
(custom.access_control.service_types), with the same modes and the
same default.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tool_classes
|
Who may add tool classes: |
required | |
max_unused_tool_classes
|
How many tool classes that no tool has a
user other than an admin may have added (as
|
required |
Attributes:
| Name | Type | Description |
|---|---|---|
tool_classes |
ToolClassesMode
|
Who may add tool classes. |
max_unused_tool_classes |
Annotated[int, Field(ge=0)] | None
|
How many unused tool classes a user other than an admin may have added. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
|
Example
AccessControlConfig(tool_classes='moderated') AccessControlConfig(tool_classes='moderated', max_unused_tool_classes=10)
IoSchemaConfig ¶
Bases: FOCABaseConfig
Model for GA4GH I/O Schema files (experimental).
I/O Schemas are files of type OTHER named ga4gh-io.schema.json (see
trs_filer.io_schema and docs/io-schema.md); they are stored and
served like other files with either setting.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
validation
|
Whether I/O Schema files are checked on registration,
against the vendored meta-schema of the GA4GH I/O Profile 0.3.0;
invalid ones are rejected with |
required | |
timeout
|
Time limit of checking an I/O Schema file, in seconds
(validation takes time that grows with the size of schemas);
files not checked within it are rejected. Defaults to |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
validation |
bool
|
Whether I/O Schema files are checked on registration. |
timeout |
float
|
Time limit of checking an I/O Schema file, in seconds. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
The time limit is not positive, or not finite. |
Example
IoSchemaConfig(validation=True) IoSchemaConfig(validation=True, timeout=10.0)
S3StorageConfig ¶
Bases: FOCABaseConfig
Model for the connection to S3-compatible object storage.
Only the S3 API is used, so any S3-compatible server or service works.
Credentials are never part of the config: they are taken from the
environment by boto3's default credential chain (e.g., the variables
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, from a Secret).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
bucket
|
Name of the bucket, which must exist. |
required | |
endpoint_url
|
URL of the S3 API (e.g., |
required | |
region
|
Region of the bucket; if not set, boto3's default (e.g., the
environment variable |
required | |
prefix
|
Prefix of the keys of the objects (e.g., |
required | |
addressing_style
|
Addressing of the bucket: |
required | |
presigned_urls
|
Whether files are served via presigned URLs of the
objects instead of through the app (see |
required | |
presigned_url_expiry
|
Validity of presigned URLs, in seconds (at most 7 days, the limit of the S3 API). |
required | |
connect_timeout
|
Timeout for connecting to the S3 API, in seconds. |
required | |
read_timeout
|
Timeout for each read from a connection, in seconds. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
bucket |
str
|
Name of the bucket. |
endpoint_url |
str | None
|
URL of the S3 API. |
region |
str | None
|
Region of the bucket. |
prefix |
str
|
Prefix of the keys of the objects. |
addressing_style |
Literal['auto', 'path', 'virtual']
|
Addressing of the bucket. |
presigned_urls |
bool
|
Whether files are served via presigned URLs. |
presigned_url_expiry |
Annotated[int, Field(strict=True, ge=1, le=604800)]
|
Validity of presigned URLs, in seconds. |
connect_timeout |
float
|
Timeout for connecting to the S3 API, in seconds. |
read_timeout |
float
|
Timeout for each read from a connection, in seconds. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
The bucket is empty, the endpoint URL is not an HTTP(S) URL, the validity of presigned URLs is out of range, or a timeout is not positive or not finite. |
Example
S3StorageConfig(bucket='trs-files', endpoint_url='https://s3.example.org') S3StorageConfig(bucket='trs-files', endpoint_url='https://s3.example.org', region=None, prefix='', addressing_style='path', presigned_urls=False, presigned_url_expiry=300, connect_timeout=5.0, read_timeout=10.0)
check_endpoint_url
classmethod
¶
Check that the endpoint URL is an HTTP(S) URL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str | None
|
Endpoint URL, or |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
The endpoint URL. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The endpoint URL is not an HTTP(S) URL. |
StorageConfig ¶
Bases: FOCABaseConfig
Model for the storage of the content of files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
backend
|
Where the content of files registered from now on is stored:
|
required | |
s3
|
Connection to S3-compatible object storage; required for backend
|
required |
Attributes:
| Name | Type | Description |
|---|---|---|
backend |
Literal['mongodb', 's3']
|
Where the content of new files is stored. |
s3 |
S3StorageConfig | None
|
Connection to S3-compatible object storage. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
The backend is |
Example
StorageConfig(backend='s3', s3=S3StorageConfig(bucket='trs-files')) StorageConfig(backend='s3', s3=S3StorageConfig(bucket='trs-files', ...))
check_s3 ¶
Check that object storage is configured for backend s3.
Returns:
| Type | Description |
|---|---|
StorageConfig
|
The storage config. |
Raises:
| Type | Description |
|---|---|
ValueError
|
The backend is |
CustomConfig ¶
Bases: FOCABaseConfig
Model for custom configuration parameters.
The service info is configured in FOCA's service_info section, no longer
in custom.service_info, which is rejected. The deprecated
toolclass.validation: true sets access_control.tool_classes to
controlled (cf. apply_tool_class_validation()).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
service
|
Service config parameters. |
required | |
tool
|
Tool config parameters. |
required | |
version
|
Version config parameters. |
required | |
toolclass
|
Tool Class config parameters. |
required | |
remote_files
|
Parameters for retrieving files from their URLs. |
required | |
access_control
|
Access control settings of the app. |
required | |
io_schema
|
Checks of GA4GH I/O Schema files (experimental). |
required | |
storage
|
Storage of the content of files. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
service |
ServiceConfig
|
Service config parameters. |
tool |
ToolConfig
|
Tool config parameters. |
version |
VersionConfig
|
Version config parameters. |
toolclass |
ToolClassConfig
|
Tool Class config parameters. |
remote_files |
RemoteFilesConfig
|
Parameters for retrieving files from their URLs. |
access_control |
AccessControlConfig
|
Access control settings of the app. |
io_schema |
IoSchemaConfig
|
Checks of GA4GH I/O Schema files (experimental). |
storage |
StorageConfig
|
Storage of the content of files. |
apply_tool_class_validation ¶
Map the deprecated toolclass.validation to the mode controlled.
toolclass.validation: true required stored tool classes for tools,
as the mode controlled does (cf. AccessControlConfig), which
replaces it: it sets the mode open to controlled; the app warns
of it on startup (cf. trs_filer.app.create_app()). The message
doesn't name the values, so that FOCA shows it on startup (FOCA
leaves out messages that quote invalid values).
Returns:
| Type | Description |
|---|---|
Self
|
The config, with mode |
Self
|
true. |
Raises:
| Type | Description |
|---|---|
ValueError
|
|
utils ¶
Typed access to the configuration and collections of the app.
get_custom_config ¶
Get the custom configuration of the current app.
Returns:
| Type | Description |
|---|---|
CustomConfig
|
Custom configuration. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
The app has no custom configuration. |
get_collection ¶
Get a collection of the app's database.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the collection, as configured. |
required |
Returns:
| Type | Description |
|---|---|
Collection[dict[str, Any]]
|
MongoDB collection. |
Controllers¶
server ¶
Controllers for TRS endpoints.
toolsIdGet ¶
List one specific tool, acts as an anchor for self references.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Tool object dict corresponding given tool id. |
Raises:
| Type | Description |
|---|---|
NotFound
|
No tool with the given identifier exists, or it is pending
and the user may not see it (cf.
|
toolsIdVersionsGet ¶
List versions of a tool.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
List of version dicts corresponding given tool id. |
Raises:
| Type | Description |
|---|---|
NotFound
|
No tool with the given identifier exists. |
toolsIdVersionsVersionIdGet ¶
List one specific tool version, acts as an anchor for self references.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Tool version identifier. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Specific version dict of the given tool. |
Raises:
| Type | Description |
|---|---|
NotFound
|
No tool with the given identifier exists, or it has no version with the given identifier. |
toolsGet ¶
toolsGet(id: str | None = None, alias: str | None = None, toolClass: str | None = None, descriptorType: str | None = None, tags: list[str] | None = None, registry: str | None = None, organization: str | None = None, name: str | None = None, toolname: str | None = None, description: str | None = None, author: str | None = None, checker: bool | None = None, limit: int = MAX_LIMIT, offset: str | None = None) -> tuple[list[dict[str, Any]], int, dict[str, Any]]
List all tools.
Filter parameters to subset the tools list can be specified. Filter
parameters are additive. Pending tools are only listed to admins and the
users with a role on them (cf.
trs_filer.ga4gh.trs.tool_classes.visible_tools()).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str | None
|
Return only entries with the given identifier. |
None
|
alias
|
str | None
|
Return only entries with the given alias. |
None
|
toolClass
|
str | None
|
Return only entries with the given subclass name. |
None
|
descriptorType
|
str | None
|
Return only entries with the given descriptor type. |
None
|
tags
|
list[str] | None
|
Return only entries that have all the given tags, of all
|
None
|
registry
|
str | None
|
Return only entries from the given registry. |
None
|
organization
|
str | None
|
Return only entries from the given organization. |
None
|
name
|
str | None
|
Return only entries with the given image name. |
None
|
toolname
|
str | None
|
Return only entries with the given tool name. |
None
|
description
|
str | None
|
Return only entries with the given description. |
None
|
author
|
str | None
|
Return only entries from the given author. |
None
|
checker
|
bool | None
|
Return only checker workflows. |
None
|
limit
|
int
|
Number of records when paginating results; at most
|
MAX_LIMIT
|
offset
|
str | None
|
Start index when paginating results. |
None
|
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
List of all tools consistent with all filters, if specified, the |
int
|
status code and the pagination headers: |
dict[str, Any]
|
|
tuple[list[dict[str, Any]], int, dict[str, Any]]
|
|
tuple[list[dict[str, Any]], int, dict[str, Any]]
|
the last tool; if |
tuple[list[dict[str, Any]], int, dict[str, Any]]
|
the sequence from 0) and |
tuple[list[dict[str, Any]], int, dict[str, Any]]
|
The links to pages keep the other query parameters of the request. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The offset is not a non-negative integer, or the limit is below 1. |
toolsIdVersionsVersionIdTypeDescriptorGet ¶
toolsIdVersionsVersionIdTypeDescriptorGet(type: str, id: str, version_id: str) -> dict[str, Any] | Response
Get the tool descriptor for the specified tool.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
type
|
str
|
The output type of the descriptor. Allowable values include
"CWL", "WDL", "NFL", "GALAXY", "SMK", and their |
required |
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Identifier to the tool version of the given tool |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Response
|
The tool descriptor. Plain types return the bare descriptor (as |
dict[str, Any] | Response
|
|
dict[str, Any] | Response
|
with metadata. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The descriptor type is invalid. |
NotAcceptable
|
The client doesn't accept the media type of the type. |
NotFound
|
The tool version or its primary descriptor of the given type does not exist, or, for plain types, its content is not available. |
toolsIdVersionsVersionIdTypeDescriptorRelativePathGet ¶
toolsIdVersionsVersionIdTypeDescriptorRelativePathGet(type: str, id: str, version_id: str, relative_path: str) -> dict[str, Any] | Response
Get additional tool descriptor files relative to the main file.
The relative path arrives percent-decoded (with / for %2F, as the
parameter has format path). Paths that match no file are decoded
once more, so that clients that encode paths twice (%252F) find them
too.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
type
|
str
|
The output type of the descriptor. Examples of allowable
values are "CWL", "WDL", "NFL", "GALAXY", "SMK", and their
|
required |
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Tool version identifier. |
required |
relative_path
|
str
|
A relative path to the additional file (same
directory or subdirectories), for example 'foo.cwl' would return
a 'foo.cwl' from the same directory as the main descriptor;
percent-encoded ( |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any] | Response
|
Additional files associated with a given descriptor type of a given |
dict[str, Any] | Response
|
tool version; for plain types, their bare content (as |
dict[str, Any] | Response
|
|
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The descriptor type is invalid. |
NotAcceptable
|
The client doesn't accept the media type of the type. |
NotFound
|
The tool version or a file with the given path and type does not exist, or, for plain types, its content is not available. |
toolsIdVersionsVersionIdTypeTestsGet ¶
toolsIdVersionsVersionIdTypeTestsGet(type: str, id: str, version_id: str) -> list[dict[str, Any]] | Response
Get a list of test JSONs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
type
|
str
|
The output type of the descriptor. Examples of allowable
values are "CWL", "WDL", "NFL", "GALAXY", "SMK", and their
|
required |
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Tool version identifier. |
required |
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]] | Response
|
List of JSONs associated with a given descriptor type of a given |
list[dict[str, Any]] | Response
|
tool version; for plain types, the same list as |
list[dict[str, Any]] | Response
|
the specification defines the plain response (an array of file |
list[dict[str, Any]] | Response
|
wrappers). |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The descriptor type is invalid. |
NotAcceptable
|
The client doesn't accept the media type of the type. |
NotFound
|
The tool version or test files of the given type do not exist. |
toolsIdVersionsVersionIdTypeFilesGet ¶
toolsIdVersionsVersionIdTypeFilesGet(type: str, id: str, version_id: str, format: str | None = None) -> list[dict[str, Any]] | Response
Get the tool_file specification(s) for the specified tool version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
type
|
str
|
The output type of the descriptor. Examples of allowable values are "CWL", "WDL", "NFL", "GALAXY", "SMK". |
required |
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Tool version identifier. |
required |
format
|
str | None
|
|
None
|
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]] | Response
|
List of file JSON responses, or, for format |
list[dict[str, Any]] | Response
|
the ZIP archive of the files. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The descriptor type is invalid. |
InternalServerError
|
The ZIP archive could not be created. |
NotFound
|
The tool version does not exist, or, for format |
toolsIdVersionsVersionIdContainerfileGet ¶
Get the container specification(s) for the specified tool version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
version_id
|
str
|
Tool version identifier. |
required |
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
List of wrapped containerfile objects. |
Raises:
| Type | Description |
|---|---|
NotFound
|
The tool version or its containerfiles do not exist. |
toolClassesGet ¶
List all tool classes.
Pending tool classes are only listed to admins and the user who added
them (cf. trs_filer.ga4gh.trs.tool_classes.visible_tool_classes()).
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
List of tool class objects. |
postTool ¶
Add tool with an auto-generated ID.
The user registering the tool becomes its owner.
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created tool. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The tool is invalid, e.g., its version identifiers are duplicated or the files of a version are invalid, or its tool class does not exist while tool classes are validated. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
putTool ¶
Add/replace tool with a user-supplied ID.
Users who may register tools (POST /tools; by default, any
authenticated user) may add a tool with the identifier of no tool, and
become its owner; only those that the policies allow (owners of the tool
and admins, by default) may replace it (cf. _check_put()).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool to be created/updated. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created/updated tool. |
Raises:
| Type | Description |
|---|---|
Conflict
|
The tool checked was deleted or registered again meanwhile. |
Forbidden
|
The user may not replace the tool, or may not add it. |
InvalidRequest
|
The tool is invalid, e.g., its version identifiers are duplicated or the files of a version are invalid, or its tool class does not exist while tool classes are validated; or the identifier of a new tool can't be stored in roles (e.g., with commas). |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
deleteTool ¶
Delete tool, with the roles granted on it.
The roles are revoked before the tool is deleted, so that nobody can act on it meanwhile, and again afterwards, as roles may have been granted meanwhile. If deleting fails, the tool is left without roles: admins can still delete it, or grant it an owner.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool to be deleted. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Previous identifier of deleted tool. |
Raises:
| Type | Description |
|---|---|
NotFound
|
No tool with the given identifier exists. |
postToolVersion ¶
Add tool version with an auto-generated ID.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool to be modified. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created tool version. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The files of the version are invalid. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
NotFound
|
No tool with the given identifier exists. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
putToolVersion ¶
Add/replace tool version with a user-supplied ID.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool to be modified. |
required |
version_id
|
str
|
Identifier of tool version to be created/updated. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created tool version. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The files of the version are invalid. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
NotFound
|
No tool with the given identifier exists. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
deleteToolVersion ¶
Delete tool version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool to be modified. |
required |
version_id
|
str
|
Identifier of tool version to be deleted. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Previous identifier of deleted tool version. The meta version of the |
str
|
tool is incremented, unless |
str
|
is |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The version is the last one of the tool. |
InternalServerError
|
The tool version could not be deleted. |
NotFound
|
No tool with the given identifier and version exists. |
postToolClass ¶
Add tool class with an auto-generated ID.
Only users who may add tool classes may (cf.
trs_filer.ga4gh.trs.tool_classes.may_add_tool_classes()); the tool
class is pending if the user's tool classes are.
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created tool class. |
Raises:
| Type | Description |
|---|---|
Forbidden
|
The user may not add tool classes. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
putToolClass ¶
Add/replace tool class with a user-supplied ID.
Users who may add tool classes (POST /toolClasses, by default any
authenticated user, and as the mode of
custom.access_control.tool_classes allows, cf.
_check_add_tool_class()) may add a tool class with the identifier of
no tool class, which is pending if the user's tool classes are; only
those that the policies allow (admins, by default) may replace it (cf.
_check_put()), which keeps whether it is pending. Adding a tool class
sets the pending flags of the tools that have it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool class to be created/updated. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Identifier of created/updated tool class. |
Raises:
| Type | Description |
|---|---|
Forbidden
|
The user may not replace the tool class (also if it was added meanwhile), or may not add it. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
deleteToolClass ¶
Delete tool class.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of tool class to be deleted. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Previous identifier of deleted tool class. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The tool class is associated with a tool. |
NotFound
|
No tool class with the given identifier exists. |
putToolClassApproval ¶
Approve a pending tool class, and so the tools that have it.
Only for admins: no policy in config.yaml allows it to other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Identifier of the tool class. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Identifier of the tool class, also if it was approved before. |
Raises:
| Type | Description |
|---|---|
NotFound
|
No tool class with the given identifier exists (also if a pending one was added while the request was processed). |
getApprovals ¶
getApprovals(limit: int = MAX_LIMIT, offset: str | None = None) -> tuple[dict[str, list[dict[str, Any]]], int, dict[str, Any]]
List the pending tool classes and tools the current user may see.
The tool classes and then the tools form one sequence, which is paged as
the tools of GET /tools (cf. toolsGet()).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
limit
|
int
|
Number of tool classes and tools when paginating results; at
most |
MAX_LIMIT
|
offset
|
str | None
|
Start index when paginating results, in the sequence of the tool classes and then the tools. |
None
|
Returns:
| Type | Description |
|---|---|
dict[str, list[dict[str, Any]]]
|
The page, the status code and the pagination headers (cf. |
int
|
|
dict[str, Any]
|
( |
tuple[dict[str, list[dict[str, Any]]], int, dict[str, Any]]
|
their versions) of the page, each sorted by identifier: all for |
tuple[dict[str, list[dict[str, Any]]], int, dict[str, Any]]
|
admins; for other users, the tool classes they added and the tools |
tuple[dict[str, list[dict[str, Any]]], int, dict[str, Any]]
|
they have a role on. |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The offset is not a non-negative integer, or the limit is below 1. |
validate_descriptor_type ¶
Validate tool descriptor type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
type
|
str
|
Descriptor type, e.g., "NFL". |
required |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
Provided descriptor type is invalid. |
permissions ¶
Roles on tools: owners, maintainers and the permission controllers.
As for services in Cloud Registry: any authenticated user may register tools
and becomes their owner; roles on tools (security.access_control.resource_roles:
owner and maintainer) are granted to users (by their identifier, the sub
claim) and to groups (values of the group claim), with FOCA's resource
ownership helpers, which keep the last owner of a tool (409 Conflict) and
log changes of roles. What each role may do is decided by the policies in
config.yaml (security.access_control.policies), which FOCA enforces;
subjects with the role admin may do everything.
The permission controllers are FOCA's ready-made ones
(foca.security.access_control.ownership_server), with a check that the tool
exists, so that admins can grant the first owner of tools stored before tools
had owners.
tool_path ¶
Get the resource path of a tool, i.e., the path roles are granted on.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Request path of the tool, including the base path of the API (e.g., |
str
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
The identifier can't be part of a resource path (cf.
FOCA's |
is_usable_id ¶
Check whether roles can be granted on a tool with an identifier.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the path of the tool is a valid resource of roles: not a |
bool
|
wildcard or dot segment of policy rules, and a valid value of policy |
bool
|
rules (e.g., without commas). |
tool_exists ¶
Check whether a tool exists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the tool is stored. |
tool_class_exists ¶
Check whether a tool class exists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool class identifier. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the tool class is stored. |
own_tool ¶
Make the subject of the current request owner of a new tool.
Roles left on the identifier (e.g., if revoking those of a deleted tool
failed) are revoked first, as they don't apply to the new tool. Nothing
is granted to requests without a subject, which only happen when
authentication is disabled (see "Running without authentication" in
docs/development.md). If the tool was deleted meanwhile (and maybe
registered again with the identifier), the role granted here is revoked
again.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
stored_id
|
Any
|
MongoDB identifier ( |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
The role can't be stored, e.g., for identifiers with
commas (cf. |
remove_roles ¶
Revoke all roles granted on a tool, e.g., when it is deleted.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
getToolPermissions ¶
List the roles granted on a tool.
See FOCA's getResourceRoles().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
**kwargs
|
Any
|
Further arguments that Connexion passes. |
{}
|
Returns:
| Type | Description |
|---|---|
list[dict[str, str]]
|
Subject type, subject and role of each role granted on the tool. |
putToolPermission ¶
Grant a user a role on a tool, replacing any other role.
See FOCA's putUserRole().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
**kwargs
|
Any
|
Further arguments that Connexion passes ( |
{}
|
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Subject type ( |
deleteToolPermission ¶
Revoke the roles of a user on a tool.
See FOCA's deleteUserRole().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
**kwargs
|
Any
|
Further arguments that Connexion passes ( |
{}
|
Returns:
| Type | Description |
|---|---|
str
|
User identifier. |
putToolGroupPermission ¶
Grant a group a role on a tool, replacing any other role.
See FOCA's putGroupRole(): owners may only grant roles to groups they
are members of, admins to any group.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
**kwargs
|
Any
|
Further arguments that Connexion passes ( |
{}
|
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Subject type ( |
deleteToolGroupPermission ¶
Revoke the roles of a group on a tool.
See FOCA's deleteGroupRole().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool identifier. |
required |
**kwargs
|
Any
|
Further arguments that Connexion passes ( |
{}
|
Returns:
| Type | Description |
|---|---|
str
|
The group. |
tool_classes ¶
Modes of adding tool classes, and pending tool classes and tools.
As the service types of Cloud Registry: who may add tool classes is
configured (custom.access_control.tool_classes). In mode open (the
default), any authenticated user, explicitly (POST /toolClasses, PUT
/toolClasses/{id} with the identifier of no tool class) or by registering a
tool with a new tool class; in mode controlled, admins only, and tools of
other users must have a stored tool class (cf. may_add_tool_classes()). In
every mode, only admins replace and delete tool classes (no policy in
config.yaml allows it to other users).
In mode moderated, any authenticated user may add tool classes, but those
that users other than admins add are pending (PENDING) until an admin
approves them (PUT /toolClasses/{id}/approval); there is no way back,
admins delete tool classes instead. Tools whose tool class is pending are
pending too: stored with an indexed flag (PENDING, as for tool classes),
which registrations set, approvals clear, and the app reconciles on startup
(cf. sync_pending_tools()). Pending tool classes are only listed (GET
/toolClasses) to admins and the user who added them (CREATED_BY), pending
tools (all reads of tools) to admins and the users with a role on them;
reads without a token have no user, so they don't list them (cf. GET
/approvals, which requires login). Pending tool classes stay pending if
the mode changes.
Users other than admins may only have added a limited number of tool
classes that no tool has (custom.access_control.max_unused_tool_classes;
cf. check_unused_tool_classes()), in every mode, so that they can't grow
the list of tool classes (or of pending tool classes) without bound.
Without authentication (disable_auth), there are no users and no admins:
anybody may add tool classes, which are never pending, and all tools are
listed.
auth_disabled ¶
Check whether authentication is disabled.
Returns:
| Type | Description |
|---|---|
bool
|
Whether the APIs of the app are served without authentication |
bool
|
( |
bool
|
without authentication" in |
bool
|
them requires it (e.g., the TRS API next to another one), so that |
bool
|
pending tools and tool classes stay hidden then. |
may_add_tool_classes ¶
Check whether the current user may add tool classes.
Returns:
| Type | Description |
|---|---|
bool
|
Whether the deployment lets any authenticated user add tool classes |
bool
|
( |
bool
|
user is an admin, or authentication is disabled. |
tool_classes_pending ¶
Check whether tool classes that the current user adds are pending.
Returns:
| Type | Description |
|---|---|
bool
|
Whether the deployment moderates tool classes |
bool
|
( |
bool
|
is not an admin, so that their new tool classes are pending until an |
bool
|
admin approves them; never without authentication. |
is_approved ¶
Check whether a tool class is stored and approved.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool class identifier. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the tool class is stored and not pending (cf. |
check_unused_tool_classes ¶
Check that the current user may add another tool class that no tool has.
Must be called in a request context, before a tool class is added (explicitly, or by registering a tool with a new tool class). Concurrent requests of a user may exceed the limit by their number.
Raises:
| Type | Description |
|---|---|
UnusedToolClassesLimit
|
The user is not an admin, and has added as
many tool classes that no tool has as allowed
( |
visible_tool_classes ¶
Get the filter for the tool classes that the current user may see.
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Filter for all tool classes if the user is an admin or authentication |
dict[str, Any]
|
is disabled; otherwise, for the approved tool classes, and the |
dict[str, Any]
|
pending ones the user added. |
own_tools ¶
Get the filter for the tools that the current user has a role on.
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Filter for all tools if authentication is disabled, or (FOCA's |
dict[str, Any]
|
|
dict[str, Any]
|
admin; otherwise, for the tools the user has a role on (also through |
dict[str, Any]
|
groups), none without a user. |
visible_tools ¶
Restrict a query of tools to those the current user may see.
The filter has the same size for any number of pending tool classes or tools.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
dict[str, Any] | None
|
Query of tools; all tools if |
None
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
The query if the user is an admin or authentication is disabled; |
dict[str, Any]
|
otherwise, always the query restricted to the tools that aren't |
dict[str, Any]
|
pending (also if none is), and, if any tool is pending (one indexed |
dict[str, Any]
|
lookup), those the user has a role on. |
sync_pending_flags ¶
Set or clear the pending flags of tools by their tool class.
Tools are pending unless their tool class is stored and approved (e.g.,
also while it is deleted, or not yet added). Writes first, then checks
the tool class again, and writes again if its state changed meanwhile
(e.g., approved, or deleted and added again as pending), so that
concurrent calls and approvals end with the flags of its last state:
each change of the state is followed by a call (cf.
putToolClassApproval(), RegisterTool.register_metadata()), and
either that call or the check here sees the other's write.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
id
|
str
|
Tool class identifier. |
required |
query
|
dict[str, Any] | None
|
Query of the tools to change (e.g., one tool); all tools of
the tool class if |
None
|
sync_pending_tools ¶
Set the pending flags of the stored tools by their tool classes.
Run on startup, so that flags missed by interrupted requests are
corrected; idempotent, and safe to run concurrently (cf.
sync_pending_flags()). Only checks the pending tool classes, and the
tool classes of the tools flagged as pending, not all tool classes.
register_objects ¶
Controllers for registering new objects.
RegisterTool ¶
RegisterTool(data: dict[str, Any], id: str | None = None, created_by: str | None = None, allow_replace: bool = True, stored_id: Any = None)
Class to register tools with the service.
Initialize tool data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
dict[str, Any]
|
Tool metadata consistent with the |
required |
id
|
str | None
|
Tool identifier. Auto-generated if not provided. |
None
|
created_by
|
str | None
|
Identifier of the user registering the tool, stored
with new tools ( |
None
|
allow_replace
|
bool
|
With |
True
|
stored_id
|
Any
|
MongoDB identifier ( |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
data |
Tool metadata. |
|
replace |
Whether it is allowed to replace an existing tool. Set
to |
|
created_by |
Identifier of the user registering the tool. |
|
was_created |
Whether the tool was created (not replaced), once it is registered. |
|
pending |
Whether the tool is pending, as its tool class is (cf.
|
|
stored_id |
|
|
id_charset |
Allowed characters for generating object identifiers. |
|
id_length |
Length of generated object identifiers. |
|
meta_version_init |
Initial value for tool meta version. |
|
meta_version_increment |
Increment of the tool meta version when
the tool is replaced, and, with
|
|
version_meta_version_init |
Initial value for version meta versions. |
|
version_meta_version_increment |
Increment of the meta version of a version when the tool replaces it. |
|
url_prefix |
URL scheme of application. For constructing tool and
version |
|
host_name |
Name of application host. For constructing tool and
version |
|
external_port |
Port at which application is served. For
constructing tool and version |
|
api_path |
Base path at which API endpoints can be reached. For
constructing tool and version |
|
db_coll_tools |
Database collection for storing tool objects. |
|
db_coll_classes |
Database collection for storing tool class objects. |
register_metadata ¶
Register tool.
Tools with a given identifier replace a stored tool with that
identifier, if any; their meta version, and those of their versions
that replace stored versions, are incremented. If the tool class does
not exist, it is created, if the user may add tool classes (cf.
trs_filer.ga4gh.trs.tool_classes): after the tool, or before it if
it is pending, so that the tool is never served as approved; existing
tool classes are not changed, and tools store a copy of them (without
internal fields). Tools with a pending tool class are pending too. New
tools store who registered them and when; replaced tools keep these,
and store when they were replaced.
Raises:
| Type | Description |
|---|---|
Conflict
|
The stored tool isn't the one that may be replaced
( |
Forbidden
|
A tool with the given identifier is stored, but may not
be replaced ( |
InvalidRequest
|
Version identifiers are duplicated, the files of a version are invalid, or the tool class does not exist and the user may not add tool classes. |
UnusedToolClassesLimit
|
The tool class does not exist, and the
user has added as many tool classes that no tool has as
allowed (cf. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
RegisterToolVersion ¶
Class to register a version with a tool object.
Initialize tool version data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
dict[str, Any]
|
Version metadata consistent with the |
required |
id
|
str
|
Tool identifer. |
required |
version_id
|
str | None
|
Version identifier. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
data |
Version metadata. |
|
id |
Tool identifer. |
|
replace |
Whether it is allowed to replace an existing version. Set
to |
|
id_charset |
Allowed characters for generating version identifiers. |
|
id_length |
Length of generated version identifiers. |
|
meta_version_init |
Initial value for version meta version. |
|
meta_version_increment |
Increment of the version meta version when the version is replaced. |
|
url_prefix |
URL scheme of application. For constructing tool and
version |
|
host_name |
Name of application host. For constructing tool and
version |
|
external_port |
Port at which application is served. For
constructing tool and version |
|
api_path |
Base path at which API endpoints can be reached. For
constructing tool and version |
|
db_coll_tools |
Database collection for storing tool objects. |
process_metadata ¶
Process version metadata.
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The files of the version are invalid. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
set_id ¶
Set the identifier of the version, and its self reference URL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
version_id
|
str
|
Version identifier. |
required |
set_tool_id ¶
Set the identifier of the tool, and the version's self reference URL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tool_id
|
str
|
Tool identifier. |
required |
process_files ¶
Process file (meta)data.
With custom.io_schema.validation, GA4GH I/O Schema files are
checked against the meta-schema of the profile (see
trs_filer.io_schema.check.check_io_schema()).
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The content of a file could not be stored in
object storage (with |
InvalidRequest
|
The files of the version are invalid, or, with
|
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
register_metadata ¶
Register version with tool.
Versions with a given identifier replace a stored version of the tool
with that identifier, if any; their meta version is incremented. The
meta version of the tool is incremented, too, unless
custom.tool.meta_version.on_version_change is false.
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The files of the version are invalid. |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
Conflict
|
The tool was deleted, or deleted and registered again, while the version was processed. |
NotFound
|
The tool does not exist. |
RemoteFileError
|
The content of a file could not be retrieved from its URL. |
now ¶
Get the current time, for timestamps of tools.
Returns:
| Type | Description |
|---|---|
datetime
|
Current time (UTC), to the millisecond, as MongoDB stores it. |
next_meta_version ¶
Get the meta version of a replaced object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
previous
|
object
|
Meta version of the stored object. |
required |
init
|
int
|
Initial meta version, for stored objects without a valid one. |
required |
increment
|
int
|
Increment of meta versions. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The previous meta version plus the increment, as a string (as in the |
str
|
TRS specification). |
tool_meta_version_change ¶
Get the change of a tool's meta version when one of its versions changes.
The next meta version is computed from the stored one, not atomically
with the update that sets it: concurrent changes of the tool may give it
the same or an earlier meta version (see docs/configuration.md).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
db_coll_tools
|
Collection[dict[str, Any]]
|
Database collection for storing tool objects. |
required |
tool_id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
The tool's next meta version (field |
dict[str, str]
|
the change of the version, or nothing if the tool doesn't exist or |
dict[str, str]
|
|
register_tool_classes ¶
Controller for registering new objects.
RegisterToolClass ¶
RegisterToolClass(data: dict[str, Any], id: str | None = None, allow_replace: bool = True, replace_only: bool = False, created_by: str | None = None, pending: bool = False)
Class to register tool classes with the service.
Initialize tool class data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
dict[str, Any]
|
Tool class metadata consistent with the |
required |
id
|
str | None
|
Tool class identifier. Auto-generated if not provided. |
None
|
allow_replace
|
bool
|
With |
True
|
replace_only
|
bool
|
With |
False
|
created_by
|
str | None
|
Identifier of the user adding the tool class, stored
with new tool classes (cf.
|
None
|
pending
|
bool
|
Whether a new tool class is pending (cf.
|
False
|
Attributes:
| Name | Type | Description |
|---|---|---|
data |
Tool metadata. |
|
replace |
Whether it is allowed to replace an existing tool class.
Set to |
|
id_charset |
Allowed characters for generating object identifiers. |
|
id_length |
Length of generated object identifiers. |
|
meta_version_init |
Initial value for tool meta version. |
|
url_prefix |
URL scheme of application. For constructing tool and
version |
|
host_name |
Name of application host. For constructing tool and
version |
|
external_port |
Port at which application is served. For
constructing tool and version |
|
api_path |
Base path at which API endpoints can be reached. For
constructing tool and version |
|
created_by |
Identifier of the user adding the tool class. |
|
pending |
Whether a new tool class is pending. |
|
db_coll_classes |
Database collection for storing tool class objects. |
register_metadata ¶
Register toolClass with TRS.
New tool classes store who added them, and whether they are pending. Replaced tool classes keep these: replacing doesn't approve them.
Raises:
| Type | Description |
|---|---|
Conflict
|
The tool class to be replaced ( |
Forbidden
|
A tool class with the given identifier is stored, but
may not be replaced ( |
InternalServerError
|
No unique identifier could be generated, or the object could not be stored. |
I/O Schemas¶
io_schema ¶
Validation of GA4GH I/O Schema files (experimental).
The GA4GH I/O Schema describes
the inputs and outputs of a workflow. It is published in TRS as a file of
type OTHER named ga4gh-io.schema.json, next to the primary descriptor; it
needs no change of the TRS specification. With custom.io_schema.validation,
such files are checked on registration against a vendored copy of the
meta-schema of the profile (trs_filer.io_schema.check, see
docs/io-schema.md).
The meta-schema and its vocabularies are vendored unchanged from
schemas/ of uniqueg/ga4gh-io-schema at commit d44e8d4 (2026-11-13, branch
v0.3, profile version 0.3.0), licensed under the Apache License 2.0, in
the directory named after the date and commit (META_SCHEMA_DIR). Their
references are resolved from these copies; nothing is fetched.
Validating against the meta-schema takes time that grows about linearly
with the size of a schema, also for deeply nested ones, but a schema that
fits in a request can still take many seconds. So it runs in a process of
its own, with a time limit (python -m trs_filer.io_schema, see
main()). This module imports only the validator's packages, so that the
process starts quickly.
is_io_schema_file ¶
Check whether a file of a tool version is an I/O Schema file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tool_file
|
dict[str, Any]
|
File type and path of the file ( |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the file has type |
bool
|
(case-sensitive), in any directory. |
find_problem ¶
Validate an I/O Schema against the meta-schema of the profile.
The reason says where the schema breaks a rule of the meta-schema (as a
JSON path, and the keyword of the meta-schema), but doesn't contain its
values (cf. trs_filer.errors.exceptions.InvalidRequest).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema
|
dict[str, Any]
|
The I/O Schema, which names the meta-schema in |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
Why the schema is invalid, or |
main ¶
Validate an I/O Schema from standard input.
Reads the schema as UTF-8 from standard input and writes the result to
standard output, as a JSON object with the reason why the schema is
invalid (problem, cf. find_problem()), or null if it is valid. Run
with python -m trs_filer.io_schema <seconds> by
trs_filer.io_schema.check.check_io_schema().
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
argv
|
list[str] | None
|
Arguments: the limit of the CPU time of the process, in whole
seconds ( |
None
|
check ¶
Checks of GA4GH I/O Schema files on registration (experimental).
Cf. custom.io_schema and docs/io-schema.md.
check_io_schema ¶
Check an I/O Schema file against the meta-schema of the profile.
The reasons for rejecting a file say where the file breaks a rule of the
meta-schema (as a JSON path, and the keyword of the meta-schema), but
don't contain its values (see InvalidRequest).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
str
|
Content of the file. |
required |
descriptor_type
|
str
|
Descriptor type that the file is registered for (a supported one). |
required |
timeout
|
float
|
Time limit of the validation against the meta-schema, in seconds. |
required |
Raises:
| Type | Description |
|---|---|
InvalidRequest
|
The file is not JSON (or not UTF-8), not a JSON
object, doesn't name the meta-schema of the supported version of
the profile in |
InternalServerError
|
The validation failed. |
Remote files¶
remote_files ¶
Retrieve the content of files from their URLs.
All retrieval of file content from URLs goes through
:py:func:fetch_remote_file, which applies the settings of the
custom.remote_files config section (cf.
:py:class:trs_filer.custom_config.RemoteFilesConfig): allowed schemes,
allowed destinations, timeouts, a maximum size and a maximum number of
redirects.
Destinations are checked when connecting, by connection classes of
urllib3 2.x (cf. pyproject.toml) that resolve host names themselves
and connect only to the checked addresses. The total timeout is enforced by
a timer that shuts down the connections of a retrieval when it expires.
InvalidURL ¶
Bases: ValueError
A URL is invalid or uses a scheme that is not allowed.
DestinationNotAllowed ¶
Bases: Exception
A destination is not allowed by the configuration.
Not an OSError, so that urllib3 and requests neither retry nor wrap
it.
RemoteFile
dataclass
¶
Content retrieved from a URL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
content
|
bytes
|
Content. |
required |
encoding
|
str | None
|
Character encoding declared by the response, if any. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
content |
bytes
|
Content. |
encoding |
str | None
|
Character encoding declared by the response, if any. |
text
property
¶
Content decoded as text.
Returns:
| Type | Description |
|---|---|
str
|
Content, decoded with the declared encoding. Without a declared |
str
|
encoding (or with an unknown one), content is decoded as UTF-8 |
str
|
if valid, else with the encoding detected by |
str
|
|
check_url ¶
Check that a URL is a valid HTTP(S) URL with an allowed scheme.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
object
|
URL. |
required |
schemes
|
Collection[str]
|
Allowed schemes, lowercase ( |
required |
Raises:
| Type | Description |
|---|---|
InvalidURL
|
The URL is not a string, is not an HTTP(S) URL with a host (e.g., has user info, whitespace or control characters, or an invalid port), or uses a scheme that is not allowed. |
is_public_address ¶
Check whether an IP address is public.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
address
|
str
|
IP address, optionally with an IPv6 zone identifier. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Whether the address is globally reachable (as per the IANA special |
bool
|
purpose address registries) and not a multicast address. IPv6 |
bool
|
addresses with embedded IPv4 addresses (IPv4-mapped, |
bool
|
IPv4-compatible, NAT64, 6to4 and Teredo addresses) are checked as |
bool
|
their IPv4 addresses (both server and client for Teredo). |
make_destination_check ¶
Make a check of destinations, as configured.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
RemoteFilesConfig
|
Settings for retrieving files. |
required |
Returns:
| Type | Description |
|---|---|
DestinationCheck
|
Function that raises :py:class: |
DestinationCheck
|
name and one of its resolved IP addresses, unless the host name or |
DestinationCheck
|
address is allowed explicitly ( |
DestinationCheck
|
public. |
fetch_remote_file ¶
Retrieve the content of a file from its URL, as configured.
The URL, and the URL of every redirect, must use an allowed scheme and
be valid (cf. :py:func:check_url); its host must resolve to
allowed addresses only. Redirects are followed up to the maximum number;
responses other than 2xx fail. Retrieval, including redirects, must
finish within the total timeout; name resolution is not limited by it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
URL of the file. |
required |
config
|
RemoteFilesConfig
|
Settings for retrieving files. |
required |
Returns:
| Type | Description |
|---|---|
RemoteFile
|
Retrieved content. |
Raises:
| Type | Description |
|---|---|
RemoteFileError
|
The content could not be retrieved, or retrieval is not allowed by the configuration. |
File storage¶
storage ¶
Storage of the content of files in S3-compatible object storage (optional).
By default, the content of files is stored in MongoDB, with the other data of
tools (file_wrapper.content of the files of versions). With
custom.storage.backend: s3, the content of files registered from then on is
stored as objects in a bucket of S3-compatible object storage instead, via
the S3 API only (boto3), and MongoDB keeps their metadata: each file of a
version records its object (field _storage, see STORAGE_FIELD), with the
object's key, the SHA-256 checksum and the size of its content, which are
checked when the content is read.
Objects are written once, under random keys, when a version is registered,
and never changed. Objects that no stored file refers to any more (of
replaced or deleted versions and tools, and of registrations that failed) are
deleted at the end of the request that released them (releases_objects()).
Files are served through the app, which reads their content from object
storage, or, with custom.storage.s3.presigned_urls, via presigned URLs of
their objects (see served_wrapper() and plain_redirect()).
ObjectMissing ¶
Bases: InternalServerError
The object of a file doesn't exist (e.g., deleted meanwhile).
FileStore ¶
Store of file contents in a bucket of S3-compatible object storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
S3StorageConfig
|
Connection to object storage. |
required |
client
|
S3Client | None
|
S3 client; created from |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
config |
Connection to object storage. |
|
client |
S3Client
|
S3 client. |
Initialize the store.
new_key ¶
Get the key of a new object: random, with the prefix.
Returns:
| Type | Description |
|---|---|
str
|
Key. |
put ¶
Store content as a new object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Key of the object (cf. |
required |
content
|
bytes
|
Content of a file. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Reference to the object: its |
dict[str, Any]
|
(hex) and |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The object could not be stored (it may have been stored nevertheless, e.g., if the response was lost). |
get ¶
Get the content of an object, checked against its reference.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ref
|
dict[str, Any]
|
Reference to the object (cf. |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
Content of the object. |
Raises:
| Type | Description |
|---|---|
ObjectMissing
|
The object doesn't exist. |
InternalServerError
|
The object could not be read, or its content doesn't match the size or checksum of the reference. |
delete ¶
Delete objects; failures are logged, not raised.
One request per object, as DeleteObjects requires checksums that
some S3-compatible servers don't support.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keys
|
Iterable[str]
|
Keys of the objects. |
required |
presigned_url ¶
Get a presigned URL to read an object.
The URL is valid for presigned_url_expiry seconds; responses are
text/plain, as when files are served through the app, and
downloads (Content-Disposition: attachment).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Key of the object. |
required |
Returns:
| Type | Description |
|---|---|
str
|
Presigned URL. |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The URL could not be created (e.g., without credentials). |
get_file_store ¶
store_file ¶
Store the content of a file of a version to be registered.
With backend s3, the content (file_wrapper.content) is stored as an
object and replaced by a reference to it (STORAGE_FIELD); the key is
remembered for the request (cf. releases_objects()). Otherwise, the
file is left as it is.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entry
|
dict[str, Any]
|
File of a version ( |
required |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The content could not be stored. |
file_content ¶
Get the content of a file from object storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entry
|
dict[str, Any]
|
File of a stored version. |
required |
Returns:
| Type | Description |
|---|---|
bytes | None
|
Content of the file, or |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The content could not be read, or doesn't match its checksum, or object storage is not configured. |
served_wrapper ¶
Get the file wrapper of a file as served.
Files in MongoDB are served as stored. For files in object storage, the
content is read from there, or, with presigned URLs, the wrapper has the
presigned URL of the object as url instead of the content, and the
SHA-256 checksum of the content (sha-256), unless given already.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entry
|
dict[str, Any]
|
File of a stored version. |
required |
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
File wrapper. |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The content could not be read, or doesn't match its checksum, or object storage is not configured. |
plain_redirect ¶
Get the redirect to the presigned URL of a file, for plain types.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
entry
|
dict[str, Any]
|
File of a stored version. |
required |
Returns:
| Type | Description |
|---|---|
Response | None
|
Redirect ( |
Response | None
|
or |
Response | None
|
are not enabled. |
Raises:
| Type | Description |
|---|---|
InternalServerError
|
The URL could not be created, or object storage is not configured. |
tool_object_keys ¶
Get the keys of the objects that the files of a stored tool refer to.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
tool_id
|
str
|
Tool identifier. |
required |
Returns:
| Type | Description |
|---|---|
set[str]
|
Keys of the objects. |
referenced_keys ¶
Get those of the given object keys that stored files refer to.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keys
|
set[str]
|
Keys of objects. |
required |
Returns:
| Type | Description |
|---|---|
set[str]
|
Keys that the files of stored tools refer to. |
retries_missing_objects ¶
Call a read controller again if an object of a file was missing.
A version may be replaced or deleted (and the objects of its files deleted) between reading its files and reading their objects; the controller is then called once more, and finds the new files, or none.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
controller
|
Callable[P, R]
|
Controller that serves files. |
required |
Returns:
| Type | Description |
|---|---|
Callable[P, R]
|
Controller that retries once. |
releases_objects ¶
Delete the objects that a write request released.
For the controllers that write tools and versions (id: the tool's
identifier, if any): after the request, successful or not, the objects of
the tool's files before the request and those written during the request
(store_file()) are deleted, unless stored files refer to them. As keys
are random and written once, no other request can refer to them again.
Without object storage, the controller is called as is.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
controller
|
Callable[P, R]
|
Controller. |
required |
Returns:
| Type | Description |
|---|---|
Callable[P, R]
|
Controller that deletes released objects. |
Errors¶
exceptions ¶
Exceptions for TRS-filer.
InvalidRequest ¶
Bases: BadRequest
The app refuses a request that matches the API specification.
E.g., a tool whose files break a rule of the app, or a query that the
app doesn't support. The description says why and is sent to clients as
detail (cf. describe_invalid_request()), so it must not contain
submitted values, which may hold credentials (e.g., URLs). Unlike FOCA's
error handler, describe_invalid_request() doesn't log: log the reason
(with identifiers, if useful) where it is raised.
UnusedToolClassesLimit ¶
Bases: Conflict
A user other than an admin has added too many unused tool classes.
Cf. custom.access_control.max_unused_tool_classes; the description
says what to do and is sent to clients as detail (cf.
describe_invalid_request(), which serves it).
Create the exception, with a description of what to do.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
limit
|
int
|
How many unused tool classes a user may have added. |
required |
RemoteFileError ¶
Bases: BadRequest
The content of a file could not be retrieved from its URL.
Raised if retrieval fails, or isn't allowed by the configuration
(custom.remote_files). The description says why; it is logged, not
returned to clients.
describe_invalid_request ¶
Answer an invalid request with its reason as detail.
Error handler of the app for InvalidRequest and
UnusedToolClassesLimit (409 Conflict; registered in
trs_filer.app.create_app()), whose descriptions say what to do:
FOCA's error handler sends the fixed
problem details of exceptions for all other exceptions, including
other bad requests (e.g., RemoteFileError, whose description is not
for clients). Like FOCA's, its responses are not validated against the
API specification. Cf. FOCA's problem_response(), which
cloud-registry's controllers return.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exc
|
InvalidRequest | UnusedToolClassesLimit
|
Invalid request, or a request over the limit of unused tool classes. |
required |
Returns:
| Type | Description |
|---|---|
Response
|
Problem details response with the status of the exception ( |
Response
|
or |