mex.backend package

Subpackages

Submodules

mex.backend.exceptions module

exception mex.backend.exceptions.BackendError

Bases: MExError

Base backend error that offer details on underlying pydantic errors.

errors() → list[ErrorDetails]

Details about underlying pydantic errors.

is_retryable() → bool

Whether the error is retryable.

class mex.backend.exceptions.DebuggingInfo(*, errors: list[dict[str, Any]], scope: DebuggingScope)

Bases: BaseModel

Debugging information for error responses.

errors: list[dict[str, Any]]
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scope: DebuggingScope
class mex.backend.exceptions.DebuggingScope(*, http_version: str, method: str, path: str, path_params: dict[str, Any], query_string: str, scheme: str)

Bases: BaseModel

Scope for debugging info of error responses.

http_version: str
method: str
model_config = {'extra': 'ignore'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: str
path_params: dict[str, Any]
query_string: str
scheme: str
class mex.backend.exceptions.DetailedError(*args, **kwargs)

Bases: Protocol

Protocol for errors that offer details.

errors() → list[ErrorDetails]

Details about each underlying error.

class mex.backend.exceptions.ErrorResponse(*, message: str, debug: DebuggingInfo)

Bases: BaseModel

Response model for user and system errors.

debug: DebuggingInfo
message: str
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

mex.backend.exceptions.handle_detailed_error(request: Request, exc: Exception) → Response

Handle detailed errors and provide debugging info.

mex.backend.exceptions.handle_uncaught_exception(request: Request, exc: Exception) → Response

Handle uncaught errors and provide debugging info.

mex.backend.helpers module

mex.backend.helpers.build_reference_filters(reference_field: ReferenceFieldName | None = None, referenced_identifier: Sequence[Identifier] | None = None, stable_target_id: Identifier | None = None) → list[ReferenceFilter]

Convert deprecated reference parameters to ReferenceFilter objects.

Parameters:

  • reference_field – Deprecated query parameter for reference field name

  • referenced_identifier – Deprecated query parameter for identifiers

  • stable_target_id – Deprecated query parameter for stableTargetId

Raises:

HTTPException – If only one of the two parameters is provided

Returns:

list with a single ReferenceFilter, or empty list if neither parameter provided

mex.backend.logging module

mex.backend.main module

mex.backend.main.lifespan(_: FastAPI) → AsyncIterator[None]

Async context manager to execute startup and teardown of the FastAPI app.

mex.backend.models module

class mex.backend.models.APIKeyDatabase(*, read: list[APIKey] = [], write: list[APIKey] = [])

Bases: BaseModel

A lookup from access level to list of allowed APIKeys.

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

read: list[APIKey]
write: list[APIKey]
class mex.backend.models.ReferenceFilter(*, field: ReferenceFieldName, identifiers: Annotated[list[Identifier | None], MinLen(min_length=1), MaxLen(max_length=100)])

Bases: BaseModel

Reference filter definition with a field and list of identifiers.

field: ReferenceFieldName
identifiers: Annotated[list[Identifier | None], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1), MaxLen(max_length=100)])]
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

mex.backend.security module

mex.backend.security.has_read_access(api_key: ~typing.Annotated[str, ~fastapi.params.Depends(dependency=<fastapi.security.api_key.APIKeyHeader object at 0x7ff354accc20>, use_cache=True, scope=None)]) → None

Verify if api key has read access or write access.

Raises:

HTTPException if APIKey has no read access. –

Parameters:

api_key – the API key

mex.backend.security.has_write_access(api_key: ~typing.Annotated[str, ~fastapi.params.Depends(dependency=<fastapi.security.api_key.APIKeyHeader object at 0x7ff354accc20>, use_cache=True, scope=None)]) → None

Verify if provided api key has write access.

Raises:

HTTPException if APIKey has no write access. –

Parameters:

api_key – the API key

mex.backend.security.is_ldap_authenticated(credentials: ~typing.Annotated[~fastapi.security.http.HTTPBasicCredentials, ~fastapi.params.Depends(dependency=<fastapi.security.http.HTTPBasic object at 0x7ff354accd70>, use_cache=True, scope=None)]) → str

Authenticate against LDAP.

Raises:

HTTPException if credentials have no LDAP write access. –

Parameters:

credentials – username and password

Returns:

username

mex.backend.settings module

class mex.backend.settings.BackendSettings(_env_file: Path | str | Sequence[Path | str] | None = PosixPath('.'), _env_file_encoding: str | None = None, _env_nested_delimiter: str | None = None, _secrets_dir: str | Path | None = None, *, MEX_DEBUG: bool = False, MEX_SINK: list[Sink] = [Sink.NDJSON], MEX_ASSETS_DIR: Path = PosixPath('/home/runner/work/mex-backend/mex-backend/assets'), MEX_OPS_DIR: Path = PosixPath('/home/runner/work/mex-backend/mex-backend/ops'), MEX_WORK_DIR: Path = PosixPath('/home/runner/work/mex-backend/mex-backend'), MEX_IDENTITY_PROVIDER: IdentityProvider = IdentityProvider.GRAPH, MEX_ASSETS_CONNECTOR: AssetsConnectorType = AssetsConnectorType.FILESYSTEM, MEX_BACKEND_API_URL: HttpUrl = HttpUrl('http://localhost:8080/'), MEX_BACKEND_API_KEY: SecretStr = SecretStr('**********'), MEX_BACKEND_API_PARALLELIZATION: int = 1, MEX_BACKEND_API_CHUNK_SIZE: int = 25, MEX_VERIFY_SESSION: bool | OpsPath = True, MEX_ORGANIGRAM_PATH: str = 'raw-data/organigram/organizational_units.json', MEX_PRIMARY_SOURCES_PATH: str = 'raw-data/primary-sources/primary-sources.json', MEX_LDAP_URL: SecretStr = SecretStr('**********'), MEX_LDAP_SEARCH_BASE: str = 'dc=ldapmock,dc=local', MEX_WIKI_API_URL: HttpUrl = HttpUrl('http://wikidata/'), MEX_WEB_USER_AGENT: str = 'rki/mex', MEX_ORCID_API_URL: HttpUrl = HttpUrl('https://orcid/'), MEX_BACKEND_HOST: Annotated[str, MinLen(min_length=1), MaxLen(max_length=250)] = 'localhost', MEX_BACKEND_PORT: Annotated[int, Gt(gt=0), Lt(lt=65536)] = 8080, MEX_BACKEND_ROOT_PATH: str = '', MEX_BACKEND_GRACEFUL_SHUTDOWN_TIMEOUT: Annotated[int, Gt(gt=0)] = 30, MEX_GRAPH_URL: str = 'neo4j://localhost:7687', MEX_GRAPH_NAME: str = 'neo4j', MEX_GRAPH_USER: SecretStr = SecretStr('**********'), MEX_GRAPH_PASSWORD: SecretStr = SecretStr('**********'), MEX_GRAPH_MAX_CONNECTION_POOL_SIZE: Annotated[int, Gt(gt=0)] = 100, MEX_GRAPH_TX_TIMEOUT: int | float = 15.0, MEX_GRAPH_SESSION_TIMEOUT: int | float = 45.0, MEX_BACKEND_NON_MERGEABLE_TYPES: list[MergedType] = [MergedType.MERGED_CONSENT, MergedType.MERGED_PERSON], MEX_BACKEND_API_KEY_DATABASE: APIKeyDatabase = APIKeyDatabase(read=[], write=[]), MEX_BACKEND_CACHE_CONNECTOR: CacheConnectorType = CacheConnectorType.MEMORY, MEX_BACKEND_VALKEY_URL: SecretStr = SecretStr('**********'))

Bases: BaseSettings

Settings definition for the backend server.

assert_cache_connector_is_shared_when_parallelized() → Self

Validate that a shared cache is configured if parallelization is > 1.

Rationale: We cache identities to make sure that multiple calls for getting an identifier receive the same identifier, even if the item with this identifier is not yet ingested in our database. If we use multiple backend instances, they must use a shared cache for storing these identities. The only shared cache is valkey, hence we make sure that valkey is configured if parallelization > 1.

assert_identity_provider_is_graph() → Self

Validate that the graph identity provider is configured.

Rationale: The backend is the service that owns the graph database and is therefore the only component that can assign and resolve identities in it. Any other provider would either delegate back to the backend itself (BACKEND) or hand out identifiers that are lost on restart (MEMORY), so we make sure the backend always uses the graph identity provider.

backend_api_key_database: APIKeyDatabase
backend_graceful_shutdown_timeout: int
backend_host: str
backend_port: int
backend_root_path: str
cache_connector: CacheConnectorType
graph_db: str
graph_max_connection_pool_size: int
graph_password: SecretStr
graph_session_timeout: int | float
graph_tx_timeout: int | float
graph_url: str
graph_user: SecretStr
identity_provider: IdentityProvider
model_config = {'arbitrary_types_allowed': True, 'case_sensitive': False, 'cli_avoid_json': False, 'cli_enforce_required': False, 'cli_exit_on_error': True, 'cli_flag_prefix_char': '-', 'cli_hide_none_type': False, 'cli_ignore_unknown_args': False, 'cli_implicit_flags': False, 'cli_kebab_case': False, 'cli_parse_args': None, 'cli_parse_none_str': None, 'cli_prefix': '', 'cli_prog_name': None, 'cli_shortcuts': None, 'cli_show_env_vars': False, 'cli_use_class_docs_for_groups': False, 'enable_decoding': True, 'env_file': '.env', 'env_file_encoding': 'utf-8', 'env_ignore_empty': False, 'env_nested_delimiter': '__', 'env_nested_max_split': None, 'env_parse_enums': None, 'env_parse_none_str': None, 'env_prefix': 'mex_', 'env_prefix_target': 'variable', 'extra': 'ignore', 'json_file': None, 'json_file_encoding': None, 'nested_model_default_partial_update': False, 'populate_by_name': True, 'protected_namespaces': ('model_validate', 'model_dump', 'settings_customise_sources'), 'secrets_dir': None, 'toml_file': None, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True, 'validate_default': True, 'yaml_config_section': None, 'yaml_file': None, 'yaml_file_encoding': None}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

non_mergeable_types: list[MergedType]
valkey_url: SecretStr

mex.backend.types module

class mex.backend.types.APIKey(secret_value: SecretType)

Bases: SecretStr

An API Key used for authenticating and authorizing a client.

class mex.backend.types.AccessLevel(*values)

Bases: Enum

Enum of access level.

READ = 'read'
WRITE = 'write'
class mex.backend.types.CacheConnectorType(*values)

Bases: Enum

Choice of available cache connectors.

MEMORY = 'memory'
VALKEY = 'valkey'
class mex.backend.types.DynamicStrEnum(name: str, bases: tuple[type], dct: EnumDict)

Bases: EnumType

Metaclass to dynamically populate an enumeration from a list of strings.

class mex.backend.types.ExtractedType(*values)

Bases: Enum

Enumeration of possible types for extracted items.

EXTRACTED_ACCESS_PLATFORM = 'ExtractedAccessPlatform'
EXTRACTED_ACTIVITY = 'ExtractedActivity'
EXTRACTED_BIBLIOGRAPHIC_RESOURCE = 'ExtractedBibliographicResource'
EXTRACTED_CONTACT_POINT = 'ExtractedContactPoint'
EXTRACTED_DISTRIBUTION = 'ExtractedDistribution'
EXTRACTED_ORGANIZATION = 'ExtractedOrganization'
EXTRACTED_ORGANIZATIONAL_UNIT = 'ExtractedOrganizationalUnit'
EXTRACTED_PERSON = 'ExtractedPerson'
EXTRACTED_PRIMARY_SOURCE = 'ExtractedPrimarySource'
EXTRACTED_RESOURCE = 'ExtractedResource'
EXTRACTED_RESOURCE_SERIES = 'ExtractedResourceSeries'
EXTRACTED_VARIABLE = 'ExtractedVariable'
EXTRACTED_VARIABLE_GROUP = 'ExtractedVariableGroup'
class mex.backend.types.MergedType(*values)

Bases: Enum

Enumeration of possible types for merged items.

MERGED_ACCESS_PLATFORM = 'MergedAccessPlatform'
MERGED_ACTIVITY = 'MergedActivity'
MERGED_BIBLIOGRAPHIC_RESOURCE = 'MergedBibliographicResource'
MERGED_CONTACT_POINT = 'MergedContactPoint'
MERGED_DISTRIBUTION = 'MergedDistribution'
MERGED_ORGANIZATION = 'MergedOrganization'
MERGED_ORGANIZATIONAL_UNIT = 'MergedOrganizationalUnit'
MERGED_PERSON = 'MergedPerson'
MERGED_PRIMARY_SOURCE = 'MergedPrimarySource'
MERGED_RESOURCE = 'MergedResource'
MERGED_RESOURCE_SERIES = 'MergedResourceSeries'
MERGED_VARIABLE = 'MergedVariable'
MERGED_VARIABLE_GROUP = 'MergedVariableGroup'
class mex.backend.types.ReferenceFieldName(*values)

Bases: Enum

Enumeration of possible field names that contain references.

ACCESS_PLATFORM = 'accessPlatform'
ACCESS_SERVICE = 'accessService'
AFFILIATION = 'affiliation'
ANALYTICS = 'analytics'
BELONGS_TO = 'belongsTo'
CONTACT = 'contact'
CONTAINED_BY = 'containedBy'
CONTRIBUTING_UNIT = 'contributingUnit'
CONTRIBUTOR = 'contributor'
CREATOR = 'creator'
DISTRIBUTION = 'distribution'
EDITOR = 'editor'
EDITOR_OF_SERIES = 'editorOfSeries'
EXTERNAL_ASSOCIATE = 'externalAssociate'
EXTERNAL_PARTNER = 'externalPartner'
FUNDER_OR_COMMISSIONER = 'funderOrCommissioner'
HAD_PRIMARY_SOURCE = 'hadPrimarySource'
HAS_DATA_SUBJECT = 'hasDataSubject'
INVOLVED_PERSON = 'involvedPerson'
INVOLVED_UNIT = 'involvedUnit'
IN_SERIES = 'inSeries'
IN_SERIES_FIRST_RESOURCE = 'inSeriesFirstResource'
IN_SERIES_LAST_RESOURCE = 'inSeriesLastResource'
IS_PART_OF = 'isPartOf'
IS_PART_OF_ACTIVITY = 'isPartOfActivity'
MEMBER_OF = 'memberOf'
PARENT_UNIT = 'parentUnit'
PUBLICATION = 'publication'
PUBLISHER = 'publisher'
RELATED_ACTIVITY = 'relatedActivity'
RELATED_RESOURCE = 'relatedResource'
RESPONSIBLE_UNIT = 'responsibleUnit'
SAMPLE = 'sample'
SOURCE = 'source'
STABLE_TARGET_ID = 'stableTargetId'
SUCCEEDS = 'succeeds'
SUPERSEDED_BY = 'supersededBy'
UNIT_IN_CHARGE = 'unitInCharge'
UNIT_OF = 'unitOf'
USED_IN = 'usedIn'
WAS_GENERATED_BY = 'wasGeneratedBy'
class mex.backend.types.RuleType(*values)

Bases: Enum

Enumeration of possible types for rule items.

ADDITIVE_ACCESS_PLATFORM = 'AdditiveAccessPlatform'
ADDITIVE_ACTIVITY = 'AdditiveActivity'
ADDITIVE_BIBLIOGRAPHIC_RESOURCE = 'AdditiveBibliographicResource'
ADDITIVE_CONTACT_POINT = 'AdditiveContactPoint'
ADDITIVE_DISTRIBUTION = 'AdditiveDistribution'
ADDITIVE_ORGANIZATION = 'AdditiveOrganization'
ADDITIVE_ORGANIZATIONAL_UNIT = 'AdditiveOrganizationalUnit'
ADDITIVE_PERSON = 'AdditivePerson'
ADDITIVE_PRIMARY_SOURCE = 'AdditivePrimarySource'
ADDITIVE_RESOURCE = 'AdditiveResource'
ADDITIVE_RESOURCE_SERIES = 'AdditiveResourceSeries'
ADDITIVE_VARIABLE = 'AdditiveVariable'
ADDITIVE_VARIABLE_GROUP = 'AdditiveVariableGroup'
PREVENTIVE_ACCESS_PLATFORM = 'PreventiveAccessPlatform'
PREVENTIVE_ACTIVITY = 'PreventiveActivity'
PREVENTIVE_BIBLIOGRAPHIC_RESOURCE = 'PreventiveBibliographicResource'
PREVENTIVE_CONTACT_POINT = 'PreventiveContactPoint'
PREVENTIVE_DISTRIBUTION = 'PreventiveDistribution'
PREVENTIVE_ORGANIZATION = 'PreventiveOrganization'
PREVENTIVE_ORGANIZATIONAL_UNIT = 'PreventiveOrganizationalUnit'
PREVENTIVE_PERSON = 'PreventivePerson'
PREVENTIVE_PRIMARY_SOURCE = 'PreventivePrimarySource'
PREVENTIVE_RESOURCE = 'PreventiveResource'
PREVENTIVE_RESOURCE_SERIES = 'PreventiveResourceSeries'
PREVENTIVE_VARIABLE = 'PreventiveVariable'
PREVENTIVE_VARIABLE_GROUP = 'PreventiveVariableGroup'
SUBTRACTIVE_ACCESS_PLATFORM = 'SubtractiveAccessPlatform'
SUBTRACTIVE_ACTIVITY = 'SubtractiveActivity'
SUBTRACTIVE_BIBLIOGRAPHIC_RESOURCE = 'SubtractiveBibliographicResource'
SUBTRACTIVE_CONTACT_POINT = 'SubtractiveContactPoint'
SUBTRACTIVE_DISTRIBUTION = 'SubtractiveDistribution'
SUBTRACTIVE_ORGANIZATION = 'SubtractiveOrganization'
SUBTRACTIVE_ORGANIZATIONAL_UNIT = 'SubtractiveOrganizationalUnit'
SUBTRACTIVE_PERSON = 'SubtractivePerson'
SUBTRACTIVE_PRIMARY_SOURCE = 'SubtractivePrimarySource'
SUBTRACTIVE_RESOURCE = 'SubtractiveResource'
SUBTRACTIVE_RESOURCE_SERIES = 'SubtractiveResourceSeries'
SUBTRACTIVE_VARIABLE = 'SubtractiveVariable'
SUBTRACTIVE_VARIABLE_GROUP = 'SubtractiveVariableGroup'
WORKFLOW_ACCESS_PLATFORM = 'WorkflowAccessPlatform'
WORKFLOW_ACTIVITY = 'WorkflowActivity'
WORKFLOW_BIBLIOGRAPHIC_RESOURCE = 'WorkflowBibliographicResource'
WORKFLOW_CONTACT_POINT = 'WorkflowContactPoint'
WORKFLOW_DISTRIBUTION = 'WorkflowDistribution'
WORKFLOW_ORGANIZATION = 'WorkflowOrganization'
WORKFLOW_ORGANIZATIONAL_UNIT = 'WorkflowOrganizationalUnit'
WORKFLOW_PERSON = 'WorkflowPerson'
WORKFLOW_PRIMARY_SOURCE = 'WorkflowPrimarySource'
WORKFLOW_RESOURCE = 'WorkflowResource'
WORKFLOW_RESOURCE_SERIES = 'WorkflowResourceSeries'
WORKFLOW_VARIABLE = 'WorkflowVariable'
WORKFLOW_VARIABLE_GROUP = 'WorkflowVariableGroup'

Module contents