mex.common.models.base package¶
Submodules¶
mex.common.models.base.container module¶
- class mex.common.models.base.container.ItemsContainer(*, items: list[_ContainerItemT])¶
Bases:
BaseModel
,Generic
[_ContainerItemT
]Generic container that contains items.
- items: list[_ContainerItemT]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'items': FieldInfo(annotation=list[~_ContainerItemT], required=True)}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.container.PaginatedItemsContainer(*, items: list[_ContainerItemT], total: int)¶
Bases:
BaseModel
,Generic
[_ContainerItemT
]Generic container that contains items and has a total item count.
- items: list[_ContainerItemT]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'items': FieldInfo(annotation=list[~_ContainerItemT], required=True), 'total': FieldInfo(annotation=int, required=True)}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- total: int¶
mex.common.models.base.extracted_data module¶
- class mex.common.models.base.extracted_data.ExtractedData(*, hadPrimarySource: MergedPrimarySourceIdentifier, identifierInPrimarySource: Annotated[str, MinLen(min_length=1), MaxLen(max_length=1000), _PydanticGeneralMetadata(pattern='^[^\\n\\r]+$')])¶
Bases:
BaseModel
Base model for all extracted item classes.
This class adds two important attributes for metadata provenance: hadPrimarySource and identifierInPrimarySource, which are used to uniquely identify an item in its original primary source. The attribute stableTargetId has to be set by each concrete subclass, like ExtractedPerson, because it needs to have the correct type, e.g. MergedPersonIdentifier.
This class also adds a validator to automatically set identifiers for provenance.
- _get_identifier(identifier_type: type[_ExtractedIdentifierT]) _ExtractedIdentifierT ¶
Consult the identity provider to get the identifier for this item.
- Parameters:
identifier_type – ExtractedIdentifier-subclass to cast the identifier to
- Returns:
Identifier of the correct type
- _get_stable_target_id(identifier_type: type[_MergedIdentifierT]) _MergedIdentifierT ¶
Consult the identity provider to get the stableTargetId for this item.
- Parameters:
identifier_type – MergedIdentifier-subclass to cast the identifier to
- Returns:
StableTargetId of the correct type
- hadPrimarySource: Annotated[MergedPrimarySourceIdentifier, FieldInfo(annotation=NoneType, required=True, description='The stableTargetId of the primary source, that this item was extracted from. This field is mandatory for all extracted items to aid with data provenance. Extracted primary sources also have this field and are all extracted from a static primary source for MEx. The extracted primary source for MEx has its own merged item as a primary source.', frozen=True)]¶
- identifierInPrimarySource: Annotated[str, FieldInfo(annotation=NoneType, required=True, description='This is the identifier the original item had in its source system. It is only unique amongst items coming from the same system, because identifier formats are likely to overlap between systems. The value for `identifierInPrimarySource` is therefore only unique in composition with `hadPrimarySource`. MEx uses this composite key to assign a stable and globally unique `identifier` per extracted item.', examples=['123456', 'item-501', 'D7/x4/zz.final3'], frozen=True, metadata=[MinLen(min_length=1), MaxLen(max_length=1000), _PydanticGeneralMetadata(pattern='^[^\\n\\r]+$')])]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'hadPrimarySource': FieldInfo(annotation=MergedPrimarySourceIdentifier, required=True, description='The stableTargetId of the primary source, that this item was extracted from. This field is mandatory for all extracted items to aid with data provenance. Extracted primary sources also have this field and are all extracted from a static primary source for MEx. The extracted primary source for MEx has its own merged item as a primary source.', frozen=True), 'identifierInPrimarySource': FieldInfo(annotation=str, required=True, description='This is the identifier the original item had in its source system. It is only unique amongst items coming from the same system, because identifier formats are likely to overlap between systems. The value for `identifierInPrimarySource` is therefore only unique in composition with `hadPrimarySource`. MEx uses this composite key to assign a stable and globally unique `identifier` per extracted item.', examples=['123456', 'item-501', 'D7/x4/zz.final3'], frozen=True, metadata=[MinLen(min_length=1), MaxLen(max_length=1000), _PydanticGeneralMetadata(pattern='^[^\\n\\r]+$')])}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
mex.common.models.base.filter module¶
- class mex.common.models.base.filter.BaseFilter¶
Bases:
BaseModel
Base class for filter implementations.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.filter.FilterField(*, fieldInPrimarySource: str | None = None, locationInPrimarySource: str | None = None, examplesInPrimarySource: list[str] | None = None, filterRules: Annotated[list[FilterRule], MinLen(min_length=1)], comment: str | None = None)¶
Bases:
BaseModel
Filter definition for one field in the primary source.
- comment: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='comment')]¶
- examplesInPrimarySource: Annotated[list[str] | None, FieldInfo(annotation=NoneType, required=True, title='examplesInPrimarySource')]¶
- fieldInPrimarySource: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='fieldInPrimarySource')]¶
- filterRules: Annotated[list[FilterRule], FieldInfo(annotation=NoneType, required=True, title='filterRules', metadata=[MinLen(min_length=1)])]¶
- locationInPrimarySource: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='locationInPrimarySource')]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'comment': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='comment'), 'examplesInPrimarySource': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None, title='examplesInPrimarySource'), 'fieldInPrimarySource': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='fieldInPrimarySource'), 'filterRules': FieldInfo(annotation=list[FilterRule], required=True, title='filterRules', metadata=[MinLen(min_length=1)]), 'locationInPrimarySource': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='locationInPrimarySource')}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.filter.FilterRule(*, forValues: list[str] | None = None, rule: str | None = None)¶
Bases:
BaseModel
A single filter rule to apply.
- forValues: Annotated[list[str] | None, FieldInfo(annotation=NoneType, required=True, title='forValues')]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'forValues': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None, title='forValues'), 'rule': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='rule')}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- rule: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='rule')]¶
mex.common.models.base.mapping module¶
- class mex.common.models.base.mapping.BaseMapping¶
Bases:
BaseModel
Base class for mapping implementations.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.mapping.MappingField(*, fieldInPrimarySource: str | None = None, locationInPrimarySource: str | None = None, examplesInPrimarySource: list[str] | None = None, mappingRules: Annotated[list[MappingRule[_MappingRuleT]], MinLen(min_length=1)], comment: str | None = None)¶
Bases:
BaseModel
,Generic
[_MappingRuleT
]Generic mapping field model.
- comment: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='comment')]¶
- examplesInPrimarySource: Annotated[list[str] | None, FieldInfo(annotation=NoneType, required=True, title='examplesInPrimarySource')]¶
- fieldInPrimarySource: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='fieldInPrimarySource')]¶
- locationInPrimarySource: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='locationInPrimarySource')]¶
- mappingRules: Annotated[list[MappingRule[_MappingRuleT]], FieldInfo(annotation=NoneType, required=True, title='mappingRules', metadata=[MinLen(min_length=1)])]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'comment': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='comment'), 'examplesInPrimarySource': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None, title='examplesInPrimarySource'), 'fieldInPrimarySource': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='fieldInPrimarySource'), 'locationInPrimarySource': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='locationInPrimarySource'), 'mappingRules': FieldInfo(annotation=list[MappingRule[~_MappingRuleT]], required=True, title='mappingRules', metadata=[MinLen(min_length=1)])}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.mapping.MappingRule(*, forValues: list[str] | None = None, setValues: _ValueT | None = None, rule: str | None = None)¶
Bases:
BaseModel
,Generic
[_ValueT
]Generic mapping rule model.
- forValues: Annotated[list[str] | None, FieldInfo(annotation=NoneType, required=True, title='forValues')]¶
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {'forValues': FieldInfo(annotation=Union[list[str], NoneType], required=False, default=None, title='forValues'), 'rule': FieldInfo(annotation=Union[str, NoneType], required=False, default=None, title='rule'), 'setValues': FieldInfo(annotation=Union[~_ValueT, NoneType], required=False, default=None, title='setValues')}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- rule: Annotated[str | None, FieldInfo(annotation=NoneType, required=True, title='rule')]¶
- setValues: Annotated[_ValueT | None, FieldInfo(annotation=NoneType, required=True, title='setValues')]¶
mex.common.models.base.merged_item module¶
- class mex.common.models.base.merged_item.MergedItem¶
Bases:
BaseModel
Base model for all merged item classes.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
mex.common.models.base.model module¶
- class mex.common.models.base.model.BaseModel¶
Bases:
BaseModel
Common base class for all MEx model classes.
- classmethod _convert_list_to_non_list(field_name: str, value: list[Any]) Any ¶
Convert a list value to a non-list value by unpacking it if possible.
- classmethod _convert_non_list_to_list(field_name: str, value: Any) list[Any] | None ¶
Convert a non-list value to a list value by wrapping it in a list.
- classmethod _fix_value_listyness_for_field(field_name: str, value: Any) Any ¶
Check actual and desired shape of a value and fix it if necessary.
- checksum() str ¶
Calculate md5 checksum for this model.
Creates a deterministic hash of the model by serializing it to JSON with sorted keys and computing an MD5 digest.
- Returns:
MD5 hexdigest string representing the model’s current state.
- classmethod fix_listyness(data: Any, handler: ValidatorFunctionWrapHandler) Any ¶
Adjust the listyness of to-be-parsed data to match the desired shape.
If that data is a Mapping and the model defines a list[T] field but the raw data contains just a value of type T, it will be wrapped into a list. If the raw data contains a literal None, but the list field is defined as required, we substitute an empty list.
If the model does not expect a list, but the raw data contains a list with no entries, it will be substituted with None. If the raw data contains exactly one entry, then it will be unpacked from the list. If it contains more than one entry however, an error is raised, because we would not know which to choose.
- Parameters:
data – Raw data or instance to be parsed
handler – Validator function wrap handler
- Returns:
data with fixed list shapes
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- classmethod model_json_schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}', schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'mex.common.models.base.schema.JsonSchemaGenerator'>, mode: ~typing.Literal['validation', 'serialization'] = 'validation') dict[str, Any] ¶
Generates a JSON schema for a model class.
- Parameters:
by_alias – Whether to use attribute aliases or not.
ref_template – The reference template.
schema_generator – Overriding the logic used to generate the JSON schema
mode – The mode in which to generate the schema.
- Returns:
The JSON schema for the given model class.
- classmethod verify_computed_field_consistency(data: Any, handler: ValidatorFunctionWrapHandler) Any ¶
Validate that parsed values for computed fields are consistent.
Parsing a dictionary with a value for a computed field that is consistent with what that field would have computed anyway is allowed. Omitting values for computed fields is perfectly valid as well. However, if the parsed value is different from the computed value, a validation error is raised.
- Parameters:
data – Raw data or instance to be parsed
handler – Validator function wrap handler
- Returns:
data with consistent computed fields.
mex.common.models.base.preview_item module¶
- class mex.common.models.base.preview_item.PreviewItem¶
Bases:
BaseModel
Base model for previews of merged items.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
mex.common.models.base.rules module¶
- class mex.common.models.base.rules.AdditiveRule¶
Bases:
BaseModel
Base rule to add values to merged items.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.rules.PreventiveRule¶
Bases:
BaseModel
Base rule to prevent primary sources for fields of merged items.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.rules.RuleSet¶
Bases:
BaseModel
Base class for a set of an additive, subtractive and preventive rule.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
- class mex.common.models.base.rules.SubtractiveRule¶
Bases:
BaseModel
Base rule to subtract values from merged items.
- model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}¶
A dictionary of computed field names and their corresponding ComputedFieldInfo objects.
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}¶
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[Dict[str, FieldInfo]] = {}¶
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.
This replaces Model.__fields__ from Pydantic V1.
mex.common.models.base.schema module¶
- class mex.common.models.base.schema.JsonSchemaGenerator(by_alias: bool = True, ref_template: str = '#/$defs/{model}')¶
Bases:
GenerateJsonSchema
Customization of the pydantic class for generating JSON schemas.
- handle_ref_overrides(json_schema: Dict[str, Any]) Dict[str, Any] ¶
Disable pydantic behavior to wrap top-level $ref keys in an allOf.
- For example, pydantic would convert
{“$ref”: “#/$defs/APIType”, “examples”: [“api-type-1”]}
- into
{“allOf”: {“$ref”: “#/$defs/APIType”}, “examples”: [“api-type-1”]}
which is in fact recommended by JSON schema, but we need to disable this to stay compatible with mex-editor and mex-model.