mex.extractors.sumo package

Subpackages

Submodules

mex.extractors.sumo.extract module

mex.extractors.sumo.extract.extract_cc1_data_model_nokeda() → Generator[Cc1DataModelNoKeda, None, None]

Extract data from cc1_data_model_NoKeda_v3.0.3.xlsx file.

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of Cc1DataModelNoKeda instances

mex.extractors.sumo.extract.extract_cc1_data_valuesets() → Generator[Cc1DataValuesets, None, None]

Extract data from cc1_data_valuesets_v3.0.3.xlsx file.

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of cc1_data_valuesets instances

mex.extractors.sumo.extract.extract_cc2_aux_mapping(sumo_cc2_aux_model: Iterable[Cc2AuxModel]) → Generator[Cc2AuxMapping, None, None]

Extract data from cc2_aux_mapping_v3.0.3.xlsx file.

Parameters:

sumo_cc2_aux_model – Cc2AuxModel variables

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of cc2_aux_mapping instances

mex.extractors.sumo.extract.extract_cc2_aux_model() → Generator[Cc2AuxModel, None, None]

Extract data from cc2_aux_model_v3.0.3.xlsx file.

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of cc2_aux_model instances

mex.extractors.sumo.extract.extract_cc2_aux_valuesets() → Generator[Cc2AuxValuesets, None, None]

Extract data from cc2_aux_valuesets_v3.0.3.xlsx file.

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of cc2_aux_valuesets instances

mex.extractors.sumo.extract.extract_cc2_feat_projection() → Generator[Cc2FeatProjection, None, None]

Extract data from cc2_feat_projection_v3.0.3.xlsx file.

Settings:

sumo.raw_data_path: path to directory holding sumo raw data.

Returns:

Generator of cc2_feat_projection instances

mex.extractors.sumo.extract.extract_ldap_contact_points_by_emails(resources: list[Any]) → Generator[LDAPActor, None, None]

Extract contact points from ldap for email in resource contacts.

Parameters:

resources – list of sumo resource mapping models

Returns:

Iterable of ldap actors

mex.extractors.sumo.extract.extract_ldap_contact_points_by_name(sumo_access_platform: Any) → Generator[LDAPPersonWithQuery, None, None]

Extract contact points from ldap for contact name in Sumo access platform.

Parameters:

sumo_access_platform – SUMO access platform mapping model

Returns:

Iterable of ldap persons with query

mex.extractors.sumo.filter module

mex.extractors.sumo.filter.filter_and_log_cc2_aux_model(extracted_cc2_aux_model: Iterable[Cc2AuxModel], extracted_primary_source: ExtractedPrimarySource) → Generator[Cc2AuxModel, None, None]

Filter out and log cc2 aux model variables which are not in_database_static.

Parameters:

  • extracted_cc2_aux_model – cc2 aux model variables

  • extracted_primary_source – primary source for report server platform

Returns:

Generator for filtered cc2 aux model variables

mex.extractors.sumo.main module

mex.extractors.sumo.settings module

class mex.extractors.sumo.settings.SumoSettings(*, raw_data_path: AssetsPath = AssetsPath('raw-data/sumo'), mapping_path: AssetsPath = AssetsPath('mappings/__final__/sumo'))

Bases: BaseModel

Settings submodel for the SUMO extractor.

mapping_path: AssetsPath
model_computed_fields: ClassVar[Dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, FieldInfo]] = {'mapping_path': FieldInfo(annotation=AssetsPath, required=False, default=AssetsPath("mappings/__final__/sumo"), description='Path to the directory with the sumo mapping files containing the default values, absolute path or relative to `assets_dir`.'), 'raw_data_path': FieldInfo(annotation=AssetsPath, required=False, default=AssetsPath("raw-data/sumo"), description='Path to the directory with the sumo excel files, absolute path or relative to `assets_dir`.')}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

raw_data_path: AssetsPath

mex.extractors.sumo.transform module

mex.extractors.sumo.transform.create_new_organization_with_official_name(name: str, extracted_primary_source: ExtractedPrimarySource) → MergedOrganizationIdentifier

Create a new extracted organization with provided name.

Parameters:

  • name – name of the organization, will be used as official name

  • extracted_primary_source – Extracted primary source

Returns:

Merged identifier of the organization

mex.extractors.sumo.transform.get_contact_merged_ids_by_emails(mex_actor_resources: Iterable[ExtractedContactPoint]) → dict[Email, MergedContactPointIdentifier]

Get merged id by emails lookup.

Parameters:

mex_actor_resources – Iterable of extracted contact points

Returns:

dict of contact merged ids by email

mex.extractors.sumo.transform.get_contact_merged_ids_by_names(mex_actors_access_platform: Iterable[ExtractedPerson]) → dict[str, MergedPersonIdentifier]

Get merged id by name lookup.

Parameters:

mex_actors_access_platform – Iterable of extracted persons

Returns:

dict of contact merged ids by name

mex.extractors.sumo.transform.transform_feat_projection_variable_to_mex_variable(extracted_cc2_feat_projection: Iterable[Cc2FeatProjection], mex_variable_groups_feat: Iterable[ExtractedVariableGroup], mex_resource_feat: ExtractedResource, extracted_primary_source: ExtractedPrimarySource) → Generator[ExtractedVariable, None, None]

Transform feat projection variable to ExtractedVariables.

Parameters:

  • extracted_cc2_feat_projection – Cc2FeatProjection variables

  • mex_variable_groups_feat – variable group feat

  • mex_resource_feat – mex resource feat

  • extracted_primary_source – Extracted report server primary source

Returns:

Generator for ExtractedVariable

mex.extractors.sumo.transform.transform_feat_variable_to_mex_variable_group(extracted_cc2_feat_projection: Iterable[Cc2FeatProjection], extracted_primary_source: ExtractedPrimarySource, mex_resource_feat: ExtractedResource) → Generator[ExtractedVariableGroup, None, None]

Transform feat projection variables to ExtractedVariableGroups.

Parameters:

  • extracted_cc2_feat_projection – Cc2FeatProjection variables

  • extracted_primary_source – Extracted primary source

  • mex_resource_feat – ExtractedResource

Returns:

Generator for ExtractedVariableGroup

mex.extractors.sumo.transform.transform_model_nokeda_variable_to_mex_variable_group(extracted_cc1_data_model_nokeda: Iterable[Cc1DataModelNoKeda], extracted_primary_source: ExtractedPrimarySource, mex_resource_nokeda: ExtractedResource) → Generator[ExtractedVariableGroup, None, None]

Transform model nokeda variables to ExtractedVariableGroups.

Parameters:

  • extracted_cc1_data_model_nokeda – Cc1DataModelNoKeda variables

  • extracted_primary_source – Extracted primary source

  • mex_resource_nokeda – ExtractedResource

Returns:

Generator for ExtractedVariableGroup

mex.extractors.sumo.transform.transform_nokeda_aux_variable_to_mex_variable(extracted_cc2_aux_model: Iterable[Cc2AuxModel], extracted_cc2_aux_mapping: Iterable[Cc2AuxMapping], extracted_cc2_aux_valuesets: Iterable[Cc2AuxValuesets], mex_variable_groups_nokeda_aux: Iterable[ExtractedVariableGroup], mex_resource_nokeda: ExtractedResource, extracted_primary_source: ExtractedPrimarySource) → Generator[ExtractedVariable, None, None]

Transform nokeda aux variable to ExtractedVariables.

Parameters:

  • extracted_cc2_aux_model – Cc2AuxModel variables

  • extracted_cc2_aux_mapping – Cc2AuxMapping variables

  • extracted_cc2_aux_valuesets – Cc2AuxValuesets variables

  • mex_variable_groups_nokeda_aux – variable group nokeda aux

  • mex_resource_nokeda – extracted resource

  • extracted_primary_source – Extracted primary source

Returns:

Generator for ExtractedVariable

mex.extractors.sumo.transform.transform_nokeda_aux_variable_to_mex_variable_group(extracted_cc2_aux_model: Iterable[Cc2AuxModel], extracted_primary_source: ExtractedPrimarySource, mex_resource_nokeda: ExtractedResource) → Generator[ExtractedVariableGroup, None, None]

Transform nokeda aux variables to ExtractedVariableGroups.

Parameters:

  • extracted_cc2_aux_model – Cc2AuxModel variables

  • extracted_primary_source – Extracted primary source

  • mex_resource_nokeda – ExtractedResource for nokeda

Returns:

Generator for ExtractedVariableGroup

mex.extractors.sumo.transform.transform_nokeda_model_variable_to_mex_variable(extracted_cc1_data_model_nokeda: Iterable[Cc1DataModelNoKeda], extracted_cc1_data_valuesets: Iterable[Cc1DataValuesets], mex_variable_groups_model_nokeda: Iterable[ExtractedVariableGroup], mex_resource_nokeda: ExtractedResource, extracted_primary_source: ExtractedPrimarySource) → Generator[ExtractedVariable, None, None]

Transform nokeda model variable to ExtractedVariables.

Parameters:

  • extracted_cc1_data_model_nokeda – Cc1DataModelNoKeda variables

  • extracted_cc1_data_valuesets – Cc1DataValuesets variables

  • mex_variable_groups_model_nokeda – variable group nokeda

  • mex_resource_nokeda – mex resource nokeda

  • extracted_primary_source – Extracted primary source

Returns:

Generator for ExtractedVariable

mex.extractors.sumo.transform.transform_resource_feat_model_to_mex_resource(sumo_resource_feat: Any, extracted_primary_source: ExtractedPrimarySource, unit_merged_ids_by_synonym: dict[str, MergedOrganizationalUnitIdentifier], contact_merged_ids_by_emails: dict[Email, MergedContactPointIdentifier], mex_resource_nokeda: ExtractedResource, transformed_activity: ExtractedActivity, sumo_access_platform: ExtractedAccessPlatform) → ExtractedResource

Transform sumo resource_feat to ExtractedResource.

Parameters:

  • sumo_resource_feat – sumo_resource_feat mapping model

  • extracted_primary_source – Extracted primary source

  • unit_merged_ids_by_synonym – Mapping from synonyms to merged IDs of units

  • contact_merged_ids_by_emails – Mapping from emails to merged IDs of contact points

  • mex_resource_nokeda – ExtractedResource for nokeda

  • transformed_activity – ExtractedActivity for sumo

  • sumo_access_platform – transformed sumo ExtractedAccessPlatform

Returns:

ExtractedResource

mex.extractors.sumo.transform.transform_resource_nokeda_to_mex_resource(sumo_resource_nokeda: Any, extracted_primary_source: ExtractedPrimarySource, unit_merged_ids_by_synonym: dict[str, MergedOrganizationalUnitIdentifier], contact_merged_ids_by_emails: dict[Email, MergedContactPointIdentifier], extracted_organization_rki: ExtractedOrganization, transformed_activity: ExtractedActivity, sumo_access_platform: ExtractedAccessPlatform) → ExtractedResource

Transform ResourceNokeda to ExtractedResource.

Parameters:

  • sumo_resource_nokeda – nokeda resource mapping model with defaults

  • extracted_primary_source – Extracted primary source

  • unit_merged_ids_by_synonym – Mapping from synonyms to merged IDs of units

  • contact_merged_ids_by_emails – Mapping from emails to merged IDs of contact points

  • extracted_organization_rki – ExtractedOrganization

  • transformed_activity – ExtractedActivity for sumo

  • sumo_access_platform – transformed sumo ExtractedAccessPlatform

Returns:

ExtractedResource

mex.extractors.sumo.transform.transform_sumo_access_platform_to_mex_access_platform(sumo_access_platform: Any, unit_merged_ids_by_synonym: dict[str, MergedOrganizationalUnitIdentifier], person_stable_target_ids_by_query_string: dict[str, MergedPersonIdentifier], extracted_primary_source: ExtractedPrimarySource) → ExtractedAccessPlatform

Transform sumo access platform info to ExtractedAccessPlatform.

Parameters:

  • sumo_access_platform – sumo_access_platform mapping model with default values

  • unit_merged_ids_by_synonym – Mapping from synonyms to merged IDs of units

  • person_stable_target_ids_by_query_string – Mapping from contact person query to person stable target ID

  • extracted_primary_source – Extracted primary source for sumo

Returns:

ExtractedAccessPlatform

mex.extractors.sumo.transform.transform_sumo_activity_to_extracted_activity(sumo_activity: Any, unit_merged_ids_by_synonym: dict[str, MergedOrganizationalUnitIdentifier], contact_merged_ids_by_emails: dict[Email, MergedContactPointIdentifier], extracted_primary_source: ExtractedPrimarySource) → ExtractedActivity

Transform sumo activity to ExtractedActivity.

Parameters:

  • sumo_activity – sumo_activity mapping model with default values

  • unit_merged_ids_by_synonym – Mapping from synonyms to merged IDs of units

  • contact_merged_ids_by_emails – Mapping from contact person query to person stable target ID

  • extracted_primary_source – Extracted primary source

Returns:

ExtractedActivity

Module contents