mex.odk package

Submodules

mex.odk.extract module

mex.odk.extract.extract_odk_raw_data() → list[ODKData]

Extract odk raw data by loading data from MS-Excel file.

Settings:

odk.raw_data_path: Path to the odk raw data,

absolute or relative to assets_dir

Returns:

list of ODK data.

mex.odk.extract.get_column_dict_by_pattern(sheet: DataFrame, pattern: str) → dict[str, list[str | float]]

Get a dict of columns by matchting pattern.

Parameters:

• sheet – sheet to extract columns from

• pattern – pattern to match column names

Returns:

dictionary of mathing columns by column names

mex.odk.extract.get_external_partner_and_publisher_by_label(odk_resource_mappings: list[dict[str, Any]]) → dict[str, WikidataOrganization]

Search and extract partner organization from wikidata.

Parameters:

odk_resource_mappings – list of resource mappings

Returns:

Dict with organization label and WikidataOrganization

mex.odk.main module

mex.odk.model module

class mex.odk.model.ODKData(*, file_name: str, hint: dict[str, list[str | float]], label_choices: dict[str, list[str | float]], label_survey: dict[str, list[str | float]], list_name: list[str | float], name: list[str | float], type: list[str | float])

Bases: BaseModel

Model class for odk data.

file_name: str

hint: dict[str, list[str | float]]

label_choices: dict[str, list[str | float]]

label_survey: dict[str, list[str | float]]

list_name: list[str | float]

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'file_name': FieldInfo(annotation=str, required=True), 'hint': FieldInfo(annotation=dict[str, list[Union[str, float]]], required=True), 'label_choices': FieldInfo(annotation=dict[str, list[Union[str, float]]], required=True), 'label_survey': FieldInfo(annotation=dict[str, list[Union[str, float]]], required=True), 'list_name': FieldInfo(annotation=list[Union[str, float]], required=True), 'name': FieldInfo(annotation=list[Union[str, float]], required=True), 'type': FieldInfo(annotation=list[Union[str, float]], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: list[str | float]

type: list[str | float]

mex.odk.settings module

class mex.odk.settings.ODKSettings(*, raw_data_path: AssetsPath = AssetsPath('raw-data/odk'), mapping_path: AssetsPath = AssetsPath('mappings/__final__/odk'))

Bases: BaseModel

Settings submodel definition for odk data extraction.

mapping_path: AssetsPath

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'str_max_length': 100000, 'str_min_length': 1, 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'mapping_path': FieldInfo(annotation=AssetsPath, required=False, default=AssetsPath("mappings/__final__/odk"), description='Path to the directory with the odk mapping files containing the default values, absolute path or relative to `assets_dir`.'), 'raw_data_path': FieldInfo(annotation=AssetsPath, required=False, default=AssetsPath("raw-data/odk"), description='Path to the directory with the odk excel files, absolute path or relative to `assets_dir`.')}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

raw_data_path: AssetsPath

mex.odk.transform module

mex.odk.transform.get_value_set(type_cell: str, choice_sheet: ODKData) → list[str]

Get value sets for types cells that start with select_one or multiple_one.

Parameters:

• type_cell – one type cell

• choice_sheet – choice sheet corresponding to type cell

Returns:

list of value sets matched to type cell

mex.odk.transform.get_variable_groups_from_raw_data(odk_raw_data: list[ODKData]) → dict[str, list[dict[str, str]]]

Get variable groups from raw data by parsing for begin_group and end_group.

Parameters:

odk_raw_data – raw data extracted from Excel files

Returns:

dictionary of odk groups by group name

mex.odk.transform.transform_odk_data_to_extracted_variables(extracted_resources_odk: list[ExtractedResource], extracted_variable_groups_odk: list[ExtractedVariableGroup], odk_variable_groups: dict[str, list[dict[str, str]]], odk_raw_data: list[ODKData], extracted_primary_source_odk: ExtractedPrimarySource) → list[ExtractedVariable]

Transform odk variables to mex variables.

Parameters:

• extracted_resources_odk – extracted mex resources

• extracted_variable_groups_odk – extracted mex variable groups

• odk_variable_groups – dictionary of odk groups by group name

• odk_raw_data – raw data extracted from Excel files

• extracted_primary_source_odk – odk primary source

Returns:

list of mex variables

mex.odk.transform.transform_odk_resources_to_mex_resources(odk_resource_mappings: list[dict[str, Any]], unit_stable_target_ids_by_synonym: dict[str, MergedOrganizationalUnitIdentifier], external_partner_and_publisher_by_label: dict[str, MergedOrganizationIdentifier], extracted_international_projects_activities: list[ExtractedActivity], extracted_primary_source_mex: ExtractedPrimarySource) → list[ExtractedResource]

Transform odk resources to mex resources.

Parameters:

• odk_resource_mappings – list of resource mappings

• unit_stable_target_ids_by_synonym – dict of OrganizationalUnitIds

• extracted_primary_source_international_projects – primary source

• external_partner_and_publisher_by_label – dict of wikidata OrganizationIDs

• extracted_international_projects_activities – list of extracted international projects activities

• extracted_primary_source_mex – mex primary source

Returns:

list of mex resources

mex.odk.transform.transform_odk_variable_groups_to_extracted_variable_groups(odk_variable_groups: dict[str, list[dict[str, str]]], extracted_resources_odk: list[ExtractedResource], extracted_primary_source_odk: ExtractedPrimarySource) → list[ExtractedVariableGroup]

Transform odk variable groups to mex variable groups.

Parameters:

• odk_variable_groups – dictionary of odk groups by group name

• extracted_resources_odk – extracted mex resources

• extracted_primary_source_odk – odk primary source

Returns:

list of mex variable groups

Module contents