Pipeline module

This module allows you to configure your model pipeline.

MLOpsPipeline

class mlops_codex.pipeline.MLOpsPipeline(*, group: str, login: str | None = None, password: str | None = None, url: str | None = None, python_version: float = 3.9)

Bases: BaseMLOps

Class to construct and orchestrates the flow data of the models inside MLOps.

Parameters:

• login (str) – Login for authenticating with the client. You can also use the env variable MLOPS_USER to set this

• password (str) – Password for authenticating with the client. You can also use the env variable MLOPS_PASSWORD to set this

• group (str) – Group the model is inserted

• url (str) – URL to MLOps Server. Default value is https://neomaril.datarisk.net/, use it to test your deployment first before changing to production. You can also use the env variable MLOPS_URL to set this

• python_version (str) – Python version for the model environment. Available versions are 3.8, 3.9, 3.10. Defaults to ‘3.9’

Example

from mlops_codex.pipeline import MLOpsPipeline

pipeline = MLOpsPipeline.from_config_file('./samples/pipeline.yml')
pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")
pipeline.start()
pipeline.run_monitoring('2', 'Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

static from_config_file(path)

Load the configuration files for orchestrate the model

Parameters:

path (str) – Path of the configuration file, but it could be a dict

Raises:

PipelineError – Undefined credentials

Returns:

The new pipeline

Return type:

MLOpsPipeline

Example

>>> pipeline = MLOpsPipeline.from_config_file('./samples/pipeline-just-model.yml')
>>> pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")
>>> pipeline.start()

property model

register_deploy_config(**kwargs) → dict

Set the files for configure the deployment

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for deploy

register_monitoring_config(**kwargs) → dict

Set the files for configure the monitoring

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for monitoring

Example

>>> pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")

register_train_config(**kwargs) → dict

Set the files for configure the training

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for training

run_deploy(training_id: str | None = None) → str

Run the deployment process

Parameters:

training_id (Optional[str], optional) – The id for the training process that you want to deploy now

Raises:

ModelError – Deploy has failed

Returns:

The new Model id (hash)

Return type:

str

Example

>>> pipeline.run_deploy('Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

run_monitoring(*, training_exec_id: str | None = None, model_id: str | None = None)

Run the monitoring process

Parameters:

• training_exec_id (Optional[str], optional) – The id for the training execution process that you want to monitore now

• model_id (Optional[str], optional) – Model hash

Example

>>> pipeline.run_monitoring('2', 'Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

run_training() → Tuple[str, str]

Run the training process

Raises:

TrainingError – Training has failed

Returns:

A tuple with the ‘training_id’ and the ‘exec_id’

Return type:

tuple[str, str]

Example

>>> pipeline.run_training()

start()

Start the pipeline for the model orchestration

Raises:

PipelineError – Cannot start pipeline without configuration

Example

>>> pipeline = MLOpsPipeline.from_config_file('./samples/pipeline.yml').start()

property training

property training_run