Pipeline module

This module alows you to configure your model pipeline.

neomaril_codex.pipeline.NeomarilPipeline

class neomaril_codex.pipeline.NeomarilPipeline(*, group: str, login: str | None = None, password: str | None = None, url: str = 'https://neomaril.staging.datarisk.net/', python_version: float = 3.9)

Bases: BaseNeomaril

Class to construct and orchestrates the flow data of the models inside Neomaril.

Atributtes

loginstr

Login for authenticating with the client. You can also use the env variable NEOMARIL_USER to set this

passwordstr

Password for authenticating with the client. You can also use the env variable NEOMARIL_PASSWORD to set this

groupstr

Group the model is inserted

urlstr

URL to Neomaril Server. Default value is https://neomaril.staging.datarisk.net, use it to test your deployment first before changing to production. You can also use the env variable NEOMARIL_URL to set this

python_versionstr

Python version for the model environment. Avaliable versions are 3.8, 3.9, 3.10. Defaults to ‘3.9’

Example

from neomaril_codex.pipeline import NeomarilPipeline

pipeline = NeomarilPipeline.from_config_file('./samples/pipeline.yml')
pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")
pipeline.start()
pipeline.run_monitoring('2', 'Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

static from_config_file(path)

Load the configuration files for orchestrate the model

Parameters:

path (str) – Path of the configuration file, but it could be a dict

Raises:

PipelineError – Undefined credentials

Returns:

The new pipeline

Return type:

NeomarilPipeline

Example

>>> pipeline = NeomarilPipeline.from_config_file('./samples/pipeline-just-model.yml')
>>> pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")
>>> pipeline.start()

register_deploy_config(**kwargs) → dict

Set the files for configure the deploy

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for deploy

register_monitoring_config(**kwargs) → dict

Set the files for configure the monitoring

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for monitoring

Example

>>> pipeline.register_monitoring_config(directory = "./samples/monitoring", preprocess = "preprocess.py", preprocess_function = "score", shap_function = "score", config = "configuration.json", packages = "requirements.txt")

register_train_config(**kwargs) → dict

Set the files for configure the training

Parameters:

kwargs (list or dict) – List or dictionary with the necessary files for training

run_deploy(training_id: str | None = None) → str

Run the deploy process

Parameters:

training_id (str, optional) – The id for the training process that you want to deploy now

Raises:

ModelError – Deploy has failed

Returns:

The new Model id (hash)

Return type:

str

Example

>>> pipeline.run_deploy('Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

run_monitoring(*, training_exec_id: str | None = None, model_id: str | None = None)

Run the monitoring process

Parameters:

• training_exec_id (str, optional) – The id for the training execution process that you want to monitore now

• model_id (str, optional) –

Example

>>> pipeline.run_monitoring('2', 'Mb29d61da4324a39a8bc2e0946f213b4959643916d354bf39940de2124f1e9d8')

run_training() → tuple[str, str]

Run the training process

Raises:

TrainingError – Training has failed

Returns:

A tuple with the ‘training_id’ and the ‘exec_id’

Return type:

tuple[str, str]

Example

>>> pipeline.run_training()

start()

Start the pipeline for the model orchestration

Raises:

PipelineError – Cannot start pipeline without configuration

Example

>>> pipeline = NeomarilPipeline.from_config_file('./samples/pipeline.yml').start()