poiesis.core.services.torc package¶

Submodules¶

poiesis.core.services.torc.torc module¶

Task orchestrator (Torc).

class poiesis.core.services.torc.torc.Torc(task)[source]¶

Bases: object

Torc service.

Parameters:: task (TesTask) – Task object from task request

task¶: Task object from task request

kubernetes_client¶: Kubernetes client

pvc_name¶: Name of the PVC created

async create_pvc(name, size)[source]¶

Create a PVC for the task.

Tif and Tof will use this PVC to read and write data, and executor will the data from the PVC for its use.

Parameters:

name (str) – Name of the PVC
size (float | None) – Size of the PVC

Raises:

Exception – If the PVC creation fails.

Return type:

None

async execute()[source]¶

Defines the template method, for each service namely Texam, Tif, Tof.

Return type:: None

async texam_execution(task)[source]¶

Execute the Texam job.

Parameters:: task (TesTask) – The TES task that needs to be executed.
Raises:: Exception – If the Texam job fails.
Return type:: None

async tif_execution(name, inputs)[source]¶

Execute the Tif job.

Parameters:

name (str) – Name of the task, will be modified to create Tif job name.
inputs (list[TesInput] | None) – List of inputs given in the task.
volumes – List of volumes given in the task.

Raises:

Exception – If the Tif job fails.

Return type:

None

async tof_execution(name, outputs)[source]¶

Execute the Tof job.

Parameters:

name (str) – Name of the task, will be modified to create Tof job name.
outputs (list[TesOutput] | None) – List of outputs given in the task.

Raises:

Exception – If the Tof job fails.

Return type:

None

poiesis.core.services.torc.torc_execution_template module¶

Torc’s template for each service.

class poiesis.core.services.torc.torc_execution_template.TorcExecutionTemplate[source]¶

Bases: ABC

TorcTemplate is a template class for the Torc service.

kubernetes_client¶: Kubernetes client.

message_broker¶: Message broker.

message¶: Message for the message broker, which would be sent to TOrc.

db¶: Database client.

id¶: Task identifier

async create_job(task_id, job_name, commands, metadata)[source]¶

Create the K8s filer job.

TIF and TOF jobs are created using this method.

Parameters:

task_id (str) – The id of the task.
job_name (str) – The name of the job, either TIF or TOF.
commands (list[str]) – The filer commands to run.
args – The arguments to pass to the filer commands.
metadata (V1ObjectMeta) – The metadata for the job to be used in K8s manifest.

Return type:

None

async execute()[source]¶

Defines the template method, for each service namely Texam, Tif, Tof.

Return type:: None

abstract property id: str¶: Task identifier - must be implemented by subclasses.

async log()[source]¶

Log the job status in TaskDB.

Return type:: None

abstractmethod async start_job()[source]¶

Create the K8s job.

It could be a Tif, Tof or Texam job.

Return type:: None

wait()[source]¶

Wait for the job to finish.

Uses message broker with task name as channel name and waits on that channel for the message.

Return type:: None

poiesis.core.services.torc.torc_texam_execution module¶

Create Texam Job and monitor it.

class poiesis.core.services.torc.torc_texam_execution.TorcTexamExecution(task)[source]¶

Bases: TorcExecutionTemplate

Tif execution class.

This class is responsible for creating the Texam Job and monitoring it.

Parameters:: task (TesTask) – The TES task that needs to be executed.

id¶: The id of the TES task.

task¶: The TES task that needs to be executed.

kubernetes_client¶: Kubernetes

message_broker¶: Message broker.

message¶: Message for the message broker.

property id: str¶: Return the task ID.

async start_job()[source]¶

Create the K8s job for Texam.

Return type:: None

poiesis.core.services.torc.torc_tif_execution module¶

Create Tif Job and monitor it.

class poiesis.core.services.torc.torc_tif_execution.TorcTifExecution(id, inputs)[source]¶

Bases: TorcExecutionTemplate

Tif execution class.

This class is responsible for creating the Tif Job.

Parameters:

name – The name of the TES task will be modified for Tif Job.
inputs (list[TesInput] | None) – The list of inputs that Tif will create and monitor.

name¶: The name of the TES task will be modified for Tif Job.

inputs¶: The list of inputs that Tif will create and monitor.

message_broker¶: Message broker client.

message¶: Message for the message broker which would to sent to TOrc.

kubernetes_client¶: Kubernetes client.

property id: str¶: Return the task ID.

async start_job()[source]¶

Create the K8s job for Tif.

Return type:: None

poiesis.core.services.torc.torc_tof_execution module¶

Create TOF Job and monitor it.

class poiesis.core.services.torc.torc_tof_execution.TorcTofExecution(id, outputs)[source]¶

Bases: TorcExecutionTemplate

TOF execution class.

This class is responsible for creating the Tof Job.

Parameters:

name – The name of the TES task will be modified for TOF Job.
outputs (list[TesOutput] | None) – The list of outputs that Tof will create and monitor.

name¶: The name of the TES task will be modified for TOF Job.

outputs¶: The list of outputs that TOF will create and monitor.

message_broker¶: Message broker client.

message¶: Message for the message broker which would to sent to TOrc.

kubernetes_client¶: Kubernetes client.

property id: str¶: Return the task ID.

async start_job()[source]¶

Create the K8s job for Tof.

Return type:: None

Module contents¶

Task orchestrator (Torc).