🎴Dflow Instruction Guide

Installing and Setting Up Dflow

1. Ensure you have Python version 3.6 or higher installed.

2. Install Dflow using pip:

pip install pydflow

1. Set up Argo Server if you don't have it already. Follow the official Argo installation guide.

Key Concepts in Dflow

Dflow uses the following key concepts:

• Parameters and Artifacts: data passed within the workflow.

• OP template: defines an operation to be performed.

• Step: an instance of an OP template with specific input data.

• Workflow: connects steps into a unified workflow.

Creating a Simple Workflow

1. Import the necessary modules:

from dflow import Workflow, Step
from dflow.python import OP, OPIO, OPIOSign, PythonOPTemplate, Artifact
from pathlib import Path

1. Define an OP template:

class SimpleExample(OP):
    @classmethod
    def get_input_sign(cls):
        return OPIOSign({
            "msg": str,
            "foo": Artifact(Path)
        })
    
    @classmethod
    def get_output_sign(cls):
        return OPIOSign({
            "msg": str,
            "bar": Artifact(Path)
        })
    
    @OP.exec_sign_check
    def execute(self, op_in: OPIO) -> OPIO:
        # Your code here
        return OPIO({"msg": op_in["msg"], "bar": Path("output.txt")})

1. Create a PythonOPTemplate:

op_template = PythonOPTemplate(SimpleExample, image="python:3.8")

1. Create a Step:

step = Step(
    name="example_step",
    template=op_template,
    parameters={"msg": "Hello, Dflow!"},
    artifacts={"foo": upload_artifact("input.txt")}
)

1. Create and submit the Workflow:

workflow = Workflow(name="example_workflow")
workflow.add(step)
workflow.submit()

Managing the Workflow

• Get the workflow ID: workflow.id

• Check status: workflow.query_status()

• Query step information: workflow.query_step(name="example_step")

• Terminate the workflow: workflow.terminate()

Additional Features

• Conditional steps: Step(..., when="condition")

• Parallel steps: Use with_param or with_sequence

• Nested workflows: Use Steps or DAG

Debugging

Dflow provides a debug mode for running workflows locally, without the need for Argo/Kubernetes.