Automation nodes

Overview

The Automation elements of the Dataloop pipeline consist of FaaS (Function as a Service) and Code nodes, which enable efficient, scalable automation within machine learning workflows.

FaaS node

The FaaS Node allows you to invoke functions defined in the Marketplace -> Applications. It's ideal for handling larger code scopes or reusable code across multiple pipelines, centralizing maintenance, version control, and providing control over computing settings and resource allocation. Learn more the FaaS.

When you select a FaaS node, its details, such as Configuration, Executions, Logs, Instances, and available actions are shown on the right-side panel.

Config Tab

• Node Name: By default, FaaS Node is displayed as name. You can change it accordingly.
• Function: Select a function from the list. The FaaS Node’s Output and Input channels are determined by the function chosen for the node.
• Node Input: It is generated automatically according to the selected FaaS.
• Node Output: It is generated automatically according to the selected FaaS.
• Trigger (Optional): An Event/Cron Triggers can be set on this node, enabling you to initiate the pipeline run from this specific point.

For information on the Executions, Logs, and Instances tabs, see the Node Details article.

FaaS node actions

The FaaS node allows you to perform the following actions:

• Open Service Page
• View Service Settings
• Download Codebase File
• Copy Service ID
• Copy Node ID
• View Analytics
• View Executions
• View Logs

Code node

The Code Node enables direct execution of simple functions within pipelines, such as uploading annotations as JSON to a dataset. When the pipeline runs, a dedicated service is launched, requiring monitoring, maintenance, and scaling as needed.

To view an example of using the Code node, see the Automatic Modalities Pipeline.

When you click on a Code Node, its details, such as Configuration, Executions, Logs, Instances, and available Actions are shown on the right-side panel. The Code Node details are presented in four tabs as follows:

Config Tab

• Node Name: By default, code is displayed as name. You can change accordingly.
• Application Name: Provide a name for the FaaS application that will be created to execute this code node. This name will be used as both the package and service names.
• Function Name: Provide a name for the function that will run your code. Make sure to use the same name as defined in the code, following the Python convention for function names. By default, run is displayed as name.
• Source Code Type: Select a source code type from the following list.
  • Code Editor (by default): The Code Editor allows you to enter the code for the code node.
  • Upload ZIP: It allows you to upload your code in a .zip file.
  • Connect GitHub: It allows you to enter the URL of your code's Git repository and Git Tag.
• Requirements (Optional): It allows you to upload the requirements file in .txt format.
  • This file is necessary when your code relies on additional Python libraries for execution, and it helps specify which Python libraries need to be installed.
  • Alternatively, a docker image can be specified in docker image URL while editing a service. Also, see the Custom Docker Image article for more information about the docker registry.
• Node Inputs: Input channels are set to be of type: item by default. A Code node can have multiple input and output channels. You can perform the following:
  • Add new inputs by clicking on the plus icon. It allows you to add a new name, new input type, and description.
  • Click on the Pencil icon to update Name and Description. You cannot edit the type.
  • Click Set Parameter to set input parameter for the Code node.
  • For more information, see the Node Inputs article.

• Node Outputs: Output channels are set to be of type: item by default. You can perform the following:
  • Add new outputs by clicking on the plus icon. It allows you to add a new name, new output type, description, and action.
  • Click on the Pencil icon to update Name, Description, and Actions. You cannot edit the type.

When adding output channels, you may need to choose a different type for your output. For example, if a code node connects to another code node that expects a boolean as an input, you will need to define the output of the first code node to be of type boolean.

For information on the Executions, Logs, and Instances tabs, see the Node Details article.

Code node actions

The Code node allows you to perform the following actions:

• Edit Service Settings
• Copy Node ID
• View Executions
• View Logs