Pipeline triggers

Overview

This document provides an overview of pipeline triggers, their execution modes, and configuration options within the Dataloop system. 🚀
The Dataloop Pipeline allows you to configure one or more triggers, enabling automatic execution of the pipeline flow in response to specific events or at predefined intervals. When a trigger is linked to a pipeline, the pipeline runs whenever the trigger is activated.

Triggers can be set based on a time pattern (cron trigger) or specific events within the Dataloop system (event trigger). Triggers can be:

• Time-based (Cron Trigger): Executes at scheduled intervals.
• Event-based (Event Trigger): Executes when a specified action occurs within the Dataloop system.

Event-based triggers

Event triggers are linked to a combination of resources and actions:

• Resources: Entities within the Dataloop system (e.g., Item, Dataset, Annotation, Task).
• Actions: Events that occur on these resources (e.g., Create, Update, Delete, Clone).

A DQL filter can be applied to filter specific resources before triggering the pipeline.

Setting triggers in a pipeline

• The triggered resource object is passed as an input parameter to the pipeline.
• A trigger can be assigned to multiple pipeline nodes, each executing independently when the trigger fires.

Event triggers - resources

The following table lists available resources and their corresponding actions:

• Created: Triggers the pipeline when a new item is added (uploaded or synced to a dataset).
• Updated: Triggers the pipeline when an item is modified. See ** for more details regarding itemStatus.Updated.
• Deleted: Triggers the pipeline when an item is removed.
• Clone: Triggers the pipeline for items generated through a "clone" operation.
• Task Assigned: Triggers the pipeline when an item is assigned to a task. To use this event, set the trigger to ItemStatus.TaskAssigned.
• Status Changed: Triggers the pipeline when a task or assignment status updates (e.g., task marked as "completed", assignment marked as "done").

** Use the trigger "itemStatus.Updated" to run your pipeline over items once their status is changed within a task context ('approved', 'discard', etc.).

Event triggers - actions

Triggers respond to specific actions taken on resources:

Execution modes

Certain events, such as item updates, may occur multiple times for the same entity. The execution mode determines whether the pipeline runs every time the event occurs or only the first time.

• Once – The pipeline executes only the first time an event is triggered for a specific resource. For example, if the resource is an "item" and the action is "Updated," the pipeline will run only for the item's first update.
• Always – The pipeline runs every time the event is triggered for a resource. For example, if the resource is an "item" and the action is "Updated," the pipeline will execute for each update. In this mode, every occurrence triggers the full pipeline flow.

Cron triggers - scheduled execution

The Cron trigger allows pipelines to execute at scheduled time intervals using Cron syntax.
For more details on Cron syntax, visit: Wikipedia - Cron.

Trigger status

Pipeline triggers have three statuses:

1. Created - A new trigger is set but not yet active.
2. Active - The trigger is running and responding to events.
3. Inactive - The trigger is paused and will not respond to events.

Trigger activation & paused pipelines

• When a pipeline is paused, its triggers can remain active to queue events.
• Any new event during pause will be added as a pending cycle.
• Cron triggers do not remain active when a pipeline is paused.

When pausing a running pipeline with active triggers (excluding cron triggers), a popup will appear, allowing you to choose whether to keep the pipeline triggers active or deactivate them.

Update pipeline settings: On pause action

Each pipeline has its own independent settings. Configuring the desired behavior for one pipeline will not impact any of your other available pipelines.

To configure pipeline behavior when paused:

1. Open Settings via the three-dot menu.
2. Choose the desired action from the dropdown list.
3. Click Save Changes the settings to apply them permanently.

Resuming pipelines with active triggers

• When resuming a paused pipeline, all triggers reactivate.
• A dialog will appear to choose whether to process pending cycles.
• Click here for more details.