Pipelines

Overview

The Pipelines tab within Marketplace displays an extensive selection of pipeline templates, enabling users to effortlessly install and construct pipelines tailored to their specific needs.

• By installing a pipeline template from the Marketplace catalog, users can utilize these templates as a foundation for creating new, customized pipelines. Installation of a template might result in the installation of a node application.
• Once installed, these pipeline templates become accessible on the pipeline page, ready for immediate use or further customization.
• Utilize the filter, search, and sort features to easily locate a specific template based on your requirements.
• Upon selecting a template card from the list, a detailed information panel will appear on the right side.

In the card view, the following details about each pipeline are presented:

• Pipeline's Image
• Pipeline's Name
• Pipeline's Category (for example, Provider, Media Type, License)
• Last Updated Timestamp

Utilize the filter, search, and sort features to easily locate a specific pipeline or template based on your requirements. Additionally, upon selecting a pipeline card from the list, the following information panel will appear on the right side.

• Pipeline Template Install action
• Use the Pipeline as Template option
• Auto Update option
• Available Actions
• A preview of the pipeline template.
• The Pipeline Description and a GitHub location link.
• Details, such as:
  • Status
  • Privacy: Public, Project, or Organization
  • Created, Installed, and Updated Date
  • Email ID of the user who installed it

The following sections will help you understand in detail.

Search a pipeline

Use the Search a pipeline field to type in the name of the pipeline template.

Filter pipelines

The Marketplace offers powerful filtering options to help you find the right pipelines. Use the left-side panel to search and refine pipelines based on various categories, installation status, and privacy.

Filter by category

The Marketplace offers powerful filtering options to help you find the right pipelines. Use the left-side panel to search and refine pipelines based on various categories, including provider, pipeline type, media type, and license. Select one or more filters to narrow your results. To remove a filter, simply click it again.

Provider

Filter by the organization or company that created the pipeline templates such as Hugging Face, Microsoft, Meta, AWS, and others.

Pipeline Type

This category emphasizes the structure and modularity of workflows. Templates provide a foundational blueprint for building pipelines tailored to different needs:

• Data Engineering and MLOps: Focused on managing data and machine learning models.
• Project Management: Designed for planning and executing projects effectively.
• Labeling Workflow: Supports data annotation processes critical to ML development.

Media type

Refine your search based on the type of media the pipeline handles, such as audio, image, text, video, or LiDAR.

License

Filter templates by their licensing terms to ensure legal and ethical compliance. Options include EULA, MIT, Apache 2.0, and more.

Filter by installation status

Marketplace allows you to filter applications based on their installation status, learn more.

Filter by privacy level

Marketplace offers a flexible filtering system that categorizes the Application according to their privacy level, enhancing navigation and selection within the Dataloop platform. Learn more

Manage pipelines

Actions based on use cases

You can perform actions that are available on each application based on the scope and installation status. Learn more section.

Install a pipeline template

1. Open the Marketplace from the left-side menu.
2. In the Pipelines tab, select a pipeline template that you want to install.
3. Click Install from the right-side panel. It creates a pipeline from the template in the active project, and it directs you to the newly created pipeline page.

Use an installed pipeline template

The Marketplace allows you to reuse installed pipeline templates for different use cases. When a template is installed, it creates a pipeline, and that's available on the Pipelines page. To use the same template again for another purpose:

1. Open the Marketplace from the left-side menu.
2. Go to the Pipelines tab and select the installed pipeline.
3. In the right-side panel, click Use Template. A new pipeline will be created based on the selected template.

Install a pipeline node

1. Open the Marketplace from the left-side menu.
2. Select the Applications tab.
3. In the left-side filter -> Application Type, select the Pipeline Node.
4. Select a pipeline node from the list according to the requirement.
5. Click Install from the right-side panel. The Install Application popup is displayed.
6. Click Install Application. The node will be installed successfully, click View Application for more details.

Activate or deactivate a pipeline application

Installing a pipeline activates the application. Once you install a pipeline in your project, the application becomes activated. Learn how to Activate or Deactivate.

More actions

For more actions, refer to the Manage Marketplace article.