Contents x
    Pipelines
    • 05 Aug 2024
    • Print
    • Dark
      Light
    • PDF
    Contents

    Pipelines

    • Updated On 05 Aug 2024
    • Print
    • Dark
      Light
    • PDF
    Article summary

    Overview

    The Pipelines tab within Marketplace displays an extensive selection of pipeline templates, enabling users to effortlessly install and construct pipelines tailored to their specific needs.

    • By installing a pipeline template from the Marketplace catalog, users can utilize these templates as a foundation for creating new, customized pipelines. Installation of a template might result in the installation of a node application.
    • Once installed, these pipeline templates become accessible on the pipeline page, ready for immediate use or further customization.
    • Utilize the filter, search, and sort features to easily locate a specific template and node based on your requirements.
    • Upon selecting a template or node card from the list, a detailed information panel will appear on the right side.

    In the card view, the following details about each pipeline are presented:

    • Pipeline's Image
    • Pipeline's Name
    • Pipeline's Category (for example, Provider, Media Type, License)
    • Last Updated Timestamp

    Utilize the filter, search, and sort features to easily locate a specific pipeline or template based on your requirements. Additionally, upon selecting a pipeline card from the list, the following information panel will appear on the right side.

    • Pipeline Template Install action
    • Use the Pipeline as Template option
    • Auto Update option
    • Available Actions
    • A preview of the pipeline template.
    • The Pipeline Description and a GitHub location link.
    • Details, such as:
      • Status
      • Privacy: Public, Project, or Organization
      • Created, Installed, and Updated Date
      • Email ID of the user who installed it

    The following sections will help you understand in detail.

    Pipeline's Search and Filters

    The methods and criteria used to efficiently locate and select specific pipeline of the Dataloop platform.

    Search a Pipeline

    Use the Search a Pipeline field to type in the name of the pipeline to search it.

    Filter Pipelines by Category

    Marketplace allows you to filter pipelines using categories. In the left-side panel, search pipelines using the following categories. Select one or more categories to refine the list of pipelines according to your choice. To remove the filter, simply click on the selected filter category once more.

    • Deployed By: Organizations or companies that provide pipeline templates and nodes. For example, Dataloop, Google, Meta, AWS, etc.
    • Pipeline Type: The Template category under Pipeline Type highlight the importance of structure and modularity in the design of workflows and pipelines. Templates offer a high-level blueprint for creating consistent and efficient pipelines, allowing for detailed customization and flexibility in processing or automation tasks.
    • Template Type: Each template type caters to a distinct aspect of technology projects, offering a foundation to build upon for specific operational needs. Data Engineering and MLOps templates address the technical requirements of handling data and machine learning models, respectively. Project Management templates provide a framework for overseeing and executing projects efficiently. Lastly, Labeling Workflow templates support the critical task of data annotation in ML projects.
    • License: Selecting AI models based on the specific licensing agreements under which they are released. This filter aids in aligning the choice of AI models with legal compliance, project needs, and ethical considerations. For example, End User License Agreement (EULA), MIT License, Apache License 2.0, etc.

    Manage Pipelines

    Actions Based on Pipeline's Installation Status and Usage Scope

    You can perform actions that are available on each Pipeline based on the scope and installation status of the Pipeline. To learn the actions' availability, refer to the Actions's Availability Based on Use Cases section.

    How to Install a Pipeline Template to the Pipelines?

    1. Open the Marketplace from the left-side menu.
    2. In the Pipelines tab, select a Pipeline that you want to install.
    3. Click Install from the right-side panel. A confirmation message is displayed and is available in the Pipeline page.
    4. Click Use Template to use it as a template. The pipeline page is displayed.
    Information
    • Once you install a pipeline, there is a green tick icon is visible on the pipeline that is installed.
    • Once you save a pipeline as a template, the scope of the DPK is Organization, and the scope of the application is the Project.
    • The application will be installed in the project where it is created.
    • The other org’s project users will be able to see the DPK, but need to install it in their project to start using the template.

    How to Use a Pipeline Template?

    1. Open the Marketplace from the left-side menu.
    2. In the Pipelines tab, select the installed Pipeline.
    3. Click on the Use Template from the right-side panel. The pipeline page is displayed.

    How to Activate or Deactivate an Application?

    Installing a pipeline activates the application

    Once you install a pipeline in your project, the application becomes activated.

    To learn the instructions, refer to the How to Activate or Deactivate an Application?

    For other actions, refer to the following:

    What's Next