Create pipelines

Overview

Dataloop enables you to create pipelines either from scratch or using a template. You can choose from your organization's templates, Dataloop’s marketplace templates, or build one entirely on your own, providing both flexibility and efficiency in developing and deploying data processing and machine learning workflows.

This article guides you through creating pipelines on the Dataloop platform.

Create a pipeline using templates

The Marketplace is a comprehensive repository that contains a wide variety of pre-defined pipeline templates. These templates are designed to cover common use cases and scenarios across different industries and applications, providing a quick and efficient way to get started with pipeline creation. Here’s how it works:

1. Open the Pipelines page from the left-side menu.

2. Click Create Pipeline and select the Use a Template from the list. It opens the Select Pipeline Template popup window.

3. Select a template from the list.
4. Click Create Pipeline. The selected pipeline template will be displayed, and you can configure the available nodes as needed.

Create a pipeline from the scratch

Creating pipelines from scratch is an approach suited for users with specific requirements that cannot be fully met by existing templates, or for those who prefer to have complete control over every aspect of their pipeline design. Here’s how it works:

1. Open the Pipelines page from the left-side menu.

3. Click Create Pipeline and select the Start from Scratch from the list. It opens the Set Pipeline Name popup window.

4. Enter a name for the new pipeline.
5. Click Create Pipeline. The Pipeline window is displayed, and you can configure the available nodes as needed.

Create a pipeline using the SDK

To learn more about creating pipelines in the SDK, click here.

Learn how to Configure & Manage pipelines.