--- title: "Overview" slug: "integrations-overview" updated: 2026-08-04T07:15:45Z published: 2026-08-04T07:15:45Z canonical: "docs.dataloop.ai/integrations-overview" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt > Use this file to discover all available pages before exploring further. # Overview ## Overview Storage Integrations act as the bridge between DDOE and your storage systems. They allow DDOE datasets, Compute Clusters, applications, and pipelines to securely access data stored in cloud or on-premises environments without requiring manual uploads or data duplication. They provide a centralized way to configure and manage storage connections while handling authentication, authorization, and secure credential management. DDOE supports integrations with: - [AWS](https://docs.dataloop.ai/docs/aws-overview) - [Azure](https://docs.dataloop.ai/docs/azure-overview) - [Google Cloud Platform (GCP)](https://docs.dataloop.ai/docs/gcp-overview) - [On-Premises Storage](https://docs.dataloop.ai/docs/on-prem-overview) - NFS - S3 API-compatible storage - Host Path ## Why Use Storage Integrations? Storage Integrations eliminate the need to manually upload and manage large volumes of data inside DDOE. Instead of copying data into the platform, DDOE accesses the data directly from its original storage location. ## Common Uses of Storage Integrations #### Dataset Creation Create datasets directly from connected storage sources such as: - AWS S3 buckets - Azure Blob Storage containers - GCP Storage buckets - On-premises NFS shares - S3-compatible object storage This allows you to work with existing data without re-uploading it. #### Cloud Storage Synchronization Connect cloud storage providers and synchronize data between DDOE and your cloud environment. Examples: - AWS S3 - Azure Blob Storage - Google Cloud Storage This keeps datasets aligned with the storage source. #### Compute Cluster Storage Access Storage Integrations are tightly integrated with [Compute Clusters](https://docs.dataloop.ai/docs/compute-cluster-1). Once attached to a Compute Cluster, the storage becomes available for: - Dataset creation through **Compute Cluster Sync** - AI applications - Services - Pipelines - FaaS workloads This enables workloads running inside Kubernetes to access data directly from connected storage systems. #### Hybrid and Multi-Cloud Environments Organizations can connect and manage storage across: - Multiple cloud providers - On-premises infrastructure - Kubernetes environments --- ## AWS: Why Cross-Account Integration is the Best Choice? AWS Cross-Account Integration is the recommended approach for securely connecting AWS resources to DDOE. Key benefits: - **Least-Privilege Access**: Grants only the permissions required for the integration, reducing security risks. - **Credential Rotation**: Uses IAM roles instead of long-lived access keys, making credential management more secure and easier to maintain. - **Improved Auditing**: Provides detailed logs of all actions performed through the integration, improving visibility and compliance. - **Separation of Responsibilities**: Allows DDOE to access only the required AWS resources without exposing AWS account credentials. Overall, AWS Cross-Account Integration provides a more secure, flexible, and manageable alternative to Access Keys or STS-based integrations. For configuration instructions, see the [AWS Cross Account Integration](https://docs.dataloop.ai/docs/aws-cross-account-integration) article. --- ## GCP: Why Cross-Project Integration is the Better Choice? GCP Cross-Project Integration is the recommended approach for securely connecting Google Cloud resources to DDOE. Key benefits: - **Least-Privilege** **Access**: Grants only the permissions required for the integration, reducing security risks. - **Credential** **Rotation**: Uses IAM roles and service accounts instead of long-lived private keys, improving security and simplifying credential management. - **Improved** **Auditing**: Provides detailed activity logs, making it easier to monitor access and investigate issues. - **Separation** **of** **Responsibilities**: Allows organizations to delegate access to specific resources without exposing project credentials. Overall, **GCP** **Cross-Project** **Integration** provides a more secure, flexible, and manageable alternative to Private Key–based integrations, helping organizations maintain stronger control over their cloud resources while simplifying access management. For configuration instructions, see the [GCP Cross Project Integration](https://docs.dataloop.ai/docs/cross-project-integration) article. --- ## Important Considerations When you set up an external cloud storage: - **Choose a nearby storage region** to improve file loading and annotation performance. - **Provide write access** so DDOE can create thumbnails, converted files, and manage the hidden `dataloop` folder. - **Synchronize file deletions**. If files are removed from external storage, configure **Upstream Sync** or manually update DDOE to keep data and metadata consistent. --- ## Create Storage Integrations This integration allows DDOE to connect to external storage services, such as cloud storage (AWS S3, Google Cloud Storage, Azure Blob Storage), and on-premises systems, for seamless access to datasets and files. Who can create? Only users with the Organization **Admin** or **Owner** roles are authorized to create integrations. - You can manage and import/export datasets between DDOE and external storage systems. - This is especially important for handling large volumes of data used in machine learning and AI projects. Clicking on the **Create Integration** -> **Create Storage Integration** allows you to create the following types of integrations: - [**AWS**](https://docs.dataloop.ai/docs/aws-overview) - [Cross Account Integration](https://docs.dataloop.ai/docs/aws-cross-account-integration) - [Access Key Integration](https://docs.dataloop.ai/docs/aws-access-key-integration) - [STS Integration](https://docs.dataloop.ai/docs/aws-sts) - [**GCP**](https://docs.dataloop.ai/docs/gcp-overview) - [Private Key Integration](https://docs.dataloop.ai/docs/private-key-integration) - [Cross Project Integration](https://docs.dataloop.ai/docs/cross-project-integration) - [**Azure**](https://docs.dataloop.ai/docs/azure-overview) - [Datalake gen2 Client Secret Integration](/docs/azure-datalake-gen2-1) - [Blob Storage Client Secret Integration](https://docs.dataloop.ai/docs/azure-cloud-storage) - [**On-Prem**](https://docs.dataloop.ai/docs/on-prem-overview) - [NFS](https://docs.dataloop.ai/docs/nfs) - [Host Path](https://docs.dataloop.ai/docs/host-path) - [S3 API](https://docs.dataloop.ai/docs/s3-api) --- ## View Your Integrations Access the Integrations page by clicking on the **Integrations** from the left-side menu. The page displays the list of Integrations available in your organization. --- ## Search and Filter Integrations The following list provides the specific criteria of search and filters for Integrations: - **To search**: Search Integrations by **Integrations Name**. - **To Filter**: Filter the listed integrations by the following criteria: - Provider: The available storage providers for the datasets. - AWS - Azure - GCP - Type: The type of the Integrations. - AWS: Cross Account, STS, and Access Key - GCP: Cross Project, and Private Key - Azure: Client Secret - Untrusted Integrations - Filter integrations based on the email ID of the creator. --- ## Integration Details Integrations page displays available Integrations in your organization in a list view. The column values are populated according to the available Integrations. | Column Name | Description | | --- | --- | | **Provider** icon | The icon of the cloud storage provider. | | **Integration Name** | The name of the integrations. | | **Integration Type** | The type of the integration depends on the cloud storage provider. | | **Completed** | It displays whether the necessary steps are completed to establish trust on the Integration. A green-tick mark is displayed if the integration is trusted. If not, a warning icon with a tool-tip explaining the current status is displayed. | | **Created At** | The creation date of the Integration. | | **Created By** | The Avatar of the user who created the Integration. You can see the email ID of the user when you hover. | Click on the **More actions** (**three dots**) to view and perform the following actions: - **Rename Integration** - **Edit Integration**: When you click, you can modify Access Key integrations' **Key** and **Secret** values. - Container Registry integration is editable only from DDOE SDK. - You cannot edit cross integration. - **Copy Integration ID** - **Delete Integration**