Overview

To integrate with Microsoft Azure storage services, platforms like Dataloop support Client Secret-based authentication using a Service Principal. This allows secure, programmatic access to either Azure Data Lake Gen2 or Blob Storage, depending on your storage architecture. Both integrations follow a similar authentication model but target different storage layers and use cases.

Azure Data Lake Gen2 – Client Secret Integration

Security Level: ⭐⭐⭐⭐⭐ (High)

This integration is designed for hierarchical storage systems, typically used for big data analytics, where file system-like structures (folders and directories) are required. Gen2 is built on top of Blob Storage and adds features like POSIX-compliant ACLs and optimized performance for analytics.

Setup Includes:

• Register an application in Azure Active Directory

• Create a Client Secret for the registered app

• Assign roles like:

  • Storage Blob Data Reader

  • Storage Blob Data Contributor

• Ensure the Hierarchical Namespace is enabled in the storage account

• Provide: Account name, Client ID, Client Secret, Tenant ID, and Container or File System name.

Best for:

• Analytics pipelines

• Structured data storage with directory-level permissions

• Integration with big data frameworks (e.g., Hadoop, Spark)

Azure Blob Storage – Client Secret Integration

Security Level: ⭐⭐⭐☆☆ (Moderate)

This integration allows Dataloop to access traditional object-based blob storage containers using a service principal. It’s ideal for general-purpose file storage and object serving, without the need for hierarchical namespaces or analytics-optimized configurations.

Setup Includes:

• App registration and service principal creation in Azure AD

• Generate a Client Secret

• Assign roles like:

  • Storage Blob Data Reader (for read access)

  • Storage Blob Data Contributor (for write access)

• Provide: Account name, Client ID, Client Secret, Tenant ID, and Container name

Best for:

• Image and video storage

• AI/ML data processing

• General file storage with flat hierarchy