Overview

To integrate with Microsoft Azure storage services, platforms like Dataloop support Client Secret-based authentication using a Service Principal. This allows secure, programmatic access to either Azure Data Lake Gen2 or Blob Storage, depending on your storage architecture. Both integrations follow a similar authentication model but target different storage layers and use cases.

1. Azure Data Lake Gen2 – Client Secret Integration

Security Level: ⭐⭐⭐⭐⭐ (High)

This integration is designed for hierarchical storage systems, typically used for big data analytics, where file system-like structures (folders and directories) are required. Gen2 is built on top of Blob Storage and adds features like POSIX-compliant ACLs and optimized performance for analytics.

🔑 Setup Includes:

• Register an application in Azure Active Directory
• Create a Client Secret for the registered app
• Assign roles like:
  • Storage Blob Data Reader
  • Storage Blob Data Contributor
• Ensure the Hierarchical Namespace is enabled in the storage account
• Provide: Account name, Client ID, Client Secret, Tenant ID, and Container or File System name.

✅ Best For:

• Analytics pipelines
• Structured data storage with directory-level permissions
• Integration with big data frameworks (e.g., Hadoop, Spark)

2. Azure Blob Storage – Client Secret Integration

Security Level: ⭐⭐⭐☆☆ (Moderate)

This integration allows Dataloop to access traditional object-based blob storage containers using a service principal. It’s ideal for general-purpose file storage and object serving, without the need for hierarchical namespaces or analytics-optimized configurations.

🔑 Setup Includes:

• App registration and service principal creation in Azure AD
• Generate a Client Secret
• Assign roles like:
  • Storage Blob Data Reader (for read access)
  • Storage Blob Data Contributor (for write access)
• Provide: Account name, Client ID, Client Secret, Tenant ID, and Container name

✅ Best For:

• Image and video storage
• AI/ML data processing
• General file storage with flat hierarchy