Contents x
    Private Key Integration
    • 01 May 2025
    • Print
    • Dark
      Light
    • PDF
    Contents

    Private Key Integration

    • Updated On 01 May 2025
    • Print
    • Dark
      Light
    • PDF
    Article summary

    To enable secure communication between Dataloop and your Google Cloud Platform (GCP) environment, the Dataloop platform supports integration using a GCP service account private key.

    This method allows Dataloop to authenticate as a service account, granting controlled access to specific GCP resources such as Cloud Storage buckets or BigQuery datasets. By uploading a private key file (in JSON format), you can establish a direct, project-level connection—enabling seamless data operations while adhering to GCP’s security and IAM best practices.

    Warning: Security Level - Low

    We recommend selecting Cross-Project as the integration type. It is GCP’s most secure option and provides enhanced protection for your data.


    The GCP Private Access Key integration process involves the following steps:

    1. Create an IAM Role

    1. Log in to the Google Cloud Console.

    2. From the left portal menu, select the IAM & admin > Roles.

    3. Click +Create Role.

    4. Enter a role Title.

    5. (Optional) Enter a Description.

    6. (Optional) Enter a role ID. By default, the role ID is generated.

    7. Click +Add Permissions, then search for and add the following permissions:

      1. storage.objects.create
      2. storage.objects.delete
      3. storage.objects.get
      4. storage.objects.list
      5. storage.buckets.get

      Note: The storage.objects.delete permission allows the Dataloop platform to delete dataset items. For more information, see the Downstream section.

    8. Click Create.

    Tip
    • See Create and manage custom roles for more information on creating an IAM role in GCP.
    • To display Roles information, you must select a project in the Google Cloud Console. If not available, create a project.

    2. Create a Service Account

    1. Log in to the Google Cloud Console.
    2. From the left portal menu, select the IAM & admin > Service accounts.
    3. Click +Create service account.
    4. (Optional) Enter a Name and Description for the account.
    5. Click Create and continue.
    6. Click Done to create the service account.
    Tip: See Create service accounts for more information on creating a service account in GCP.

    3. Create a Private Key for the Service Account

    1. Log in to the Google Cloud Console.
    2. From the left portal menu, select the IAM & admin > Service accounts.
    3. Find the desired service account for which you want to add a private key, and click on its name to open its details.
      Note: The Service Account you recently created.
    4. Select the Keys tab.
    5. Click Add key.
    6. Click Create new key from the list. A popup window is displayed.
    7. Ensure the Key type is in JSON format.
    8. Click Create. The "Private key saved to your computer" message is displayed.
    Note
    • Use the key to authenticate as the service account on the Dataloop platform.
    • Save the JSON file in a convenient location, it is required at the integration phase.

    4. Create a Cloud Storage Bucket

    1. Log in to Google Cloud Console.
    2. From the left portal menu, go to the Cloud Storage > Buckets.
    3. Click Create bucket.
    4. Enter a Name for the storage bucket, and click Continue.
    5. In the Choose where to store your data, select a location for the bucket, and click Continue.
    6. Click Create bucket.
    Note: For all other optional settings, use the default values.

    5. Grant Permissions to the Cloud Storage Bucket

    1. Log in to Google Cloud Console.
    2. From the left portal menu, go to the Cloud Storage > Buckets.
    3. Select the Storage bucket for which you want to add permissions, and then click on it.
    4. Select the Permissions tab.
    5. Click Grant access. The Add principals dialog box appears.
    6. Under the Add principals, add your Service account principal.
    7. Under the Assign roles, choose custom and choose the role you recently created.
    8. Click Save. A confirmation message is displayed.
    Tip: See Use IAM with buckets for more information on adding permissions to a storage bucket in GCP.

    6. Create the GCP Private Key Integration

    1. Log in to the Dataloop platform.
    2. From the left-side panel, select Data Governance.
    3. Click Create Integration -> Create Storage Integration. A pop-up window is displayed on the right-side.
    1. Integration Name: Enter a Name for the integration.
    2. Provider: Choose GCP from the list.
    3. Integration Type: Select the Private Key from the list.
    4. Click Import JSON file, and import the GCS Private Key JSON file that you downloaded at Step 3.
    5. Click Create Integration. A confirmation message is displayed.

    7. Create a GCS Storage Drivers

    For more information, see the Create a GCS Storage Driver topic.

    What's Next